Confocal laser endomicroscope with distal MEMS scanner for real-time histopathology

Optical design

Ray-trace simulations were performed using ZEMAX (ver 2013) optical design software to define the parameters for the focusing optics. The design criteria included near diffraction-limited resolution on axis, working distance = 0 μm, and field-of-view (FOV) greater than 250×250 μm2. A single mode fiber (SMF) was used to deliver excitation at λex = 488 nm. Achromatic doublets were used to mitigate chromatic dispersion for fluorescence collection (Fig. 5a). The beam was delivered through a SMF with a mode-field diameter of 3.5 μm, and passes the center of the reflector with 50 μm diameter aperture without truncation. A solid immersion (half-ball) lens with high refractive index (n = 2.03) was used to minimize spherical aberrations from the incident beam and provide comprehensive contact with the mucosal surface. The focusing optics provides a total NA = 0.41, where NA = nsinα, n is the index of refraction of tissue, and α is the maximum convergence angle of the beam. The diffraction limit for lateral and axial resolution is 0.44 and 6.65 μm, respectively, using values of NA = 0.41, λ = 488 nm, and n = 1.3313. Only commercially available lenses with outer diameter (OD) ≤ 2 mm were considered. The optical path was folded whereby the beam exiting the SMF passed through a central aperture in the scanner, and was reflected backwards by a fixed mirror (0.29 mm diameter). This configuration shortens the length of the rigid distal tip to facilitate forward passage of the endomicroscope through a standard (3.2 mm diameter) working channel in medical endoscopes. This capability allows for seamless use as an accessory during routine endoscopy.

Figure 5
figure 5

Folded optical path and endomicroscope packaging. (a) The excitation beam exits the SMF, and passes through the center aperture of the scanner. The beam expands and reflects off a fixed circular mirror back toward the scanner for lateral deflections. The focusing optics consists of a pair of achromatic doublets and a solid immersion (half-ball) lens that provides contact with the mucosal surface. ZEMAX 2013 ( was used for the optical design and ray-trace simulation. (b) Packaging arrangement for the individual components of instrument, including single mode fiber (SMF), scanner, mirror, and lenses, is shown. Solidworks 2016 ( was used for the 3D modeling of the endomicroscope packaging.


A SMF with 3.5 μm mode-field diameter at 488 nm (#460HP, Thorlabs) was used to serve as a “pinhole” to spatially filter out-of-focus light (Fig. 5b). The SMF was enclosed in a flexible polymer tube (#Pebax 72D, Nordson MEDICAL). A length of ~4 meters was used to provide sufficient distance between the patient and imaging system. A pair of 2 mm diameter MgF2 coated achromatic doublets (#65568, #65567, Edmund Optics) and an uncoated 2 mm diameter half-ball lens (#90858, Edmund Optics) were used to focus the beam and collect fluorescence. A stainless steel end tube (4 mm length, 2.0 mm OD, 1.6 mm ID) was inserted in between the polymer and outer tubes to isolate the scanner from vibrations. A medical-grade adhesive was applied to seal the instrument from bodily fluids and during reprocessing. A heat shrink tube was used to protect the interface.


A compact scanner was fabricated based on the principle of parametric resonance14. A 50 μm aperture was etched in the center of the reflector to pass the excitation beam. The expanded beam was deflected laterally in orthogonal directions (XY-plane) in a Lissajous pattern using a set of orthogonal comb-drive actuators. A data acquisition board (#DAQ PCI-6115, NI) was used to create the analog waveform to drive the scanner. Power was provided by high voltage amplifiers (#PDm200, PiezoDrive) via fine electrical wires (#B4421241, MWS Wire Industries). Wiring was performed on electrode anchors. The scanner was driven at frequencies near 15 kHz (fast-axis) and 4 kHz (slow-axis) to achieve a FOV up to 250 μm × 250 μm. Videos could be collected at frame rates of 10, 16, or 20 Hz. These frame rates were used to match the repetition rate of the Lissajous scan pattern, which depends on the value of the scanner X- and Y- driving frequencies29. Details on the tradeoffs among the frame rate, pixel resolution, and scan pattern density are provided in our previous work14.

Imaging system

A solid-state laser (#OBIS 488 LS, Coherent) delivered λex = 488 nm to excite fluorescein for image contrast (Fig. 6a). The fiber pigtail was coupled to an optical filter block via an FC/APC connector (1.82 dB loss) (Fig. 6b). The beam was deflected by a dichroic mirror (#WDM-12P-111-488/500:600, Oz Optics) into the SMF via another FC/APC connector. The power incident on tissue was limited to a maximum of 2 mW to meet the FDA requirements for non-significant risk per 21 CFR 812. Fluorescence passed through the dichroic mirror and a long-pass filter (#BLP01-488R, Semrock). A ~1 m long multimode fiber with 50 μm core diameter was used to transmit fluorescence via a FC/PC connector to the photomultiplier tube (PMT) detector (#H7422-40, Hamamatsu). Fluorescence signal was amplified by a high-speed current amplifier (#59-179, Edmund Optics). Custom software (LabVIEW 2021, NI) was developed to perform real-time data acquisition and image processing. The laser power and PMT gain settings were determined by a microcontroller (#Arduino UNO, Arduino) using a custom-printed circuit board. The SMF and wires were terminated by connectors, and inserted into the fiber (F) and wire (W) ports of the base station (Fig. 6c). The imaging system was contained on a portable cart (Fig. 6d). An isolation transformer was used to limit the leakage current to <500 μA.

Figure 6
figure 6

Imaging system. (a) The PMT, laser, and amplifiers are contained within the base station. (b) In the filter block, the laser (blue) delivers excitation via a fiber pigtail via a FC/APC connector. The beam is deflected by a dichroic mirror (DM) into a single mode fiber (SMF) via a second FC/APC connector. Fluorescence (green) passes through the DM and a long pass filter (LPF) to the PMT via a multimode fiber (MMF). (c) The proximal end of the endomicroscope connects to the fiber (F) and wire (W) ports of the base station. (d) The endomicroscope, monitor, base station, computer, and isolation transformer are contained on a portable cart. (a, c) Solidworks 2016 was used for 3D modelling of the imaging system assembly and the endomicroscope.

Imaging performance

The lateral and axial resolution of the focusing optics was measured from the point-spread-function of 0.1 μm diameter fluorescent microspheres (#F8803, Thermo Fisher Scientific). Images were collected by translating the microspheres in the horizontal and vertical direction in 1 μm increments using a linear stage (#M-562-XYZ, DM-13, Newport). Images were stacked using ImageJ2 to obtain cross-sectional images from the microspheres.

System software

Custom software (LabVIEW 2021, NI) was developed to perform real-time data acquisition and image processing. An overview of the routines used for system operation is shown in Fig. 7. The user interface consists of data acquisition (DAQ), main, and controller panels. The DAQ panel communicates with the main panel to acquire and save raw data, provides input for the user-defined data acquisition settings, and controls the scanner actuation parameters. The main panel allows the user to select the desired configuration to use the endomicroscope, including drive signals to the scanner, video frame rate, and data acquisition parameters. This panel also allows users to display and control image brightness and contrast. Using raw data as the input, an algorithm computes the optimal gain setting for the PMT, and automatically adjusts this parameter using a proportional-integral (PI) based feedback control system16. The controller panel communicates with both the main and DAQ panels to control the laser power and PMT gain.

Figure 7
figure 7

System software architecture. The user interface includes modules for (1) data acquisition (DAQ), (2) main panel, and (3) controller panel. The programs run simultaneously and communicate with each other via message queuing. Key – MEMS: microelectromechanical systems, TDMS: technical data management streaming, PI: proportional-integral, PMT: photomultiplier tube. Image and video files are saved in BMP and AVI format, respectively.

Image processing

A phase correction algorithm was used to compute the variance in image pixel intensities at various values for phase to determine the largest value to sharpen the image15. For real-time correction, the phase was swept over a range of ±2.86° with a relatively large step size of 0.286° to reduce computation time. Also, a partial region of the image with a smaller number of samples was used to further reduce the computation time from 7.5 s (1 M samples) to 1.88 s (250 K samples) per image frame at 10 Hz. These input parameters were chosen to provide sufficient image quality with minimal delay during in vivo imaging. Real-time images and videos are recorded in BMP and AVI formats, respectively. Raw data was saved in Technical Data Management Streaming (TMDS) format.

The in vivo images were post-processed to enhance quality using LabVIEW 2021. The phase correction algorithm had limited accuracy when used during in vivo imaging because of the long computation times required. Only a limited image area and number of samples were used. Moreover, the algorithm was less effective for images with either motion artifact or low contrast, and led to phase miscalculations30. Individual frames with high contrast and without motion artifact were manually selected to fine tune the phase with a phase sweep range of ±0.75° and a step size of 0.01°. The full image area (e.g., 1 M samples for images recorded at 10 Hz) was used. Details on the image parameters used for the real-time and post-processing are summarized in Table S2. After phase correction, image noise was further reduced using a median filter. Brightness and contrast were further enhanced through histogram stretching and gamma correction31.

In vivo confocal images

The clinical study was approved by the Institutional Review Board at Michigan Medicine, and was performed in the Medical Procedures Unit. The study was registered online at (NCT03220711, date of registration: 18/07/2017). Inclusion criteria included patients (ages from 18 years to 100 years) previously scheduled for routine colonoscopy, at increased risk for colorectal cancer, and with a history of inflammatory bowel disease. Informed consent was obtained from each subject who agreed to participate. Exclusion criteria included patients who were pregnant, had known allergy to fluorescein, or were on active chemotherapy or radiation therapy. Consecutive patients scheduled for routine colonoscopy were recruited for this study, and represented the population seen at Michigan Medicine. The study was performed in accordance with the Declaration of Helsinki.

Prior to the procedure, the endomicroscope was calibrated using 10 μm diameter fluorescent beads (#F8836, Thermo Fisher Scientific) fixed in a silicone mold. A translucent silicone sealant (#RTV108, Momentive) was poured into an 8 cm3 3D printed plastic mold. Aqueous fluorescent beads were dropped on top of the silicone and were left until the aqueous media dried out.

A standard medical colonoscope (Olympus, CF-HQ190L) was used to examine the entire colon using white light illumination. Once the endoscopist identified a region suspicious for disease, the site was rinsed with 5-10 mL of 5% acetic acid followed by sterile water to remove mucus and debris. A 5 mL dose of 5 mg/mL fluorescein (Alcon, Fluorescite) was either injected intravenously or topically sprayed onto the mucosa using a standard cannula (M00530860, Boston Scientific) passed through the working channel.

An irrigator was used to wash away any excess dye or debris on the mucosal surface. The spray catheter was removed, and the endomicroscope was then passed through the working channel to collect in vivo images. The distal tip was positioned onto the target region using wide-field endoscopy for guidance. The total time used to collect confocal images was <10 min. White light endoscopy videos were processed with the Olympus EVIS EXERA III (CLV-190) imaging system and recorded using Elgato HD video recorder. Endomicroscopy videos were recorded and saved using LabVIEW 2021. After completion of imaging, the endomicroscope was removed, and either a biopsy forceps or snare was used to resect the tissues imaged. The tissues were processed for routine histology (H&E), and evaluated by an expert GI pathologist (H.D.A.). The spectral properties of fluorescein were confirmed using a spectrometer (USB2000+, Ocean Optics) as shown in Fig. S2.

Endoscope sterilization

The endomicroscope were sterilized after each use in human subjects (Fig. 8). The cleaning procedure was performed under guidance and with approval of the Department of Infection Control and Epidemiology and the Central Sterile Reprocessing Department at Michigan Medicine. Prior to the study, the instruments were tested and validated for sterilization by Advanced Sterilization Products (ASP, Johnson & Johnson), a commercial entity that provides infection prevention and sterilization validation services.

Figure 8
figure 8

Instrument reprocessing. (a) The endomicroscope was placed in a tray after each procedure for sterilization with the STERRAD reprocessing process. (b) The SMF and wires were terminated with fiber and electrical connectors, respectively, that were capped prior to reprocessing.

The endomicroscope was reprocessed using the following steps: (1) The endomicroscope was wiped from the proximal to the distal end with a lint-free cloth soaked in enzymatic detergent; (2) The instrument was immersed in an enzymatic detergent solution for 3 min, rinsed with water, and dried with a lint-free cloth. The electrical and fiber connectors were capped, and left out of the solution; (3) The endomicroscope was wrapped and placed in an instrument tray for sterilization using STERRAD 100NX, a reprocessing system that uses hydrogen peroxide gas plasma technology at relatively low temperature and in a low moisture environment for a standard cycle of 47 min.

Source link

Back to top button