Lens-free Dual-color Fluorescent CMOS Image Sensor for F?rster Resonance Energy Transfer Imaging

In this study, a lens-free dual-color bio-imaging system is developed. This system utilizes a small CMOS image sensor chip as the imager. By coupling alternate lines of blue and yellowgreen filters coated on photodiodes, we can simultaneously detect cyan and yellow fluorescent lights using the device. By equipping the device with a hybrid filter made of a bandpass interference filter, a fiber optic plate (FOP), and a pale-yellow absorption filter, we can achieve an excitation light extinction ratio of approximately −57 dB. By using fluorescent beads of two different colors, we can confirm the operation of the image sensor. Individual images from the cyan and yellow channels can be obtained after data processing, and subsequently the yellow-tocyan image ratio can also be determined. The results show that our device can be used to study Förster resonance energy transfer (FRET) and can thus be applied to investigate the progression of diseases such as cancer.


Introduction
A lens-free imaging system is expected to be used in new applications in bio-imaging. (1)(2)(3)(4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14) This is because a conventional fluorescence microscope is large, bulky, costly, and has a limited field of view. (4,7) A lens-free imaging system, on the other hand, does not include any lenses in its design. The image is directly recorded on a digital image sensing array without being optically focused by a lens. (9,14) This makes the imaging system much smaller and more compact and gives a wider field of view while maintaining relatively good resolution. (4,7,9) Previous research has already shown that the fabrication of small yet high-resolution image sensors is possible for various purposes, such as observing the brain activity of rodents, (15) illuminating and detecting light in deep biological tissues for optical therapy and diagnosis, (16) and optogenetics. (17) Moreover, it is possible to insert a small image sensor in an incubator to observe cultured cells for a long period of time, (3,6,8,11,12) and also possible to develop a similar image sensor for long-term Förster resonance energy transfer (FRET) observation in a small animal.
FRET involves a photophysical mechanism, first discovered by the German scientist Theodor Förster in 1948. (18) Here, an excited donor fluorophore nonradiatively transfers its excitation energy to an acceptor fluorophore in close proximity, and this is usually accompanied by fluorescence emission from the acceptor fluorophore. However, without the presence of an acceptor fluorophore in close proximity, the fluorescence will only be emitted by the donor fluorophore. (18)(19)(20) As FRET is highly sensitive to small changes in distance, typically within 1-10 nm, it has been extensively used as a "ruler" to measure intermolecular and intramolecular distances, enabling the interaction between the molecules to be deduced in cell signaling and biochemistry research. (19) For example, the FRET technique has been applied to study the spatiotemporal dynamics of GTPases, (21) kinases, (22) and proteins. (23) The developments in the aforementioned research can lead to improved knowledge of various diseases such as cancer. Subsequently, a possible method of reversing the development of such diseases might also be developed. (24) Although the FRET technique has successfully been applied to various in vitro studies, applying it to study a living organism is more difficult. The visualization of FRET activity involves determining the ratio of yellow fluorescent protein (YFP) fluorescence intensity (or also known as FRET intensity) to cyan fluorescent protein (CFP) fluorescence intensity before and after stimulation. (22) Small changes in fluorescence signal intensity must be detected to accurately determine the fluorescence intensity ratio. However, the technique is often applied without proper elimination of excitation light and other noises, such as autofluorescence from the materials used and electric signals from the body and devices. Moreover, the fluorescence signal intensity also decreases rapidly with increasing distance between the sample and the sensing region. (14) This makes it difficult to detect small changes in fluorescence emission from FRET activity.
From the viewpoint of devices, the application of lens-free imaging to study FRET is also not an easy task. To observe fluorescence with a known wavelength, the image sensor must be equipped with optical filters, specifically either an interference filter or an absorption filter. Generally, these filters block excitation light and transmit the target fluorescent light. However, utilizing an interference filter in a lens-free setup will cause the transmission spectrum to shift to a lower wavelength when the incident light is not normal, (25) while the absorption filter will usually emit fluorescence that is outside of the rejection band of the absorption filter, (11,12) which might affect the results of experiments.
However, the problem can be solved by combining both an interference filter and an absorption filter to form a hybrid filter. (11,12) This hybrid filter can then be applied to a CMOS image sensor to develop a small and a compact in vivo image sensor that is capable of simultaneously detecting both cyan and yellow fluorescent lights emitted as a result of FRET activity.

Chip design
The image sensor chip is designed and fabricated using 0.35 µm 2-poly 4-metal standard CMOS technology (AMS). The sensing region is made up of an array of 60 × 134, threetransistor active pixel sensors, each with a size of 15 × 15 µm 2 . Further details of the image sensor specifications are listed in Table 1 and a microscope image of the image sensor chip is shown in Fig. 1. The photodiodes occupied most of the area of the chip, while the four squares near the bottom are the bonding pads. These pads are connected to the clock signal (input signals), VDD (positive terminal), GND (ground terminal), and output (output terminal) on the control board.
As the detectors are Si photodiodes, they will detect photons of all wavelengths from the visible to near-infrared range. Hence, additional filters must be added to the image sensor to allow it to selectively detect a target fluorescent light. In addition, the filters are also responsible for blocking excitation light to prevent it from reaching the photodiodes.

Filter design
To choose the best filters for our purpose, a few criteria must be fulfilled. First, all the filters must be capable of transmitting the target fluorescent light. In our work, the target FRET probe comprises two fluorescent proteins: an enhanced cyan fluorescent protein (ECFP, hereafter CFP) and a yellow fluorescent protein for energy transfer (Ypet, hereafter YFP). The fluorescence emission originates from the CFP, with a peak near 474 nm, and the YFP, with a peak near 530 nm. Next, the filters should also be capable of blocking the excitation light to prevent its detection by the photodiodes. This is crucial because the excitation light is very strong and it will saturate the image sensor if not treated properly. Moreover, leaked excitation light will considerably increase the photon shot noise when the signal is weak, for example, during fluorescence imaging, as the photon shot noise is given by the square root of the signal intensity.
To initiate the FRET activity for the CFP and YFP pair, a 435 nm excitation light is chosen as it is the peak excitation wavelength for CFP. Hence, the filters must show a high extinction ratio near this wavelength. Lastly, the filters also should not emit strong fluorescence. As the autofluorescence emitted by the filters is normally outside the rejection band of the filters, this fluorescence will be detected directly by the image sensor in a lens-free design because the filters are in the vicinity of the sensor. This situation is not desirable as it will affect the results of experiments.
By considering the criteria stated previously, we developed a hybrid filter for our device. The hybrid filter structure of our device consists of two main layers. The lower layer is a selective detection layer that consists of alternate lines of blue and yellow-green absorption filters. In contrast, the upper layer is the excitation light removal layer. This layer consists of a 460 nm long-pass interference filter, a fiber optic plate (FOP), a 550 nm short-pass interference filter (the long-pass and short-pass interference filters form a bandpass interference filter), and a yellow absorption filter from the top to the bottom of the layer. This filter structure is illustrated in Fig. 2. The most important function of the excitation light removal layer is to reduce the amount of excitation light so that it is not detected by the image sensor or saturates it. The excitation light removal layer is also responsible for transmitting the target fluorescent light to the lower layer. In our design, the interference filters are on the frontline to remove most of the excitation light illuminated on the image sensor.
However, when using an interference filter, there is a problem in that spectral shifting will occur when the incident light is not normal. (24) As the excitation light might be randomly scattered by an observation target, such as cells or microbeads, the scattered excitation light is no longer normal. This will cause the interference filter to transmit the light. Most of the time, the intensity of the scattered light component will still be much higher than that of the target fluorescence, and thus this component cannot be ignored during fluorescence imaging. (11,12) However, this situation is addressed by the yellow absorption filter in our design.
Here, the yellow absorption filter acts as an emission filter. It absorbs the remaining excitation light that passes through the previous layer, which is either not fully removed by the interference filters or transmitted because of the spectral shifting effect. This filter transmits the target fluorescent light to the lowermost layer for selective detection. Although the absorption filter tends to emit fluorescent light after absorbing the excitation light, this can be prevented by placing the absorption filter at the lowermost layer of the hybrid filter. As the interference filter at the upper layer reflects most of the incident excitation light, the transmitted component will have much lower intensity than the incident excitation light, and the intensity of autofluorescence from the absorption filter will also be extremely low. The transmission spectra of the filters used in the excitation light removal layer with respect to the emission of the fluorescent proteins are shown in Fig. 3.
Here, the transmission spectrum of each filter was measured using a spectrophotometer (JASCO V-670 UV-VIS-NIR Spectrophotometer). The filters were casted on microscope cover glass (Matsunami Glass) under the conditions stated in Sects. 3.1 and 3.2. On the other hand, the fluorescence emission spectra for CFP (26) and YFP (26) were obtained from their database. According to Fig. 3, the bandpass interference filter and the yellow absorption filter are both capable of blocking excitation light near 435 nm and allow cyan and yellow fluorescent lights from the CFP and YFP, respectively, to pass through them, hence fulfilling our filter selection criteria.
Lastly, in the selective detection layer, alternate lines of blue and yellow-green absorption filters are directly coated on the photodiodes. The blue absorption filters are responsible for transmitting cyan fluorescent light and absorbing yellow fluorescent light. The opposite applies for the yellow-green absorption filters. In this way, two different types of filter will coat alternate neighboring pixels. This makes it possible to selectively detect two different fluorescent lights in the same area. The transmission spectra of the filters used in the selective detection layer with respect to the emission of the fluorescent proteins are shown in Fig. 4.
As previously described, the transmission spectrum of each individual filter were measured by using a spectrophotometer (JASCO V-670 UV-VIS-NIR Spectrophotometer). The filters were casted on microscope cover glass (Matsunami Glass) under the conditions stated in Sects. 3.1 and 3.2. According to Fig. 4, the blue absorption filter will transmit most of the cyan fluorescent light emitted by the CFP and block most of the yellow fluorescent light emitted by the YFP. The opposite applies to the yellow-green absorption filter. This shows that the blue and yellow-green filters are capable of selectively detecting fluorescent light in the desired region of the image sensor when they are coated on the sensor.

Fabrication of the selective detection layer
First, a yellow dye solution is prepared by mixing a yellow dye powder (Valifast Yellow 3120, Orient Chemical Corporation) with cyclopentanone and epoxy resin (GA, Cannon Chemical) at a weight ratio of 1:8:1. Then, some of the yellow dye solutions is added dropwise to the image  sensor and spin-coated at a speed of 1500 rpm for 30 s. The dye is then illuminated with UV for 60 s, followed by baking at 150 °C under vacuum for 45 min to form a layer acting as a yellow filter on the image sensor. After that, some green dye (SG-3000L, Fujifilm) is added dropwise to the image sensor and spin-coated at a speed of 3000 rpm to form a layer acting as a green filter with a thickness of about 1.96 µm on the yellow filter, effectively making them a layer of yellow-green filter.
Next, a laser patterning process is carried out using UV pulses of a Nd:YAG laser at 266 nm (Callisto VL-C30, V Technology Co., Ltd.). This laser is used to remove alternate lines of the yellow-green filter from the pixel array, where each line is 15 µm wide.
A cyclo-olefin polymer (E2110-10, Zeocoat, Zeon) is spin-coated at a rotation speed of 4000 rpm followed by baking at 150 °C to form a thin and hard layer of surface protectant. Thereafter, a blue dye (SB-3000L, Fujifilm) is added dropwise to the image sensor and spin-coated at a speed of 4000 rpm for 30 s. Next, a layer acting as blue filter with thickness of about 1.56 µm is coated on the photodiodes previously coated with the yellow-green filter, as well as on the photodiodes that are not coated with a filter.
Lastly, laser patterning is carried out again with green pulses of the Nd:YAG laser at 532 nm. This green laser illuminates the photodiodes coated with the blue and yellow-green filters, and it only removes the blue filter as the yellow-green filter transmits most of the laser light. After the laser patterning process, the pixel array is coated with alternate lines of blue and yellowgreen filters.

Fabrication of the excitation light removal layer
A high-resolution FOP (J5734, Hamamatsu Photonics) is used in our design. The core pitch of this FOP is 3 µm, much finer than the pixel size. Long-pass interference filters of 460 nm wavelength and short-pass interference filters of 550 nm wavelength are coated on the FOP. This process is performed by a company (Tac Coat). After that, some pale-yellow dye solution is added dropwise on the opposite side of the short-pass filter and is spin-coated at a speed of 1000 rpm, followed by UV curing for 60 s and baking at 60 °C under vacuum for 3 h. This forms a layer acting as a pale-yellow absorption filter on the opposite side of the short-pass interference filter. The aforementioned pale-yellow dye solution is prepared by mixing a paleyellow dye powder (FDB-003, Yamada Chemical Co., Ltd.) with cyclopentanone and NOA 63 at a weight ratio of 1:30:30. The thickness of the pale-yellow absorption filter was found to be about 5 µm.

Filter thickness measurement
In order to measure the thickness of a filter, the dye solution of that particular filter is prepared accordingly. Then, the dye solution is added dropwise on microscope cover glass (Matsunami Glass) and spin-coated. The cover glass was illuminated with UV and baked if needed to form a layer acting as that particular filter on the cover glass.
Next, the thickness of the filter casted on the cover glass is measured using a surface profiler (Kosaka Laboratory Ltd., ET200). The surface profiler will record the difference in height between the cover glass and the filter, hence determining the thickness of the filter. For each filter, 5 measurements were taken at different sides of the filter and their average was taken to obtain a more accurate measurement.

Assembly process
The image sensor chip is attached to a printed circuit board (PCB) using epoxy resin, and wire bonding is carried out to connect the image sensor chip to the PCB. The epoxy resin is cured by heating for 10 min at 120 °C. Next, the area surrounding the chip is painted with black antireflection paint (CS-37, Cannon Chemical) to prevent light from being reflected by the PCB during the experiment. Lastly, a piece of thinned high-resolution FOP with a thickness of 0.5 mm is attached to the image sensor chip using epoxy resin to act as a protection layer. This time the epoxy resin is cured for 24 h at room temperature to avoid air bubble formation between the image sensor chip and the FOP. Then, four pieces of wire are bonded to the designated pins on the PCB so that it can be connected to the imaging interface. The excitation light removal layer is stacked on top of the thinned FOP during the experiment. Figures 5(a)-5(c) show photographs of the fabricated device with and without the hybrid filter and a microscopy image of the photodiode region after the fabrication process, respectively.

Excitation light extinction ratio for the excitation light removal layer
The excitation light removal layer consists of a bandpass interference filter and the yellow absorption filter. The extinction ratio for the bandpass interference filter is essentially fixed, and hence we can only modify the yellow absorption filter to achieve the desired extinction  ratio. For fluorescence detection, an excitation light extinction of less than 10 −6 or −60 dB is preferred. Generally, the light extinction ratio of an absorption filter is directly proportional to its thickness. However, up to a certain point, its light extinction ratio increases very slowly, even when its thickness is increased substantially. This is explained by Lambert's law, where the transmittance of monochromatic light through a homogeneous, low-concentration medium becomes asymptotic when the light path length continues to increase. In this case, the optimum thickness of the absorption filter should be determined to achieve a balance between the excitation light extinction ratio and the thickness. For this, an experiment is carried out to determine the excitation light transmission through the excitation light removal layer with different thicknesses of the yellow absorption filter. Here, the filter is placed on top of a power meter (Thorlabs, S130VC), and the excitation light is illuminated from the top. The excitation light is from a fluorescence microscope (Olympus, BX51WI) equipped with a mercury burner (Olympus, U-RFL-T) and an objective lens (Olympus, MPlan N 5x/0.10na). A piece of excitation filter (Thorlabs, MF434-17) is used together with the microscope to limit the central wavelength of the excitation light to 434 nm. During the experiment, another piece of excitation filter (Semrock, FF01-434/17-25) is placed directly below the objective lens to reduce the fluorescence emission from the optical system. A drawing showing the path of light of this experiment setup and the arrangement of the samples is shown in Fig. 6. Note that the longpass and short-pass interference filters were coated on the glass substrate, while the absorption filter (FDB-003) was casted on a microscope cover glass. The filters are added or removed when needed during the experiment. As this setup essentially eliminates most, if not all, of the unwanted fluorescence, the excitation light performance is comparable to that of a fluorescence microscope. The results of excitation light transmission through the hybrid filter are shown in Fig. 7.
From Fig. 7, it was found that when using the absorption filter alone, the excitation light transmission is on the order of 10 −2 . This is far from sufficient for fluorescence imaging. Moreover, when using the absorption filter alone, the absorption filter emits autofluorescence by absorbing the excitation light. The intensity of autofluorescence is usually greater than that of the target fluorescence of CFP or YFP from the cells. We added an interference filter with a cut-on wavelength of 460 nm on top to reduce the autofluorescence. The interference filter reflects the excitation light and does not show autofluorescence. Thus, the excitation light transmission is reduced to over 10 −5 . Although this design considerably increased the performance of the hybrid filter, the excitation light extinction capability of the hybrid filter was further improved by sandwiching a piece of 550 nm short-pass interference filter between the long-pass filter and the absorption filter because the FOP shows some fluorescence. Furthermore, the hybrid filter can transmit the cyan and yellow fluorescent lights, so it will not affect the results of experiments. Lastly, it also reflects any unwanted autofluorescence with wavelengths greater than 550 nm from, for example, cells, living tissues, or chambers. This further reduces the likelihood of capturing unwanted autofluorescence during the experiment. For this hybrid filter design, the excitation light transmission increases to approximately 2 × 10 −6 or −57 dB if we calculate the extinction ratio instead. This excitation light extinction performance should be sufficient for fluorescence imaging.
Next, by examining the relationship between the transmission of excitation light through the filters and the thickness of the absorption filter, we found that for all three filter designs, the excitation light transmission stopped decreasing when the thickness of the absorption filter increased beyond 5 µm. Hence, the optimum thickness of the yellow absorption filter is found to be approximately 5 µm. This thickness can be achieved by spin-coating the yellow dye solution at a speed of 1000 rpm for 30 s.

Angle-dependent sensitivity spectrum of the full device
As the image sensor is equipped with a bandpass interference filter, the incident angle of light affects the sensitivity of the image sensor. Hence, an experiment is carried out to determine the effect of the incident angle of light on the sensitivity of the image sensor with all filters attached to it. The results of this experiment are shown in Fig. 8. According to Fig. 8, there is a decrease in the normalized pixel reading when the incident angle of light increases. The reduction in the normalized pixel reading is more significant for pixels coated with the yellow-green filter than for those coated with the blue filter. This is because the sensitivity of each channel is limited by these two different filters. Referring to Fig. 3, the lower limit of light transmission is limited by the absorption filter (FDB-003). Because the transmission spectrum is shifted to a lower wavelength by tilting the incident light, the spectral shape of the blue channel is almost unaffected. In contrast, the longer-wavelength limit of light transmission for pixels coated with the yellow-green filter is determined by the short-pass interference filter. Thus, the transmission spectrum is significantly affected by the angle because of spectral shift. The sensitivity peak of the yellow-green pixel shifted by approximately 15 nm to a shorter wavelength when the incident angle of light was increased from 0 to 45°. In contrast, there were almost no changes for the peak of the normalized pixel reading spectrum of pixels coated with the blue filter when the incident angle of light was increased. The peak efficiency of both blue and yellow-green pixels was approximately 6%. This value can be improved by reducing the thickness of the FOPs. The total thickness of the FOPs was 7.6 mm. The transmittance of the FOP is estimated to be less than 50% for normal incident light.
Because the target light arrives from all directions during fluorescence imaging, the image sensor will detect oblique fluorescent light with a lower intensity than the actual intensity. As the sensitivity of yellow-green pixels decreases more significantly than that of blue pixels, two neighboring pixels coated with different filters will detect yellow fluorescent light rather than cyan fluorescent light when these lights arrive from the same source. As the ratio of yellow fluorescent light to cyan fluorescent light is needed to determine the FRET activity, this might cause the calculated ratio to be lower than the actual ratio. Although the performance of the image sensor decreases when the incident light is not normal, it can work very well when most of the fluorescent light is from the normal direction. However, when the fluorescent light is expected to arrive from all directions, some countermeasures can be taken. For example, the high-resolution FOP can be replaced with a low-numerical-aperture (low-NA) FOP to reduce light with angles greater than a certain threshold, a light guide array can be used, (24) or the device can be equipped with an angle-selective grating (12) to transmit normal incident light only.

Experimental results of imaging two-color fluorescent microbeads
In this experiment, the fluorescent light emitted by fluorescent microbeads of two different colors is observed using our device. The fluorescent microbeads we used were blue-green (F8830, Thermo Fisher Scientific) and yellow-green (F8844, Thermo Fisher Scientific). The nominal bead diameter of the blue-green and yellow-green microbeads are 10 and 15 µm, respectively. On the other hand, the absorption and emission spectra of the blue-green and yellow-green microbeads are 430/465 nm and 505/515 nm respectively. Both types of microbead are added dropwise to a piece of microscope cover glass (Matsunami Glass) with a thickness of approximately 0.05 mm and mixed. The cover glass containing the microbeads is then placed on the device after the device is set up under a fluorescence microscope. Excitation light was irradiated from above by a Hg lamp built in the fluorescence microscope (Olympus, BX51WI). The objective lens was an Olympus MPlan N 5x/0.10NA. A piece of excitation filter (Thorlabs, MF434-17) was used together with the microscope to limit the central wavelength of the excitation light to 434 nm. The fluorescence from the microbeads was captured by using both our devices as well as the fluorescence microscope. Here, our imaging device can obtain a twocolor image with a single shot, although the filter cubes must be exchanged for blue and yellow channels when using the fluorescence microscope. The microscopy images serve as a reference for the image captured by our device.
Initially, the image captured by our device consists of the data in both the cyan and yellow channels. Postprocessing work was carried out using MATLAB (MATLAB R2018b). The cyan-and yellow-channel images were extracted as shown in Fig. 9. In contrast to the microscopy images, the results show that our lens-free device can detect and capture fluorescent light in the cyan and yellow color regions separately. As the pixel size of our device is 15 µm and alternate lines of blue and yellow-green filters are coated on the pixel, the spatial resolution of the cyan and yellow channels of our device is reduced to 30 µm in the vertical direction in the image. However, this spatial resolution is still sufficient for bio-imaging purposes.
After extracting the cyan and yellow channel data, the ratio of the yellow-to-cyan channel data is obtained. To obtain a clearer ratio image, the base 10 log of the ratio of the yellow-tocyan channel data is taken and plotted. In this image, a region with a value greater than 0 is occupied by yellow-green beads, while a region with a value smaller than 0 is occupied by bluegreen beads. Next, the cyan-and yellow-channel images obtained using our device are merged and overlaid on the ratio image to obtain a reference for the type of microbead that occupied a certain region on the device. Figure 10(a) shows a base 10 log plot of the ratio of yellow-to-cyan channel data. This image indicates the color of the fluorescence but not the intensity. The dark area is noisy but also colored. Figure 10(b) shows the overlaid image obtained by combining the ratio image with the cyan-and yellow-channel images. In this image, both the position and color of the beads can be verified. According to Fig. 10, our device is capable of obtaining the ratio of yellow-to-cyan channel data and displaying it in the postprocessed image. This is  particularly important as this device will be used to study FRET activity. To determine the spatiotemporal dynamics of a pair of molecules, the FRET-to-CFP ratio is needed, and this ratio will be given by the ratio of yellow-to-cyan channel data obtained by our device. By continuously collecting all the pixel readings, i.e., "taking a movie" using the device, we can obtain the changes in the ratio with time.

Conclusions
We reported a lens-free dual-color CMOS image sensor equipped with a high-performance hybrid filter to reject excitation light. This device has an excitation light extinction ratio of approximately −57 dB and is capable of simultaneously detecting cyan and yellow fluorescent lights with peaks near 474 and 530 nm, respectively. The operation of the device is verified with fluorescent beads of two different colors. The device successfully captured cyan and yellow fluorescent lights emitted by the beads. The fluorescent lights captured by the device can later be separated into the cyan-channel only and yellow-channel only by postprocessing. Subsequently, the ratio image of yellow-to-cyan channel data can be obtained. This shows that our image sensor can be used to study FRET activity, which requires the simultaneous detection of two different fluorescent lights and their ratio. This device will be applied to in vitro FRET imaging to observe the FRET activity in a cultured cell. We also hope to fabricate a similar device with a smaller thickness and implant it into the body of a small animal for in vivo FRET imaging.