Pixel Aperture Technique in CMOS Image Sensors for 3D Imaging

We propose a pixel aperture technique in a complementary metal oxide semiconductor (CMOS) image sensor for 3D imaging. In conventional camera systems, the aperture is located between the object and the CMOS image sensor (CIS); this type of image sensor consists of a pixel array with red, green, and blue (RGB) Bayer pattern color filters. Our proposed image sensor uses red, green, blue, and white (RGBW) (without color filter) filters, and the aperture is located on the W pixel. A sharp image can be obtained from the W pixels, and the RGB pixels produce a defocused image with blurring. The sharp image can be compared with the defocused image to obtain depth information for 3D imaging. A metal layer, such as aluminum in the conventional CIS process, is used for the aperture on the white pixel. We designed and simulated a pixel model for the pixel aperture technique using a 0.11 μm CIS process and evaluated the performance of the proposed technique using finite-difference time-domain (FDTD) analysis.


Introduction
Many techniques such as stereo vision, structured light, and time-of-flight (TOF) are used for 3D imaging. However, these techniques require multiple cameras, a light source, or complex systems. (1)(2)(3)(4)(5)(6) Therefore, we proposed a pixel aperture technique, which has many advantages compared with conventional 3D imaging techniques. The most important advantage of the pixel aperture technique is that it requires only a single image sensor and can simultaneously acquire both color and depth images. The proposed technique is based on the depth from the defocus (DFD) method. (7)(8)(9) Additionally, the proposed technique is implemented in the active pixel sensor (APS) array of the complementary metal oxide semiconductor (CMOS) image sensor (CIS); this feature offers significant advantages, such as low power, low cost, and the possibility to integrate circuits on the same chip. (10,11) Color filters with a Bayer arrangement are typically integrated in the APS array. However, a white filter with a pixel aperture is used as one of the color filters in the proposed image sensor; the use of a white filter increases sensitivity. (12)(13)(14) The proposed image sensor has numerous applications, such as human-motion detection, face recognition, and object reconstruction.
A light-field image sensor in a 180-nm CMOS which was previously proposed also uses both sharp and blurred images to obtain depth information. (15) However, the image sensor proposed in this paper is more effective for color implementation, and the micolenses are used for higher sensitivity of the pixels. In addition, it is possible to implement a high resolution image sensor with a 4-transistor pinned photodiode. Figure 1 shows a conceptual diagram of the proposed camera system. The parallel rays are incident on the camera lens and are focused on a single point F. The image sensor is located behind F. With a normal image sensor, a blurred image with a diameter of 5 pixels would be formed. However, with the proposed image sensor, rays from the outer part of the camera lens are partially or completely blocked by the pixel aperture depending on the location of the pixels. Therefore, a sharp image can be obtained when a pixel aperture image sensor is used instead of a normal image sensor. Figure 2 shows the pixel array with red, green, blue, and white (RGBW) color filters in the pixel aperture technique. The size of a pixel is 2.8 × 2.8 μm 2 in the 0.11-μm CIS process, and the  aperture is placed on the white pixel, which does not have a color filter. A white filter is ideal for the masked pixel because such a filter ensures maximum light transmission. The image constructed by the white pixel array is less blurred than the color image. The two images can be compared to determine the depth of objects. The sharp image from the white pixels can be used as reference image to obtain a depth image by DFD method. Depth calculation is implemented by depth estimation using interpolation of a point spreading function (PSF). The spread factor of a PSF is defined as the standard deviation of the distribution of the PSF. The spread factor of the PSF of a defocused image corresponds to an object's distance. Therefore, the distance of the object can be calculated using the DFD method.

Pixel aperture design
Figures 3(a) and 3(b) show the designs for the RGB pixel and the white pixel. The RGB pixel is constructed vertically using a microlens, a planar layer, a color filter, a metal layer, and a photodiode on a substrate. When compared with the RGB pixel, the white pixel without a color filter has an aperture designed with a metal layer. Optical photons are absorbed by the photodiode, and absorbed photons are converted to electrical signals by a pinned photodiode in the 4-transistor (4-Tr) APS.
A simple model of the pixel aperture can be described as follows: The incident light on the pixel has an effective incidence angle (θ 0 /2) and an effective f-number (F 0 ). The relationship between the aperture diameter (w) and the microlens height (h) is given by where n is the refractive index of the oxide layer. Equation 1 indicates that a smaller pixel aperture diameter corresponds to a larger effective f-number.  The models for the pixel array with and without microlenses are shown in Figs. 4(a) and 4(b), respectively. A finite-difference time-domain (FDTD) simulation is implemented in the pixel aperture region of the design model. The structure is designed in the CIS process, and the refractive index of each layer is considered. The overall characteristics of the proposed image sensor are summarized in Table 1. Figure 5 shows simulation results that demonstrate the variation of the optical power with the incidence angle (θ/2) as a function of the aperture diameter (w). The range of the incidence angle in relation to the f-number is −15° to 15°; in addition, the optical power in the photodiode surface as a function of the incidence angle is simulated by FDTD analysis. (16) The optical power of the pixel with an aperture is maximum at 0°, and most of the light passes through the pixel aperture. The optical power decreases when the incidence angle increases because the light is gradually blocked by the pixel aperture. Thus, light having a high incidence angle is reflected by the metal layer of the pixel aperture. The optical power increases with aperture diameter; an aperture diameter of 0.7-0.82 μm yields reasonable results for the pixel aperture technique owing to the sharp slope.

Results and Discussion
The 2D structure of an APS array with a pixel aperture is shown in Fig. 6. In the 2D structure, the light rays near F are approximated by a Gaussian beam. The distance D between the focal point and the imaging plane is 120 μm, and the size of the 21-pixel array is 58.8 μm. Figure 7 shows    the light distribution using a Gaussian beam approximation. The results confirm that more light is reflected by the pixel aperture at the side of the pixel array than at the center of the pixel array. Figure 8 shows the results for optical power as a function of pixel position analyzed by the FDTD method. The optical power of an array with a pixel aperture has a sharper slope than the optical power of an array without a pixel aperture. The optical power of the array with a pixel aperture decreases as a function of the pixel position, as described in Fig. 8. Evaluation of the optical power based on pixel position shows that the optical power decreases by 80% at the side of the pixel array when compared to the corresponding value at the center of the pixel array. The optical power of an array with a pixel aperture is higher than that of an array without a pixel aperture in the center of the pixel array because of the effect of reabsorbed photons by the pixel aperture.  The results in terms of the graphs and the images of a PSF representing characteristics of image blur distribution are shown in Fig. 9. The graphs and images of the PSF are obtained using a 2D interpolation algorithm. When compared with the optical power with a pixel aperture, optical power without a pixel aperture yields a flatter graph and a blurrier image of the PSF.

Conclusions
In this study, we proposed a pixel aperture technique to obtain a sharp image from W pixels and a defocused image with blur from RGB pixels. Both the sharp image and the defocused image can be used to obtain depth information on the object. In the proposed CIS, we use RGB pixels and a W pixel in which the aperture is located. The performance of the pixel aperture technique was evaluated by FDTD analysis.
We have designed the pixel aperture using a metal layer based on the 0.11-μm CIS process and confirmed that the pixel aperture technique is appropriate for 3D imaging by evaluating optical characteristics of the pixel array.