High Fidelity Full-Color Optical Sectioning Structured Illumination Microscopy by Fourier Domain Based Reconstruction

Abstract

The natural color of biological specimens plays a crucial role in body protection, signaling, physiological adaptations, etc. Full-color optical sectioning structured illumination microscopy (OS-SIM) color is a promising approach that can reconstruct biological specimens in three-dimension meanwhile maintaining their natural color. Full-color OS-SIM takes the advantages of rapid imaging speed, compatibility with fluorescence and non-fluorescence samples, compact configuration, and low cost. However, the commonly used HSV-RMS reconstruction algorithm for full-color OS-SIM faces two issues to be improved. One is the RMS (root-mean-square) OS reconstruction algorithm is prone to background noise, and the other is the reconstruction is bound in RGB and HSV color spaces, consuming more reconstructing time. In this paper, we propose a full-color Fourier-OS-SIM method that allows for the OS reconstruction using the high-frequency spectrum of the sample and thus is immune to the low-frequency background noise. The full-color Fourier-OS-SIM directly runs in the RGB color space, providing an easy way to restore the color information. Simulation and experiments with various samples (pollen grains and tiny animals) demonstrate that the full-color Fourier-OS-SIM method is superior to the HSV-RMS method regarding background noise suppression. Moreover, benefiting from the background noise suppression merit, the quantitative morphological height map analysis with the full-color Fourier-OS-SIM method is more accurate. The proposed full-color Fourier-OS-SIM method is expected to find broad applications in biological and industrial fields where the 3D morphology and the color information of objects both need to be recovered.

1. Introduction

Insects use colors for various reasons, including body protection, signaling, and physiological adaptations [1,2,3,4]. Colors convey multiple channels of information, which are valuable for species identification, distinguishing individual quality, and revealing ecological or evolutionary aspects of animals’ lives. For example, ants might use colors principally for camouflage, while wasps’ and bees’ colors have confirmed linkages to thermoregulation, which is increasingly important in the face of global climate change. Moreover, wasps use colors for various types of signaling. Color variations of well-chemically defended social insects are the mimetic model for unprotected organisms. Insect coloration is an excellent source for finding new ecological indicators and biomarkers closely interconnected with physiological and biochemical processes. Due to novel digital imaging techniques, software, and artificial intelligence, there are emerging possibilities for new advances in this topic.

Full-color optical sectioning structured illumination microscopy (OS-SIM) is an emerging digital imaging technique that can reconstruct tiny samples, e.g., insects, in three dimensions (3D) with natural color [5]. With the merits of rapid imaging speed, compatibility with fluorescence and label-free samples, and low-cost, the full-color OS-SIM has found applications in various studies [6,7,8]. Two steps are essential in the full-color OS-SIM method: (1) utilizing the routine OS-SIM method to produce a monochrome OS image, and (2) colorizing the OS image in a color space.

In the OS image production step, the initial OS-SIM reported by Neil et al. employs a root-mean-square (RMS) algorithm to process the three raw phase-shifted fringed images in a shift interval of 2π/3 to generate a slice of OS image [9]. Despite the simple procedure and fast running speed, the RMS algorithm disregards the interference of the background noise, which in each raw image is also involved in the RMS operation, being amplified in the OS image reconstruction. Especially in the case of raw images with a low signal-to-noise ratio (SNR) and low fringe contrast, the OS image reconstructed by the RMS algorithm often exhibits a low SNR and even residual fringes. To address the RMS algorithm shortage, researchers have proposed various improved algorithms. Santos et al. introduced a HiLo microscopy to obtain optically sectioned images using both a structured and a uniform illumination [10,11]. By combining the high-frequency component from the uniform illumination image with the low-frequency component from the structured illumination image, the HiLo microscopy makes a proper adjustment to ensure a seamless fusion of both high- and low-frequency components, giving rise to an in-focus image that contains all frequency components. However, the adjustment of the ratio of the high- and low-frequency components is always empirical and varies with samples [12]. Patorski et al. proposed a FABEMD-HS reconstruction framework using two raw phase-shifted fringed images with a shift interval of π [13]. The two raw images are subtracted from each other, and then the subtraction is decomposed in a series of bi-dimensional intrinsic mode functions (BIMFs) according to the fast and adaptive bi-dimensional empirical mode decomposition (FABEMD) [14]. The BIMFs are selectively filtered and finally used to reconstruct the OS image through a 2D spiral Hilbert transform. Even though the background noise is reduced by selective filtration, the decomposition and 2D spiral Hilbert transform are complex and time-consuming. Zhou et al. improved the FABEMD-HS method and proposed a sequence Hilbert transform (SHT) based on the 1D Hilbert transform [15]. SHT requires two arbitrary phase-shifted raw fringe images, releasing the burden that the phase-shifting must be as accurate as π for the FABEMD-HS. The complex decomposition and 2D spiral Hilbert transform in the FABEMD-HS are replaced by a simple 1D Hilbert transform. However, the SHT algorithm still suffers from background noise interference due to the lack of a noise suppression scheme. Nowadays, OS image reconstruction is greatly facilitated by deep learning technologies [16,17]. Yet, the robustness and precision of the reconstruction remain challenging.

In the colorizing step of OS image reconstruction, Qian et al. proposed an HSV (hue, saturation, value) color restoration algorithm for the gray OS-SIM image [5]. A white light LED source and a color camera were used in the DMD (digital micromirror device)-based SIM setup to acquire the raw color-fringed images. All raw images are transformed from the RGB to the HSV color space. Then, each of the HSV channels is processed independently with the RMS algorithm to reconstruct the OS image in the HSV color space. Finally, the HSV OS image is transformed back to the RGB color space for display. As mentioned above, the RMS algorithm is prone to background noise. The image transformations between RGB and HSV color spaces slow down the reconstruction speed.

In this paper, we propose a full-color OS-SIM method in the frequency domain, termed full-color Fourier-OS-SIM. By using the high-frequency spectrum of the sample to reconstruct the OS image, the Fourier-OS-SIM algorithm avoids interference from the low-frequency background noise. The signal-to-noise ratio is further improved with a Fourier filter. Implementation-wise, the computational simplicity in RGB color space surpasses the complexity of the HSV-RMS algorithm.

2. Principle of Full-Color Fourier-OS-SIM

SIM generally uses sinusoidal or cosine fringe patterns for illumination, which are the fundamental modes of the Fourier realm. According to the Fourier series expansion theory, any function can be expressed by the combination of serial sinusoidal and cosine functions. The intensity distribution of the fringe pattern is expressed as:

$$I(r)=I_0[1+m\mathit{cos}(2\pi k_{0}r+\phi )]$$

(1)

where r indicates the coordinate in the object plane; I₀, m, k₀, and φ represent the mean intensity, modulation depth, spatial frequency, and initial phase, respectively.

On a light microscope, the signal light (reflection light or fluorescence) intensity F(r) is linearly proportional to the illumination intensity I(r) and the distribution of object S(r), that is, F(r) = I(r)·S(r). Because the imaging is a convolution process, the recorded image D(r) can be expressed as:

$$D(r)=F(r)⨂PSF(r)=[S\left(r\right)I(r)]⨂PSF(r)$$

(2)

Here, PSF(r) is the point spread function of the microscope, and the symbol ⊗ denotes the convolution operation. Substitution of Equation (1) into Equation (2) results in:

$$D(r)=I_0\left\{S\left(r\right)\right[1+m\mathit{cos}(2\pi k_{0}r+\phi )]\}⨂PSF(r)$$

(3)

Applying Fourier transform to Equation (3), we obtain the spectrum of the structured illumination image:

$$\tilde{D}\left(k\right)=I_0\left[\tilde{S}\left(k\right)+\frac{m}{2}{e}^{i\phi }\tilde{S}\left(k\right)⨂\delta \left(k-k_0\right)+\frac{m}{2}{e}^{-i\phi }\tilde{S}\left(k\right)⨂\delta \left(k+k_0\right)\right]OTF\left(k\right)$$

(4)

Here, the symbol ~ represents the corresponding Fourier spectrum, k denotes the spatial frequency, OTF(k) is the optical transfer function by the Fourier transform of PSF(r). $\tilde{S}(k)$ refers to the conventional Wide-Field sample spectrum. $\tilde{S}(k+k_0)$ and $\tilde{S}(k-k_0)$ correspond to the sample’s spectra with central frequency shifted by a distance of ±k₀, respectively.

In order to solve the three unknown spectral components of $\tilde{S}(k)$, $\tilde{S}(k-k_0)$ and $\tilde{S}(k-k_0)$ in Equation (4), at least three independent equations are needed, which are generally obtained by varying the initial phase φ by three times. Specifically, the three-step phase-shifting has an interval of 2π/3. Thus, the following matrix equation is created:

$$\left[\begin{array}{c}{\tilde{D}}_1(k)\\ {\tilde{D}}_2(k)\\ {\tilde{D}}_3(k)\end{array}\right]=I_{0}OTF(k)\left[\begin{array}{ccc}1& \frac{m}{2}{e}^{-i\phi }& \frac{m}{2}{e}^{i\phi }\\ 1& \frac{m}{2}{e}^{-i(\phi +\frac{2\pi }{3})}& \frac{m}{2}{e}^{i(\phi +\frac{2\pi }{3})}\\ 1& \frac{m}{2}{e}^{-i(\phi +\frac{4\pi }{3})}& \frac{m}{2}{e}^{i(\phi +\frac{4\pi }{3})}\end{array}\right]\left[\begin{array}{l}\tilde{S}(k)\\ \tilde{S}(k+k_0)\\ \tilde{S}(k-k_0)\end{array}\right]$$

(5)

The solution of Equation (5) can be rewritten as:

$$\left\{\begin{array}{l}\tilde{S}(k)=\frac{1}{3I_0}\left[{\tilde{D}}_1(k)+{\tilde{D}}_2(k)+{\tilde{D}}_3(k)\right]/OTF(k)\\ \tilde{S}(k+k_0)=\frac{2e^{i{\phi }_0}}{3m{I}_0}\left[{\tilde{D}}_1(k)+e^{i\frac{2\pi }{3}}{\tilde{D}}_2(k)+e^{i\frac{4\pi }{3}}{\tilde{D}}_3(k)\right]/OTF(k)\\ \tilde{S}(k-k_0)=\frac{2e^{-i{\phi }_0}}{3m{I}_0}\left[{\tilde{D}}_1(k)+e^{i\frac{4\pi }{3}}{\tilde{D}}_2(k)+e^{i\frac{2\pi }{3}}{\tilde{D}}_3(k)\right]/OTF(k)\end{array}\right.$$

(6)

Making the inverse Fourier transform for Equation (6), we obtain:

$$\left\{\begin{array}{l}S(r)\otimes PSF(r)=\frac{1}{3I_0}\left[D_1(r)+D_2(r)+D_3(r)\right]\\ F{T}^{-1}\left\{\tilde{S}(k+k_0)\right\}\otimes PSF(r)=\frac{2e^{i{\phi }_0}}{3m{I}_0}\left[D_1(r)+e^{i\frac{2\pi }{3}}{D}_2(r)+e^{i\frac{4\pi }{3}}{D}_3(r)\right]\\ F{T}^{-1}\left\{\tilde{S}(k-k_0)\right\}\otimes PSF(r)=\frac{2e^{i{\phi }_0}}{3m{I}_0}\left[D_1(r)+e^{i\frac{4\pi }{3}}{D}_2(r)+e^{i\frac{2\pi }{3}}{D}_3(r)\right]\end{array}\right.$$

(7)

According to the OS-SIM principle by Neil et al., an optically sectioned image can be reconstructed by the equation [11]:

$$\begin{array}{l}{D}_{os}(r)=\left|D_1(r)+e^{i\frac{2\pi }{3}}{D}_2(r)+e^{i\frac{4\pi }{3}}{D}_3(r)\right|\\ \hspace{1em}\hspace{1em}\hspace{1em}=\left|D_1(r)+e^{i\frac{4\pi }{3}}{D}_2(r)+e^{i\frac{2\pi }{3}}{D}_3(r)\right|\\ \hspace{1em}\hspace{1em}\hspace{1em}=\frac{\sqrt{2}}{2}\sqrt{{\left(D_1(r)-D_2(r)\right)}^2+{\left(D_2(r)-D_3(r)\right)}^2+{\left(D_3(r)-D_1(r)\right)}^2}\\ \end{array}$$

(8)

Here, the symbol | | denotes the modulus operation. Equation (8) is the core of the so-called root-mean-square (RMS) algorithm. The Wide-Field (WF) image paired with the optically sectioned image in Equation (8) is in the form:

$$D_{WF}\left(r\right)={\frac{1}{3}[D}_1\left(r\right)+D_2\left(r\right)+D_3\left(r\right)]$$

(9)

Combining Equations (7) and (8), the OS image can be rewritten as:

$$\begin{array}{c}{D}_{os}\left(r\right)=\left|\frac{3m{I}_0}{2e^{i{\phi }_0}}F{T}^{-1}\left\{\tilde{S}\left(k+k_0\right)\right\}\otimes PSF\left(r\right)\right|\\ \hspace{1em}\hspace{1em}\hspace{1em}=\left|\frac{3m{I}_0}{2e^{-i{\phi }_0}}F{T}^{-1}\left\{\tilde{S}\left(k-k_0\right)\right\}\otimes PSF\left(r\right)\right|\\ \end{array}$$

(10)

where FT⁻¹ represents the inverse Fourier transform. Equation (10) provides a new OS image reconstruction scheme with $\tilde{S}(k+k_0)$ or $\tilde{S}(k-k_0)$ in the Fourier domain (termed Fourier-OS-SIM). This algorithm possesses two significant advantages. On the one hand, $\tilde{S}(k+k_0)$ and $\tilde{S}\left(k-k_0\right)$ are in high frequency regions; thus, the reconstruction process surpasses the primary low-frequency background noise interference. On the other hand, the Fourier-OS-SIM runs in the frequency domain, facilitating the introduction of frequency filters to further suppress the low-frequency background.

For the colorization of the OS image, the HSV-RMS algorithm adopts the general monochrome RMS algorithm (Equation (8)) in the HSV color space, as illustrated in Figure 1a. Firstly, the three phase-shifted color-fringed images are acquired and transformed into the HSV color space. Then, the RMS algorithm and the WF image reconstruction (Equation (9)) are applied to each raw image in the H, S, and V channels, resulting in the OS and WF images in the HSV color space. Finally, the OS and WF HSV images are transformed back to the RGB color space and multiplied with each other in the R, G, and B channels, producing the full-color OS-SIM image.

The HSV-RMS OS-SIM algorithm must transform the images between RGB and HSV spaces in split color channels, lagging the reconstruction speed. Moreover, the RMS algorithm is prone to background noise due to lacking a noise suppression mechanism. To address these problems, we propose the full-color Fourier-OS-SIM framework, whose workflow is presented in Figure 1b. The three phase-shifted color-fringed images are firstly changed into gray intensity images, and simultaneously, they are cloned to reconstruct the color WF image (Equation (9)). The full-color OS image is produced by the Fourier-OS-SIM (Equation (10)) and then colorized by matching the WF image color. It is worth noting that the coloring process is realized by storing the R:G:B ratio in each pixel of the WF image and accordingly copied to the gray OS image pixel by pixel. Compared to the HSV-RMS algorithm, the full-color Fourier-OS-SIM algorithm directly runs in the RGB color space with a concise procedure. Moreover, the full-color Fourier-OS-SIM can yield higher SNR OS images than the HSV-RMS, because it intrinsically avoids the primary low-frequency background noise interference and is easily adopted with the noise suppression filters.

3. Simulation of Full-Color Fourier-OS-SIM

To highlight the better performance of the Fourier-OS-SIM in suppressing background noise, full-color imaging simulations were conducted using both the Fourier-OS-SIM and the HSV-RMS for comparison. A ground-truth image of the DNP (Dai Nippon Printing, Tokyo, Japan) standard color bar chart with 1024 × 768 pixels is taken as the target [18]. To demonstrate the optical sectioning effect, we apply a steady blur filter along rows of the target image. Specifically, a Gaussian blur kernel size is proportionally increased with the row number and then convoluted with the target image to generate an image with a clear upper part and blurred lower part. The fringed raw images are created by multiplying the blurred target image with sinusoidal fringes with three phase shifts of 0, 2π/3, and 4π/3. To be more practical, we perform the simulation in a noisy scenario with the Gaussian noise model. The noise strength is measured by the standard variance σ² of the Gaussian function (σ² = 0.02, 0.04, 0.08, and 0.16, respectively).

The Wide-Field, HSV-RMS, and Fourier-OS-SIM images are reconstructed by using the three raw images as input according to the workflows in Figure 1. Figure 2 presents the ground truth image and one of the structured illumination raw images, as well as the Wide-Field, HSV-RMS, and Fourier-OS-SIM images at various noise variances. It can be seen that both the HSV-RMS and the Fourier-OS-SIM images retain the clear upper regions indicated by the sinusoidal patterns while removing the blurred lower parts (becomes black). This reflects that both the HSV-RMS and the Fourier-OS-SIM have the optical sectioning ability. The calculated SNR values suggest that the Fourier-OS-SIM images have higher SNR values than the HSV-RMS images in each case. This proves that the Fourier-OS-SIM can yield higher quality OS images than the HSV-RMS with a better background noise suppression merit. At the color facet, both the HSV-RMS and the Fourier-OS-SIM correctly retrieve the target color displayed as the Wide-Field images.

4. Experiment Setup

In order to validate the full-color Fourier-OS-SIM algorithm, a digital micromirror device (DMD)-based SIM (DMD-SIM) setup with LED illumination was built previously [5]. The light source is a high-luminance white LED (SOLIS-3C, Thorlabs Inc., Newton, New Jersey, USA) with a liquid light guide. After being collimated, the beam illuminates the DMD chipset of 1024 × 768 pixels with 13.7 µm pitch (V7000, ViALUX GmbH, Chemnitz, Germany) at an angle of 24⁰ with respect to the normal of the chipset plane. The structured pattern generated by the DMD outputs in the normal direction of the DMD. It passes through an achromatic lens (f = 200 mm), a 50:50 beam splitter (for non-fluorescence) or a dichroic mirror (for fluorescence, 490 nm long-pass), the objective lens, and then projects the structured pattern onto the sample. The DMD patterns are designed in binary to reach the maximum modulation speed, which can be automatically filtered into sinusoidal patterns onto the sample by the objective lens. The three-step phase-shifting is realized by addressing the three pre-designed patterns on the DMD sequentially. A color CMOS camera of 2048 × 2048 pixels with a maximum full-frame rate of 80 fps (UI-3370CP-HQ, IDS GmbH, Germany) is employed for acquiring raw images with a tube lens (f = 200 mm). Two types of objective lenses (20×/NA 0.75, Nikon Inc., Tokyo, Japan; 4×/NA 0.2, Thorlabs Inc., Newton, New Jersey, USA) are used in the experiment. An XYZ motorized piezo stage with a 25 mm travel range and 50 nm settle accuracy (3-ESC3040, Attocube GmbH, Haar, Germany) is used for sample mounting, target searching, field-of-view (FOV) translation and axial scanning.

The sample size in the experiment is usually beyond the FOV of the objective lens. Thus, a FOV stitching strategy is adopted for the large field full-color OS-SIM. For each FOV, three-step phase-shifted raw structured illumination images are recorded by the camera. Then, the raw images are respectively processed by the full-color HSV-RMS or the Fourier-OS-SIM algorithms to yield the full-color OS images, according to the workflows shown in Figure 1. The adjacent FOV is addressed by controlling the XY stage, and the overlap ratio between two adjacent FOVs is 20%~30% of the image size for stitching. A customer cross-correlation-based image stitching program is developed by using Matlab (R2022a, MathWorks Inc., Natick, Massachusetts, USA) to fuse the resulting FOV-related full-color OS images. By axially scanning the sample and repeating the above FOV stitching procedure, a large FOV full-color OS image stack can be obtained and visualized in 3D.

5. Results and Discussion

5.1. Fluorescence Imaging of Mixed Pollen Grains

The first experiment demonstrates the applicability of the Fourier-OS-SIM to fluorescent samples by observing auto-fluorescent pollen grains. An excitation filter (475 nm short-pass) and a fluorescence filter (490 nm long-pass) are matched with the dichroic mirror (490 nm long-pass). The objective lens is 20×/NA 0.75 so that the single FOV and the resolution are about 560 µm × 560 µm and 330 nm, respectively. We acquire the raw structured illumination images and process them according to the procedure described in Section 4. In this case, the stitched FOV is 902 µm × 1403 µm in size, scanned by 2(x) × 3(y) steps. The z-stack involves 463 slices with an interval of 200 nm. The exposure time of the camera is set to 80 ms. The total raw data acquisition time costs 21.8 min (≈[(80 ms exposure time + 60 ms image storage time) × 3 phase-shift steps × 463 slices + (50 ms z-stage settling time × 462 z-steps)] × 6 sub-FOVs + (200 ms × 5 XY-stage movement)), ignoring 0.031 ms of the DMD refreshing time. The total data capacity is 13.02 GB (≈1.6 MB of each raw image × 3 phase-shifting × 6 sub-FOVs × 463 slices).

Figure 3a exhibits the full-color panoramic maximum intensity projection (MIP) image of the slice stack reconstructed by the Fourier-OS-SIM approach. The mixed pollen grains appear in various colors due to their own specific auto-fluorescence spectrum. A region of interest (ROI) indicated in Figure 3a is selected, and the corresponding image stacks of the Wide-Field, HSV-RMS, and Fourier-OS-SIM are reconstructed, respectively. Figure 3b–d present the same index images from the Wide-Field, HSV-RMS, and Fourier-OS-SIM stacks, respectively. We further zoom in Figure 3b–d in the boxes, as shown in Figure 3e–g. It can be seen that the HSV-RMS and the Fourier-OS-SIM image colors are consistent with the Wide-Field image color. This color consistency indicates that both the HSV-RMS and the Fourier-OS-SIM can correctly restore the color information. Secondly, the HSV-RMS and the Fourier-OS-SIM images look sharper than the Wide-Field image, indicating both of them own the OS capability to remove blurry backgrounds. Moreover, the Fourier-OS-SIM image is obviously clearer than the HSV-RMS image, implying that the Fourier-OS-SIM is superior to the HSV-RMS in OS image quality. The histograms of Figure 3f,g also agree that the Fourier-OS-SIM image quality is better than that of HSV-RMS, as shown in Figure 3h,i. The curve in Figure 3h is more discrete than in Figure 3i, meaning the background noise in the Fourier-OS-SIM image is significantly lower than in the HSV-RMS image. We further draw a line in Figure 3f,g and plot the intensity curves along the line shown in Figure 3j,k. It again proves that the background noise level in the Fourier-OS-SIM image is much lower than in the HSV-RMS image.

5.2. Non-Fluorescence Imaging of Tiny Animals

To demonstrate the availability of the full-color Fourier-OS-SIM method for non-fluorescence samples, we select a tiny tiger beetle as the sample, which is about centimeters long and observable by the naked eye. A 50:50 beam splitter is used to depart the illumination and the reflected signal beams. The objective lens is 4×/NA0.2 so that the single FOV and the resolution are about 2.8 mm × 2.8 mm and 1.4 µm, respectively. After FOV registration, the whole FOV size is 12.94 mm × 20.81 mm, scanned by 7(x) × 11(y) sub-FOVs. The slices are obtained by 1041 steps of z-scanning with an interval of 5 µm. The camera exposure time is 26 ms. In this experiment, the raw data acquisition time is 6.86 h (≈[26 ms exposure time + 60 ms image storage time) × 3 phase-shift steps × 1041 slices + (50 ms z-stage settling time × 1040 z-steps)] × 77 sub-FOVs + (200 ms × 76 XY-stage movement)). The total raw data capacity is 375.74 GB (≈1.6 MB of each image × 3 phase-shifting × 77 sub-FOVs × 1041 slices).

The entire FOV full-color MIP image of the slice stack reconstructed by the Fourier-OS-SIM approach is presented in Figure 4a. Other image stacks processed by the HSV-RMS and the Wide-Field are compared. Figure 4b–d show a slice of images of the Wide-Field, HSV-RMS, and Fourier-OS-SIM stacks for the zoom-in area labeled by the white dashed box in Figure 4a, respectively. The further enlarged parts of Figure 4b–d are presented in Figure 4e–g, in which it can be seen that the Fourier-OS-SIM image performs the best fidelity because the background area is the darkest. The histograms of Figure 4f,g shown in Figure 4h,i also imply the Fourier-OS-SIM can effectively reduce the background noise interference over the HSV-RMS. The intensity profiles are shown in Figure 4j,k along the dashed lines in Figure 4f,g further manifest that the Fourier-OS-SIM has an advantage in background noise suppression.

5.3. Quantitative Height Map Analysis

The height map is a direct view of the 3D morphology of a sample that can be extracted from the slice stack [19]. For each lateral pixel position in the slice stack, a pixel array along the axial z is selected and plotted as an intensity line varied with z. The peak intensity of the line assigns a z-value that is regarded as the sample height on such an xy position. Each intensity line is splined using the linear regression method to obtain an accurate height value. By going all over the lateral pixel positions, we can depict the height map of the sample.

Higher SNR slice images will produce a finer and more accurate height map. Benefiting from the background noise suppression of the full-color Fourier-OS-SIM method, a better height map for the existing 3D slice stack is predictable. To prove that, we choose a ROI of the tiger beetle in Figure 4a, and compare the height maps resulting from both the full-color Fourier-OS-SIM and the HSV-RMS. Figure 5a,b show the full-color MIP images of the compound eye’s structure of the tiger beetle by using the HSV-RMS and the Fourier-OS-SIM, respectively, and the height maps are shown in Figure 5c,d. We draw a line in Figure 5c,d and plot the intensity curves along the lines, as presented in Figure 5e. It is clear that the HSV-RMS curve is more rippled than the Fourier-OS-SIM curve, because it is disturbed by the random background noise. The root mean square errors (RMSE) of the height profiles for the Fourier-OS-SIM and the HSV-RMS curves are calculated to be 0.10 µm and 0.39 µm, respectively. This further proves that the Fourier-OS-SIM method can obtain higher fidelity and accurate height maps than the HSV-RMS method.

6. Conclusions

We developed a full-color Fourier-OS-SIM method that is capable of obtaining high-fidelity 3D color images of micro- to meso-scale objects. The full-color Fourier-OS-SIM reconstructs the optical sectioning image using the high-frequency spectrum of samples, intrinsically avoiding the low-frequency background noise. The colorization of the optical sectioning image is directly performed in the RGB color space rather than bound in RGB and HSV color spaces, providing an easy way to restore the color information. The experiments with fluorescence and non-fluorescence biological specimens (mixed pollen grains and tiny tiger beetles) prove that the full-color Fourier-OS-SIM method can achieve higher SNR 3D images than the HSV-RMS method. The quantitative morphological height map analysis is more accurate with the full-color Fourier-OS-SIM method than the HSV-RMS method. This proposed full-color 3D imaging method can be extended to other fields where both the 3D morphology and the color information of objects need to be recovered.