Adaptive Fourier Single-Pixel Imaging Based on Probability Estimation

Wei Lun Tey

, Mau-Luen Tham

1 a

, Yeong-Nan Phua

1 b

and Sing Yee Chua

1,2 c

Lee Kong Chian Faculty of Engineering and Science, Universiti Tunku Abdul Rahman,

Bandar Sungai Long, Selangor, Malaysia

Centre for Photonics and Advanced Materials Research (CPAMR), Universiti Tunku Abdul Rahman,

Bandar Sungai Long, Selangor, Malaysia

Keywords:

Single-Pixel Imaging, Fourier Imaging, Compressed Sensing, Variable Density Sampling.

Abstract:

Fourier single-pixel imaging (FSI) is able to reconstruct images by sampling the information in the Fourier

domain. The conventional sampling method of FSI acquires the low frequency Fourier coefﬁcients to obtain

the image outlines but misses out on the image details in high frequency bands. The variable density sampling

method improves the image quality but follows a predeﬁned mechanism where the power of image information

decreases when frequency increases. In this paper, an adaptive approach is proposed to sample the Fourier

coefﬁcients based on probability estimation. While the low frequency Fourier coefﬁcients are fully sampled

to secure the image outlines, the high frequency Fourier coefﬁcients are sparsely sampled adaptively, and the

image is reconstructed through Compressed sensing (CS) algorithm. Results show that the proposed adaptive

FSI sampling method improves the image quality with sampling ratio ranging from 0.05 to 0.25, as compared

to the commonly used conventional low frequency sampling and variable density sampling methods.

1 INTRODUCTION

Single-pixel imaging (SPI) is a paradigm that is only

equipped with a spatially unresolved detector i.e.

single-pixel detector, as compared to a conventional

imaging system which employs a pixelated detector

i.e. charge-coupled device or complementary metal

oxide semiconductor (Gibson et al., 2020). SPI sys-

tem enables the possibility of building a compact and

fast imaging device at low cost, serving as an im-

portant alternative, especially in low light or unusual

wavelength conditions. Choices for a single-pixel de-

tector include a photon multiplier tube, a photodiode

or a single-pixel of an image sensor (Qiu et al., 2020).

Since the SPI technique was founded, it has been

used in various applications such as terahertz imag-

ing, 3D imaging (Sun et al., 2013), gas leaking detec-

tion (Gibson et al., 2017), underwater imaging (Wu

et al., 2020), etc (Yu et al., 2016).

SPI samples a scene using only a single-pixel

detector and a series of light modulation patterns.

The image is then reconstructed from the measure-

ments acquired by the detector with the help of recon-

struction algorithms. In general, image quality and

https://orcid.org/0000-0003-4600-9839

https://orcid.org/0000-0002-7120-6626

https://orcid.org/0000-0001-6327-4592

computational complexity are the major concerns in

SPI (Shin et al., 2021a; Woo et al., 2022). By in-

troducing compressed sensing (CS) techniques, the

number of measurements needed is greatly reduced

to sub-Nyquist sampling rate. Pseudo-random pat-

terns are commonly used in SPI while determinis-

tic patterns have become popular more recently (Shin

et al., 2021b). Hadamard single-pixel Imaging (HSI)

and Fourier single-pixel Imaging (FSI) are the most

well-known deterministic models-based techniques,

using Hadamard and Fourier basis patterns respec-

tively (Zhang et al., 2017; Yu et al., 2020). FSI is

more advantageous in term of the image energy con-

centration as compared to HSI. Under low-sampling

condition, FSI is more efﬁcient and performs better

than HSI (Zhang et al., 2017).

Zhang et al. introduced SPI through the means

of Fourier spectrum acquisition (Zhang et al., 2015)

and later presented different sampling strategies i.e.

orderings of the Fourier basis patterns used. Conven-

tionally, only the low frequency Fourier coefﬁcients

are sampled (Zhang et al., 2017). A sparse Fourier

single-pixel imaging (S-FSI) sampling method was

proposed by Meng et al. (Wenwen et al., 2019), where

the sampling probability for a sampling point is in-

versely proportional to the distance from the center

of the Fourier spectrum, namely the variable density

sampling method. It improves the reconstructed im-

212

Tey, W., Tham, M., Phua, Y. and Chua, S.

Adaptive Fourier Single-Pixel Imaging Based on Probability Estimation.

DOI: 10.5220/0011658300003417

In Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 4: VISAPP, pages

212-219

ISBN: 978-989-758-634-7; ISSN: 2184-4321

 2023 by SCITEPRESS – Science and Technology Publications, Lda. Under CC license (CC BY-NC-ND 4.0)

age in terms of signal-to-noise ratio (SNR) and object

details compared to conventional FSI sampling (Hu

et al., 2021). The downside of the existing variable

density sampling method is that since the probability

of sampling is predetermined by the distance of the

sampling points from the center of Fourier spectrum,

the sampling pattern is always ﬁxed regardless of the

nature and characteristics of the image. Clearly, this

sampling mechanism might not be a suitable solution

for all kinds of images.

The adaptive sampling method has been studied

recently to address the aforementioned problem. In

2020, Liang et al. proposed an adaptive sampling

trajectory based on a rough estimation of spectrum

energy distribution. This method requires additional

sampling and interpolation processes for the estima-

tion (Liang et al., 2020). An adaptive sampling

method based on Gaussian random sampling was pro-

posed, where the density of the variable density sam-

pling is based on the importance of Fourier coefﬁ-

cients according to 1D Gaussian function (Qiu et al.,

2021). Another adaptive sampling method based

on radial correlations has been proposed to produce

decent image reconstruction without CS (He et al.,

2021). In addition, dynamic ordering of sampling

patterns based on an image dictionary was proposed

(Yuan et al., 2021). Although this method is able to

provide adaptive and dynamic ordering of sampling

patterns based on the feedback of the recorded mea-

surements, it requires a huge dataset to be trained be-

fore applying the image dictionary.

In this paper, an adaptive FSI sampling method

based on the estimation of probability sampling

is proposed without the need for training datasets.

Fourier coefﬁcients are fully sampled in the low fre-

quency bands while the high frequency coefﬁcients

are sparsely sampled adaptively. The probability of

sampling the next sampling point is decided based

on the Fourier coefﬁcient of the point sampled. This

estimation is repeated in the sampling process until

the targeted sampling ratio is met. Finally, the im-

age is reconstructed using the CS algorithm. The re-

sult shows that the proposed adaptive FSI sampling

method is able to produce better image quality overall

compared to the conventional sampling and variable

density sampling methods that are commonly imple-

mented.

2 METHODOLOGY

In this section, the operating principle of conven-

tional FSI and variable density sampling method is

introduced to provide necessary background infor-

mation. Subsequently, the proposed adaptive sam-

pling method is presented. In addition, the evaluation

method is explained for the performance comparison.

2.1 Conventional FSI

FSI utilizes Fourier basis patterns to acquire the spec-

trum of the scene and then utilizes inverse fast Fourier

transform (IFFT) to reconstruct the scene. The inter-

nal products of the targeted scene and the projected

Fourier patterns are measured using a single-pixel

detector, and all Fourier coefﬁcients are processed

through phase-shifts as shown in Eq. (1) (Qiu et al.,

2021).

(x,y; f

, f

) = a + b ·cos(2π f

x + 2π f

y + φ) (1)

where x and y are the 2D Cartesian coordinates from

the captured scene, while ( f

, f

) denotes the spatial

frequency, a represents the DC component, b is the

contrast, and φ indicates the initial phase. Eq. (2) can

be utilized to determine the reﬂected intensity from

the targeted scene.

( f

, f

) =

Ω

r(x, y)P

(x,y; f

, f

)dxdy (2)

where Ω denotes the region of the targeted scene,

while r represents the reﬂectivity distribution of the

target object from the captured scene. As ambient

lights ﬂuctuate, the overall response of the detector

is given in Eq. (3).

( f

, f

) = T

+ k I

( f

, f

) (3)

where k is decided based on the size of the detector,

while T

indicates the ambient light measured.

Both the 4-step FSI and the 3-step FSI are ac-

ceptable but 3-step FSI reduces the measurement time

by 25%. Therefore, despite the fact that 4-step FSI

is more robust, 3-step FSI is preferred in practice.

With a ﬁxed phase of △

(i.e.,0, 2π/3,4π/3), 3-step

FSI enables the acquisition of each Fourier coefﬁcient

based on every three corresponding illumination pat-

terns (Hu et al., 2021). Each complex Fourier coefﬁ-

cient F( f

, f

) can therefore be obtained as:

F( f

, f

) = (2T

−T

2π/3

−T

4π/3

√

3 j(T

2π/3

−T

4π/3

)

(4)

The image can then be reconstructed by utilizing

IFFT, as shown in Eq. (5).

∗

= IFFT (F + n) (5)

where r

∗

is the under-sampled reconstructed image

with noise n.

The conventional sampling method concentrates

on the information in the low frequency bands of the

Fourier space. Hence, only low frequency Fourier

coefﬁcients are acquired. Ringing artifacts and blur

can occur since r

∗

lacks of high frequency coefﬁcients

(Hu et al., 2021; Mdraﬁ and Gurbuz, 2020).

Adaptive Fourier Single-Pixel Imaging Based on Probability Estimation

213

2.2 Variable Density Sampling Method

Variable density FSI proposes a sampling probability

based on the distance from the centre of the Fourier

spectrum, as shown in Eq. (6) (Wenwen et al., 2019).

As such, the power of image information slowly de-

creases from low to high frequencies.

ρ =



1, r ≤ R

(1 −r)

,r > R

(6)

r denotes the distance of the sampling point to the spa-

tial center of Fourier spectrum, R is the radius thresh-

old in terms of sampling ratio, and positive coefﬁcient

ε is to adjust the sampling probability (Wenwen et al.,

2019).

The image reconstructed using variable density

FSI tends to have more noise compared to the con-

ventional sampling method but the conventional sam-

pling method observes signiﬁcant oscillation. By us-

ing the CS technique, the excessive noise can be sup-

pressed with the cost of computation time and re-

sources.

2.3 Proposed Adaptive Sampling

Method

The proposed adaptive sampling method determines

the probability density, ρ as shown in Eq. (7).

ρ =



1, r ≤ R

coe f f

,r > R

(7)

The proposed method suggests to fully sample the low

frequency bands, represented by R. r denotes the dis-

tance of the sampling point to the spatial center of the

Fourier spectrum. The probability of sampling the

next sampling point is decided based on the Fourier

coefﬁcient of the point sampled coe f f and thresh-

old X which can be determined based on the low fre-

quency coefﬁcients sampled, as shown in Eq. (8). The

threshold X is calculated based on the processed in-

tensity matrix I which is unique per image.

X = I(:, :,1) −I(:, :,2) + I(:,:,3)i −I(:,:, 4) (8)

As such, the probability density, ρ can be determined.

Basically, Fourier coefﬁcients in the low frequency

bands are fully sampled meanwhile the coefﬁcients in

the high frequency bands are sparsely sampled adap-

tively. This estimation is repeated in the sampling

process until the targeted sampling ratio is met.

2.4 Evaluation Method

Peak Signal to Noise Ratio (PSNR) represents the ra-

tio between the maximum power possible of a signal

and noise corruption that affects the ﬁdelity of the im-

age’s representation and are used to measure the qual-

ity of the reconstructed image. PSNR is deﬁned base

on Mean Square Error(MSE):

MSE =

M ∗N

M−1,N−1

∑

i=0, j=0

[I(i, j) −K(i, j)]

(9)

PSNR is calculated as follow:

PSNR = 10 log



MAX

MSE



(10)

where MAX

represents the maximum possible value

of the pixels in the image and MSE is the mean square

error. The higher the PSNR, the better the quality of

the reconstructed image.

Root Mean Square Error (RMSE) represents the

differences between the reconstructed image and the

captured scene. It is calculated as:

RMSE =

√

MSE (11)

where lower values of RMSE indicates a better ﬁt be-

tween two images with the value of 0 implying that

ground truth and reconstructed image are identical.

Structural Similarity Index (SSIM) is a well-

known perceptual metric that is commonly used to

quantify image quality degradation. Unlike PSNR,

SSIM indicates the image’s visible structure. SSIM

is calculated as:

SSIM(x,y) =

(2µ

)(2σ

)

(µ

+ µ

)(σ

+ σ

)

(12)

where µ

and µ

denote the pixel sample means of x

and y respectively, σ

and σ

indicates the variance of

both x and y. SSIM value ranges from 0 to 1, with the

value of 1 implying an identical match between the

ground truth and the reconstructed image.

3 RESULTS AND DISCUSSION

The study was performed using a computer equipped

with an Intel(R) Core (TM) i7-11700F CPU, 32 GB

RAM, and MATLAB 2021a. L1-Magic library was

employed for the post-IFFT reconstruction with CS.

All images were processed in a resolution of 256x256.

The performance of the proposed method is ana-

lyzed for different sampling ratios SR ranging from

0.05 to 0.25 and compared with three existing meth-

ods i.e. conventional sampling method based on cir-

cular path, variable density method (polynomial) and

line mask sampling method (Qiu et al., 2021; Candes

et al., 2006). Figure 1 illustrates the sampling pattern

for the (a) conventional sampling method, (b) variable

VISAPP 2023 - 18th International Conference on Computer Vision Theory and Applications

214

Figure 1: Sampling pattern illustration for (a) conventional

sampling based on circular path, (b) variable density sam-

pling (polynomial), (c) line mask sampling, and (d) the pro-

posed adaptive sampling method.

density method, (c) line mask sampling method, and

(d) the proposed adaptive method.

The radius threshold R is set to the same value

for both the variable density sampling method and the

proposed method. For SR = 0.05, R is set to 0.03 sam-

pling ratio while for the other SR > 0.05, R is set to

0.05 sampling ratio. Besides, the value of ε for the

variable density sampling method is set as 2. With

the radius R of the fully sampled low frequency bands

being set to 0.05 sampling ratio, a unique threshold

value, X for a particular image can be determined as

shown in Figure 2.

The results comparison for two test images, ‘Pi-

rate’ and ‘USAF chart’ are shown in Figure 3 and

Figure 4. The reconstructed images using four dif-

ferent methods with sampling ratio SR ranging from

0.05 to 0.25 are shown respectively. The image qual-

ity is indicated by the PSNR, SSIM, and RMSE value,

which serves as the quantitative comparison between

the methods. It can be seen that when the sampling ra-

tio SR increases, all four methods reconstruct images

with better quality. For test image ‘Pirate’, the con-

ventional sampling method which only samples the

low frequency bands performs relatively better than

the variable density sampling. This could be due to

the characteristics of the test image which has more

signiﬁcant information in the low frequency bands.

The proposed adaptive sampling method shows the

best image quality among all four methods. For test

Figure 2: Threshold value, X estimated in the proposed

adaptive FSI method for different test images.

image ‘USAF chart’, the variable density and the pro-

posed adaptive sampling method outperform the con-

ventional FSI and line mask sampling method in low

sampling ratio, while line mask sampling method is

comparable in high sampling ratio. This is because

these methods are able to obtain high frequency de-

tails at the same time securing most image outline in-

formation in the low frequency bands.

Additional images, ‘Boat’, ‘Cameraman’, ‘Moon’

and ‘Fruits’ were tested and the results comparison

using different sampling methods is summarized in

Table 1. Note that the conventional sampling method

outperforms the variable density sampling method on

images with a more center focused spectrum such as

‘Boat’ and ‘Fruits’, while the variable density sam-

pling method performs better on a sparser spectrum

such as ‘Cameraman’ and ‘Moon’. It is worth noting

that the proposed adaptive sampling approach gives

the best image quality for all cases as compared to the

variable density sampling method.

Adaptive Fourier Single-Pixel Imaging Based on Probability Estimation

215

Figure 3: Results comparison of different sampling methods for test image ‘Pirate’. Note that the conventional sampling

method which focuses on sampling the low frequency Fourier coefﬁcients performs relatively better than the variable density

sampling. The proposed adaptive sampling method shows the best image quality among all four methods.

VISAPP 2023 - 18th International Conference on Computer Vision Theory and Applications

216

Figure 4: Results comparison of different sampling methods for test image ‘USAF chart’. Note that the variable density

sampling outperforms the conventional sampling method while the proposed adaptive sampling method still shows the best

image quality among all four methods overall although line mask sampling method yield higher PSNR and lower RMSE when

sampling ratio increases.

Adaptive Fourier Single-Pixel Imaging Based on Probability Estimation

217

Table 1: Results comparison of different sampling methods for test image ‘Boat’, ‘Cameraman’, ‘Moon’ and ‘Fruits’. Note

that the conventional sampling method outperforms the variable density sampling method on images with a more center-

focused spectrum such as ‘Boat’ and ‘Fruits’, while the variable density sampling method performs better on a sparser spec-

trum such as ‘Cameraman’ and ‘Moon’. The proposed adaptive sampling method still shows the best image quality among

all four methods.

4 CONCLUSIONS

This paper proposes an adaptive approach to sample

an image in the Fourier domain, which is proven to re-

construct high quality images with sub-Nyquist sam-

pling rate. The proposed method suggests to fully

sample the low frequency Fourier coefﬁcients to se-

cure the image outlines and adaptively sample the

high frequency coefﬁcients in a sparse manner based

on the estimation of probability sampling. Accord-

ingly, the image is reconstructed using the CS algo-

rithm. Based on the results obtained, the proposed

adaptive sampling method gives better image quality

with sampling ratio ranging from 0.05 to 0.25 as com-

pared to the existing methods i.e. conventional sam-

pling strategy based on circular path, variable density

strategy (polynomial) and line mask sampling. With

the adaptive characteristic of the proposed sampling

approach, it is able to perform well for various im-

ages while both the conventional sampling and vari-

able density sampling methods do not work well in

some images with peculiar Fourier spectrum patterns.

However, there are some possible improvements

to be considered for the proposed method in the fu-

ture. First of all, the mechanism to decide the sam-

pling probability for spectrum points can be enhanced

by considering other conditions such as the difference

in Fourier spectrum patterns. Furthermore, the CS al-

gorithm to reconstruct the images is computationally

expensive. By optimizing the algorithms to reduce the

computational time and complexity, it is hoped to im-

prove the overall efﬁciency of the proposed adaptive

sampling method.

ACKNOWLEDGEMENTS

The research was supported by the Ministry of Higher

Education (MoHE) through Fundamental Research

Grant Scheme (FRGS/1/2021/TK0/UTAR/02/9).

REFERENCES

Candes, E. J., Romberg, J. K., and Tao, T. (2006). Sta-

ble signal recovery from incomplete and inaccurate

measurements. Communications on Pure and Applied

Mathematics: A Journal Issued by the Courant Insti-

tute of Mathematical Sciences, 59(8):1207–1223.

Gibson, G. M., Johnson, S. D., and Padgett, M. J. (2020).

Single-pixel imaging 12 years on: a review. Optics

express, 28(19):28190–28208.

VISAPP 2023 - 18th International Conference on Computer Vision Theory and Applications

218

Gibson, G. M., Sun, B., Edgar, M. P., Phillips, D. B.,

Hempler, N., Maker, G. T., Malcolm, G. P., and Pad-

gett, M. J. (2017). Real-time imaging of methane

gas leaks using a single-pixel camera. Optics express,

25(4):2998–3005.

He, R., Weng, Z., Zhang, Y., Qin, C., Zhang, J., Chen, Q.,

and Zhang, W. (2021). Adaptive fourier single pixel

imaging based on the radial correlation in the fourier

domain. Optics Express, 29(22):36021–36037.

Hu, Y., Cheng, Z., Fan, X., Liang, Z., and Zhai, X.

(2021). Optimizing the quality of fourier single-pixel

imaging via generative adversarial network. Optik,

227:166060.

Liang, Z., Yu, D., Cheng, Z., and Zhai, X. (2020). Adaptive

fourier single-pixel imaging sampling based on fre-

quency coefﬁcients prediction. Optical Engineering,

59(7):073105.

Mdraﬁ, R. and Gurbuz, A. C. (2020). Joint learning of mea-

surement matrix and signal reconstruction via deep

learning. IEEE Transactions on Computational Imag-

ing, 6:818–829.

Qiu, Z., Guo, X., Lu, T., Qi, P., Zhang, Z., and Zhong,

J. (2021). Efﬁcient fourier single-pixel imaging with

gaussian random sampling. In Photonics, volume 8,

page 319. MDPI.

Qiu, Z., Zhang, Z., Zhong, J., et al. (2020). Comprehensive

comparison of single-pixel imaging methods. Optics

and Lasers in Engineering, 134:106301.

Shin, Z., Chai, T.-Y., Pua, C. H., Wang, X., and Chua, S. Y.

(2021a). Efﬁcient spatially-variant single-pixel imag-

ing using block-based compressed sensing. Journal of

Signal Processing Systems, 93(11):1323–1337.

Shin, Z., Lin, H. S., Chai, T.-Y., Wang, X., and Chua, S. Y.

(2021b). Programmable spatially variant single-pixel

imaging based on compressive sensing. Journal of

Electronic Imaging, 30(2):021004.

Sun, B., Edgar, M. P., Bowman, R., Vittert, L. E., Welsh, S.,

Bowman, A., and Padgett, M. J. (2013). 3d compu-

tational imaging with single-pixel detectors. Science,

340(6134):844–847.

Wenwen, M., Dongfeng, S., Jian, H., Kee, Y., Yingjian, W.,

and Chengyu, F. (2019). Sparse fourier single-pixel

imaging. Optics express, 27(22):31490–31503.

Woo, B. H., Tham, M.-L., and Chua, S. Y. (2022). Deep

learning based single pixel imaging using coarse-to-

ﬁne sampling. In 2022 IEEE 18th International Col-

loquium on Signal Processing & Applications (CSPA),

pages 127–131. IEEE.

Wu, H., Zhao, M., Li, F., Tian, Z., and Zhao, M.

(2020). Underwater polarization-based single pixel

imaging. Journal of the Society for Information Dis-

play, 28(2):157–163.

Yu, W.-K., Yao, X.-R., Liu, X.-F., Lan, R.-M., Wu, L.-A.,

Zhai, G.-J., and Zhao, Q. (2016). Compressive micro-

scopic imaging with “positive–negative” light modu-

lation. Optics Communications, 371:105–111.

Yu, X., Yang, F., Gao, B., Ran, J., and Huang, X. (2020).

Deep compressive single pixel imaging by reordering

hadamard basis: a comparative study. IEEE Access,

8:55773–55784.

Yuan, A. Y., Feng, J., Jiao, S., Gao, Y., Zhang, Z., Xie,

Z., Du, L., and Lei, T. (2021). Adaptive and dynamic

ordering of illumination patterns with an image dictio-

nary in single-pixel imaging. Optics Communications,

481:126527.

Zhang, Z., Ma, X., and Zhong, J. (2015). Single-pixel imag-

ing by means of fourier spectrum acquisition. Nature

communications, 6(1):1–6.

Zhang, Z., Wang, X., Zheng, G., and Zhong, J. (2017).

Hadamard single-pixel imaging versus fourier single-

pixel imaging. Optics Express, 25(16):19619–19639.

Adaptive Fourier Single-Pixel Imaging Based on Probability Estimation

219