×

Image deblurring with coupled dictionary learning. (English) Zbl 1398.94055

Summary: Image deblurring is a challenging problem in vision computing. Traditionally, this task is addressed as an inverse problem that is enclosed into the image itself. This paper presents a learning-based framework where the knowledge hidden in huge amounts of available data is explored and exploited for image deblurring. To this end, our algorithm is developed under the conceptual framework of coupled dictionary learning. Specifically, given pairs of blurred image patches and their corresponding clear ones, a learning model is constructed to learn a pair of dictionaries. Among them, one dictionary is responsible for the representation of clear images, while the other is responsible for that of the blurred images. Theoretically, the learning model is analyzed with coupled sparse representations for training samples. As the atoms of these dictionaries are coupled together one-by-one, the reconstruction information can be transmitted between the clear and blurry images. In application phase, the blurry dictionary is employed to reconstruct linearly the blurry image to be restored. Then, the reconstruction coefficients are kept unchanged along with the clear dictionary to restore the final results. The main advantage of our approach lies in that it works in the case of unknown blur kernels. Comparative experiments indicate the validity of our approach.

MSC:

94A08 Image processing (compression, reconstruction, etc.) in information and communication theory
68T05 Learning and adaptive systems in artificial intelligence

Software:

BM3D; TwIST; iCoseg
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Afonso, MV; Bioucas-Dias, JM; Figueiredo, M, Fast image recovery using variable splitting and constrained optimization, IEEE Transactions on Image Processing, 19, 2345-2356, (2010) · Zbl 1371.94018 · doi:10.1109/TIP.2010.2047910
[2] Aharon, M; Elad, M; Bruckstein, AM, K-svd: an algorithm for designing of over-complete dictionaries for sparse representationn, IEEE Transactions on Signal Processing, 54, 4311-4322, (2006) · Zbl 1375.94040 · doi:10.1109/TSP.2006.881199
[3] Bar, L., Sochen, N., & Kiryati, N. (2007). Restoration of images with piecewise space-variant blur. In International conference on scale space and variational methods in computer vision (pp. 533-544). Ischia, Italy.
[4] Barbero, A., & Sra, S. (2011). Fast newton-type methods for total variation regularization. In International conference on machine learning (pp. 313-320). Bellevue, Washington, USA.
[5] Batra, D., Kowdle, A., Parikh, D., Luo, J., & Chen, T. (2010). icoseg: Interactive co-segmentation with intelligent scribble guidance. In IEEE conference on computer vision and pattern recognition (pp. 3169-3176). San Francisco, CA, USA.
[6] Batra, D; Kowdle, A; Parikh, D; Luo, J; Chen, T, Interactively co-segmentating topically related images with intelligent scribble guidance, International Journal of Computer Vision, 93, 273-292, (2011) · doi:10.1007/s11263-010-0415-x
[7] Beck, A; Teboulle, M, Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems, IEEE Transactions on Image Processing, 18, 2419-2434, (2009) · Zbl 1371.94049 · doi:10.1109/TIP.2009.2028250
[8] Beck, A; Teboulle, M, A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM Journal on Imaging Sciences, 2, 183-202, (2009) · Zbl 1175.94009 · doi:10.1137/080716542
[9] Bioucas-Dias, JM; Figueiredo, M, A new twist: two-step iterative shrinkage/thresholding algorithms for image restoration, IEEE Transactions on Image Processing, 16, 2992-3004, (2007) · doi:10.1109/TIP.2007.909319
[10] Bronstein, MM; Bronstein, AM; Zibulevsky, M; Zeevi, YY, Blind deconvolution of images using optimal sparse representations, IEEE Transactions on Image Processing, 14, 726-736, (2005) · doi:10.1109/TIP.2005.847322
[11] Chantas, GK; Galatsanos, NP; Molina, R; Katsaggelos, AK, Variational Bayesian image restoration with a product of spatially weighted total variation image priors, IEEE Transactions on Image Processing, 19, 351-362, (2010) · Zbl 1371.94079 · doi:10.1109/TIP.2009.2033398
[12] Couzinie-Devy, F., Mairal, J., Bach, F., & Ponce, J. (2011). Dictionary learning for deblurring and digital zoom. CoRR. arXiv:1110.0957.
[13] Dabov, K; Foi, A; Katkovnik, V; Egiazarian, K, Image denoising by sparse 3d transform-domain collaborative filtering, IEEE Transactions on Image Processing, 16, 2080-2095, (2007) · doi:10.1109/TIP.2007.901238
[14] Danielyan, A; Katkovnik, V; Egiazarian, K, Bm3d frames and variational image deblurring, IEEE Transactions on Image Processing, 21, 1715-1728, (2012) · Zbl 1373.94096 · doi:10.1109/TIP.2011.2176954
[15] Dong, W; Zhang, L; Shi, G; Li, X, Nonlocally centralized sparse representation for image restoration, IEEE Transactions on Image Processing, 22, 1620-1630, (2013) · Zbl 1373.94104 · doi:10.1109/TIP.2012.2235847
[16] Dong, W; Zhang, L; Shi, G; Wu, X, Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization, IEEE Transactions on Image Processing, 20, 1838-1857, (2011) · Zbl 1372.94072 · doi:10.1109/TIP.2011.2108306
[17] Elad, M; Aharon, M, Image denoising via sparse and redundant representations over learned dictionaries, IEEE Transactions on Image Processing, 15, 3736-3745, (2006) · doi:10.1109/TIP.2006.881969
[18] Elad, M; Figueiredo, M; Ma, Y, On the role of sparse and redundant representations in image processing, Proceedings of the IEEE, 98, 972-982, (2010) · doi:10.1109/JPROC.2009.2037655
[19] Engan, K; Aase, SO; Husoy, JH, Multi-frame compression: theory and design, EURASIP Signal Processing, 80, 2121-2140, (2000) · Zbl 1098.94538 · doi:10.1016/S0165-1684(00)00072-4
[20] Fergus, R; Singh, B; Hertzmann, A; Roweis, ST; Freeman, WT, Removing camera shake from a single photograph, ACM Transactions on Graphics (SIGGRAPH), 25, 787-794, (2006) · Zbl 1371.94125 · doi:10.1145/1141911.1141956
[21] Goldluecke, B., & Cremers, D. (2010). An approach to vectorial total variation based on geometric measure theory. In IEEE international conference on computer vision and pattern recognition (pp. 327-333). San Francisco, CA, USA.
[22] Gorodnitsky, IF; Rao, BD, Sparse signal reconstruction from limited data using focuss: A re-weighted norm minimization algorithm, IEEE Transactions on Signal Processing, 45, 600-616, (1997) · doi:10.1109/78.558475
[23] He, L., Qi, H., & Zaretzki, R. (2013). Beta process joint dictionary learning for coupled feature spaces with application to single image super-resolution. In IEEE conference on computer vision and pattern recognition (pp. 345-352). Portland, Oregon, USA.
[24] Hu, Z., Huang, J. B., & Yang, M. H. (2010). Single image deblurring with adaptive dictionary learning. In IEEE international conference on image processing (pp. 1169-1172). Hong Kong, China.
[25] Jenatton, R; Mairal, J; Obozinski, G; Bach, F, Proximal methods for hierarchical sparse coding, Journal of Machine Learning Research, 12, 2297-2334, (2011) · Zbl 1280.94029
[26] Jia, K; Wang, X; Tang, X, Image transformation based on learning dictionaries across image spaces, IEEE Transactions on Pattern Analysis and Machine Intelligence, 35, 367-380, (2013) · doi:10.1109/TPAMI.2012.95
[27] Joshi, N., Kang, S. B., Zitnick, C. L., & Szeliski, R. (2010). Image deblurring using inertial measurement sensors. In SIGGRAPH (pp. 1-8). Los Angeles, CA, USA.
[28] Joshi, N., Zitnick, C. L., Szeliski, R., & Kriegman, D. J. (2009). Image deblurring and denoising using color priors. In International conference on computer vision and pattern recognition (pp. 1-8). Miami, FL, USA.
[29] Kamilov, U; Bostan, E; Unser, M, Wavelet shrinkage with consistent cycle spinning generalizes total variation denoising, IEEE Signal Processing Letters, 19, 187-190, (2012) · doi:10.1109/LSP.2012.2185929
[30] Kenig, T; Kam, Z; Feuer, A, Blind image deconvolution using machine learning for three-dimensional microscopy, IEEE Transactions on Pattern Analysis and Machine Intelligence, 32, 2191-2204, (2010) · doi:10.1109/TPAMI.2010.45
[31] Levin, A; Fergus, R; Durand, F; Freeman, WT, Coded exposure photography: motion deblurring using fluttered shutter, ACM Transactions on Graphics (SIGGRAPH), 26, 70:1-70:9, (2007) · doi:10.1145/1276377.1276464
[32] Levin, A., Weiss, Y., Durand, F., & Freeman, W. T. (2009). Understanding and evaluating blind deconvolution algorithms. In International conference on computer vision and pattern recognition (pp. 1-8). Miami, FL, USA.
[33] Li, D; Mersereau, RM; Simske, S, Blind image deconvolution through support vector regression, IEEE Transactions on Neural Networks, 18, 931-935, (2007) · doi:10.1109/TNN.2007.891622
[34] Lou, Y; Bertozzi, AL; Soatto, S, Direct sparse deblurring, Journal of Mathematical Imaging and Vision, 39, 1-12, (2011) · Zbl 1255.94022 · doi:10.1007/s10851-010-0220-8
[35] Lucy, L, An iterative technique for the rectification of observed distributions, The Astronomical Journal, 79, 745-754, (1974) · doi:10.1086/111605
[36] Ma, L; Moisan, L; Yu, J; Zeng, T, A dictionary learning approach for Poisson image deblurring, IEEE Transactions on Image Processing, 32, 1277-1289, (2013) · doi:10.1109/TMI.2013.2255883
[37] Ma, S., Yin, W., Zhang, Y., & Chakraborty, A. (2008). An efficient algorithm for compressed MR imaging using total variation and wavelets. In IEEE international conference on computer vision and pattern recognition (pp. 1-8). Anchorage, Alaska, USA.
[38] Mairal, J; Bach, F; Ponce, J, Task-driven dictionary learning, IEEE Transactions on Pattern Analysis and Machine Intelligence, 34, 791-804, (2012) · doi:10.1109/TPAMI.2011.156
[39] Mairal, J; Bach, F; Ponce, J; Sapiro, G, Online learning for matrix factorization and sparse coding, Journal of Machine Learning Research, 11, 19-60, (2010) · Zbl 1242.62087
[40] Mairal, J., Bach, F., Ponce, J., Sapiro, G., & Zisserman, A. (2009). Non-local sparse models for image restoration. In IEEE international conference on computer vision (pp. 2272-2279). Kyoto, Japan.
[41] Mairal, J; Elad, M; Sapiro, G, Sparse representation for color image restoration, IEEE Transactions on Image Processing, 17, 53-69, (2008) · doi:10.1109/TIP.2007.911828
[42] Mairal, J; Jenatton, R; Obozinski, G; Bach, F, Convex and network flow optimization for structured sparsity, Journal of Machine Learning Research, 12, 2681-2720, (2011) · Zbl 1280.68179
[43] Peleg, T; Eldar, Y; Elad, M, Exploiting statistical dependencies in sparse representations for signal recovery, IEEE Transactions on Signal Processing, 60, 2286-2303, (2013) · Zbl 1393.94704 · doi:10.1109/TSP.2012.2188520
[44] Portilla, J. (2009). Image restoration through l0 analysis-based sparse optimization in tight frames. In IEEE international conference on image processing (pp. 3909-3912). Cairo, Egypt.
[45] Richardson, WH, Bayesian-based iterative method of image restoration, Journal of the Optical Society of America, 62, 55-59, (1972) · doi:10.1364/JOSA.62.000055
[46] Rodriguez, P; Wohlberg, B, Efficient minimization method for a generalized total variation functional, IEEE Transactions on Image Processing, 18, 322-332, (2009) · Zbl 1371.94316 · doi:10.1109/TIP.2008.2008420
[47] Rubinstein, R; Peleg, T; Elad, M, Analysis k-svd: A dictionary-learning algorithm for the analysis sparse model, IEEE Transactions on Signal Processing, 61, 661-677, (2013) · Zbl 1393.94416 · doi:10.1109/TSP.2012.2226445
[48] Rubinstein, R; Zibulevsky, M; Elad, M, Double sparsity: learning sparse dictionaries for sparse signal approximation, IEEE Transactions on Signal Processing, 58, 1553-1564, (2010) · Zbl 1392.94427 · doi:10.1109/TSP.2009.2036477
[49] Rudin, LI; Osher, S; Fatemi, E, Nonlinear total variation based noise removal algorithms, Physica D, 60, 259-268, (1992) · Zbl 0780.49028 · doi:10.1016/0167-2789(92)90242-F
[50] Shan, Q; Jia, J; Agarwala, A, High-quality motion deblurring from a single image, ACM Transactions on Graphics (SIGGRAPH), 27, 73:1-73:10, (2008) · doi:10.1145/1360612.1360672
[51] Su, M; Basu, M, A hybrid learning system for image deblurring, Pattern Recognition, 35, 2881-2894, (2002) · Zbl 1007.68930 · doi:10.1016/S0031-3203(01)00239-4
[52] Takeda, H; Farsiu, S; Milanfar, P, Deblurring using regularized locally adaptive kernel regression, IEEE Transactions on Image Processing, 17, 550-563, (2008) · doi:10.1109/TIP.2007.918028
[53] Tropp, JA, Greed is good: algorithmic results for sparse approximation, IEEE Transactions on Information Theory, 50, 2231-2242, (2004) · Zbl 1288.94019 · doi:10.1109/TIT.2004.834793
[54] Wang, S., Zhang, L., Liang, Y., & Pan, Q. (2012). Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis. In IEEE conference on computer vision and pattern recognition (pp. 2216-2223). Providence, Rhode Island, USA.
[55] Wang, Z; Li, Q, Information content weighting for perceptual image quality assessment, IEEE Transactions on Image Processing, 20, 1185-1198, (2011) · Zbl 1372.94266 · doi:10.1109/TIP.2010.2092435
[56] Wen, YW; Chan, RH; Yip, AM, A primal-dual method for total-variation-based wavelet domain inpainting, IEEE Transactions on Image Processing, 21, 106-114, (2012) · Zbl 1372.94268 · doi:10.1109/TIP.2011.2159983
[57] Whyte, O; Sivic, J; Zisserman, A; Ponce, J, Non-uniform deblurring for shaken images, International Journal of Computer Vision, 98, 168-186, (2012) · Zbl 1254.68287 · doi:10.1007/s11263-011-0502-7
[58] Xiang, S., Nie, F., & Zhang, C. (2010). Semi-supervised classification via local spline regression. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(11), 2039-2053.
[59] Xu, L., Zheng, S., & Jia, J. (2013). Unnatural l0 sparse representation for natural image deblurring. In IEEE conference on computer vision and pattern recognition (pp. 1107-1114). Portland, Oregon, USA.
[60] Yang, J., Wang, Z., Lin, Z., Cohen, S., & Huang, T. S. (2012). Coupled dictionary training for image super-resolution. IEEE Transactions on Image Processing, 21(8), 3467-3478. · Zbl 1373.68441
[61] Yang, J., Wright, J., Huang, T. S., & Ma, Y. (2010). Image super-resolution via sparse representation. IEEE Transactions on Signal Processing, 19(11), 2861-2873. · Zbl 1371.94429
[62] Yuan, L; Sun, J; Quan, L; Shum, HY, Image deblurring with blurred/noisy image pairs, ACM Transactions on Graphics (SIGGRAPH), 26, 1-10, (2007) · doi:10.1145/1276377.1276379
[63] Zeyde, R., Elad, M., & Protter, M. (2010). On single image scale-up using sparse-representations. In Proceedings of the 7th international conference on curves and surfaces (pp. 711-730). Avignon, France. · Zbl 1314.94018
[64] Zhang, H., Wipf, D. P., & Zhang, Y. (2013). Multi-image blind deblurring using a coupled adaptive sparse prior. In IEEE conference on computer vision and pattern recognition (pp. 1051-1058). Portland, Oregon, USA.
[65] Zhang, H., Yang, J., Zhang, Y., & Huang, T. S. (2011). Sparse representation based blind image deblurring. In IEEE international conference on multimedia and expo (pp. 1-6). Barcelona, Spain.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.