zbMATH — the first resource for mathematics

Variational texture synthesis with sparsity and spectrum constraints. (English) Zbl 1332.62371
Summary: This paper introduces a new approach for texture synthesis. We propose a unified framework that both imposes first order statistical constraints on the use of atoms from an adaptive dictionary, as well as second order constraints on pixel values. This is achieved thanks to a variational approach, the minimization of which yields local extrema, each one being a possible texture synthesis. On the one hand, the adaptive dictionary is created using a sparse image representation rationale, and a global constraint is imposed on the maximal number of use of each atom from this dictionary. On the other hand, a constraint on second order pixel statistics is achieved through the power spectrum of images. An advantage of the proposed method is its ability to truly synthesize textures, without verbatim copy of small pieces from the exemplar. In an extensive experimental section, we show that the resulting synthesis achieves state of the art results, both for structured and small scale textures.

62M40 Random fields; image analysis
68U10 Computing methodologies for image processing
Full Text: DOI
[1] Aguerrebere, C; Gousseau, Y; Tartavel, G, Exemplar-based texture synthesis: the efros-leung algorithm, Image Processing On Line, 2013, 213-231, (2013)
[2] Aharon, M., Elad, M., Bruckstein, A.: K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on, signal processing, 54(11), 4311-4322 (2006) · Zbl 1375.94040
[3] Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.: PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics (Proceedings SIGGRAPH) 28(3) (2009)
[4] Bauschke, HH; Combettes, PL; Luke, DR, Phase retrieval, error reduction algorithm, and fienup variants: a view from convex optimization, JOSA A, 19, 1334-1345, (2002)
[5] Bonneel, N., Rabin, J., Peyré, G., Pfister, H.: Sliced and radon wasserstein barycenters of measures. to appear in Journal of Mathematical Imaging and Vision (2014). doi:10.1007/s10851-014-0506-3. http://hal.archives-ouvertes.fr/hal-00881872/ · Zbl 1332.94014
[6] Briand, T., Vacher, J., Galerne, B., Rabin, J.:The Heeger and Bergen pyramid based texture synthesis algorithm. Image Processing On Line, preprint (2013)
[7] Cross, G., Jain, A.: Markov random field texture models. IEEE Transactions on, Pattern Analysis and Machine Intelligence 5(1), 25-39 (1983)
[8] Desolneux, A., Moisan, L., Ronsin, S.: A compact representation of random phase and gaussian textures. In: Acoustics, Speech and Signal Processing (ICASSP), 2012 IEEE International Conference on, pp. 1381-1384. IEEE (2012) · Zbl 1169.49030
[9] Efros, A., Freeman, W.: Image quilting for texture synthesis and transfer. In: Proceedings of the 28th annual conference on Computer graphics and interactive techniques, pp. 341-346. ACM (2001) · Zbl 0143.41905
[10] Efros, A., Leung, T.: Texture synthesis by non-parametric sampling. In: Computer Vision, 1999. The Proceedings of the Seventh IEEE International Conference on, vol. 2, pp. 1033-1038. IEEE (1999)
[11] Elad, M., Aharon, M.: Image denoising via sparse and redundant representations over learned dictionaries. IEEE Transactions on, Image Processing 15(12), 3736-3745 (2006)
[12] Engan, K., Aase, S., Hakon Husoy, J.: Method of optimal directions for frame design. In: Acoustics, Speech, and Signal Processing, 1999. Proceedings., 1999 IEEE International Conference on, vol. 5, pp. 2443-2446. IEEE (1999)
[13] Galerne, B., Gouseau, Y., Morel, J.M.: Micro-texture synthesis by phase randomization. Image Processing On Line (2011) · Zbl 0473.62110
[14] Galerne, B., Gousseau, Y., Morel, J.: Random phase textures: Theory and synthesis. IEEE Transactions on, Image Processing 20(1), 257-267 (2011) · Zbl 1372.94086
[15] Galerne, B; Lagae, A; Lefebvre, S; Drettakis, G, Gabor noise by example, ACM Trans. Graph., 31, 73, (2012)
[16] Heeger, D., Bergen, J.: Pyramid-based texture analysis/synthesis. In: SIGGRAPH ’95, pp. 229-238 (1995) · Zbl 1255.94025
[17] Julesz, B, Visual pattern discrimination, IRE Trans. Inf. Theor., 8, 84-92, (1962)
[18] Julesz, B, A theory of preattentive texture discrimination based on first-order statistics of textons, Biol Cybern, 41, 131-138, (1981) · Zbl 0473.62110
[19] Kuhn, H, The Hungarian method for the assignment problem, Nav. res. logist. q., 2, 83-97, (1955) · Zbl 0143.41905
[20] Kwatra, V; Essa, I; Bobick, A; Kwatra, Kwatra, Texture optimization for example-based synthesis, ACM Trans. Graph., 24, 795-802, (2005)
[21] Lagae, A., Lefebvre, S., Cook, R., Derose, T., Drettakis, G., Ebert, D., Lewis, J., Perlin, K., Zwicker, M.: State of the art in procedural noise functions. EG 2010-State of the Art Reports (2010)
[22] Lagae, A., Lefebvre, S., Drettakis, G., Dutré, P.: Procedural noise using sparse gabor convolution. In: ACM Transactions on Graphics (TOG), vol. 28, p. 54. ACM (2009)
[23] Lefebvre, S; Hoppe, H, Parallel controllable texture synthesis, ACM Trans. Grap. (TOG), 24, 777-786, (2005)
[24] Lewis, AS; Luke, DR; Malick, J, Local linear convergence for alternating and averaged nonconvex projections, Found. Comput. Math., 9, 485-513, (2009) · Zbl 1169.49030
[25] Mairal, J; Bach, F; Ponce, J; Sapiro, G, Online learning for matrix factorization and sparse coding, J Mach. Learn. Res., 11, 19-60, (2010) · Zbl 1242.62087
[26] Mallat, S., Zhang, Z.: Matching pursuits with time-frequency dictionaries. IEEE Transactions on Signal Processing 41(12), 3397-3415 (1993) · Zbl 0842.94004
[27] Moisan, L, Periodic plus smooth image decomposition, J. Math. Imag. Vis., 39, 161-179, (2011) · Zbl 1255.94025
[28] Olshausen, B; Field, D, Natural image statistics and efficient coding, Network, 7, 333-339, (1996)
[29] Perlin, K.: An image synthesizer. SIGGRAPH Comput. Graph. 19(3), 287-296 (1985) · Zbl 0473.62110
[30] Peyré, G, Sparse modeling of textures, J. Math. Imag. Vis., 34, 17-31, (2009)
[31] Pickard, R., Graszyk, C., Mann, S., Wachman, J., Pickard, L., Campbell, L.: Vistex Database. Media Lab, MIT, Cambridge, Massachusetts (1995)
[32] Pitie, F., Kokaram, A., Dahyot, R.: N-dimensional probability density function transfer and its application to color transfer. In: Computer Vision, 2005. ICCV 2005. Tenth IEEE International Conference on, vol. 2, pp. 1434-1439. IEEE (2005)
[33] Portilla, J; Simoncelli, E, A parametric texture model based on joint statistics of complex wavelet coefficients, Int. J. Comput. Vis., 40, 49-70, (2000) · Zbl 1012.68698
[34] Ramanarayanan, G., Bala, K.: Constrained texture synthesis via energy minimization. IEEE Transactions on Visualization and Computer Graphics 13(1), 167-178 (2007)
[35] Simoncelli, E., Freeman, W., Adelson, E., Heeger, D.: Shiftable multiscale transforms. IEEE Transactions on Information Theory 38(2), 587-607 (1992)
[36] Tartavel, G., Gousseau, Y., Peyré, G.: Constrained sparse texture synthesis. In: Proceedings of SSVM’13 (2013) · Zbl 1332.62371
[37] Tropp, J.: Greed is good: Algorithmic results for sparse approximation. IEEE Transactions on Information Theory 50(10), 2231-2242 (2004) · Zbl 1288.94019
[38] Wei, L., Lefebvre, S., Kwatra, V., Turk, G.: State of the art in example-based texture synthesis. In: Eurographics 2009, State of the Art Report, EG-STAR. Eurographics Association (2009)
[39] Wei, L., Levoy, M.: Fast texture synthesis using tree-structured vector quantization. In: SIGGRAPH ’00, pp. 479-488. ACM Press/Addison-Wesley Publishing Co. (2000)
[40] Wexler, Y., Shechtman, E., Irani, M.: Space-time video completion. In: Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on, vol. 1, pp. I-120. IEEE (2004)
[41] Xia, G.S., Ferradans, S., Peyré, G., Aujol, J.F.: Synthesizing and mixing stationary gaussian texture models. SIAM J. Imag. Sci. 7(1), 476-508 (2014). doi:10.1137/130918010. http://hal.archives-ouvertes.fr/hal-00816342/ · Zbl 1391.94108
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.