Encoding and decoding V1 fMRI responses to natural images with sparse nonparametric models. (English) Zbl 1454.62413

Summary: Functional MRI (fMRI) has become the most common method for investigating the human brain. However, fMRI data present some complications for statistical analysis and modeling. One recently developed approach to these data focuses on estimation of computational encoding models that describe how stimuli are transformed into brain activity measured in individual voxels. Here we aim at building encoding models for fMRI signals recorded in the primary visual cortex of the human brain. We use residual analyses to reveal systematic nonlinearity across voxels not taken into account by previous models. We then show how a sparse nonparametric method [P. Ravikumar et al., J. R. Stat. Soc., Ser. B, Stat. Methodol. 71, No. 5, 1009–1030 (2009; Zbl 1411.62107)] can be used together with correlation screening to estimate nonlinear encoding models effectively. Our approach produces encoding models that predict about 25% more accurately than models estimated using other methods [the fourth author et al., “Identifying natural images from human brain activity”, Nature 452, No. 7185, 352–355 (2008; doi:10.1038/nature06713)]. The estimated nonlinearity impacts the inferred properties of individual voxels, and it has a plausible biological interpretation. One benefit of quantitative encoding models is that estimated models can be used to decode brain activity, in order to identify which specific image was seen by an observer. Encoding models estimated by our approach also improve such image identification by about 12% when the correct image is one of 11,500 possible images.


62P10 Applications of statistics to biology and medical sciences; meta analysis


Zbl 1411.62107
Full Text: DOI arXiv


[1] Adelson, E. H. and Bergen, J. R. (1985). Spatiotemporal energy models for the perception of motion. J. Opt. Soc. Amer. A 2 284-299.
[2] Albrecht, D. G. and Hamilton, D. B. (1982). Striate cortex of monkey and cat: Contrast response function. Journal of Neurophysiology 48 217-237.
[3] Buxton, R. B., Wong, E. C. and Frank, L. R. (1998). Dynamics of blood flow and oxygenation changes during brain activation: The balloon model. Magnetic Resonance in Medicine 39 855-864.
[4] Buxton, R. B., Uludag, K., Dubowitz, D. J. and Liu, T. T. (2004). Modeling the hemodynamic response to brain activation. NeuroImage 23 S220-S233.
[5] Carandini, M., Heeger, D. J. and Movshon, J. A. (1997). Linearity and normalization in simple cells of the macaque primary visual cortex. Journal of Neuroscience 17 8621-8644.
[6] Cleveland, W. S. and Devlin, S. J. (1988). Locally weighted regression: An approach to regression analysis by local fitting. J. Amer. Statist. Assoc. 83 596-610. · Zbl 1248.62054
[7] De Valois, R. L. and De Valois, K. K. (1990). Spatial Vision . Oxford Univ. Press, New York. · Zbl 1216.92026
[8] Frahm, H. D., Stephan, H. and Stephan, M. (1982). Comparison of brain structure volumes in Insectivora and Primates. I. Neocortex. Journal für Hirnforschung 23 375-389.
[9] Friedman, J. H. and Popescu, B. E. (2004). Gradient directed regularization for linear regression and classification. Technical report, Dept. Statistics, Stanford Univ.
[10] Friedman, J. H. and Stuetzle, W. (1981). Projection pursuit regression. J. Amer. Statist. Assoc. 76 817-823. · doi:10.1080/01621459.1981.10477729
[11] Friston, K. J., Jezzard, P. and Turner, R. (1994). Analysis of functional MRI time-series. Human Brain Mapping 1 153-171.
[12] Hastie, T. and Tibshirani, R. (1990). Generalized Additive Models . Chapman & Hall, Boca Raton, FL. · Zbl 0747.62061
[13] Heeger, D. J. (1992). Normalization of cell responses in cat striate cortex. Visual Neuroscience 9 181-197.
[14] Hofman, M. A. (1989). On the evolution and geometry of the brain in mammals. Progress in Neurobiology 32 137-158.
[15] Jones, J. P. and Palmer, L. A. (1987). An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. Journal of Neurophysiology 58 1233-1258.
[16] Kafadar, K. and Wegman, E. J. (2006). Visualizing “typical” and “exotic” internet traffic data. Comput. Statist. Data Anal. 50 3721-3743. · Zbl 1445.62324
[17] Kay, K. N., Naselaris, T., Prenger, R. J. and Gallant, J. L. (2008a). Identifying natural images from human brain activity. Nature 452 352-355.
[18] Kay, K. N., David, S. V., Prenger, R. J., Hansen, K. A. and Gallant, J. L. (2008b). Modeling low-frequency fluctuation and hemodynamic response timecourse in event-related fMRI. Human Brain Mapping 29 142-156.
[19] Lauritzen, M. (2005). Reading vascular changes in brain imaging: Is dendritic calcium the key? Nat. Rev. Neurosci. 6 77-85.
[20] Naselaris, T., Prenger, R. J., Kay, K. N., Oliver, M. and Gallant, J. L. (2009). Bayesian reconstruction of natural images from human brain activity. Neuron 63 902-915.
[21] Naselaris, T., Kay, K. N., Nishimoto, S. and Gallant, J. L. (2011). Encoding and decoding in fMRI. NeuroImage 56 400-410.
[22] Olman, C. A., Ugurbil, K., Schrater, P. and Kersten, D. (2004). BOLD fMRI and psychophysical measurements of contrast response to broadband images. Vision Research 44 669-683.
[23] Olshausen, B. A. and Field, D. J. (1996). Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381 607-609.
[24] Radic, P. (1995). A small step for the cell, a giant leap for mankind: A hypothesis of neocortical expansion during evolution. Trends in Neurosciences 18 383-388.
[25] Raizada, R. D. S., Tsao, F.-M., Liu, H.-M. and Kuhl, P. K. (2010). Quantifying the adequacy of neural representations for a cross-language phonetic discrimination task: Prediction of individual differences. Cerebral Cortex 20 1-12.
[26] Ravikumar, P., Vu, V. Q., Yu, B., Naselaris, T., Kay, K. and Gallant, J. (2009a). Nonparametric sparse hierarchical models describe V1 fMRI responses to natural images. In Advances in Neural Information Processing Systems ( D. Koller, D. Schuurmans, Y. Bengio and L. Bottou, eds.) 21 1337-1344. Curran Associates, Inc., Redhook, NY. · Zbl 1454.62413
[27] Ravikumar, P., Lafferty, J., Liu, H. and Wasserman, L. (2009b). Sparse additive models. J. Roy. Statist. Soc. Ser. B 71 1009-1030. · doi:10.1111/j.1467-9868.2009.00718.x
[28] Sclar, G., Maunsell, J. H. R. and Lennie, P. (1990). Coding of image contrast in central visual pathways of the macaque monkey. Vision Research 30 1-10.
[29] Sharpee, T. O., Miller, K. D. and Stryker, M. P. (2008). On the importance of static nonlinearity in estimating spatiotemporal neural filters with natural stimuli. Journal of Neurophysiology 99 2496-2509.
[30] Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58 267-288. · Zbl 0850.62538
[31] Touryan, J., Lau, B. and Dan, Y. (2002). Isolation of relevant visual features from random stimuli for cortical complex cells. Journal of Neuroscience 22 10811-10818.
[32] Van Essen, D. C. (1997). A tension-based theory of morphogenesis and compact wiring in the central nervous system. Nature 385 313-318.
[33] Vinje, W. E. and Gallant, J. L. (2000). Sparse coding and decorrelation in primary visual cortex during natural vision. Science 287 1273-1276.
[34] Walther, D. B., Caddigan, E., Fei-Fei, L. and Beck, D. M. (2009). Natural scene categories revealed in distributed patterns of activity in the human brain. Journal of Neuroscience 29 10573-10581.
[35] Williams, M. A., Dang, S. and Kanwisher, N. G. (2007). Only some spatial patterns of fMRI response are read out in task performance. Nature Neuroscience 10 685-686.
[36] Zhang, K. and Sejnowski, T. J. (1999). Neuronal tuning: To sharpen or broaden? Neural Comput. 11 75-84.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.