×

Varying-coefficient models for geospatial transfer learning. (English) Zbl 1460.62156

Summary: We study prediction problems in which the conditional distribution of the output given the input varies as a function of task variables which, in our applications, represent space and time. In varying-coefficient models, the coefficients of this conditional are allowed to change smoothly in space and time; the strength of the correlations between neighboring points is determined by the data. This is achieved by placing a Gaussian process (GP) prior on the coefficients. Bayesian inference in varying-coefficient models is generally intractable. We show that with an isotropic GP prior, inference in varying-coefficient models resolves to standard inference for a GP that can be solved efficiently. MAP inference in this model resolves to multitask learning using task and instance kernels. We clarify the relationship between varying-coefficient models and the hierarchical Bayesian multitask model and show that inference for hierarchical Bayesian multitask models can be carried out efficiently using graph-Laplacian kernels. We explore the model empirically for the problems of predicting rent and real-estate prices, and predicting the ground motion during seismic events. We find that varying-coefficient models with GP priors excel at predicting rents and real-estate prices. The ground-motion model predicts seismic hazards in the State of California more accurately than the previous state of the art.

MSC:

62M30 Inference from spatial processes
62M20 Inference from stochastic processes and prediction
62F15 Bayesian inference
62H30 Classification and discrimination; cluster analysis (statistical aspects)
62P20 Applications of statistics to economics
86A15 Seismology (including tsunami modeling), earthquakes
86A32 Geostatistics

Software:

GPML; PMTK; BayesDA
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Abrahamson, N. A., Silva, W. J., & Kamai, R. (2014). Summary of the ASK14 ground motion relation for active crustal regions. Earthquake Spectra, 30(3), 1025-1055. · doi:10.1193/070913EQS198M
[2] Adams, R. P., & Stegle, O. (2008). Gaussian process product models for nonparametric nonstationarity. In Proceedings of the 25th international conference on machine learning.
[3] Akkar, S., & Cagnan, Z. (2010). A local ground-motion predictive model for turkey, and its comparison with other regional and global ground-motion models. Bulletin of the Seismological Society of America, 100(6), 2978-2995. · doi:10.1785/0120090367
[4] Al-Atik, L., Abrahamson, N., Bommer, J. J., Scherbaum, F., Cotton, F., & Kuehn, N. (2010). The variability of ground-motion prediction models and its components. Seismological Research Letters, 81(5), 794-801. · doi:10.1785/gssrl.81.5.794
[5] Álvarez, M. A., Rosasco, L., & Lawrence, N. D. (2011). Kernels for vector-valued functions: A review. Technical report MIT-CSAIL-TR-2011-033, Massachusetts Institute of Technology. · Zbl 1301.68212
[6] Ancheta, T. D., Darragh, R. B., Stewart, J. P., Seyhan, E., Silva, W. J., Chiou, B. S.-J., et al. (2014). NGA-West2 database. Earthquake Spectra, 30(3), 989-1005. · doi:10.1193/070913EQS197M
[7] Anderson, J. D., & Brune, J. N. (1999). Probabilistic seismic hazard analysis without the ergodic assumption. Seismological Research Letters, 70(1), 19-28. · doi:10.1785/gssrl.70.1.19
[8] Atkinson, G. M., & Morrison, M. (2009). Observations on regional variability in ground-motion amplitudes for small-to-moderate earthquakes in North America. Bulletin of the Seismological Society of America, 99(4), 2393-2409. · doi:10.1785/0120080223
[9] Bindi, D., Pacor, F., Luzi, L., Puglia, R., Massa, M., Ameri, G., et al. (2011). Ground motion prediction equations derived from the italian strong motion database. Bulletin of Earthquake Engineering, 9(6), 1899-1920. · doi:10.1007/s10518-011-9313-z
[10] Bonilla, E. V., Agakov, F. V., & Williams, C. K. I. (2007). Kernel multi-task learning using task-specific features. In Proceedings of the 11th international conference on artificial intelligence and statistics.
[11] Bragato, P. L., & Slejko, D. (2005). Empirical ground-motion attenuation relations for the eastern Alps in the magnitude range 2.5-6.3. Bulletin of the Seismological Society of America, 95(1), 252276. · doi:10.1785/0120030231
[12] Chiou, B., Youngs, R., Abrahamson, N., & Addo, K. (2010). Ground-motion attenuation model for small-to-moderate shallow crustal earthquakes in California and its implications on regionalization of ground-motion prediction models. Earthquake Spectra, 26(4), 907-926. · doi:10.1193/1.3479930
[13] City of New York. (2013). NYC Open Data. https://nycopendata.socrata.com/.
[14] Cressie, N. (2015). Statistics for spatial data. New York: Wiley. · Zbl 1347.62005
[15] Danciu, L., & Tselentis, G. A. (2007). Engineering ground-motion parameters attenuation relationships for greece. Bulletin of the Seismological Society of America, 97(1), 162183.
[16] Daumé III, H. (2009). Frustratingly easy domain adaptation. arXiv preprint arXiv:0907.1815.
[17] Estes, J., Nguyen, D., Dalrymple, L., Mu, Y., & Sentürk, D. (2014). Cardiovascular event risk dynamics over time in older patients on dialysis: A generalized multiple-index varying coefficient model approach. Biometrics, 70(3), 751-761. · Zbl 1299.62118 · doi:10.1111/biom.12176
[18] Evgeniou, T., Micchelli, C. A., & Pontil, M. (2005). Learning multiple tasks with kernel methods. Journal of Machine Learning Research, 6(1), 615-637. · Zbl 1222.68197
[19] Fan, J., & Huang, T. (2005). Profile likelihood inferences on semiparametric varying-coefficient partially linear models. Bernoulli, 11(6), 1031-1057. · Zbl 1098.62077 · doi:10.3150/bj/1137421639
[20] Fan, J., & Zhang, W. (2008). Statistical methods with varying coefficient models. Statistics and Its Interface, 1(1), 179-195. · Zbl 1230.62031 · doi:10.4310/SII.2008.v1.n1.a15
[21] Finkel, J. R., & Manning, C. D. (2009). Hierarchical Bayesian domain adaptation. In Proceedings of the annual conference of the North American chapter of the association for computational linguistics.
[22] Gelfand, A. E., Kim, H., Sirmans, C. F., & Banerjee, S. (2003). Spatial modeling with spatially varying coefficient processes. Journal of the American Statistical Association, 98(462), 387-396. · Zbl 1041.62041 · doi:10.1198/016214503000170
[23] Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. (1995). Bayesian data analysis. London: Chapman & Hall. · Zbl 0914.62018
[24] Gianniotis, N., Kuehn, N., & Scherbaum, F. (2014). Manifold aligned ground motion prediction equations for regional datasets. Computers and Geosciences, 69, 72-77. · doi:10.1016/j.cageo.2014.04.014
[25] Hastie, T., & Tibshirani, R. (1993). Varying-coefficient models. Journal of the Royal Statistical Society, 55(4), 757-796. · Zbl 0796.62060
[26] Higdon, D. (2002). Space and space-time modeling using process convolutions. In Quantitative methods for current environmental issues (pp. 37-56). Berlin: Springer. · Zbl 1255.86016
[27] Journel, A. G., & Huijbregts, C. J. (1978). Mining geostatistics. London: Academic Press.
[28] Landwehr, N., Kuehn, N., Scheffer, T., & Abrahamson, N. (2016). A non-ergodic ground-motion model for California with spatially varying coefficients. Bulletin of the Seismological Society of America, 106, 2574-2583. · doi:10.1785/0120160118
[29] Lin, P.-S., Chiou, B., Abrahamson, N., Walling, M., Lee, C.-T., & Cheng, C.-T. (2011). Repeatable source, site, and path effects on the standard deviation for empirical ground-motion prediction models. Bulletin of the Seismological Society of America, 101(5), 2281-2295. · doi:10.1785/0120090312
[30] Matheron, G. (1963). Principles of geostatistics. Economic Geology, 58(8), 1246-1266. · doi:10.2113/gsecongeo.58.8.1246
[31] Murphy, K. P. (2012). Machine learning: A probabilistic perspective. Cambridge, MA: MIT Press. · Zbl 1295.68003
[32] Rasmussen, C. E., & Nickisch, H. (2010). Gaussian processes for machine learning (GPML) toolbox. Journal of Machine Learning Research, 11, 3011-3015. · Zbl 1242.68242
[33] Schölkopf, B., & Smola, A. J. (2002). Learning with kernels. Cambridge, MA: MIT Press. · Zbl 1019.68094
[34] Snelson, E., & Ghahramani, Z. (2005). Sparse Gaussian processes using pseudo-inputs. In Proceedings of the 19th annual conference on neural information processing systems.
[35] Stafford, P. J. (2014). Crossed and nested mixed-effects approaches for enhanced model development and removal of the ergodic assumption in empirical ground-motion models. Bulletin of the Seismological Society of America, 104(2), 702-719. · doi:10.1785/0120130145
[36] Tolvanen, V., Jylanki, P., & Vehtari, A. (2014). Expectation propagation for nonstationary heteroscedastic Gaussian process regression. In Proceedings of the 2014 IEEE international workshop on machine learning for signal processing.
[37] US Census Bureau. Public use microdata areas. https://www.census.gov/geo/reference/puma.html (2010).
[38] US Census Bureau. (2013) American Community Surveys ASC public use microdata sample files. http://factfinder.census.gov/faces/affhelp/jsf/pages/metadata.xhtml?lang=en&type=document&id=document.en.ACS_pums_csv_2013#main_content.
[39] Ver Hoef, J. M., & Barry, R. P. (1998). Constructing and fitting models for cokriging and multivariable spatial prediction. Journal of Statistical Planning and Inference, 69(2), 275-294. · Zbl 0935.62110 · doi:10.1016/S0378-3758(97)00162-6
[40] Wang, C., & Neal, R. M. (2012). Gaussian process regression with heteroscedastic or non-Gaussian residuals. Technical Report CoRR abs/1212.6246, University of Toronto.
[41] Wang, J. M., Fleet, D. J., & Hertzmann, A. (2007). Multifactor Gaussian process models for style-content separation. In Proceedings of the 24th international conference on machine learning.
[42] Wheeler, D. C., & Calder, C. A. (2006). Bayesian spatially varying coefficient models in the presence of collinearity. Spatial Modeling Section: American Statistical Association.
[43] Wilson, A. G., Knowles, D. A., & Ghahramani, Z. (2011). Gaussian process regression networks. arXiv preprint arXiv:1110.4411.
[44] Wu, C. O., & Chiang, C. T. (2000). Kernel smoothing on varying coefficient models with longitudinal dependent variable. Statistica Sinica, 10(1), 433-456. · Zbl 0945.62047
[45] Yan, R., & Zhang, J. (2009). Transfer learning using task-level features with application to information retrieval. In Proceedings of the 21st international joint conference on artificial intelligence.
[46] Zhang, Y., & Yeung, D. (2010). A convex formulation for learning task relationships in multi-task learning. In Proceedings of the 26th conference on uncertainty in artificial intelligence.
[47] Zhu, H., Fan, J., & Kong, L. (2014). Spatially varying coefficient model for neuroimaging data with jump discontinuities. Journal of the American Statistical Association, 109(507), 1084-1098. · Zbl 1368.62297 · doi:10.1080/01621459.2014.881742
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.