Hierarchical probabilistic forecasting of electricity demand with smart meter data. (English) Zbl 1457.62286

Summary: Decisions regarding the supply of electricity across a power grid must take into consideration the inherent uncertainty in demand. Optimal decision-making requires probabilistic forecasts for demand in a hierarchy with various levels of aggregation, such as substations, cities, and regions. The forecasts should be coherent in the sense that the forecast of the aggregated series should equal the sum of the forecasts of the corresponding disaggregated series. Coherency is essential, since the allocation of electricity at one level of the hierarchy relies on the appropriate amount being provided from the previous level. We introduce a new probabilistic forecasting method for a large hierarchy based on UK residential smart meter data. We find our method provides coherent and accurate probabilistic forecasts, as a result of an effective forecast combination. Furthermore, by avoiding distributional assumptions, we find that our method captures the variety of distributions in the smart meter hierarchy. Finally, the results confirm that, to ensure coherency in our large-scale hierarchy, it is sufficient to model a set of lower-dimension dependencies, rather than modeling the entire joint distribution of all series in the hierarchy. In achieving coherent and accurate hierarchical probabilistic forecasts, this work contributes to improved decision-making for smart grids.


62M20 Inference from stochastic processes and prediction
62P30 Applications of statistics in engineering and industry; control charts
62M10 Time series, auto-correlation, regression, etc. in statistics (GARCH)


Forecast; forecast
Full Text: DOI


[1] Technical Report, Energy Demand Research Project: Final Analysis (2011), AECOM House: AECOM House, Hertfordshire, UK
[2] AECOM Building Engineering, “Energy Demand Research Project: Early Smart Meter Trials, 2007-2010,” (2014)
[3] Arbenz, P.; Hummel, C.; Mainik, G., “Copula Based Hierarchical Risk Aggregation Through Sample Reordering, Insurance, Mathematics & Economics, 51, 122-133 (2012) · Zbl 1284.91198
[4] Arora, S.; Taylor, J. W., “Forecasting Electricity Smart Meter Data Using Conditional Kernel Density Estimation, Omega, 59, 47-59 (2016)
[5] Ben, Taieb; Huser, R.; Hyndman, R. J.; Genton, M. G., “Forecasting Uncertainty in Electricity Smart Meter Data by Boosting Additive Quantile Regression, IEEE Transactions on Smart Grid, 7, 2448-2455 (2016)
[6] Berrocal, V. J.; Raftery, A. E.; Gneiting, T.; Steed, R. C., “Probabilistic Weather Forecasting for Winter Road Maintenance, Journal of the American Statistical Association, 105, 522-537 (2010) · Zbl 1392.62330
[7] Bessa, R. J.; Trindade, A.; Miranda, V., “Spatial-Temporal Solar Power Forecasting for Smart Grids, IEEE Transactions on Industrial Informatics, 11, 232-241 (2015)
[8] Borenstein, S.; Bushnell, J., “The US Electricity Industry After 20 Years of Restructuring, Annual Review of Economics, 7, 437-463 (2015)
[9] Borges, C. E.; Penya, Y. K.; Fernandez, I., “Evaluating Combined Load Forecasting in Large Power Systems and Smart Grids, IEEE Transactions on Industrial Informatics, 9, 1570-1577 (2013)
[10] Cabrera, B. L.; Schulz, F., “Forecasting Generalized Quantiles of Electricity Demand: A Functional Data Approach, Journal of the American Statistical Association, 112, 127-136 (2017)
[11] Cho, H.; Goude, Y.; Brossat, X.; Yao, Q., “Modeling and Forecasting Daily Electricity Load Curves: A Hybrid Approach, Journal of the American Statistical Association, 108, 7-21 (2013) · Zbl 1379.62091
[12] Cottet, R.; Smith, M., “Bayesian Modeling and Forecasting of Intraday Electricity Load, Journal of the American Statistical Association, 98, 839-849 (2003)
[13] De Livera, A. M.; Hyndman, R. J.; Snyder, R., “Forecasting Time Series With Complex Seasonal Patterns Using Exponential Smoothing, Journal of the American Statistical Association, 106, 1513-1527 (2011) · Zbl 1234.62123
[14] Dowell, J.; Pinson, P., “Very-Short-Term Probabilistic Wind Power Forecasts by Sparse Vector Autoregression,”, IEEE Transactions on Smart Grid, 7, 763-770 (2016)
[15] Gijbels, I.; Herrmann, K., “On the Distribution of Sums of Random Variables With Copula-Induced Dependence, Insurance, Mathematics & Economics, 59, 27-44 (2014) · Zbl 1323.62020
[16] Gneiting, T., “Making and Evaluating Point Forecasts, Journal of the American Statistical Association, 106, 746-762 (2011) · Zbl 1232.62028
[17] Gneiting, T.; Balabdaoui, F.; Raftery, A. E., “Probabilistic Forecasts, Calibration and Sharpness, Journal of the Royal Statistical Society, Series B, 69, 243-268 (2007) · Zbl 1120.62074
[18] Gneiting, T.; Katzfuss, M., “Probabilistic Forecasting, Annual Review of Statistics and Its Application, 1, 125-151 (2014)
[19] Gneiting, T.; Ranjan, R., “Comparing Density Forecasts Using Threshold- and Quantile-Weighted Scoring Rules, Journal of Business & Economic Statistics, 29, 411-422 (2011) · Zbl 1219.91108
[20] Grimit, E. P.; Gneiting, T.; Berrocal, V. J.; Johnson, N. A., “The Continuous Ranked Probability Score for Circular Variables and Its Application to Mesoscale Forecast Ensemble Verification, Quarterly Journal of the Royal Meteorological Society, 132, 2925-2942 (2006)
[21] Haben, S.; Ward, J.; Vukadinovic Greetham, D.; Singleton, C.; Grindrod, P., “A New Error Measure for Forecasts of Household-Level, High Resolution Electrical Energy Consumption, International Journal of Forecasting, 30, 246-256 (2014)
[22] Hall, P.; Racine, J.; Li, Q., “Cross-Validation and the Estimation of Conditional Probability Densities, Journal of the American Statistical Association, 99, 1015-1026 (2004) · Zbl 1055.62035
[23] Hong, T.; Pinson, P.; Fan, S.; Zareipour, H.; Troccoli, A.; Hyndman, R. J., “Probabilistic Energy Forecasting: Global Energy Forecasting Competition 2014 and Beyond, International Journal of Forecasting, 32, 896-913 (2016)
[24] Hyndman, R.; Athanasopoulos, G.; Bergmeir, C.; Caceres, G.; Chhay, L.; O’Hara-Wild, M.; Petropoulos, F.; Razbash, S.; Wang, E.; Yasmeen, F., “forecast: Forecasting functions for time series and linear models.” R package version 8.9 (2019)
[25] Hyndman, R. J.; Ahmed, R. A.; Athanasopoulos, G.; Shang, H. L., “Optimal Combination Forecasts for Hierarchical Time Series, Computational Statistics & Data Analysis, 55, 2579-2589 (2011) · Zbl 1464.62095
[26] Hyndman, R. J.; Khandakar, Y., “Automatic Time Series for Forecasting: The Forecast Package for R, Journal of Statistical Software, 27, 1-22 (2008)
[27] Hyndman, R. J.; Lee, A. J.; Wang, E., “Fast Computation of Reconciled Forecasts for Hierarchical and Grouped Time Series, Computational Statistics & Data Analysis, 97, 16-32 (2016) · Zbl 1468.62086
[28] Iman, R. L.; Conover, W. J., “A Distribution-Free Approach to Inducing Rank Correlation Among Input Variables, Communications in Statistics—Simulation and Computation, 11, 311-334 (1982) · Zbl 0496.65071
[29] Jeon, J.; Taylor, J. W., “Using Conditional Kernel Density Estimation for Wind Power Density Forecasting, Journal of the American Statistical Association, 107, 66-79 (2012) · Zbl 1261.62031
[30] Kremer, M.; Siemsen, E.; Thomas, D. J., “The Sum and Its Parts: Judgmental Hierarchical Forecasting, Management Science, 62, 2745-2764 (2016)
[31] Mainik, G., “Risk Aggregation With Empirical Margins: Latin Hypercubes, Empirical Copulas, and Convergence of Sum Distributions, Journal of Multivariate Analysis, 141, 197-216 (2015) · Zbl 1327.62274
[32] Mirowski, P.; Chen, S.; Ho, T. K.; Yu, C.-N., “Demand Forecasting in Smart Grids, Bell Labs Technical Journal, 18, 135-158 (2014)
[33] Misiti, M.; Misiti, Y.; Oppenheim, G.; Poggi, J.-M., Optimized Clusters for Disaggregated Electricity Load Forecasting, REVSTAT—Statistical Journal, 8, 105-124 (2010) · Zbl 1297.62253
[34] Nelsen, R. B., An Introduction to Copulas (2007), New York: Springer, New York
[35] Patton, A. J., “Modelling Asymmetric Exchange Rate Dependence, International Economic Review, 47, 527-556 (2006)
[36] Ramchurn, S. D.; Vytelingum, P.; Rogers, A.; Jennings, N. R., “Putting the ‘Smarts’ Into the Smart Grid: A Grand Challenge for Artificial Intelligence, Communications of the ACM, 55, 86-97 (2012)
[37] Ramdas, A.; Trillos, N. G.; Cuturi, M., “On Wasserstein Two-Sample Testing and Related Families of Nonparametric Tests, Entropy, 19, 47 (2017)
[38] Rüschendorf, L., “On the Distributional Transform, Sklar’s Theorem, and the Empirical Copula Process, Journal of Statistical Planning and Inference, 139, 3921-3927 (2009) · Zbl 1171.60313
[39] Schäfer, J.; Strimmer, K., “A Shrinkage Approach to Large-Scale Covariance Matrix Estimation and Implications for Functional Genomics, Statistical Applications in Genetics and Molecular Biology, 4, 32 (2005)
[40] Schefzik, R.; Thorarinsdottir, T. L.; Gneiting, T., “Uncertainty Quantification in Complex Simulation Models Using Ensemble Copula Coupling, Statistical Science, 28, 616-640 (2013) · Zbl 1331.62265
[41] Sklar, M. (1959), “Fonctions de répartition ‘̀a n dimensions et leurs marges,” Université Paris 8. · Zbl 0100.14202
[42] Smith, M.; Min, A.; Almeida, C.; Czado, C., “Modeling Longitudinal Data Using a Pair-Copula Decomposition of Serial Dependence, Journal of the American Statistical Association, 105, 1467-1479 (2010) · Zbl 1388.62171
[43] Sun, X.; Luh, P. B.; Cheung, K. W.; Guan, W.; Michel, L. D.; Venkata, S. S.; Miller, M. T., “An Efficient Approach to Short-Term Load Forecasting at the Distribution Level, IEEE Transactions on Power Systems, 31, 2526-2537 (2016)
[44] Taylor, J. W., “Exponentially Weighted Methods for Forecasting Intraday Time Series With Multiple Seasonal Cycles, International Journal of Forecasting, 26, 627-646 (2010)
[45] Taylor, J. W.; Buizza, R., “Neural Network Load Forecasting With Weather Ensemble Predictions, IEEE Transactions on Power Systems, 17, 626-632 (2002)
[46] Taylor, J. W.; McSharry, P.; El-Hawary, M., Advances in Electric Power and Energy; Power Systems Engineering, Power Engineering Series of IEEE Press, Univariate Methods for Short-Term Load Forecasting,”, 17-40 (2012), Hoboken, NJ: Wiley, Hoboken, NJ
[47] van Erven, T.; Cugliari, J.; Antoniadis, A.; Poggi, J. M.; Brossat, X., Modeling and Stochastic Learning for Forecasting in High Dimensions, Game-Theoretically Optimal Reconciliation of Contemporaneous Hierarchical Time Series Forecasts,”, 297-317 (2015), Cham: Springer, Cham
[48] Wickramasuriya, S. L.; Athanasopoulos, G.; Hyndman, R. J., “Optimal Forecast Reconciliation for Hierarchical and Grouped Time Series Through Trace Minimization, Journal of the American Statistical Association, 114, 804-819 (2018) · Zbl 1420.62402
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.