×

Estimating robot strengths with application to selection of alliance members in FIRST Robotics Competitions. (English) Zbl 1510.62280

Summary: Since the inception of the FIRST Robotics Competition (FRC) and its special playoff system, robotics teams have longed to appropriately quantify the strengths of their designed robots. The FRC includes a playground draft-like phase (alliance selection), arguably the most game-changing part of the competition, in which the top-8 robotics teams in a tournament based on the FRC’s ranking system assess potential alliance members for the opportunity of partnering in a playoff stage. In such a three-versus-three competition, several measures and models have been used to characterize actual or relative robot strengths. However, existing models are found to have poor predictive performance due to their imprecise estimates of robot strengths caused by a small ratio of the number of observations to the number of robots. A more general regression model with latent clusters of robot strengths is, thus, proposed to enhance their predictive capacities. Two effective estimation procedures are further developed to simultaneously estimate the number of clusters, clusters of robots, and robot strengths. Meanwhile, some measures are used to assess the predictive ability of competing models, the agreement between published FRC measures of strength and model-based robot strengths of all, playoff, and FRC top-8 robots, and the agreement between FRC top-8 robots and model-based top robots. Moreover, the stability of estimated robot strengths and accuracies is investigated to determine whether the scheduled matches are excessive or insufficient. In the analysis of qualification data from the 2018 FRC Houston and Detroit championships, the predictive ability of our model is also shown to be significantly better than those of existing models. Teams who adopt the new model can now appropriately rank their preferences for playoff alliance partners with greater predictive capability than before.

MSC:

62H30 Classification and discrimination; cluster analysis (statistical aspects)
62P30 Applications of statistics in engineering and industry; control charts
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Akaike, H., A new look at statistical model identification, IEEE Trans. Automat. Control, 19, 6, 716-723 (1974) · Zbl 0314.62039
[2] Aldous, D., Elo ratings and the sports model: a neglected topic in applied probability?, Stat. Sci., 32, 4, 616-629 (2017) · Zbl 1383.60057
[3] Bradley, R. A., Some statistical methods in taste testing and quality evaluation, Biometrics, 9, 1, 22-38 (1953)
[4] Bradley, R. A.; Terry, M. E., Rank analysis of incomplete block designs: I. the method of paired comparisons, Biometrika, 39, 3/4, 324-345 (1952) · Zbl 0047.12903
[5] Cattelan, M.; Varin, C.; Firth, D., Dynamic Bradley-Terry modelling of sports tournaments, J. R. Stat. Soc. Ser. C, 62, 1, 135-150 (2013)
[6] Chen, Z.; Sun, Y.; El-nasr, M. S.; Nguyen, T. D., Player skill decomposition in multiplayer online battle arenas (2017)
[7] Chiang, C.-T.; Chiu, C.-H., Nonparametric and semiparametric optimal transformations of markers, J. Multivariate Anal., 103, 1, 124-141 (2012) · Zbl 1231.62051
[8] Clark, N.; Macdonald, B.; Kloo, I., A Bayesian adjusted plus-minus analysis for the esport Dota 2 (2020)
[9] Collins, L. M.; Lanza, S. T., Latent Class and Latent Transition Analysis: with Applications in the Social, Behavioral, and Health Sciences (2010), Wiley: Wiley New York
[10] Deshpande, S. K.; Jensen, S. T., Estimating an NBA players impact on his teams chances of winning, J. Quant. Anal. Sports, 12, 2, 51-72 (2016)
[11] Diggle, P. J.; Heagerty, P.; Liang, K. Y.; Zeger, S. L., Analysis of Longitudinal Data (2002), Oxford: Oxford Oxford
[12] Elo, A. E., The Rating of Chess Players, Past and Present (1978), Arco: Arco New York
[13] Fahrmeir, L.; Tutz, G., Multivariate Statistical Modelling Based on Generalized Linear Models (2001), Springer: Springer New York · Zbl 0980.62052
[14] Fang, E., The math behind OPR - an introduction. The Blue Alliance Blog (2017), https://blog.thebluealliance.com/2017/10/05/the-math-behind-opr-an-introduction/
[15] Feamhead, P.; Taylor, B., On estimating the ability of NBA players, J. Quant. Anal. Sports, 7, 3 (2011)
[16] 2017 FIRST robotics competition STEAMWORKS game animation (2017), https://www.youtube.com/watch?v=EMiNmJW7enI
[17] 2017 game and season manual (2017), https://firstfrc.blob.core.windows.net/frc2017/Manual/2017FRCGameSeasonManual.pdf
[18] Gardner, W., An overview and analysis of statistics used to rate FIRST robotics teams (2015), https://www.chiefdelphi.com/uploads/default/original/3X/e/b/ebd6f208f128bcc2d44267c7b6d25a4ae53aa8a4.pdf
[19] Giraud, C., Introduction to High-Dimensional Statistics (2015), Chapman & Hall/CRC: Chapman & Hall/CRC Florida · Zbl 1341.62011
[20] Goodman, L. A., Exploratory latent structure analysis using both identifiable and unidentifiable models, Biometrika, 61, 215-231 (1974) · Zbl 0281.62057
[21] Gordon, A. D., A review of hierarchical classification, J. R. Stat. Soc. A, 150, 2, 119-137 (1987) · Zbl 0616.62086
[22] Han, A. K., Non-parametric analysis of a generalized regression model: the maximum rank correlation estimator, J. Econom., 35, 2, 303-316 (1987) · Zbl 0638.62063
[23] Hass, Z.; Craig, B., Exploring the potential of the plus/minus in NCAA women’s volleyball via the recovery of court presence information, J. Sports Anal., 4, 285-295 (2018)
[24] Hochberg, Y.; Tamhane, A. C., Multiple Comparison Procedures (1987), Wiley: Wiley New York · Zbl 0731.62125
[25] Hothorn, T.; Bretz, F.; Westfall, P., Simultaneous inference in general parametric models, Biom. J., 50, 3, 346-363 (2008) · Zbl 1442.62415
[26] Hsu, J. C., Multiple comparisons: theory and methods (1996), Chapman & Hall: Chapman & Hall London · Zbl 0898.62090
[27] Huang, T. K.; Lin, C. J.; Weng, R. C., Ranking individuals by group comparisons, J. Mach. Learn. Res., 9, 2187-2216 (2008) · Zbl 1225.68187
[28] Hvattum, L. M., A comprehensive review of plus-minus ratings for evaluating individual players in team sports, Int. J. Comput. Sci. Sports, 18, 1, 1-23 (2019)
[29] Ilardi, S.; Barzilai, A., Adjusted plus-minus ratings: new and improved for 2007-2008 (2008), https://www.82games.com/ilardi2.htm
[30] Islam, M. M.; Khan, J. R.; Raheem, E., Bradley-Terry model for assessing the performance of ten ODI cricket teams adjusting for home ground effect, Data Sci. J., 16, 657-668 (2017)
[31] Kendall, M., A new measure of rank correlation, Biometrika, 30, 1/2, 81-89 (1938) · Zbl 0019.13001
[32] Kovalchik, S., Extension of the Elo rating system to margin of victory, Int. J. Forecast., 36, 1329-1341 (2020)
[33] Krzanowski, W. J.; Lai, Y. T., A criterion for determining the number of clusters in a data set, Biometrics, 44, 23-34 (1985) · Zbl 0707.62122
[34] Laird, N. M.; Ware, J. H., Random-effects models for longitudinal data, Biometrics, 38, 4, 963-974 (1982) · Zbl 0512.62107
[35] Law, E., New scouting database from team 2834 (2008), https://www.chiefdelphi.com/forums/showpost.php?p=835222&postcount=48
[36] Lazarsfeld, P. F.; Henry, N. W., Latent Structure Analysis (1968), Houghton Mifflin: Houghton Mifflin Boston · Zbl 0182.52201
[37] Macdonald, B., A regression-based adjusted plus-minus statistic for NHL players, J. Quant. Anal. Sports, 7, 3 (2011)
[38] Mammen, E., Empirical process of residuals for high-dimensional linear models, Ann. Statist., 24, 1, 307-325 (1996) · Zbl 0853.62042
[39] Marx, I., Transformation of series by a variant of Stirling’s numbers, Am. Math. Mon., 69, 6 (1962) · Zbl 0136.35601
[40] Mosteller, F., Remarks on the method of paired comparisons: I. the least squares solution assuming equal standard deviations and equal correlations, Psychometrika, 16, 1, 3-9 (1951)
[41] Perry, J. W.; Kent, A.; Berry, M. M., Machine literature searching X. Machine language; factors underlying its design and development, Am. Doc., 6, 4, 242-254 (1955)
[42] Portnoy, S., Asymptotic behavior of M-estimators of \(p\) regression parameters when \(p^2 / n\) is large. I. Consistency, Ann. Statist., 12, 1298-1309 (1984) · Zbl 0584.62050
[43] Rosenbaum, D. T., Measuring how NBA players help their teams win (2004), https://www.82games.com/comm30.htm
[44] Sæbø, O. D.; Hvattum, L. M., Evaluating the efficiency of the association football transfer market using regression based player ratings, (NIK-2015 Conference, Vol. 4 (2015), Bibsys Open Journal Sysytems), 12
[45] Salmeri, A., Introduzione alla teoria dei coefficienti fattoriali, G. Mat. Battagl., 90, 10, 44-54 (1962) · Zbl 0124.25102
[46] Schuckers, M. E.; Lock, D. F.; Wells, C.; Knickerbocker, C. J.; Lock, R. H., National Hockey League skater ratings based upon on-ice events: an adjusted minus/plus probability (AMPP) approach (2011), https://www.researchgate.net/profile/Michael_Schuckers/publication/266493342_Copyright_C_2011_Michael_Schuckers_National_Hockey_League_Skater_Ratings_Based_upon_All_On-Ice_Events_An_Adjusted_MinusPlus_Probability_AMPP_Approach/links/55251f210cf2caf11bfd146c.pdf
[47] Schwarz, G., Estimating the dimension of a model, Ann. Statist., 6, 2, 461-464 (1978) · Zbl 0379.62005
[48] Sen, A.; Srivastava, M., Regression analysis: theory, methods, and applications (1990), Springer: Springer Berlin · Zbl 0714.62057
[49] Sill, J., Improved NBA adjusted +/- using regularization and out-of-sample testing, (MIT Sloan Sports Analytics Conference (2010), MIT)
[50] Sismanis, Y., How i won the “Chess Ratings - Elo vs the Rest of the World” competition (2010)
[51] Slowinski, S., What is WAR? (2010), https://library.fangraphs.com/misc/war/
[52] Sokal, R. R.; Michener, C. D., (A Statistical Method for Evaluating Systematic Relationships (1958)), 1409-1438
[53] Spearman, C., The proof and measurement of association between two things, Am. J. Psychol., 15, 1, 72-101 (1904)
[54] Sugar, C.; James, G., Finding the number of clusters in a data set: an information theoretic approach, J. Amer. Statist. Assoc., 98, 463, 750-763 (2003) · Zbl 1046.62064
[55] NBA team possessions per game (2019), https://www.teamrankings.com/nba/stat/possessions-per-game
[56] Hopper division 2018 (2018), https://www.thebluealliance.com/event/2018hop
[57] Thurstone, L. L., A law of comparative judgment, Psychol. Rev., 34, 4, 273-286 (1927)
[58] Tibshirani, R.; Walther, G.; Hastie, T., Estimating the number of clusters in a data set via the gap statistic, J. R. Stat. Soc. Ser. B Stat. Methodol., 63, 2, 411-423 (2001) · Zbl 0979.62046
[59] Vecchio, T. J., Predictive value of a single diagnostic test in unselected populations, New Engl. J. Med., 274, 21, 1171-1173 (1966)
[60] Wang, J., Consistent selection of the number of clusters via crossvalidation, Biometrika, 97, 4, 893-904 (2010) · Zbl 1204.62104
[61] Weingart, S., Offense/defense rankings for 1043 teams (2006), https://www.chiefdelphi.com/forums/showpost.php?p=484220&postcount=19
[62] Weng, R. C.; Lin, C. J., A Bayesian approximation method for online ranking, J. Mach. Learn. Res., 12, 267-300 (2011) · Zbl 1280.68215
[63] FIRST robotics competition. Retrieved August 30, 2018 from (2018), https://en.wikipedia.org/wiki/FIRST_Robotics_Competition
[64] Woolner, K., Introduction to VORP: value over replacement player (2001), https://web.archive.org/web/20070928064958/http://www.stathead.com/bbeng/woolner/vorpdescnew.htm
[65] Woolner, K., VORP: measuring the value of a baseball player’s performance (2001), https://web.archive.org/web/20070928064958/http://www.stathead.com/bbeng/woolner/vorpdescnew.htm
[66] Yerushalmy, J., Statistical problems in assessing methods of medical diagnosis with special reference to x-ray techniques, Public Health Rep., 62, 2, 1432-1439 (1947)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.