Skip sequencing: A decision problem in questionnaire design. (English) Zbl 1137.62304

Manski, Charles F.; Molinari, Francesca

Ann. Appl. Stat. 2, No. 1, 264-285 (2008).

Summary: This paper studies questionnaire design as a formal decision problem, focusing on one element of the design process: skip sequencing. We propose that a survey planner uses an explicit loss function to quantify the trade-off between cost and informativeness of the survey and aims to make a design choice that minimizes loss. We pose a choice between three options: ask all respondents about an item of interest, use skip sequencing, thereby asking the item only of respondents who give a certain answer to an opening question, or do not ask the item at all.
The first option is most informative but also most costly. The use of skip sequencing reduces respondent burden and the cost of interviewing, but may spread data quality problems across survey items, thereby reducing informativeness. The last option has no cost but is completely uninformative about the item of interest. We show how the planner may choose among these three options in the presence of two inferential problems, item nonresponse and response error.

MSC:

62D05	Sampling theory, sample surveys
62C99	Statistical decision theory

Keywords:

item nonresponse; response error; partial identification

References:

[1]	Beresteanu, A. and Molinari, F. (2008). Asymptotic properties for a class of partially identified models., Econometrica . · Zbl 1274.62136 · doi:10.1111/j.1468-0262.2008.00859.x
[2]	Blundell, R., Gosling, A., Ichimura, H. and Meghir, C. (2007). Changes in the distribution of male and female wages accounting for employment composition using bounds., Econometrica 75 323-363. · Zbl 1132.91515 · doi:10.1111/j.1468-0262.2006.00750.x
[3]	Bound, J., Brown, C. and Mathiowetz, N. A. (2001). Measurement error in survey data. In, Handbook of Econometrics 5 Chapter 59 (J. Heckman and E. Leamer, eds.) 3705-3843. North-Holland, Amsterdam.
[4]	Chernozhukov, V., Hong, H. and Tamer, E. (2007). Estimation and confidence regions for parameter sets in econometric models., Econometrica 75 1243-1284. · Zbl 1133.91032 · doi:10.1111/j.1468-0262.2007.00794.x
[5]	Connor, J. T., Fienberg, S. E., Erashova, A. E. and White, T. (2006). Towards a restructuring of the national long term care survey: A longitudinal perspective. Prepared for presentation at an Expert Panel Meeting on the National Long Term Care Survey, Committee on National Statistics, National Research, Council.
[6]	Groves, R. M. (1987). Research on survey data quality., The Public Opinion Quarterly 51 S156-S172.
[7]	Groves, R. M. (1989)., Survey Errors and Survey Costs . Wiley, New York.
[8]	Groves, R. M. and Heeringa, S. G. (2006). Responsive design for household surveys: Tools for actively controlling survey errors and costs., J. Roy. Statist. Soc. Ser. A 169 437-457. · doi:10.1111/j.1467-985X.2006.00423.x
[9]	Hill, D. H. (1991). Interviewer, respondent, and regional office effects on response variance: A statistical decomposition. In, Measurement Errors in Surveys (P. Biemer, R. Groves, L. Lyberg, N. Mathiowetz and S. Sudman, eds.) 463-483. Wiley, New York.
[10]	Hill, D. H. (1993). Response and sequencing errors in surveys: A discrete contagious regression analysis., J. Amer. Statist. Assoc. 88 775-781.
[11]	Huber, P. (1964). Robust estimation of a location parameter., Ann. Math. Statist. 35 73-101. · Zbl 0136.39805 · doi:10.1214/aoms/1177703732
[12]	Horowitz, J. L. and Manski, C. F. (1995). Identification and robustness with contaminated and corrupted data., Econometrica 63 281-302. JSTOR: · Zbl 0820.62030 · doi:10.2307/2951627
[13]	Imbens, G. and Manski, C. F. (2004). Confidence intervals for partially identified parameters., Econometrica 72 1845-1857. JSTOR: · Zbl 1091.62015 · doi:10.1111/j.1468-0262.2004.00555.x
[14]	Krosnick, J. (1999). Survey research., Ann. Rev. Psychology 50 537-567.
[15]	Little, R. J. and Rubin, D. B. (1987)., Statistical Analysis with Missing Data . Wiley, New York. · Zbl 0665.62004
[16]	Manski, C. F. (1989). Anatomy of the selection problem., J. Human Resources 24 343-360.
[17]	Manski, C. F. (1994). The selection problem. In, Advances in Econometrics , Sixth World Congress I (C. Sims, ed.) 143-170. Cambridge Univ. Press. · Zbl 1066.91084 · doi:10.1111/0034-6527.00291
[18]	Manski, C. F. (2003)., Partial Identification of Probability Distributions . Springer, New York. · Zbl 1047.62001 · doi:10.1007/b97478
[19]	Mathiowetz, N. A. and Groves, R. M. (1985). The effects of respondent rules on health survey reports., American J. Public Health 75 639-644.
[20]	Mathiowetz, N. A. and Lair, T. J. (1994). Getting better? Change or error in the measurement of functional limitations., J. Economic and Social Measurement 20 237-262.
[21]	Mathiowetz, N. A. and Wunderlich, G. S., eds. (2000)., Survey Measurement of Work Disability . National Academy Press, Washington, DC.
[22]	Messmer, D. and Seymour, D. (1982). The effect of branching on item nonresponse., Public Opinion Quarterly 46 270-277.
[23]	Miller, K. and DeMaio, T. J. (2006). Report of cognitive research on proposed American community survey disability questions. U.S. Census Bureau, Statistical Research Division Report, #SSM2006/06.
[24]	Molinari, F. (2003). Contaminated, corrupted, and missing data. Ph.D. thesis, Northwestern Univ. Available at, http://www.arts.cornell.edu/econ/fmolinari/dissertation.pdf.
[25]	Molinari, F. (2008). Partial identification of probability distributions with misclassified data., J. Econometrics . · Zbl 1418.62522 · doi:10.1016/j.jeconom.2007.12.003
[26]	Moore, J. C. (1988). Self/proxy response status and survey response quality., J. Official Statistics 4 155-172.
[27]	Rodgers, W. L. and Miller, B. (1997). A comparative analysis of ADL questions in surveys of older people., J. Gerontology Ser. B 52B (Special Issue) 21-36.
[28]	Rubenstein, L. Z., Schairer, C., Wieland, G. D. and Kane, R. (1984). Systematic biases in functional status assessment of elderly adults: Effects of different data sources., J. Gerontology 39 686-691.
[29]	Spencer, B. D. (1980)., Benefit-Cost Analysis of Data Used to Allocate Funds . Springer, New York. · Zbl 0925.62534
[30]	Spencer, B. D. (1985). Optimal data quality., J. Amer. Statist. Assoc. 80 564-573. JSTOR: · Zbl 0582.62007 · doi:10.2307/2288471
[31]	Spencer, B. D. (1994). Sensitivity of benefit-cost analysis of data programs to monotone misspecification., J. Statist. Plann. Inference 39 19-31. · Zbl 0803.90083 · doi:10.1016/0378-3758(94)90109-0
[32]	Stoye, J. (2005). Partial identification of spread parameters when some data are missing. Dept. Economics, New York, Univ.
[33]	Wald, A. (1950)., Statistical Decision Functions . Wiley, New York. · Zbl 0040.36402