×

Data aggregation in evaluating psychological constructs: Multivariate and logical deductive considerations. (English) Zbl 0705.62104

Summary: The value of a number of constructs in psychology has been impugned because of weak empirical associations between measures of the constructs and measures of relevant behaviours. Weakness of association has taken the forms of inconsistent statistical significance of construct-behaviour associations across studies and unacceptably low magnitudes of association within studies.
To redeem empirical support for the value of such constructs, aggregation of the behavioural measures by computation of simple sums prior to calculation of associations with the construct has been advocated. The validity of this procedure for the stated purposes has been the subject of considerable verbal debate, but sparse analytical treatment. The procedure is examined here in the light of analyses of simultaneous multiple observations using techniques that have formal mathematical foundations. Substantial differences between the two sets of procedures bearing directly on empirical support for the value of constructs are demonstrated. In turn, the validity of data aggregation for supporting the value of constructs is rejected when the espoused role of such support is subjected to tenets of logical deductive scientific systems.

MSC:

62P15 Applications of statistics to psychology
91E99 Mathematical psychology
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] (Blalock, H. M., Causal models in the social sciences (1985), Aldine: Aldine New York)
[2] Bock, R. D., Multivariate statistical methods in behavioral research (1975), McGraw-Hill: McGraw-Hill New York · Zbl 0398.62086
[3] (Borgatta, E. F.; Jackson, P. J., Aggregate data: Analysis and interpretation (1980), Sage: Sage Beverly Hills, CA)
[4] Braithwaite, R. B., Scientific explanation (1968), Cambridge Univ. Press: Cambridge Univ. Press London · Zbl 0052.00401
[5] Cattell, R. B., Evaluating therapy as a total personality change: Theory and available measurements, American Journal of Psychotherapy, 20, 69-88 (1966)
[6] Cattell, R. B., The voyage of a laboratory, 1928-1984, Multivariate Behavioural Research, 19, 121-174 (1984)
[7] Cattell, R. B.; Bjerstedt, A., The structure of depression by factoring \(Q\)-data in relation to general personality source traits in normal and pathological subjects, Scandinavian Journal of Psychology, 8, 17-24 (1964)
[8] Cattell, R. B.; Bolton, L. S., What pathological dimensions lie beyond the normal dimensions of the 16PF, Journal of Consulting and Clinical Psychology, 33, 18-29 (1969)
[9] Cohen, J., Statistical power analysis for the behavioral sciences (1969), Academic Press: Academic Press New York
[10] Cohen, J., Set correlation as a general multivariate data-analytic method, Multivariate Behavioral Research, 17, 301-341 (1982)
[11] Cramer, E. M.; Nicewander, W. A., Some symmetric, invariant measures of multivariate association, Psychometrika, 44, 43-54 (1979) · Zbl 0419.62048
[12] Cronbach, L. J.; Meehl, P. E., Construct validity in psychological tests, Psychological Bulletin, 52, 281-302 (1955)
[13] Day, H. D.; Marshall, D.; Hamilton, B.; Christy, J., Some cautionary notes regarding the use of aggregated scores as a measure of behavioural stability, Journal of Research in Personality, 17, 97-109 (1983)
[14] Embretson (Whitely), S., Construct validity: Construct representation versus nomothetic span, Psychological Bulletin, 93, 179-197 (1983)
[15] Epstein, S., Traits are alive and well, (Magnusson, D.; Endler, N. S., Personality at the crossroads: Current issues in interactional psychology (1977), Erlbaum: Erlbaum Hillsdale, NJ), 83-98
[16] Epstein, S., Does aggregation produce spuriously high estimates of behavior stability?, Psychological Bulletin, 53, 134-140 (1986)
[17] Epstein, S.; Obrien, E. J., The person-situation debate in historical and current perspective, Psychological Bulletin, 98, 513-517 (1985)
[18] Estes, W. K., The problem of inference from curves based on group data, Psychological Bulletin, 53, 134-140 (1956)
[19] Gardner, R. C.; Erdle, S., Aggregating scores: To standardise or not to standardise?, Educational and Psychological Measurement, 44, 813-821 (1984)
[20] Gardner, R. C.; Erdle, S., Aggregating standard scores or raw scores: Whither the controversy?, Educational and Psychological Measurement, 46, 533-536 (1986)
[21] Giere, R. N., Testing theoretical hypothesis, (Earman, J., Testing scientific theories. Minnesota studies in the philosophy of science, Vol. X (1983), Univ. of Minnesota Press: Univ. of Minnesota Press Minneapolis), 269-298
[22] Glass, G. V.; McGraw, B.; Smith, M. L., Meta-analysis in social research (1981), Sage: Sage Beverly Hills, CA
[23] Gleason, T. C., On redundancy in canonical analysis, Psychological Bulletin, 83, 1004-1006 (1976)
[24] Hannon, M. T., Problems of aggregation, (Blalock, H. M., Causal models in the social sciences (1985), Aldine: Aldine New York), 403-440
[25] Harris, R. J., A primer of multivariate statistics (1985), Academic Press: Academic Press New York
[26] Hedges, L. V.; Olkin, I., Statistical methods for meta analysis (1985), Academic Press: Academic Press New York · Zbl 0666.62002
[27] Israels, A. Z., Redundancy analysis of qualitative variables, Psychometrika, 49, 331-346 (1984)
[28] Johansson, J. K., An extension of Wollenberg’s redundancy analysis, Psychometrika, 46, 93-103 (1981)
[29] Keeping, E. S., Introduction to statistical significance (1962), Van Nostrand: Van Nostrand New York · Zbl 0045.41204
[30] Kettenring, J. R., Canonical analysis of several sets of variables, Biometrika, 58, 433-451 (1971) · Zbl 0225.62072
[31] Lambert, Z. V.; Wildt, A. R.; Durand, R. M., Redundancy analysis: An alternative to canonical correlation and multivariate multiple regression in exploring interset associations, Psychological Bulletin, 104, 282-289 (1988)
[32] McDonald, R. P.; Mulaik, S. A., Determinancy of common factors: A nontechnical review, Psychological Bulletin, 86, 297-306 (1979)
[33] Messick, S., Constructs and their vicissitudes in educational and psychological measurement, Psychological Bulletin, 89, 575-588 (1981)
[34] Mischel, W.; Peake, P. K., Beyond deja vu in the search for cross-situational consistency, Psychological Review, 89, 730-755 (1982)
[35] Moorman, J. E., Aggregation bias: An empirical demonstration, (Borgatta, E. F.; Jackson, D. J., Aggregate data: Analysis and interpretation (1980), Sage: Sage Beverly Hills, CA), 131-156
[36] Morrison, D. F., Multivariate statistical methods (1976), McGraw-Hill: McGraw-Hill San Francisco · Zbl 0183.20605
[37] Neufeld, R. W.J.; Broga, M. I., Relations between diagnostic-demographic variables and thought-disorder symptomatology among process schizophrenic patients, Multivariate Experimental Clinic Research, 3, 133-145 (1977)
[38] Nicewander, W. A.; Price, J. M., Reliability of measurement and the power of statistical tests: Some new results, Psychological Bulletin, 94, 524-533 (1983)
[39] Nunnally, J. C., Psychometric theory (1978), McGraw-Hill: McGraw-Hill New York
[40] Olson, C. L., A Monte Carlo investigation of the robustness of multivariate analysis of variance, (Unpublished Ph.D. dissertation (1973), University of Toronto)
[41] Olson, C. L., Comparative robustness of six tests in multivariate analysis of variance, Journal of the American Statistical Association, 69, 894-908 (1974) · Zbl 0302.62026
[42] Olson, C. L., On choosing a test statistic in multivariate analysis of variance, Psychological Bulletin, 83, 579-586 (1976)
[43] Ossenkopp, K. P.; Mazmanian, D. S., The measurement and integration of behavioural variables: Aggregation and complexity as important isues, Neurobehavioural Toxicology and Teratology, 7, 95-100 (1985)
[44] Perloff, J. M.; Persons, J. B., Biases resulting from the use of indexes: An application to attributional style and depression, Psychological Bulletin, 103, 95-104 (1988)
[45] Robinson, W. S., Ecological correlations and the behaviour of inviduals, American Sociological Review, 15, 351-357 (1950)
[46] Rozeboom, W. W., Linear correlations between sets of variables, Psychometrika, 30, 57-71 (1965) · Zbl 0127.10302
[47] Rozeboom, W. W., Foundations of the theory of prediction (1966), Dorsey Press: Dorsey Press Homewood, IL · Zbl 0154.45603
[48] Rozeboom, W. W., Factor indeterminacy: The saga continues, British Journal of Mathematical and Statistical Psychology, 41, 209-226 (1988) · Zbl 0718.62131
[49] Rushton, J. P.; Brainerd, C. J.; Pressley, M., Behavioural development and construct validity: The principle of aggregation, Psychological Bulleton, 94, 18-36 (1983)
[50] Smith, I. L., The eta coefficient in MANOVA, Multivariate Behavioural Research, 7, 361-372 (1972)
[51] Stevens, J. P., Power of multivariate analysis of variance tests, Psychological Bulletin, 88, 728-737 (1980)
[52] Stewart, D.; Love, W. A., A general canonical correlation index, Psychological Bulletin, 70, 160-163 (1968)
[53] Wainer, H., Estimating coefficients in linear models: It don’t make no never mind, Psychological Bulletin, 83, 213-217 (1976)
[54] Wainer, H., On the sensitivity of regression and regressors, Psychological Bulletin, 85, 267-273 (1978)
[55] Yule, G. U.; Kendall, M. G., An introduction to the theory of statistics (1950), Charles Griffin: Charles Griffin London · Zbl 0039.13802
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.