[R-sig-ME] constructed level 2 predictors/random effects

[Steven J. Pierce]

Paul,

I don’t know any R packages that are specific to this issue, but there are interesting theoretical and methods papers on measuring constructs at a higher level of analysis with via aggregation in the organizational psychology literature. Chan (1998) wrote some very interesting bits on conceptualizing constructs at multiple levels of analysis. Kozlowski & Klein (2000) discuss that further. The other papers listed below get more into specifics of measurement & validating constructs. One issue that Croon (2007) brought up was treating the aggregation as a latent variable via SEM techniques; subsequently Lüdtke et al (2008) refined that idea, as did Marsh et al (2012). 

Chan, D. (1998). Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models. Journal of Applied Psychology, 83(2), 234-246. doi: 10.1037/0021-9010.83.2.234

Kozlowski, S. W. J., & Klein, K. J. (2000). A multilevel approach to theory and research in organizations: Contextual, temporal, and emergent processes. In K. J. Klein & S. W. J. Kozlowski (Eds.), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 3-90). San Francisco, CA: Jossey-Bass.

Cole, M. S., Bedeian, A. G., Hirschfeld, R. R., & Vogel, B. (2011). Dispersion-composition models in multilevel research: A data-analytic framework. Organizational Research Methods, 14(4), 718-734. doi: 10.1177/1094428110389078

Conway III, L. G., & Schaller, M. (1998). Methods for the measurement of consensual beliefs within groups. Group Dynamics: Theory, Research, and Practice, 2(4), 241-252. doi: 10.1037/1089-2699.2.4.241

Croon, M. A. (2007). Predicting group-level outcome variables from variables measured at the individual level: A latent variable multilevel model. Psychological Methods, 12(1), 45-57. doi: 10.1037/1082-989X.12.1.45

Liska, A. E. (1990). The significance of aggregate dependent variables and contextual independent variables for linking macro and micro theories. Social Psychology Quarterly, 53(4), 292-301.

Lüdtke, O., Marsh, H. W., Robitzsch, A., Trautwein, U., Asparouhov, T., & Muthén, B. (2008). The multilevel latent covariate model: A new, more reliable approach to group-level effects in contextual studies. Psychological Methods, 13(3), 203-229. doi: 10.1037/a0012869

Marsh, H. W., Lüdtke, O., Nagengast, B., Trautwein, U., Morin, A. J. S., Abduljabbar, A. S., & Köller, O. (2012). Classroom climate and contextual effects: Conceptual and methodological issues in the evaluation of group-level effects. Educational Psychologist, 47(2), 106-124. doi: 10.1080/00461520.2012.670488

Meade, A. W., & Eby, L. T. (2007). Using indices of group agreement in multilevel construct validation. Organizational Research Methods, 10(1), 75-96. doi: 10.1177/1094428106289390

Roberson, Q. M., Sturman, M. C., & Simons, T. L. (2007). Does the measure of dispersion matter in multilevel research? A comparison of the relative performance of dispersion indexes. Organizational Research Methods, 10(4), 564-588. doi: 10.1177/1094428106294746

van Mierlo, H., Vermunt, J. K., & Rutte, C. G. (2009). Composing group-level constructs from individual-level survey data. Organizational Research Methods, 12(2), 368-392. doi: 10.1177/1094428107309322



Steven J. Pierce, Ph.D.
Associate Director
Center for Statistical Training & Consulting (CSTAT)
Michigan State University

-----Original Message-----
From: Paul Johnson [mailto:pauljohn32 at gmail.com] 
Sent: Monday, June 29, 2015 3:35 PM
To: R-SIG-Mixed-Models at r-project.org
Subject: [R-sig-ME] constructed level 2 predictors/random effects

I see people wanting to average survey responses to manufacture
contextual variables. They just take the average of individual level
scores and treat it as if it were context.  Or in studies of
education, they average class Socio Economic or religious variables
and use the means for context.   Have you ever seen R packages that
try to facilitate this kind of work?

I am thinking about problems like this.

1. Can we account for the standard error of the mean at the group
level?  Will the come back to "not with lme4, but the old lme had
varIdent for weights?"

Of course, if there is 1 or 2 people within a cluster, and 50 in
another, we'd have an especially big reason to try to fix this.

2. It seems to me they should calculate a "leave one out" estimate for
each row, excluding that case's impact on the group-level average.

I'm thinking about the education studies that want to both create mean
SES as a predictor, and then look at individual variations against
that predictor.  If there are 100 people within each classroom, I
don't guess it matters.  But sometimes they have 2 or 5 people within
each group.

3. They are using raw averages, not pooled estimates for these
constructed level 2 variables.  It looks to me like we ought to fit a
multi level model on those variables, using the group as a random
effect. Then take the BLUPs as estimates of the context.  Otherwise,
these means at the group level are just as inefficent as the
one-regression-per group approach.

Even if you don't know of R work on this, I'd appreciate any pointers
to literature or such.

-- 
Paul E. Johnson
Professor, Political Science       Director
1541 Lilac Lane, Room 504      Center for Research Methods
University of Kansas                 University of Kansas
http://pj.freefaculty.org               http://crmda.ku.edu