[R-meta] terminologies of multilevel and multivariate model when accounting for correlated errors

[Yefeng Yang]

Hi all (especially Wofgang & James),

My questions: I am confused about whether should we call a multilevel model with a VCV matrix accounting for sampling variances still a multilevel model OR should we call it a multivariate model

I elaborate on my questions as follows:

For statistically dependent effect sizes, we usually have two[1] 'typical' models to deal with.


  1.  For dependence due to multilevel/nested structure (one study contributes more than one effect size estimate), we usually use a multilevel model (with a nested random effect structure) to account for the non-independence if there are 'overlapping individuals' (no correlated sampling errors).
  2.
  3.  For dependence due to multivariate structure (one study contributes more than one response variable or outcome), we usually use a multivariate model (with a correlated random effect structure) to account for the non-independence. Also, we should use a variance-covariance matrix to account for the independent sampling errors (either guessing within-study correlation or using formulas).

[1] robust variance estimation (RVE) is also a good approach to dealing with dependent effect sizes in terms of estimating fixed effects (overall effect intercept beta0 or moderator effect slope beta1).  The combination of the RVE with either multilevel or multivariate is also an elegant approach. But RVE is not the focus of my question.

However, sometimes we want to use the multilevel model to deal with all types of independence. By doing so, we reformulate the multivariate structure of the data as multilevel/nested data. I mean we: (1) use dummy codes to denote different types of response variables/outcomes, (2) impute or calculate a VCV matrix, and (3) fit a multilevel model.  Through (1) - (3) steps, I account for all types of independence: the correlations between true outcomes and sampling errors. Not 100% sure, but this approach should work well.

So my question comes: I use a multilevel model but I also use a VCV matrix. What will a multilevel model with a VCV be called? Still a multilevel model, but a multilevel model assumes independent sampling errors (but we have a VCV in the model). Should it be a multivariate model, but we did not account for the correlated random effects only account for the correlated sampling errors? Hope my question is clear.

Best,

Yefeng Yang PhD
Research Associate
UNSW, Sydney




	[[alternative HTML version deleted]]