[R-meta] Replicating Three-Level Model with Fixed Effects

[Andrew Loignon]

Hi Michael and Wolfgang,

Thanks for your prompt replies. That certainly helps clarify things.

Best,

Andrew

Andrew C. Loignon
Assistant Professor
2716 Business Education Complex
Rucks Department of Management
E. J. Ourso College of Business
Louisiana State University
Baton Rouge, LA 70808
Email: aloignon using lsu.edu
Phone: 225-578-6148

-----Original Message-----
From: Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer using maastrichtuniversity.nl> 
Sent: Tuesday, December 11, 2018 07:11
To: Michael Dewey <lists using dewey.myzen.co.uk>; Andrew Loignon <aloignon using lsu.edu>; r-sig-meta-analysis using r-project.org
Subject: RE: [R-meta] Replicating Three-Level Model with Fixed Effects

This may be one explanation. But I think the district-level value of 'year' was used to center. So:

> mean(aggregate(dat$year, list(dat$district), FUN=mean)$x)
[1] 1990.591

Then:

> res <- rma.mv(yi, vi, mods = ~ I(year-1990.591), random = ~ 1 | 
> district/study, data=dat, method="ML") print(res, digits=3)

Multivariate Meta-Analysis Model (k = 56; method: ML)

Variance Components:

           estim   sqrt  nlvls  fixed          factor 
sigma^2.1  0.056  0.238     11     no        district 
sigma^2.2  0.033  0.181     56     no  district/study 

Test for Residual Heterogeneity:
QE(df = 54) = 550.260, p-val < .001

Test of Moderators (coefficient 2):
QM(df = 1) = 0.355, p-val = 0.551

Model Results:

                    estimate     se   zval   pval   ci.lb  ci.ub 
intrcpt                0.183  0.080  2.298  0.022   0.027  0.340  * 
I(year - 1990.591)     0.005  0.009  0.596  0.551  -0.012  0.022    

---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

These are now the same results as in Table 5.

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-
>project.org] On Behalf Of Michael Dewey
>Sent: Tuesday, 11 December, 2018 10:13
>To: Andrew Loignon; r-sig-meta-analysis using r-project.org
>Subject: Re: [R-meta] Replicating Three-Level Model with Fixed Effects
>
>Dear Andrew
>
>The difference you report is about 2.5%. With possibly different 
>hardware, different fitting algorithm, ..., I would not be too 
>concerned myself. Other readers on this site might have a different view though.
>
>Michael
>
>On 10/12/2018 18:01, Andrew Loignon wrote:
>> Hi everyone,
>>
>> As part of a separate project, I've been familiarizing myself with
>multi-level meta-analyses. While working through the example provided 
>on the metafor package's website (http://www.metafor- 
>project.org/doku.php/analyses:konstantopoulos2011), I ran into some 
>difficulties replicating the three-level model described by 
>Konstantopolous (2011, "Fixed effects and variance components 
>estimation in
>> three-level meta-analysis").
>>
>> Specifically, I can replicate Konstantopolous' two-level model using
>year as a fixed effect (Table 4 in the research article) using the 
>following code:
>>
>> m0.ml <- rma.mv(yi, vi,
>>                  random = ~ 1| study,
>>                  mods = ~ I(year-mean(year)),
>>                  method="ML",
>>                  digits=3,
>>                  data=dat.konstantopoulos2011
>>                  )
>> summary(m0.ml)
>>
>> However, when I extend the model to include the higher-level nesting
>variable (i.e., district), the intercept diverges from what 
>Konstantopolous reports (Table 5 in the research article). Right now, 
>I'm using the following code:
>>
>> m1.ml <- rma.mv(yi, vi,
>>                  random = ~ 1| district/study,
>>                  mods = ~ I(year-mean(year)),
>>                  method="ML",
>>                  digits = 3,
>>                  data=dat.konstantopoulos2011)
>> summary(m1.ml)
>>
>> With this model, I find an intercept of .178, while Konstantopolous
>reports a value of .183.
>>
>> Any suggestions for how to reconcile these differences would be very
>much appreciated.
>>
>> Best,
>>
>> Andrew
>> Andrew C. Loignon
>> Assistant Professor
>> 2716 Business Education Complex
>> Rucks Department of Management
>> E. J. Ourso College of Business
>> Louisiana State University
>> Baton Rouge, LA 70808
>> Email: aloignon using lsu.edu<mailto:aloignon using lsu.edu>
>> Phone: 225-578-6148