[R-meta] Questions About Multilevel Models

[Lukasz Stasielowicz]

Dear Tori,

In case you have not seen it already, Wolfgang's site has a vignette 
showing how to apply the Hunter and Schmidt method using the metafor 
package: https://www.metafor-project.org/doku.php/tips:hunter_schmidt_method
As Wolfgang has explained, it is straightforward to generalize it to 
models with multiple levels.


Obligatory word of caution: Accounting for measurement error within the 
Hunter and Schmidt framework relies on some bold assumptions.
A couple of references:
Brakenhoff, T. B., Van Smeden, M., Visseren, F. L. J., & Groenwold, R. 
H. H. (2018). Random measurement error: Why worry? An example of 
cardiovascular risk factors. PLoS ONE, 13(2), 1–8. 
https://doi.org/10.1371/journal.pone.0192298
Flegal, K. M., Keyl, P. M., & Nieto, F. J. (1991). Differential 
misclassification arising from nondifferential errors in exposure 
measurement. American Journal of Epidemiology, 134(10), 1233–1246. 
https://doi.org/10.1093/oxfordjournals.aje.a116026
Loken, E., & Gelman, A. (2017). Measurement error and the replication 
crisis. Science, 355(6325), 584–585. https://doi.org/10.1126/science.aal3618
van Smeden, M., Lash, T. L., & Groenwold, R. H. H. (2020). Reflection on 
modern methods: Five myths about measurement error in epidemiological 
research. International Journal of Epidemiology, 49(1), 338–347. 
https://doi.org/10.1093/ije/dyz251


Hox and colleagues (2017) chose a different approach and included 
reliability as a moderator variable. This means that the corrected mean 
effect size can be smaller than the uncorrected effect, which is 
impossible within the Hunter and Schmidt framework when accounting for 
measurement error. However, such regression-based adjustments can 
sometimes lead to implausible estimates, e.g., corrected correlations 
larger than 1.
Hox, J. J., Moerbeek, M., & Van de Schoot, R. (2017). Multilevel 
analysis: Techniques and applications. Routledge.


Best,
Lukasz
-- 
Lukasz Stasielowicz
Osnabrück University
Institute for Psychology
Research methods, psychological assessment, and evaluation
Lise-Meitner-Straße 3
49076 Osnabrück (Germany)
Twitter: https://twitter.com/l_stasielowicz

On 15.11.2022 12:00, r-sig-meta-analysis-request using r-project.org wrote:
> Send R-sig-meta-analysis mailing list submissions to
> 	r-sig-meta-analysis using r-project.org
> 
> To subscribe or unsubscribe via the World Wide Web, visit
> 	https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> or, via email, send a message with subject or body 'help' to
> 	r-sig-meta-analysis-request using r-project.org
> 
> You can reach the person managing the list at
> 	r-sig-meta-analysis-owner using r-project.org
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of R-sig-meta-analysis digest..."
> 
> 
> Today's Topics:
> 
>     1. Re: Questions About Multilevel Models (Viechtbauer, Wolfgang (NP))
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Mon, 14 Nov 2022 11:12:22 +0000
> From: "Viechtbauer, Wolfgang (NP)"
> 	<wolfgang.viechtbauer using maastrichtuniversity.nl>
> To: "r-sig-meta-analysis using r-project.org"
> 	<r-sig-meta-analysis using r-project.org>
> Subject: Re: [R-meta] Questions About Multilevel Models
> Message-ID: <6537e0b3d22b456a9ed735ae51eb94de using UM-MAIL3213.unimaas.nl>
> Content-Type: text/plain; charset="utf-8"
> 
> Dear Tori,
> 
> 1) Corrections like this are done typically on the effect size estimates (and corresponding sampling variances) before fitting the model (unless one uses methods like those described by Hunter and Schmidt using 'artifact distributions' and correcting estimates coming from the model, but this is really something very specific to the 'Hunter and Schmidt method'). So, you can apply any corrections you deem necessary and then use those in whatever model you like.
> 
> 2) Yes, predict() will provide those.
> 
> Best,
> Wolfgang
> 
>> -----Original Message-----
>> From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>> Behalf Of Tori Peña
>> Sent: Friday, 11 November, 2022 17:23
>> To: r-sig-meta-analysis using r-project.org
>> Subject: [R-meta] Questions About Multilevel Models
>>
>> Hi everyone -
>>
>> Thank you for all your guidance in advance!
>>
>> I have a couple of questions -- (1) is there a way to correct for range
>> restriction and/or measurement error in multilevel analyses and (2) is
>> there a way to calculate prediction intervals using the metafor package for
>> multilevel models?  Thanks again for your help!
>>
>> Best,
>> Tori
>>
>> --
>> *Tori Peña <https://www.linkedin.com/in/toripena/> *(she/her/ella)
>> Doctoral Candidate, Cognitive Psychology
>> Dept. of Psychology
>> Stony Brook University
>> Stony Brook, NY 11790-2500
>> [image: Stony Brook University logo]
> 
> 
> ------------------------------
> 
> Subject: Digest Footer
> 
> _______________________________________________
> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> To manage your subscription to this mailing list, go to:
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> 
> 
> ------------------------------
> 
> End of R-sig-meta-analysis Digest, Vol 66, Issue 6
> **************************************************