[R-meta] Using rcalc() in metafor

[Lukasz Stasielowicz]

Dear Yuhang,

1. From what I've seen in the SEM literature, a non-positive definite 
matrix can lead to results that are not trustworthy. The algorithms can 
have problems finding a trustworthy numerical solution. Perhaps a 
visualization of these issues will be helpful (section Visualizing 
definiteness): 
https://gregorygundersen.com/blog/2022/02/27/positive-definite/

My experience with such issues in the meta-analytic context is very 
limited. A few months ago, I played around with a fancy 
variance-covariance matrix that was non-positive definite. The 
meta-analytic results (brms package) were nonsensical (e.g., the 
magnitude or sign of the mean effect, the amount of estimated 
variability), so it was easy to spot a problem. However, I've seen 
several SEM analyses where the results might seem plausible but are 
nevertheless biased.

In some SEM-based studies, the authors report the results based on the 
non-positive definite matrix but emphasize the limitations. One could 
also run a sensitivity analysis with a different V matrix to check 
whether the results change substantially. That could reduce the uncertainty.

However, avoiding a non-positive definite matrix (e.g., playing around 
with V imputation) is usually the preferred approach, at least in the 
SEM context.



2. I'm glad to hear that you found a solution. Wolfgang has kindly added 
so many functionalities over the years, so it was naive of me to think 
that "predict" yields the desired output for every model.
Fortunately, there are other functions such as transf.ztor or tanh.

vcov: Hafdahl reports some formulas in the appendix (p. 201).
I don't know whether the respective function is available in R or if one 
needs to write the code from scratch.
Hafdahl, A. R. (2007). Combining correlation matrices: Simulation 
analysis of improved fixed-effects methods. Journal of Educational and 
Behavioral Statistics, 32(2), 180-205.



Best,
Lukasz
-- 
Lukasz Stasielowicz
Osnabrück University
Institute for Psychology
Research methods, psychological assessment, and evaluation
Lise-Meitner-Straße 3
49076 Osnabrück (Germany)
Twitter: https://twitter.com/l_stasielowicz

On 05.08.2023 12:00, r-sig-meta-analysis-request using r-project.org wrote:
> Send R-sig-meta-analysis mailing list submissions to
> 	r-sig-meta-analysis using r-project.org
> 
> To subscribe or unsubscribe via the World Wide Web, visit
> 	https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> or, via email, send a message with subject or body 'help' to
> 	r-sig-meta-analysis-request using r-project.org
> 
> You can reach the person managing the list at
> 	r-sig-meta-analysis-owner using r-project.org
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of R-sig-meta-analysis digest..."
> 
> 
> Today's Topics:
> 
>     1. Re: Using rcalc() in metafor (Lukasz Stasielowicz)
>     2. Re: Using rcalc() in metafor (Yuhang Hu)
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Fri, 4 Aug 2023 13:23:52 +0200
> From: Lukasz Stasielowicz <lukasz.stasielowicz using uni-osnabrueck.de>
> To: r-sig-meta-analysis using r-project.org
> Cc: yh342 using nau.edu
> Subject: Re: [R-meta] Using rcalc() in metafor
> Message-ID: <3e3b0e91-c2ee-1d5e-5767-6299ec0656fd using uni-osnabrueck.de>
> Content-Type: text/plain; charset="utf-8"; Format="flowed"
> 
> Dear Yuhang,
> 
> 
> 1. Playing around with the imputation settings for the V matrix is one
> option, e.g.,
> impute_covariance_matrix from the clubSandwich package
> or
> vcalc from the metafor package.
> 
> There is certainly a trade-off. The risk of getting non-positive
> definite matrices can be reduced by simplifying the covariance
> structure, but it will probably lead to less realistic assumptions.
> However, one could conduct sensitivity analyses by using different
> imputation settings for the V matrix and checking whether the main
> findings change substantially.
> 
> 
> 2. The mean effect size and its confidence interval can be transformed
> using predict:
> predict(res, transf=transf.ztor)
> 
> 
> 
> Best,
> 
> Lukasz