[R-meta] query re: multi-level analysis in metafor with missing standard errors - p.s.

[Jennifer Oser]

Hi all, I write with a p.s. to my post from last week (Sep 29) copied below
that my co-authors and I are still trying to resolve:

In our dataset, 83 of the total 185 coded effects are in this problematic
category for multi-level analysis in metafor, i.e., a coefficient reported
in a study's regression table as a standardized coefficient with no
reported standard error.

My guess is that this must be a common issue for applied researchers like
myself and my co-authors who are interested in using this approach, as our
dataset includes a nice distribution of studies according to standard
indicators of article quality (we coded ISI impact factor quartile).

As noted in my original post, we are very keen to try to resolve this asap
as the rest of the manuscript is in final draft form and we've received
very favorable comments on the substantive academic contribution.

Any and all tips and hints are welcome!
Jenny


Dr. Jennifer Oser
Department of Politics & Government
Ben-Gurion University of the Negev
https://www.jenniferoser.com/

****************************************************************
Jennifer Oser o using er @end|ng |rom po using t@bgu@@c@||
Tue Sep 29 18:13:17 CEST 2020

Hi all,

My co-authors and I are conducting a multi-level meta-analysis, and I write
with a question about calculating the sampling variance to use in the rma.mv
argument. We followed the approach of calculating the sampling variance (v)
by squaring the standard error, as documented in "Doing Meta-Analysis in R"
here:
https://bookdown.org/MathiasHarrer/Doing_Meta_Analysis_in_R/fitting-a-three-level-model.html
.
However, most of the regressions in the studies that meet our inclusion
criteria report on standardized coefficients without reporting standard
errors, and therefore cannot be included in the multi-level analysis using
this approach.

Our question is therefore: Is there a way to conduct a multi-level analysis
that calculates the relevant sampling variance based on regression outputs
that report standardized coefficients but do not report standard errors? We
know that it is possible to do conversions for an effect size in
non-multi-level studies, but we have not yet seen documentation on how to
do this sort of conversion for multi-level meta-analysis based on
regression outputs.

If the answer is a definitive “No, this is not possible (at least at this
time)” – this would also be a useful answer, as we have conducted
vote-counting tests in our paper, and can report on those results along
with the multi-level findings from the smaller number of effects that do
include the necessary information.

If the answer is “Yes, this is possible but it’s complicated…” - any advice
and/or references for addressing this issue would be greatly appreciated,
and we are happy to provide more information on the analysis as useful. We
are in the final stages of revising a paper that has received very positive
feedback in various conferences (on the topic of political efficacy and
online political participation), and would be grateful for leads that would
help us conduct as rigorous and comprehensive an analysis as possible.

Best wishes,
Jenny

Dr. Jennifer Oser
Department of Politics & Government
Ben-Gurion University of the Negev
https://www.jenniferoser.com/

	[[alternative HTML version deleted]]