[R-meta] Calculating effect size for subsets of data
Viechtbauer, Wolfgang (SP)
wo||g@ng@v|echtb@uer @end|ng |rom m@@@tr|chtun|ver@|ty@n|
Thu Dec 3 09:10:01 CET 2020
Dear Tarun,
(please post in plain text; when converted to plain text, the table becomes unreadable)
Indeed, coefficients with overlapping CIs may still be significantly different from each other. You can use anova() with the 'L' argument to test pairs of coefficients against each other. See:
http://www.metafor-project.org/doku.php/tips:testing_factors_lincoms#testing_linear_combinations
and
http://www.metafor-project.org/doku.php/tips:multiple_factors_interactions
for illustrations.
Best,
Wolfgang
>-----Original Message-----
>From: Tarun Khanna [mailto:khanna using hertie-school.org]
>Sent: Thursday, 03 December, 2020 8:02
>To: Viechtbauer, Wolfgang (SP); r-sig-meta-analysis using r-project.org
>Subject: Re: Calculating effect size for subsets of data
>
>Dear Wolfgang,
>
>I am following up on a question that we discussed a few weeks ago regarding
>meta-analysis for different combinations of data. This is regarding
>interpreting the results.
>
>label
> beta
> se
> pvalues
> upper_lim
> lower_lim
>Social Comparison
> 0.102
> 0.057
> 0.077
> 0.214
> (0.011)
>Feedback
> 0.076
> 0.033
> 0.020
> 0.140
> 0.012
>Feedback+Social
> 0.104
> 0.043
> 0.016
> 0.189
> 0.020
>Monetary Incentives
> 0.261
> 0.042
> 0.000
> 0.344
> 0.178
>Social+Monetary
> 0.034
> 0.081
> 0.674
> 0.193
> (0.125)
>Feedback+Monetary
> 0.176
> 0.060
> 0.003
> 0.293
> 0.060
>Social+Monetary+Feedback
> 0.338
> 0.139
> 0.015
> 0.611
> 0.065
>Motivation
> 0.131
> 0.052
> 0.012
> 0.233
> 0.029
>Feedback+Motivation
> 0.152
> 0.047
> 0.001
> 0.243
> 0.061
>Social+Feedback+Motivation
> 0.212
> 0.087
> 0.015
> 0.383
> 0.041
>
>I ran the model as you suggested. The model reveals differences in the
>average effect size the different combinations but the condifence levels of
>these estimates overlap. In my opinion that does not mean that the
>differences are not statistically significant as we don't necessarily test
>for significance of differences. Or do these results mean we can't say
>anything about the differences? In a regression model I would run a F test
>with Ho : b1-b2 = 0. Can we do the same here?
>
>Best
>Tarun
>
>Tarun Khanna
>Research Associate
>
>Hertie School
>
>Friedrichstraße 180
>10117 Berlin ∙ Germany
>khanna using hertie-school.org ∙ www.hertie-school.org
>________________________________________
>From: Viechtbauer, Wolfgang (SP)
><wolfgang.viechtbauer using maastrichtuniversity.nl>
>Sent: 02 September 2020 11:40:27
>To: Tarun Khanna; r-sig-meta-analysis using r-project.org
>Subject: RE: Calculating effect size for subsets of data
>
>Dear Tarun,
>
>If I understand you correctly, then there should be 16 different
>combinations of A, B, C, and D but one of them (A=B=C=D=0) cannot occur, so
>essentially there are 15 combinations that were observed. As a result, you
>should have gotten a warning when fitting the model that a redundant
>predictor was dropped from the model. Let's consider a simpler case with
>just A and B:
>
>set.seed(1234)
>k <- 900
>A <- c(rep(0,k/3), rep(1,k/3), rep(1,k/3))
>B <- c(rep(1,k/3), rep(0,k/3), rep(1,k/3))
>vi <- rep(.01, k)
>yi <- rnorm(k, 0.5 * A + 0.1 * B + 0.3*A*B, sqrt(vi))
>
>A <- factor(A)
>B <- factor(B)
>
>res <- rma(yi, vi, mods = ~ A*B)
>res
>
>These are the model results:
>
> estimate se zval pval ci.lb ci.ub
>intrcpt -0.3019 0.0100 -30.1904 <.0001 -0.3215 -0.2823 ***
>A1 0.7963 0.0082 97.5319 <.0001 0.7803 0.8123 ***
>B1 0.4032 0.0082 49.3825 <.0001 0.3872 0.4192 ***
>
>The results are a bit tricky to interpret, so I would suggest a different
>parameterization:
>
>res <- rma(yi, vi, mods = ~ A:B + 0)
>res
>
> estimate se zval pval ci.lb ci.ub
>A1:B0 0.4944 0.0058 85.6397 <.0001 0.4831 0.5058 ***
>A0:B1 0.1013 0.0058 17.5462 <.0001 0.0900 0.1126 ***
>A1:B1 0.8976 0.0058 155.4771 <.0001 0.8863 0.9090 ***
>
>Now we can clearly see that A1:B0 is the estimated effect when A is given
>alone, A0:B1 is the estimated effect when B is given alone, and A1:B1 is the
>estimated effect when A and B are given together.
>
>Best,
>Wolfgang
>
>>-----Original Message-----
>>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-
>project.org]
>>On Behalf Of Tarun Khanna
>>Sent: Monday, 31 August, 2020 13:11
>>To: r-sig-meta-analysis using r-project.org
>>Subject: [R-meta] Calculating effect size for subsets of data
>>
>>Dear all,
>>
>>I am conducting a meta-analysis of effect of certain interventions on
>>household energy consumption. In my data set I have a dummy variable for
>>each of the sub-interventions: A,B,C,D such that intersection of A=0 & B=0
>&
>>C=0 & D=0 is zero. Each effect size may be associated with multiple
>>interventions though.
>>
>>I have calculated an aggregate effect size across interventions and then
>>effect size by sub-intervention. But I also want to compare if the effect
>of
>>the sub-interventions differs from each other. I thought about including
>the
>>sub-regression dummies as controls in the meta regression:
>>
>>rma (yi, vi, method = "REML", data = data, mods ~ A*B*C*D)
>>
>>The problem in interpreting the output of this regression is that there is
>>no base category left for the intercept to denote. Can I perhaps run the
>>model by supressing the intercept? Or what would be the interpretation of
>>the intercept in this case?
>>
>>Thanks in advance!
>>
>>Best
>>
>>Tarun
>>Tarun Khanna
>>PhD Researcher
>>Hertie School
>>
>>Friedrichstraße 180
>>10117 Berlin ∙ Germany
>>khanna using hertie-school.org ∙ www.hertie-school.org<http://www.hertie-
>>school.org/>
More information about the R-sig-meta-analysis
mailing list