[R] Regression and Sub-Groups Analysis in Metafor

[Bert Gunter]

Briefly, as this is off-topic, and inline:
Bert Gunter

"The trouble with having an open mind is that people keep coming along
and sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Tue, May 31, 2016 at 11:32 AM, Dan Kolubinski <kolubind at lsbu.ac.uk> wrote:
> That makes perfect sense.  Thank you, Michael.  I take your point about not
> chasing the data and definitely see the risks involved in doing so.  Our
> hypothesis was that the first, second and fourth variables would be
> significant, but the third one (intervention) would not be.

That is **not** a legitimate scientific hypothesis. Post to a
statistical list like stats.stackexchange.com to learn why not.

Cheers,
Bert



 I will
> double-check the dataset to make sure that there are not any errors and
> will report the results as we see them.  I much appreciate you taking the
> time!
>
> Best wishes,
> Dan
>
> On Tue, May 31, 2016 at 12:02 PM, Michael Dewey <lists at dewey.myzen.co.uk>
> wrote:
>
>> In-line
>>
>> On 30/05/2016 19:27, Dan Kolubinski wrote:
>>
>>> I am completing a meta-analysis on the effect of CBT on low self-esteem
>>> and
>>> I could use some help regarding the regression feature in metafor.  Based
>>> on the studies that I am using for the analysis, I identified 4 potential
>>> moderators that I want to explore:
>>> - Some of the studies that I am using used RCTs to compare an intervention
>>> with a waitlist and others used the pre-score as the control in a
>>> single-group design.
>>> - Some of the groups took place in one day and others took several weeks.
>>> - There are three discernible interventions being represented
>>> - The initial level of self-esteem varies
>>>
>>> Based on the above, I used this command to conduct a meta-analysis using
>>> standarized mean differences:
>>>
>>>
>>>
>>> MetaMod<-rma(m1i=m1, m2i=m2, sd1i=sd1, sd2i=sd2, n1i=n1, n2i=n2,
>>> mods=cbind(dur, rct, int, level),measure = "SMD")
>>>
>>>
>> You could also say mods = ~ dur + rct + int + level
>>
>>
>>>
>>> Would this be the best command to use for what I described?  Also, what
>>> could I add to the command so that the forest plot shows a sub-group
>>> analysis using the 'dur' variable as a between-groups distinction?
>>>
>>>
>> You have to adjust the forest plot by hand and then use add.polygon to
>> add the summaries for each level of dur.
>>
>>
>>> Also, with respect to the moderators, this is what was delivered:
>>>
>>>
>>>
>>> Test of Moderators (coefficient(s) 2,3,4,5):
>>> QM(df = 4) = 8.7815, p-val = 0.0668
>>>
>>> Model Results:
>>>
>>>          estimate      se     zval    pval    ci.lb   ci.ub
>>> intrcpt    0.7005  0.6251   1.1207  0.2624  -0.5246  1.9256
>>> dur        0.5364  0.2411   2.2249  0.0261   0.0639  1.0090  *
>>> rct       -0.3714  0.1951  -1.9035  0.0570  -0.7537  0.0110  .
>>> int        0.0730  0.1102   0.6628  0.5075  -0.1430  0.2890
>>> level     -0.2819  0.2139  -1.3180  0.1875  -0.7010  0.1373
>>>
>>> ---
>>> Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>>>
>>>
>>>
>> So the totality of moderators did not reach an arbitrary level of
>> significance.
>>
>>
>>> From this, can I interpret that the variable 'dur' (duration of
>>>>
>>> intervention) has a significant effect and the variable 'rct' (whether a
>>> study was an RCT or used pre-post scores) was just shy of being
>>> statistically significant?  I mainly ask, because the QM-score has a
>>> p-value of 0.0668, which I thought would mean that none of the moderators
>>> would be significant.  Would I be better off just listing one or two
>>> moderators instead of four?
>>>
>>>
>> At the moment you get an overall test of the moderators which you had a
>> scientific reason for using. If you start selecting based on the data
>> you run the risk of ending up with confidence intervals and significance
>> levels which do not have the meaning they are supposed to have.
>>
>>
>> Much appreciated,
>>> Dan
>>>
>>>       [[alternative HTML version deleted]]
>>>
>>> ______________________________________________
>>> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide
>>> http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>>
>>>
>> --
>> Michael
>> http://www.dewey.myzen.co.uk/home.html
>>
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.