[R-meta] metagen / low heterogeneity

[Michael Dewey]

Dear Sean

Some comments in-line. It is difficult to read your output because you 
posted in HTML so I will leave that to people more familiar with the 
software. Next time it would help to set your mailer to use plain text 
so your message does not get mangled.

On 11/01/2021 14:56, Sean wrote:
> Hello Meta-analysis Community,
> 
> I've been using the metagen function in the meta package for a
> meta-analysis on fungicide efficacy to control a foliar pathogen in
> cucumbers. I'm using pre-calculated Hedge's G as my effect size and it's
> standard error. I'm not really a statistician, so I've been using this
> resource to hold my hand through the process (
> https://bookdown.org/MathiasHarrer/Doing_Meta_Analysis_in_R/random.html).
> I've run into a bit of a rut and I'm having a hard time getting help to
> interpret my results. I'm dealing with the issue of some of my dataset
> heterogeneity being nearly 0 (which could just be the case).
> 
> *Here is an example of my output:*
> 
> Number of studies combined: k = 288
> 
>                                      SMD      95%-CI                t
> p-value
> Random effects model 0.3309 [ 0.2866; 0.3751] 14.72 < 0.0001
> Prediction interval                     [-0.2216; 0.8834]

The fact that your prediction interval is so much wider than the 
confidence interval does suggest there is heterogeneity here.
> 
> Quantifying heterogeneity:
>   tau^2 = 0.0783 [<0.0000; <0.0000]; tau = 0.2798 [<0.0000; <0.0000];
>   I^2 = 0.0% [0.0%; 0.0%]; H = 1.00 [1.00; 1.00]
> 
> Test of heterogeneity:
>        Q     d.f.   p-value
>   165.46  287  1.0000
> 
> *Here is the code:*
> 
> metamkt <- metagen(G,
>                      seG,
>                      data = mkt,
>                      studlab = paste(Study),
>                      comb.fixed = FALSE,
>                      comb.random = TRUE,
>                      method.tau = "SJ",
>                      hakn = TRUE,
>                      prediction = TRUE,
>                      sm = "SMD")
> 
> My first red flag is of course "I^2 = 0.0%", then that my Q p-value is 1.
> The interpretation being that the observed heterogeneity is completely
> random. I have a couple datasets, with the highest I^2 = 17.4%. The reason
> I find it odd, is that when I do subgroup analysis (even though I'm not
> supposed to with such low / non-existat heterogeneity), the results make
> biological sense.

No, no, a thousand times no. You use a moderator if there is a 
scientific hypothesis which justifies it not because of observed 
heterogeneity. In this case if there is a biological theory behind a 
moderator then use it.

Michael

  My data spans the last decade and the results are also
> similar with a meta-analysis done in the previous decade on the same topic.
> This makes me feel like I've made some sort of error at some point in my
> workflow and I was wondering if you have any diagnostic recommendations for
> me? One thing that worries me is that my standard errors for my Hedge's G
> values are so similar since all treatments in each study have 4
> replications, but maybe it shouldn't.
> 
> Best,
> 
> Sean
> 
> 	[[alternative HTML version deleted]]
> 
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> 

-- 
Michael
http://www.dewey.myzen.co.uk/home.html