[R-meta] Fixed vs Random Effects

[Célia Sofia Moreira]

Dear Professor Michael Dewey,

Thank you very much for your helpful comments. I really appreciate your
attention. I just didn't understand the meaning of "Point 1 is indeed
true". We know that fixed effects models give more weight to larger
studies. But do you think this is correct? That is, do you think that
weights' distribution in fixed-effects is better/fairer than weights'
distribution in random-effects modelling?

Also, don't weights in random effects modelling also take into account the
sample size [although not as strongly (proportionally) as in fixed-effects
modelling]?

Kind regards,
 celia

2018-04-12 13:48 GMT+01:00 Michael Dewey <lists at dewey.myzen.co.uk>:

> Comments in line. Perhaps even more necessary than usual to stress these
> are personla views and others will differ.
>
> On 11/04/2018 23:59, Célia Sofia Moreira wrote:
>
>> Dear all,
>>
>> I'm needing your help on the decision between fixed- or random-effects. I
>> know that most of you are reviewers in top and respectable journals on
>> meta-analysis, and so I will take your opinion very seriously. The
>> question
>> is the following:
>>
>> My "favourite" papers recommend the use of random effects when you want to
>> make inferences about the average effect to the entire population of
>> studies from which the included studies are assumed to be a random
>> selection (including "studies that have been conducted, that could have
>> been conducted, or that may be conducted in the future"). Others
>> (Cochrane)
>> recommend the use of random-effect when samples/experiments/designs/...
>> have different features. All of them say that the choice should not be
>> decided on the basis of presence/absence of heterogeneity, and the
>> researcher should decide on the type of inference desired before examining
>> the data.
>>
>
> That summary is what I believe too and I think the last sentence is one
> which is very important.
>
>
>> Papers included in 'my' meta-analysis have very different
>> samples/experimental features, as the majority of studies in social
>> sciences. Moreover, I consider that is advantageous to make inferences to
>> the entire population, instead of making inferences only to the set of
>> studies included in 'my' meta-analysis; it is a wider approach. Therefore,
>> I decided to perform random-effects models. In most cases, the results
>> showed only small heterogeneity (and thus the results for fixed effects
>> are
>> similar).
>>
>> Now, a co-author disagrees with my point of view and says that the
>> meta-analysis should be performed using fixed-effects models because (his
>> main reasons):
>> 1) "larger studies should have more weight" (sample sizes range from 25 to
>> 65),
>> 2) "choosing a random-effects model introduces an error in each study",
>> 3) "fixed effects provide narrower CI intervals and, as such, more precise
>> results".
>>
>
> Point 1 is indeed true. I do not understand point 2. Point 3 is also true
> but misses the point as to whether that narrowness is appropriate or not.
> If you want to choose a third option there is the method by Hemni and Copas
> which is available in  metafor. This was designed for situations of small
> study bias and basically gives you the fixed effect summary (which gets rid
> of point 1) while making the CI more wide.
>
>
>> He also gave me a reference of an article that was published in the same
>> journal we are planning to submit 'our' meta-analysis, in which
>> fixed-effects were preferred. The authors used the following argument:
>>
>> "Studies on the effect of medications were combined using a fixed-effect
>> model (Borenstein et al., 2010). We expected the final model to include
>> only a small number of studies and estimation of random-effects models
>> with
>> few studies has been shown to be unreliable (Guolo and Varin, 2017).
>> However, random-effects models were carried out in a sensitivity
>> analysis."
>>
>
> It is true that the estimate of tau^2 is quite imprecise but I would have
> thought it more logical to do the analyses the other way round (random
> primary, fixed sensitivity).
>
> There is also the issue f whether in the face of extreme heterogeneity it
> makes sense to give any summary estimate at all. I recently reviewed an
> article where they used random effects to combine two estimates, one from
> each sex. Apart from the issue of whether you can generalise to a
> population of other sexes there were some pairs of estimates which clearly
> looked different and where combining them obscured rather than illuminated.
>
>
>> I have confirmed that results from random- and fixed-effects models are
>> similar in most of the cases (usually <= .01; narrower CI but the
>> significance does not change), and even when the difference is higher
>> (=.04) there is no "small-studies effect" (i.e., small studies are not
>> consistently more positive, or negative).
>>
>> What is your opinion on his arguments and on the argument used in that
>> paper (i.e., the estimation of fixed-effects models is more reliable than
>> the random-effects when there are only few studies)?
>>
>> Kind regards,
>> celia
>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-meta-analysis mailing list
>> R-sig-meta-analysis at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>
>>
> --
> Michael
> http://www.dewey.myzen.co.uk/home.html
>

	[[alternative HTML version deleted]]