[R-meta] Assessing publication bias from multilevel modelling

[James Pustejovsky]

A funnel plot is simply a scatter-plot of effect size estimates versus
their standard errors (or some other measure of precision). Modeling
assumptions about the dependencies among the effect size estimates don't
affect the plot itself, and so it makes sense that you would get the same
result from both of those functions.

On Wed, Feb 14, 2018 at 11:56 AM, Célia Sofia Moreira <
celiasofiamoreira at gmail.com> wrote:

> Dear Michael and James,
>
> Thank you very much for your nice recommendations. Meanwhile, I don't know
> if it is just by chance, but the funnel plots of
>
> mm<-rma.mv(y ~ 1, V=Vlist, random = ~ 1 |Study, struct = "UN", data =
> base); #Vlist <- impute_covariance_matrix(vi = base$v, cluster =
> base$Study, r = .5)
> m<-rma(y ~ 1, v,  data = base);
>
> are precisely the same (although the models' summaries are different)! I
> was not expecting this result.
> Does this make sense to you? Maybe "funnel(mm)" is not taking into account
> the random part of the model, I don't know... Can you please comment about
> this fact?
>
>
>
> 2018-02-14 17:36 GMT+00:00 James Pustejovsky <jepusto at gmail.com>:
>
>> I think regtest() is appropriate only for effect sizes that are
>> statistically independent. It would not be appropriate for Celiia's case,
>> where there is dependence among multiple effects from a given study.
>>
>> Egger's regression test can be conducted simply by using the standard
>> errors of the effect size estimates (i.e., the "sei") as a predictor in a
>> meta-regression. Using cluster-robust variance estimation with this
>> meta-regression will take care of the dependence issue. A significant
>> coefficient on the standard error predictor would indicate funnel plot
>> asymmetry. (Although as always, a lack of statistically significance does
>> not *prove* that the funnel plot is symmetric.) When I've used this
>> technique, I fit the meta-regression without any random effects in the
>> model so that larger studies are given relatively more weight for
>> estimating the meta-regression coefficients.
>>
>> The Henmi-Copas model is limited to univariate models. I don't know of a
>> way to generalize it for multi-variate meta-analysis.
>>
>> James
>>
>> On Wed, Feb 14, 2018 at 11:21 AM, Michael Dewey <lists at dewey.myzen.co.uk>
>> wrote:
>>
>>> Dear Célia
>>>
>>> As far as I can see regtest can accept the yi, vi or sei parameters so
>>> you should be OK there. I do not think the HC method is going to be easy to
>>> do if it is even possible.
>>>
>>> Michael
>>>
>>>
>>> On 14/02/2018 16:14, Célia Sofia Moreira wrote:
>>>
>>>> Hi!
>>>>
>>>> I am studying a pretest-posttest controlled design and I'm using metafor
>>>> and clubSandwich packages. In some papers, I collected more than one
>>>> effect
>>>> size (from the same study sample). For this reason, I did multilevel
>>>> modelling using rma.mv function, with
>>>> random = ~ 1 |Study, struct = "UN".
>>>>
>>>> I would like to perform a publication bias analysis, using funnel plots,
>>>> tests for funnel plots asymmetry, and Hemni and copas (HC) model.
>>>> However,
>>>> since I am using rma.mv function, I can only depict (multilevel) funnel
>>>> plots; unfortunately, tests for funnel plots asymmetry and HC model are
>>>> not
>>>> available for rma.mv....
>>>>
>>>> Do you know any alternative way to test the asymmetry of my "multilevel"
>>>> funnel plots, as well as to compare with the HC modelling?
>>>>
>>>> If not, can anybody please suggest other methods to assess publication
>>>> bias
>>>> in this multilevel case?
>>>>
>>>> Thank you very much,
>>>> celia
>>>>
>>>>         [[alternative HTML version deleted]]
>>>>
>>>> _______________________________________________
>>>> R-sig-meta-analysis mailing list
>>>> R-sig-meta-analysis at r-project.org
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>>>
>>>>
>>> --
>>> Michael
>>> http://www.dewey.myzen.co.uk/home.html
>>>
>>>
>>> _______________________________________________
>>> R-sig-meta-analysis mailing list
>>> R-sig-meta-analysis at r-project.org
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>>
>>
>>
>

	[[alternative HTML version deleted]]