[R-meta] rma, sandwich correction and very small data sets

[Valeria Ivaniushina]

Michael, Wolfgang,
Thank you!
This list is a huge help

Best,
Valeria

On Sat, Dec 12, 2020, 00:04 Viechtbauer, Wolfgang (SP) <
wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:

> Just want to add to this an answer to question 5.
>
> robust() from metafor with adjust=TRUE does use a small-sample adjustment,
> but a very simple one. clubSandwich has much more sophisticated adjustments
> and I would recommend to use them (CR2 is the recommended one).
>
> Best,
> Wolfgang
>
> >-----Original Message-----
> >From: Michael Dewey [mailto:lists using dewey.myzen.co.uk]
> >Sent: Wednesday, 09 December, 2020 17:47
> >To: Valeria Ivaniushina; Viechtbauer, Wolfgang (SP)
> >Cc: R meta
> >Subject: Re: [R-meta] rma, sandwich correction and very small data sets
> >
> >Dear Valeria
> >
> >I think as a general principle you are entitled to do your analysis even
> >on a small data-set as long as you accept that your results may not be
> >very precise. There seems to be a general feeling among analysts in the
> >area in which I work (health) that looking for small study effects is
> >not worth trying with fewer than ten studies and even with more may well
> >be uninformative. I am personally rather sceptical about identifying
> >observations as outliers in the absence of a scientific reason for doing
> so.
> >
> >Michael
> >
> >On 09/12/2020 15:21, Valeria Ivaniushina wrote:
> >> Dear Wolfgang,
> >>
> >> Thank you VERY much!
> >> Thank you for correcting my code -- indeed, random effect on the 1st
> level
> >> is totally needed!
> >>
> >> A couple more questions, if I may
> >>
> >> 1. There are too little cases for such a complex data structure, and
> it's
> >a
> >> serious limitation.
> >> But I hope that even if the results may be considered only as
> descriptive,
> >> they still point out in the correct direction?
> >> Especially taking into account that all three subsamples show quite
> >similar
> >> results.
> >> Is it a valid interpretation?
> >>
> >> 2. Considering that the sample is small (and 3-level!),  I guess that
> >> analysis of outliers would be excessive. Is it right?
> >>
> >> 3. The same goes for publication bias analysis? (as James points out,
> >these
> >> tests do not have strong power:
> >> www.jepusto.com/publication/selective-reporting-with-dependent-effects/
> )
> >>
> >> 4. and there is no power for mediation analysis, so I don't have to even
> >> attempt to do it?
> >>
> >> 5. Estimators question:
> >>   "robust" function in rma is using sandwich-type estimator, and with
> >adjust
> >> = TRUE it does a small-sample adjustment
> >> In the clubSandwich library there are a bunch of estimators with
> different
> >> small sample corrections. They give somewhat different results, some are
> >> very close to "robust" output
> >> Is clubSandwich  CR2 (for example) better than robust.rma?
> >> Or, if CR estimators from clubSandwich are not definitely preferable,
> can
> >I
> >> just use robust.rma?
> >>
> >> Best,
> >> Valeria
>

	[[alternative HTML version deleted]]