[R-meta] Wald_test - is it powerful enough?

[Cátia Ferreira De Oliveira]

Dear Wolfgang,

Thank you for your thorough response. I really appreciate it!

Best wishes,

Catia

On Wed, 22 Sept 2021 at 12:26, Viechtbauer, Wolfgang (SP) <
wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:

> Dear Cátia,
>
> To be precise: RVE does not penalize the (fixed effect) estimates at all,
> as they are unchanged. It only affects the SEs of the fixed effects and,
> when using clubSandwich, it can also use a Satterthwaite approximation for
> the degrees of freedom of the test statistics.
>
> With respect to the SEs:
>
> The degree to which the SEs differ between those from the working model
> and those you obtain after using RVE depends on the degree of discrepancy
> between the sources of heterogeneity/dependency that are accounted for in
> the working model and the actual sources of heterogeneity/dependency
> underlying the data. If this discrepancy is large, then this will also lead
> to large differences between the SEs even in large samples.
>
> However, in smaller samples, all estimates (including the SEs) will be
> unstable and so one might also tend to see larger differences there (but
> then it is less clear whether those differences really reflect
> discrepancies or just instabilities).
>
> With respect to the dfs:
>
> Depending on the structure and size of the data, the estimated dfs from
> the Satterthwaite approximation can also be quite small, which can have a
> noticeable impact on the significance of the tests (rightly so!). This
> should become less of an issue in larger datasets.
>
> Best,
> Wolfgang
>
> >-----Original Message-----
> >From: Cátia Ferreira De Oliveira [mailto:cmfo500 using york.ac.uk]
> >Sent: Wednesday, 22 September, 2021 9:00
> >To: James Pustejovsky
> >Cc: Viechtbauer, Wolfgang (SP); R meta
> >Subject: Re: [R-meta] Wald_test - is it powerful enough?
> >
> >Dear James and Wolfgang,
> >
> >Would you say that the RVE models are more penalising of estimates when
> we have
> >small samples?
> >I wonder if that also could explain the disparity between wald tests and
> rma.mv
> >since for the rest of the analyses when I have bigger sample sizes the
> results
> >seem to converge a lot more.
> >
> >Best wishes,
> >
> >Catia
>


-- 
Cátia Margarida Ferreira de Oliveira
Psychology PhD Student
Department of Psychology, Room A105
University of York, YO10 5DD

Twitter: @CatiaMOliveira
pronouns: she, her

	[[alternative HTML version deleted]]