[R-meta] Random-effects meta-analyses with inverse-variance weights cannot include studies with sample sizes of two?

[Viechtbauer, Wolfgang (SP)]

I can't give any specific advice here. This would require knowing more details of the studies, the data collection procedures, and the goals of the meta-analysis. I believe this article is quite relevant though:

Bakbergenuly, I., Kulinskaya, E., & Morgenthaler, S. (2016). Inference for binomial probability based on dependent Bernoulli random variables with applications to meta-analysis and group level studies. Biometrical Journal, 58(4), 896-914. doi:10.1002/bimj.201500115

Best,
Wolfgang

-----Original Message-----
From: Patrizio Tressoldi [mailto:patrizio.tressoldi using unipd.it] 
Sent: Friday, 09 November, 2018 16:20
To: Viechtbauer, Wolfgang (SP); r-sig-meta-analysis using r-project.org
Subject: Re: [R-meta] Random-effects meta-analyses with inverse-variance weights cannot include studies with sample sizes of two?

Thank you very much Wolfgang for your exaustive information.

X is based on the performance of a single participant.

Given you raised doubts about the independence assumption, what do you
suggest as a best effect size and variance alternative in similar cases?

Best

Patrizio

Il 09/11/2018 14:33, Viechtbauer, Wolfgang (SP) ha scritto:
> I see. This assumes independence between trials and is typically used when measuring a dichotomous outcome on N different subjects. But here it seems like a single subject went through N different trials. In that case, the independence assumption seems tenuous.
>
> How does this relate to the two subjects you talked about earlier? Or is X based on the sum of two subjects?
>
> Also, I don't see how you then derived the equation for the variance of z/sqrt(N). You wrote:
>
> Variance = Sqr(1/(1-q)*(1-1/(1-q))/N
>
> But let's go through this step-by-step:
>
> Leaving aside the independence issue, we assume (for large enough N):
>
> p ~ N(π, π(1-π)/N)
>
> Then we compute:
>
> z = (p - π₀) / sqrt(π₀(1-π₀)/N)
>
> Under H0: π = π₀, it is easy to see that z ~ N(0,1), so z/sqrt(N) ~ N(0, 1/N).
>
> But if H0 is not true, then some simple algebra shows:
>
> Var(z) = π(1-π) / (π₀(1-π₀))
>
> and hence Var(z/sqrt(N)) = π(1-π) / (N * π₀(1-π₀)), which we could estimate with:
>
> Var(z/sqrt(N)) = p(1-p) / (N * π₀(1-π₀)).
>
> Best,
> Wolfgang
>
> -----Original Message-----
> From: Patrizio Tressoldi [mailto:patrizio.tressoldi using unipd.it]
> Sent: Friday, 09 November, 2018 13:01
> To: Viechtbauer, Wolfgang (SP); r-sig-meta-analysis using r-project.org
> Subject: Re: [R-meta] Random-effects meta-analyses with inverse-variance weights cannot include studies with sample sizes of two?
>
> Il 09/11/2018 12:05, Viechtbauer, Wolfgang (SP) ha scritto:
>
> Maybe I am a bit dense here, but I still do not fully understand what you are computing. You wrote earlier that there are two participants that "contributed to a percentage of hits, e.g. .30 and .40". Ok, that sounds like these participants did a series of trials that could yield hits/successes. I assume N is the number of trials. So the first participant had .3*N hits and the second participant had .4*N hits. So far so good. But what is 'z = binomial z'? Where does that equation for the variance come from? It would also help if you could provide a fully reproducible example of the computations.
>
> This is the z score obtained from a normal approximation of the binomial test binomial test: z= (X - µ)/σ ; where X = observed percentage;  µ = chance percentage; σ = Sqr(µ(1-µ)/N
>
> ex. X= 31; N= 49; p = .5 = (31/46 - .5)/Sqr(.5*.5/49) = 2.4
>
> Patrizio
>
> --
> Patrizio E. Tressoldi Ph.D.
> Dipartimento di Psicologia Generale
> Università di Padova
> via Venezia 8
> 35131 Padova - ITALY
> http://www.patriziotressoldi.it
> https://orcid.org/0000-0002-6404-0058
>
> Science of Consciousness Research Group
> http://dpg.unipd.it/en/soc
>
> Make war history
> support http://www.emergency.it