[R] Meta-analysis on a repeated measures design with multiple trials per subject using metafor

[Viechtbauer Wolfgang (STAT)]

It seems to me that the most straightforward approach would be to pool the b coefficients directly. I assume you used some kind of mixed-effects logistic model to analyze those data for each study with random effects for subjects and a dummy variable for treatment (besides possibly some other covariates) and the coefficient of interest is the one corresponding to the dummy for treatment. Given those 5 coefficients and the corresponding standard errors, you can then pool them with rma(b, sei=se), where 'b' is the vector with the coefficients and 'se' is the vector with the corresponding standard errors.

Careful: rma(b, se) would not be correct, since the second argument is assumed to represent the *sampling variances*:

> library(metafor)
> args(rma)
function (yi, vi, sei, [SNIP])

So, due to positional matching of arguments, rma(b, se) would be treated as rma(yi=b, vi=se), which is not what you want (you would not be the first person to fall into that trap). 

And yes, if you include additional covariates in the mixed-effects logistic model, then of course the b's and se's will change, but I see no issue here. It's just a question of whether you want to pool raw or adjusted coefficients (if you do adjust though, it would be best to adjust for the same covariates, so that the adjusted coefficients are directly comparable).

Aside from all of that, there is another approach you could take. Since you have the raw data from all 5 studies, you could just as well pool the raw data into one dataset and analyze that. This is essentially an "individual person (or patient) data meta-analysis" (IPDMA). You would then include either fixed or random effects for studies and you probably would also want the treatment effect to vary randomly across studies. That approach should give you similar results as pooling the 5 coefficients (which is a sort of two-step approach).

Best,
Wolfgang

--   
Wolfgang Viechtbauer, Ph.D., Statistician   
Department of Psychiatry and Psychology   
School for Mental Health and Neuroscience   
Faculty of Health, Medicine, and Life Sciences   
Maastricht University, P.O. Box 616 (VIJV1)   
6200 MD Maastricht, The Netherlands   
+31 (43) 388-4170 | http://www.wvbauer.com   


> -----Original Message-----
> From: Marc Heerdink [mailto:m.w.heerdink at uva.nl]
> Sent: Sunday, July 07, 2013 23:51
> To: Michael Dewey
> Cc: Viechtbauer Wolfgang (STAT); r-help at r-project.org
> Subject: Re: [R] Meta-analysis on a repeated measures design with multiple
> trials per subject using metafor
> 
> Dear Michael and other readers,
> 
> Please see below for my answers to your questions about my data.
> 
> On 07/06/2013 02:56 PM, Michael Dewey wrote:
> [..]
> >> Because everything was randomized, I can only calculate the total
> >> number of times a certain response was used under each type of trial.
> >> There is no pairing of trials under two treatments, so I am forced to
> >> use the marginal totals from your table.
> >
> > But presumably you could calculate some statistic suitable for
> > summarising the relevant features here? Difference in proportions, odds
> > ratio, ...
> 
> Using the totals, it is indeed easy to calculate the difference in
> proportions or odds ratio based on these totals. However, I am not sure
> how I should calculate a study-level statistic suitable for
> meta-analysis on the basis of these participant-level proportion
> differences.
> 
> So, for instance, I have the following table;
> 
> pp	proportion_difference
> 1	0.1
> 2	0.05
> 3	0.08
> 4	0.02
> ..
> N	..
> 
> Can I just calculate the mean and standard deviation of these proportion
> differences -- mean(proportion_difference) and sd(proportion_difference)
> -- and use these for meta-analysis? If yes, what escalc measure should I
> use?
> 
> [..]
> >> One alternative that I have tried over the last few days, is to use
> >> the b parameter of interest and it's corresponding standard error from
> >> the lme4 regression output that I use to analyse the individual
> >> experiments. Then, I use rma(yi, sei) to do a meta-analysis on these
> >> parameters. I am not sure this is correct though, since it takes into
> >> account between-subjects variance (through a random effect for
> >> subject), and it is sensitive to the covariates/moderators I include
> >> in the models that I get the b parameters from.
> >
> > So you end up with 5 values of b? The fact that they adjust for
> > different moderators does not seem an issue to me, indeed it could be
> > argued to be an advantage of the meta-analytic approach here.
> 
> OK, thank you for your comment on this one. I think the results of a
> meta-analysis using these 5 b values are indeed more or less sensible,
> which is encouraging. I think I will go this way if it turns out I
> cannot find a simpler approach, as a simpler approach would be easier to
> sell to potential reviewers.
> 
> [..
> > I think we are all assuming you have different participants in each
> > experiment but I thought I would raise that as a question.
> 
> You are right in assuming this, I have different participants in all 5
> experiments.
> 
> Thanks all for the help so far, your suggestions are highly appreciated!
> 
> Regards,
> Marc