[R-meta] Question regarding Generalized Linear Mixed-effects Model for Meta-analysis

[James Pustejovsky]

Wolfgang,

Please forgive me for following up with questions that are pure statistical
geekery. Do you have a reference for the formula you gave on estimating the
sampling variance of a mean proportion? I haven't seen it before and was
curious to know its development. Also, is there a problem with simply using
s^2 / n? This is the unbiased variance estimator under simple random
sampling, and so I would have thought that it would work adequately here.

Best,
James

On Wed, Jan 3, 2018 at 4:18 AM, Viechtbauer Wolfgang (SP) <
wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:

> Two additions:
>
> 1) Estimation of the sampling variance of a mean proportion is a bit more
> complex.
>
> Assume that in a given study there are n subjects, each of which completes
> t trials. So, for each subject, there is a proportion, p_i = x_i/t, where
> x_i denotes the number of 'successes' on the t trials. Let p = sum p_i / n
> denote the mean proportion and s^2 the variance of the proportions. Then
> the sampling variance of p can be estimated with:
>
> v = (p*(1-p) - s^2) / (n*t).
>
> So, when meta-analyzing values of p from multiple studies, the sampling
> variances should be computed in this way.
>
> 2) Instead of meta-analyzing values of p directly (which indeed might lead
> to predicted values outside of the 0-1 range), we can meta-analyze
> ln(p/(1-p)) values, which are unbounded and back-transformed values will
> always be in the 0-1 range. The sampling variance of ln(p/(1-p)) can be
> estimated with:
>
> v = 1/(p*(1-p))^2 * (p*(1-p) - s^2) / (n*t)
>
> Best,
> Wolfgang
>
> -----Original Message-----
> From: Michael Dewey [mailto:lists at dewey.myzen.co.uk]
> Sent: Wednesday, 03 January, 2018 10:59
> To: Akifumi Yanagisawa; Viechtbauer Wolfgang (SP)
> Cc: r-sig-meta-analysis at r-project.org
> Subject: Re: [R-meta] Question regarding Generalized Linear Mixed-effects
> Model for Meta-analysis
>
> Dear Aki
>
> In that case why not just use the mean and its sampling variance in the
> usual way? This may lead to impossible predictions as there will be no
> way of specifying that the means are bounded above and below but it may
> be the best you can do with what they have published.
>
> Michael
>
> On 02/01/2018 20:48, Akifumi Yanagisawa wrote:
> > Thank you for your reply, Wolfgang.
> >
> > Your guess is right. I do not have a single count out of a total number
> of trials in each study. What I am using is the mean proportion and SD
> among the proportions.
> >
> > I am sad to hear that I cannot use the binomial distribution in glmer()
> in this case, and weights argument cannot be used as usual weights.
> >
> > Do you have any ideas on how to deal with this type of data?
> >
> > Thank you very much.
> >
> > Best regards,
> >
> > Aki
> >
> > On Jan 2, 2018, at 4:03 AM, Viechtbauer Wolfgang (SP) <
> wolfgang.viechtbauer at maastrichtuniversity.nl<mailto:wolfgang.viechtbauer@
> maastrichtuniversity.nl>> wrote:
> >
> > Dear Aki,
> >
> > Before I could even suggest a modeling approach, I would need to better
> understand your dependent variable. You say that you have 'proportional
> data', but then also mention 'means' and 'SDs'. So, it seems to me that you
> do not have proportions per se (that is, you do not have a single count out
> of a total number of trials in each study -- which we could indeed model
> using a binomial GLMM with a logit link).
> >
> > Maybe you have studies where each participant conducted a number of
> trials, so that there is a proportion per participant and what is reported
> is the mean proportion and the SD among the proportions. But now I am just
> guessing.
> >
> > In either case, your glmer() syntax doesn't make sense. For a binomial
> GLMM, the 'weights' argument is used to give the number of trials when the
> response is the proportion of successes, but you are using 1/vi as weights.
> >
> > Best,
> > Wolfgang
> >
> > -----Original Message-----
> > From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-
> bounces at r-project.org] On Behalf Of Akifumi Yanagisawa
> > Sent: Wednesday, 20 December, 2017 19:57
> > To: r-sig-meta-analysis at r-project.org<mailto:r-sig-meta-
> analysis at r-project.org>
> > Subject: [R-meta] Question regarding Generalized Linear Mixed-effects
> Model for Meta-analysis
> >
> > Dear all,
> >
> > I am having some difficulty dealing with proportional data; the
> dependent variable is learning gain from an activity, in which means and
> SDs are converted into proportion. The learning gains are nested in each
> article; each article examined the learning gains from different types of
> activities and measured the learning gain at different timing (i.e.,
> immediate post and delayed post). The main thing I would like to do is to
> get the estimated learning gain percentage and its confidence interval for
> each activity.
> >
> > Using the rma.mv() function, I noticed that estimation values go over
> 100% sometimes; then I thought I should use generalized linear mixed
> effects model. On the metafor’s webpage (http://www.metafor-project.
> org/doku.php/todo), I found that the rma.glmm() command does not support
> Multilevel Models so far and suggested using the LME4 package. I have been
> trying to figure out how to do this by myself, but I am not sure if I am
> doing this right. I would appreciate it if you could see if my approach is
> appropriate and answer to some of my questions.
> >
> > (1) The approach I tried was, (1) calculated variance from means, SDs,
> and the numbers of participants by using the escalc function, and (2) then
> I tried ‘results <- glmer (learning_gain ~ ACTIVITY * TEST_TYPE *
> TEST_TIMING + (1|article_number/participant_group) +
> (1|TEST_TIMING:participant_group), weights = 1/vi, family = binomial
> (link = logit))’. I use the sjPlot package for plotting and the emmeans
> package to get estiamted learning gain percentages.  Does this sound like
> the proper approach? Are there other options should I add?
> >
> > (2) Is it possible for me to get I^2 and H^2 values? I would like to
> know the proportion of variance explained by each the moderator.
> >
> > (3) Is there anyway I can conduct (a) Test for Residual Heterogeneity
> and (b) Test of Moderators? If so, which R package would you recommend? I
> noticed that the anova function does not provide p-values for the test, and
> the LmerTest package does not work with the glmer function, either.
> >
> > Any suggestions and comments will be greatly appreciated. Thank you for
> your help.
> >
> > Aki
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>

	[[alternative HTML version deleted]]