[R-meta] Calculating the covariance structure for multilevel meta-analysis using the raw mean

[Viechtbauer Wolfgang (SP)]

Thanks for clarifying. For a within-subjects design, you would need the correlation between the measurements under the two treatments in order to compute the covariance of the means. The same applies to the case where a group is measured twice.

The problem is of course that these correlations are not reported. Strategies for dealing with this problem have been covered extensively in the past on this mailing list. The most straightforward approach would be to ignore the covariances during the model fitting and then use cluster-robust inference methods.

Best,
Wolfgang

>-----Original Message-----
>From: Akifumi Yanagisawa [mailto:ayanagis at uwo.ca]
>Sent: Monday, 26 February, 2018 15:52
>To: Viechtbauer Wolfgang (SP)
>Cc: r-sig-meta-analysis at r-project.org
>Subject: Re: Calculating the covariance structure for multilevel meta-
>analysis using the raw mean
>
>Dear Wolfgang,
>
>Thank you for replying to my question!
>
>To your first question, yes for some studies and no for the other
>studies. Some studies used within-participant design, where the same
>group of people engaged in several different treatments (i.e., word
>learning activities). The other studies used between-participant design,
>where one group of people only experienced one treatment.
>
>For your second question, yes to your assumption, the same group of
>people was involved in both ‘measurement1’ and ‘measurement2’, and the
>group sizes are the same.
>But, not to your question. If studies used both ‘measurement1’ and
>‘measurement2’, all of the participants’ learning gains were tested with
>both measurements. So the 28 people involved in 'measurement1’ are the
>SAME group of people as the 30 (OR, 28) people involved in
>‘measurement2’.
>(Sorry for the confusion. The number of participants should have been the
>same; let’s say 28 for here.)
>
>I am glad to hear that I can regard the learning gains from studies that
>used between-participant design as independent observations and include
>them for analysis. Do you think it would be possible for me to include
>studies that used within-participant design as well?
>
>Thank you very much for your time and support.
>Best regards,
>Aki
>
>> On Feb 26, 2018, at 9:17 AM, Viechtbauer Wolfgang (SP)
><wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
>>
>> Dear Aki,
>>
>> For the first case, are the 28 people in row 1 a different group of
>people than the 30 people in row 2? Then the sampling errors for the two
>means are independent and you do not have to worry about the covariance.
>>
>> For the second case (with multiple measurements), I would have assumed
>that the same group of people was involved in 'measurement1' and
>'measurement2' but (assuming no missing data) then the group sizes should
>be the same, which they are not in your illustrative data. So, then I
>would have the same question: Are the 28 people involved in
>'measurement1' a different group of people than the 30 people involved in
>'measurement2'?
>>
>> Best,
>> Wolfgang
>>
>>> -----Original Message-----
>>> From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-
>>> project.org] On Behalf Of Akifumi Yanagisawa
>>> Sent: Monday, 19 February, 2018 14:57
>>> To: r-sig-meta-analysis at r-project.org
>>> Subject: [R-meta] Calculating the covariance structure for multilevel
>>> meta-analysis using the raw mean
>>>
>>> Dear all,
>>>
>>> I am currently conducting a multilevel meta-analysis with the metafor
>>> package using the raw mean.
>>> The following is sample data which describes the characteristics of my
>>> data. Please note the actual data set includes more studies.
>>>
>>> ## data structure ##
>>> dat <- data.frame(
>>> id = c(1,2,3,4,5, 6, 7),
>>> study = c("study1","study1", "study2", "study2", "study3", "study3",
>>> "study4"),
>>> treatment = c("treatment1","treatment1", "treatment2", "treatment3",
>>> "treatment3", "treatment4", "treatment4"),
>>> mi = c(5.3, 5.1, 2.2, 3.4, 2.5, 5.1, 3.3),
>>> sdi = c(1.2, 1.3, 0.5, 3.1, 0.7, 2.1, 0.8),
>>> ni = c(28, 30, 20, 20, 42, 43, 120))
>>> require(metafor)
>>> dat <- escalc(data = dat, measure = "MN", mi = mi, sdi = sdi, ni = ni)
>>> dat
>>>
>>> As you can see in the above data, some studies examined only one type
>of
>>> treatment. I am trying to specify the model using the the following
>code.
>>>
>>> res <- rma.mv (yi, vi, mods = ~ treatment - 1, random = ~treatment |
>>> study, struct = "DIAG", data = dat)
>>>
>>> However, I cannot figure out how to calculate the covariance structure
>>> for this case, i.e., some studies include only one treatment. I am
>>> thinking that I probably cannot use the same approach as network meta-
>>> analysis. I would appreciate it if I could learn how to calculate the
>>> covariance matrix for this case.
>>>
>>> Also, some of the studies measured the outcome using two types of
>>> measurements. None of these studies reported a correlation between
>>> measurement 1 and measurement 2. Would it be possible for me to
>include
>>> this type of measurement as a factor while somehow specifying the
>>> covariance matrix for this as well? Or, should I just analyze the data
>>> separately for each type of measurement?
>>>
>>> ## data structure 2 ##
>>> dat <- data.frame(
>>> id = c(1,2,3,4,5, 6, 7),
>>> study = c("study1","study1", "study2", "study2", "study3", "study3",
>>> "study4"),
>>> treatment = c("treatment1","treatment1", "treatment2", "treatment3",
>>> "treatment3", "treatment4", "treatment4"),
>>> measurement = c("measurement1","measurement2", "measurement1",
>>> "measurement1", "measurement2", "measurement2", "measurement1"),
>>> mi = c(5.3, 5.1, 2.2, 3.4, 2.5, 5.1, 3.3),
>>> sdi = c(1.2, 1.3, 0.5, 3.1, 0.7, 2.1, 0.8),
>>> ni = c(28, 30, 20, 20, 42, 43, 120))
>>> dat <- escalc(data = dat, measure = "MN", mi = mi, sdi = sdi, ni = ni)
>>> res <- rma.mv(data = dat ,yi, vi, mods = ~ treatment + measurement -1,
>>> random = ~treatment|study, struct = "DIAG”)
>>>
>>> Thank you very much for your help,
>>> Aki