[R-meta] Calculating the covariance structure for multilevel meta-analysis using the raw mean

[Viechtbauer Wolfgang (SP)]

I am not sure I understand your question. The covariance I was talking about goes into the V matrix. What you specify for 'struct' is something totally different. Regardless of what you put into V, it makes very little sense to set struct = "DIAG", since that assumes that true effects for the same study are independent.

Best,
Wolfgang

>-----Original Message-----
>From: Akifumi Yanagisawa [mailto:ayanagis at uwo.ca]
>Sent: Monday, 26 February, 2018 16:34
>To: Viechtbauer Wolfgang (SP)
>Cc: r-sig-meta-analysis at r-project.org
>Subject: Re: Calculating the covariance structure for multilevel meta-
>analysis using the raw mean
>
>Dear Wolfgang,
>
>Thank you for your quick response and providing very straightforward
>answers. I will read the past post on the mailing list to learn how to
>use cluster-robust inference methods.
>
>Just a curiosity, I initially thought I could just fit multilevel models
>specifying covariance structure as diagonal, instead of specifying V (Or,
>the exact covariance structure).
>
>>>>> res <- rma.mv (yi, vi, mods = ~ treatment - 1, random = ~treatment |
>study, struct = "DIAG", data = dat)
>
>What do you think would be the problem with this approach? Do you think
>it could be acceptable to some extent?
>
>Thank you,
>Aki
>
>> On Feb 26, 2018, at 10:03 AM, Viechtbauer Wolfgang (SP)
><wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
>>
>> Thanks for clarifying. For a within-subjects design, you would need the
>correlation between the measurements under the two treatments in order to
>compute the covariance of the means. The same applies to the case where a
>group is measured twice.
>>
>> The problem is of course that these correlations are not reported.
>Strategies for dealing with this problem have been covered extensively in
>the past on this mailing list. The most straightforward approach would be
>to ignore the covariances during the model fitting and then use cluster-
>robust inference methods.
>>
>> Best,
>> Wolfgang
>>
>>> -----Original Message-----
>>> From: Akifumi Yanagisawa [mailto:ayanagis at uwo.ca]
>>> Sent: Monday, 26 February, 2018 15:52
>>> To: Viechtbauer Wolfgang (SP)
>>> Cc: r-sig-meta-analysis at r-project.org
>>> Subject: Re: Calculating the covariance structure for multilevel meta-
>>> analysis using the raw mean
>>>
>>> Dear Wolfgang,
>>>
>>> Thank you for replying to my question!
>>>
>>> To your first question, yes for some studies and no for the other
>>> studies. Some studies used within-participant design, where the same
>>> group of people engaged in several different treatments (i.e., word
>>> learning activities). The other studies used between-participant
>design,
>>> where one group of people only experienced one treatment.
>>>
>>> For your second question, yes to your assumption, the same group of
>>> people was involved in both ‘measurement1’ and ‘measurement2’, and the
>>> group sizes are the same.
>>> But, not to your question. If studies used both ‘measurement1’ and
>>> ‘measurement2’, all of the participants’ learning gains were tested
>with
>>> both measurements. So the 28 people involved in 'measurement1’ are the
>>> SAME group of people as the 30 (OR, 28) people involved in
>>> ‘measurement2’.
>>> (Sorry for the confusion. The number of participants should have been
>the
>>> same; let’s say 28 for here.)
>>>
>>> I am glad to hear that I can regard the learning gains from studies
>that
>>> used between-participant design as independent observations and
>include
>>> them for analysis. Do you think it would be possible for me to include
>>> studies that used within-participant design as well?
>>>
>>> Thank you very much for your time and support.
>>> Best regards,
>>> Aki
>>>
>>>> On Feb 26, 2018, at 9:17 AM, Viechtbauer Wolfgang (SP)
>>> <wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
>>>>
>>>> Dear Aki,
>>>>
>>>> For the first case, are the 28 people in row 1 a different group of
>>> people than the 30 people in row 2? Then the sampling errors for the
>two
>>> means are independent and you do not have to worry about the
>covariance.
>>>>
>>>> For the second case (with multiple measurements), I would have
>assumed
>>> that the same group of people was involved in 'measurement1' and
>>> 'measurement2' but (assuming no missing data) then the group sizes
>should
>>> be the same, which they are not in your illustrative data. So, then I
>>> would have the same question: Are the 28 people involved in
>>> 'measurement1' a different group of people than the 30 people involved
>in
>>> 'measurement2'?
>>>>
>>>> Best,
>>>> Wolfgang
>>>>
>>>>> -----Original Message-----
>>>>> From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-
>>>>> project.org] On Behalf Of Akifumi Yanagisawa
>>>>> Sent: Monday, 19 February, 2018 14:57
>>>>> To: r-sig-meta-analysis at r-project.org
>>>>> Subject: [R-meta] Calculating the covariance structure for
>multilevel
>>>>> meta-analysis using the raw mean
>>>>>
>>>>> Dear all,
>>>>>
>>>>> I am currently conducting a multilevel meta-analysis with the
>metafor
>>>>> package using the raw mean.
>>>>> The following is sample data which describes the characteristics of
>my
>>>>> data. Please note the actual data set includes more studies.
>>>>>
>>>>> ## data structure ##
>>>>> dat <- data.frame(
>>>>> id = c(1,2,3,4,5, 6, 7),
>>>>> study = c("study1","study1", "study2", "study2", "study3", "study3",
>>>>> "study4"),
>>>>> treatment = c("treatment1","treatment1", "treatment2", "treatment3",
>>>>> "treatment3", "treatment4", "treatment4"),
>>>>> mi = c(5.3, 5.1, 2.2, 3.4, 2.5, 5.1, 3.3),
>>>>> sdi = c(1.2, 1.3, 0.5, 3.1, 0.7, 2.1, 0.8),
>>>>> ni = c(28, 30, 20, 20, 42, 43, 120))
>>>>> require(metafor)
>>>>> dat <- escalc(data = dat, measure = "MN", mi = mi, sdi = sdi, ni =
>ni)
>>>>> dat
>>>>>
>>>>> As you can see in the above data, some studies examined only one
>type
>>> of
>>>>> treatment. I am trying to specify the model using the the following
>>> code.
>>>>>
>>>>> res <- rma.mv (yi, vi, mods = ~ treatment - 1, random = ~treatment |
>>>>> study, struct = "DIAG", data = dat)
>>>>>
>>>>> However, I cannot figure out how to calculate the covariance
>structure
>>>>> for this case, i.e., some studies include only one treatment. I am
>>>>> thinking that I probably cannot use the same approach as network
>meta-
>>>>> analysis. I would appreciate it if I could learn how to calculate
>the
>>>>> covariance matrix for this case.
>>>>>
>>>>> Also, some of the studies measured the outcome using two types of
>>>>> measurements. None of these studies reported a correlation between
>>>>> measurement 1 and measurement 2. Would it be possible for me to
>>> include
>>>>> this type of measurement as a factor while somehow specifying the
>>>>> covariance matrix for this as well? Or, should I just analyze the
>data
>>>>> separately for each type of measurement?
>>>>>
>>>>> ## data structure 2 ##
>>>>> dat <- data.frame(
>>>>> id = c(1,2,3,4,5, 6, 7),
>>>>> study = c("study1","study1", "study2", "study2", "study3", "study3",
>>>>> "study4"),
>>>>> treatment = c("treatment1","treatment1", "treatment2", "treatment3",
>>>>> "treatment3", "treatment4", "treatment4"),
>>>>> measurement = c("measurement1","measurement2", "measurement1",
>>>>> "measurement1", "measurement2", "measurement2", "measurement1"),
>>>>> mi = c(5.3, 5.1, 2.2, 3.4, 2.5, 5.1, 3.3),
>>>>> sdi = c(1.2, 1.3, 0.5, 3.1, 0.7, 2.1, 0.8),
>>>>> ni = c(28, 30, 20, 20, 42, 43, 120))
>>>>> dat <- escalc(data = dat, measure = "MN", mi = mi, sdi = sdi, ni =
>ni)
>>>>> res <- rma.mv(data = dat ,yi, vi, mods = ~ treatment + measurement -
>1,
>>>>> random = ~treatment|study, struct = "DIAG”)
>>>>>
>>>>> Thank you very much for your help,
>>>>> Aki