[R-meta] Covariance-variance matrix when studies share multiple treatment x control comparison

[Ju Lee]

Dear Wolfgang,

I deeply appreciate your time looking into this issue, and this has been immensely helpful.
I was able to incorporate all possible inter-dependence among effect sizes by adding different layers of non-independence to our dataframe.

I manually calculated hedges'd based on based on Hedges and Olkin (1985), and it generates exactly same value as hedges' g in escalc() "SMD" function. So hopefully I am doing everything right using the equation we've discussed earlier.

I have been also wondering if it is possible to account of this variance-covariance structure that I've constructed when running publication bias analysis, for example, when using fsn() function or modified egger's regression test (looking at intercept term of residual ~ precision meta-regression using rma.mv). I had no luck so far finding information on this, and I would appreciate if you have any suggestions related to this

Thank you for all of your valuable helps!
Best regards,
JU
________________________________
From: Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
Sent: Tuesday, September 24, 2019 3:59 AM
To: Ju Lee <juhyung2 using stanford.edu>; r-sig-meta-analysis using r-project.org <r-sig-meta-analysis using r-project.org>
Subject: RE: Covariance-variance matrix when studies share multiple treatment x control comparison

Hi Ju,

The equation for the covariances applies to the standardized mean difference for two independent groups (so, what one could call 'classical' Cohen's d or Hedges' g if one uses the bias correction). It does not apply to other types of standardized mean differences, such as those for dependent/paired groups. Also, the equation assumes that the pooled variance 'sdp' (that is used for computing pairwise SMDs, such as (m1 - m3) / sdp and (m2 - m3) / sdp for three groups, with group 3 being the reference group) has been computed by pooling the SDs of all three groups. Doing so isn't standard practice and takes some extra effort to implement.

Just a sidenote: When pasting tables like this, please use a fixed width font (and don't use tabs). Without that, the formating is all off and makes the table hard to read.

Even study 1 is much more complicated. It seems that rows 1 and 2 share a common group (mc=0.68 with nc=6). Then rows 3 and 4 share a common group (mt=0.84 with nt=9). But rows 1 and 4 and 2 and 3 also share a common group each. So, the var-cov matrix for this study would look like this:

[ 0.370 cov12     0 cov14 ]
[       0.278 cov23     0 ]
[             0.234 cov34 ]
[                   0.291 ]

In study 2, it would be:

[ 0.056 cov12     0 ]
[       0.070 cov23 ]
[             0.052 ]

Leaving aside the issue of whether SDs are getting pooled across all groups within a study or not, the covariance terms should be computable with the equation (19.19) from Gleser and Olkin (2009). I am not 100% sure though if the fact that the reference group is sometimes T and sometimes C matters. In any case, you should treat the group (whether it is actually a 'treatment' or a 'control' group as the 'control' group in equation (19.19)).

As for your ID variable and how to code it -- a single ID variable cannot capture these complexities. You will need 2 ID variables, indicating which group is the T and which is the C group. So, for study 1, it would be:

Study   mt      mc      nt      nc      TID CID    hedges.d    var
1       0.92    0.68    6       6       1   3      0.937       0.370
1       0.69    0.68    9       6       2   3      0.038       0.278
1       0.84    0.69    9       9       4   2      0.659       0.234
1       0.84    0.92    9       6       4   1     -0.623      0.291

Best,
Wolfgang

-----Original Message-----
From: Ju Lee [mailto:juhyung2 using stanford.edu]
Sent: Thursday, 19 September, 2019 1:24
To: Viechtbauer, Wolfgang (SP); r-sig-meta-analysis using r-project.org
Subject: Re: Covariance-variance matrix when studies share multiple treatment x control comparison

Dear Wolfgang and all,

Please ignore my previous e-mail,  and refer to this one instead:

Thank you very much for your response. I am using Hedges' d and I assumed this code applies to all SMD type of ES including Hedges' d?

I am still little baffled how I can apply this to more complex combinations. Below table looks busy, but I would appreciate if you can answer the following two questions. FYI, C_ID1 is the shared group ID.

1) In study 1, treatment 0.84 is shared in 3th and 4th response. To account for this inter-dependence, should I use "nt"  instead of "nc" for calculating cov.d matrix?

2) I am not sure how to proceed if treatments and controls are shared in fashion as study 2. Clearly, 1st/2nd response shares treatment (1.00) and 2nd/3rd shares  control (2.00), but 1st and 3rd do not share any experimental unit. I've categorized 2nd and 3rd with same C_ID1 because they share control, but am unclear if I should assign 1st response the same ID as 2nd and 3rd (c3) since it is still technically not independent from 2nd response (shared control) OR do I assign it a complete different control ID?

Study   mt     mc     nt    nc   C_ID1   hedges.d    var
1       0.92    0.68    6       6       c1      0.937       0.370
1       0.69    0.68    9       6       c1      0.038       0.278
1       0.84    0.69    9       9       c2      0.659       0.234
1       0.84    0.92    9       6       c2      -0.623      0.291
2       1.00   0.32    10     20      c?     0.760       0.056
2       1.00   2.00    10      5       c3     0.885       0.070
2       0.61   2.00    15      5       c3      0.209       0.052

I hope I am not asking too much of your time, but if you can answer this specific question, it will clarify all the doubts I have with this stage of analysis.

Thank you very much, and I hope to hear from you.

Best wishes,
JU

	[[alternative HTML version deleted]]