[R-meta] Subgroup comparison for non-linear meta-regression: Clarification
Mustafa Kilickap
mkilickap at yahoo.com
Thu Oct 26 15:48:52 CEST 2017
Dear all,
In my previous e-mail I wrote a formula that requires clarification:
The formula was z=(p1-p1)/sqrt(se1^2 +se2^2)
where, p1 and p2 denotes beta estimates of the non-linear fit for females and males; se1 and se2 denote corresponding standard errors.
Sorry for that problem.
Regards
Mustafa Kilickap
On Thursday, October 26, 2017 4:21 PM, "r-sig-meta-analysis-request at r-project.org" <r-sig-meta-analysis-request at r-project.org> wrote:
Send R-sig-meta-analysis mailing list submissions to
r-sig-meta-analysis at r-project.org
To subscribe or unsubscribe via the World Wide Web, visit
https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
or, via email, send a message with subject or body 'help' to
r-sig-meta-analysis-request at r-project.org
You can reach the person managing the list at
r-sig-meta-analysis-owner at r-project.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of R-sig-meta-analysis digest..."
Today's Topics:
1. Subgroup comparison for non-linear meta-regression
(Mustafa Kilickap)
2. Re: complex data structure for a meta-analysis (Yogev Kivity)
3. Re: Subgroup comparison for non-linear meta-regression
(Viechtbauer Wolfgang (SP))
4. Re: complex data structure for a meta-analysis
(Viechtbauer Wolfgang (SP))
----------------------------------------------------------------------
Message: 1
Date: Thu, 26 Oct 2017 12:44:58 +0000 (UTC)
To: "r-sig-meta-analysis at r-project.org"
<r-sig-meta-analysis at r-project.org>
Subject: [R-meta] Subgroup comparison for non-linear meta-regression
Message-ID: <1926775132.5242675.1509021898318 at mail.yahoo.com>
Content-Type: text/plain; charset=UTF-8
Dear all,
I did a ?non-linear? meta-regression for prevalence (proportion) on time variable (year) using a power function. The studies consist of information for male and female participants. I would like to assess
whether the non-linear fit was similar for male and female. In other words, I would like to
make a comparison for male and female.
I think it is more appropriate to do a meta-regression including gender*time interaction, however, number of the studies is small. My questions are:
1) Is it acceptable just to compare the subgroups (as stated below) without using interaction term. If it is acceptable;
2) Does metafor package has a command like ?byvar? that makes a statistical comparison of two subgroups possible? If not,
2) Is it appropriate to calculate z=(p1-p1)/sqrt(se1^2 +se2^2) and then p for a ?non-linear? relationship?
Kind regards
Mustafa Kilickap
------------------------------
Message: 2
Date: Thu, 26 Oct 2017 12:53:36 +0000 (UTC)
To: "r-sig-meta-analysis at r-project.org"
<r-sig-meta-analysis at r-project.org>, "Viechtbauer Wolfgang (SP)"
<wolfgang.viechtbauer at maastrichtuniversity.nl>
Subject: Re: [R-meta] complex data structure for a meta-analysis
Message-ID: <1171473548.4810383.1509022416823 at mail.yahoo.com>
Content-Type: text/plain; charset="UTF-8"
Hi Wolfgang!
[[elided Yahoo spam]]
I want to elaborate more on 1a. I think what I'm struggling to understand is how our design of 4-level model affects the way I should construct the V matrix. Again, we have multivariate effects that are nested within group, which in turn, are nested within studies. Can I assume that effect sizes of different groups within the same study are independent as far as their sampling errors are concerned?
Best,Yogev
--
Yogev Kivity, Ph.D.
Postdoctoral Fellow
Department of Psychology
The Pennsylvania State University
Bruce V. Moore Building
University Park, PA 16802
Office Phone: (814) 867-2330
On Thursday, October 26, 2017, 4:35:45 AM EDT, Viechtbauer Wolfgang (SP) <wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
1) Multivariate data have in essence automatically a multilevel structure. When I use the term 'multivariate', I use it to denote a dataset where multiple estimates are computed for the same sample (e.g., due to multiple dependent variables, multiple time points, multiple treatment groups compared against a single control group) and hence the sampling errors are no longer independent. Besides the dependency in the sampling errors, the data have a multilevel structure, since the estimates are nested within studies.
a) In multivariate data, there are two sources of dependency: The dependency in the sampling errors (which should be incorporated into the V matrix) and potential dependency in the underlying true effects -- the latter can be accounted for by using appropriate random effects. So, in your case, it seems like you would want random intercepts for study, group, and each individual estimate.
b) Yes, that is fine. Berkey et al. (1998) is a special case where we have both outcomes for each study. But that's not necessary (and in fact rather unusual).
If there are a lot of different outcome types, you may not be able to use a fully unstructured var-cov matrix for the random effects (i.e., struct="UN").
2) You can always add up or take the average of variance components if you want to compute some kind of overall I^2-type measure. I^2 is just a proportion (out of some measure of total variability), so the principle is easily extended to pretty much any model.
Best,
Wolfgang
-----Original Message-----
Sent: Wednesday, 25 October, 2017 21:56
To: r-sig-meta-analysis at r-project.org; Viechtbauer Wolfgang (SP)
Subject: Re: RE: [R-meta] complex data structure for a meta-analysis
Dear Wolfgang and all,
Thanks a lot for your detailed reply - this is very helpful! I have read the posts in the links and I feel like I have a better sense of the issues I am dealing with now. I do have several outstanding issues that I am hoping to get peoples? opinion on.
1. Yes, my data are indeed multivariate, but I think it is also multilevel, because most studies had multiple groups (i.e. treatment arms) nested within the study. We are interested in examining the correlation between attachment style and treatment outcome within each group, and then as a second step to examine whether the effect size differs by treatment type (group level moderator). In addition, there is substantial variability across studies in the number of outcome measures (e.g., some report only one while others report 4-5) as well as the construct being measured (e.g., some measure depression and anxiety, while others measure functioning and quality of life). These issues raise several questions:
a. is there a way to account for the dependency between groups (in addition to simply including group as a level in the model)?
b. Say I choose to specify a ?guestimated? var-cov matrix to account for the dependencies among sampling variances, is it ok to have a different number of outcome measures, and different types of outcome measures in each study? The Berkey et al., 1998 study, for example, had the same number and type of outcome measures for all studies.
2. regarding I^2 ? again, in my case (see above) I have both multilevel and multivariate data so I am not sure if the approaches mentioned in the link may be appropriate for my data. In general, I am interested in a general I^2 across outcome measures, and not necessarily by measure.
Thanks in advance,
Yogev
--
Yogev Kivity, Ph.D.
Postdoctoral Fellow
Department of Psychology
The Pennsylvania State University
Bruce V. Moore Building
University Park, PA 16802
Office Phone: (814) 867-2330
On Saturday, September 23, 2017, 5:51:56 PM EDT, Viechtbauer Wolfgang (SP) <wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
1) You do have what I would call 'multivariate' data -- that is, multiple outcomes for the same sample. Since you say that you do not have the information needed in order to compute the covariances between multiple outcomes for the same sample, a possible strategy to account for the dependency is the use cluster robust inferences. This has been discussed at quite some length in previous posts on this mailing list, so I would encourage you to look through the archives.
2) Similarly, this has come up before. There is no general concensus on how I^2 can be extended to more complex models. I have written down some ideas here:
http://www.metafor-project.org/doku.php/tips:i2_multilevel_multivariate
I cannot tell you whether this is the best/right way.
3) A standard funnel plot isn't wrong, it just does not give any indication of what points are independent vs dependent. I am not aware of any suggestions on how a funnel plot could be drawn that does provide such an indication (maybe connecting dependent estimates with lines?).
Best,
Wolfgang
-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On Behalf Of Yogev Kivity
Sent: Wednesday, 20 September, 2017 13:42
To: r-sig-meta-analysis at r-project.org
Subject: [R-meta] complex data structure for a meta-analysis
Hi everyone,I am running a meta-analysis using 'metafor' and I came across several questions that I could not find answers
for in 'metafor's documentation.
In short, were are examining psychotherapy data, and how a specific measure collected at the beginning of treatment (attachment style in relationships) predicts outcome of therapy as measured at post-treatment. Both measures are usually dimensional, so we are using Pearson's r which we then convert to Fisher's z.
The design of the meta-analysis is multilevel and multivariate in that each study usually includes several different treatment groups with different patients, as well as several subscales of attachment (e.g., level of anxiety in attachment and level of avoidance in attachment) and several measures of outcome at post-treatment (e.g., anxiety, depression etc.). This is complicated by the fact that studies rarely use the same attachment and outcome measures, and for the most part, we do not have data on the covariance among these measures.
I am assuming that our design is most similar to Konstantopoulos (2011), but we have an additional level of effect sizes repeated within groups, so basically we have multiple effect size per treatment arm, nested within treatment arm, which in are turn nested within study. Would that be correct?
My main questions are:
1. what would be the best approach for modeling all of these levels of analyses, while taking into account the fact that the effect sizes within treatment arm are likely no independent. My understanding is that usually multivariate is interpreted to mean multiple outcome measures, but in our case we have multiple outcome as well as multiple predictors.
2. How should I squared be calculated for such models?
3. is there an extension of funnel plots to multi-level models that could reliably represent the data? I guess that using the standard funnel plot ignores the mutilevel structure of the data, is that correct?
Best,
Yogev
_______________________________________________
R-sig-meta-analysis mailing list
R-sig-meta-analysis at r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
[[alternative HTML version deleted]]
------------------------------
Message: 3
Date: Thu, 26 Oct 2017 13:12:35 +0000
From: "Viechtbauer Wolfgang (SP)"
<wolfgang.viechtbauer at maastrichtuniversity.nl>
To: "r-sig-meta-analysis at r-project.org"
<r-sig-meta-analysis at r-project.org>
Subject: Re: [R-meta] Subgroup comparison for non-linear
meta-regression
Message-ID: <92bb53c534c4422793f17f489b3bb00d at UM-MAIL3216.unimaas.nl>
Content-Type: text/plain; charset="utf-8"
Dear Mustafa,
Subgrouping and then comparing the estimates across subgroups is in no way more efficient than running the model with the interaction. In fact, if one runs the model with the interaction and allow the amount of heterogeneity to differ across subgroups, then the two approaches are identical. See here:
http://www.metafor-project.org/doku.php/tips:comp_two_independent_estimates
Best,
Wolfgang
-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On Behalf Of Mustafa Kilickap
Sent: Thursday, 26 October, 2017 14:45
To: r-sig-meta-analysis at r-project.org
Subject: [R-meta] Subgroup comparison for non-linear meta-regression
Dear all,
I did a ?non-linear? meta-regression for prevalence (proportion) on time variable (year) using a power function. The studies consist of information for male and female participants. I would like to assess
whether the non-linear fit was similar for male and female. In other words, I would like to
make a comparison for male and female.
I think it is more appropriate to do a meta-regression including gender*time interaction, however, number of the studies is small. My questions are:
1) Is it acceptable just to compare the subgroups (as stated below) without using interaction term. If it is acceptable;
2) Does metafor package has a command like ?byvar? that makes a statistical comparison of two subgroups possible? If not,
2) Is it appropriate to calculate z=(p1-p1)/sqrt(se1^2 +se2^2) and then p for a ?non-linear? relationship?
Kind regards
Mustafa Kilickap
------------------------------
Message: 4
Date: Thu, 26 Oct 2017 13:21:40 +0000
From: "Viechtbauer Wolfgang (SP)"
<wolfgang.viechtbauer at maastrichtuniversity.nl>
To: "r-sig-meta-analysis at r-project.org"
<r-sig-meta-analysis at r-project.org>
Subject: Re: [R-meta] complex data structure for a meta-analysis
Message-ID: <648fe8477a4d415394699929c4eb77f8 at UM-MAIL3216.unimaas.nl>
Content-Type: text/plain; charset="utf-8"
Yes, as long as the different groups consist of different subjects (no overlap at all), then the sampling errors can be assumed to be independent. So, for example, if, for a given study, we have an effect size estimate for the subgroup of males and an effect size estimate for the subgroup of females, then there is no overlap and the sampling errors are independent.
On the other hand, when the same group is measured with respect to multiple outcomes or at multiple time points, then there is obviously overlap (in fact, it's the exact same group). Another case of overlap occurs when you have a group of subjects subdivided into, let's say, 3 groups and one computes two effect sizes (e.g., standardized mean differences, log odds/risk ratios, etc.) that contrasts group 1 with group 3 and group 2 with group 3 (i.e., the 'multiple treatment group case'). In all those cases, sampling errors are no longer independent.
Best,
Wolfgang
-----Original Message-----
Sent: Thursday, 26 October, 2017 14:54
To: r-sig-meta-analysis at r-project.org; Viechtbauer Wolfgang (SP)
Subject: Re: [R-meta] complex data structure for a meta-analysis
Hi Wolfgang!
[[elided Yahoo spam]]
I want to elaborate more on 1a. I think what I'm struggling to understand is how our design of 4-level model affects the way I should construct the V matrix. Again, we have multivariate effects that are nested within group, which in turn, are nested within studies. Can I assume that effect sizes of different groups within the same study are independent as far as their sampling errors are concerned?
Best,
Yogev
--
Yogev Kivity, Ph.D.
Postdoctoral Fellow
Department of Psychology
The Pennsylvania State University
Bruce V. Moore Building
University Park, PA 16802
Office Phone: (814) 867-2330
On Thursday, October 26, 2017, 4:35:45 AM EDT, Viechtbauer Wolfgang (SP) <wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
1) Multivariate data have in essence automatically a multilevel structure. When I use the term 'multivariate', I use it to denote a dataset where multiple estimates are computed for the same sample (e.g., due to multiple dependent variables, multiple time points, multiple treatment groups compared against a single control group) and hence the sampling errors are no longer independent. Besides the dependency in the sampling errors, the data have a multilevel structure, since the estimates are nested within studies.
a) In multivariate data, there are two sources of dependency: The dependency in the sampling errors (which should be incorporated into the V matrix) and potential dependency in the underlying true effects -- the latter can be accounted for by using appropriate random effects. So, in your case, it seems like you would want random intercepts for study, group, and each individual estimate.
b) Yes, that is fine. Berkey et al. (1998) is a special case where we have both outcomes for each study. But that's not necessary (and in fact rather unusual).
If there are a lot of different outcome types, you may not be able to use a fully unstructured var-cov matrix for the random effects (i.e., struct="UN").
2) You can always add up or take the average of variance components if you want to compute some kind of overall I^2-type measure. I^2 is just a proportion (out of some measure of total variability), so the principle is easily extended to pretty much any model.
Best,
Wolfgang
-----Original Message-----
Sent: Wednesday, 25 October, 2017 21:56
To: r-sig-meta-analysis at r-project.org; Viechtbauer Wolfgang (SP)
Subject: Re: RE: [R-meta] complex data structure for a meta-analysis
Dear Wolfgang and all,
Thanks a lot for your detailed reply - this is very helpful! I have read the posts in the links and I feel like I have a better sense of the issues I am dealing with now. I do have several outstanding issues that I am hoping to get peoples? opinion on.
1. Yes, my data are indeed multivariate, but I think it is also multilevel, because most studies had multiple groups (i.e. treatment arms) nested within the study. We are interested in examining the correlation between attachment style and treatment outcome within each group, and then as a second step to examine whether the effect size differs by treatment type (group level moderator). In addition, there is substantial variability across studies in the number of outcome measures (e.g., some report only one while others report 4-5) as well as the construct being measured (e.g., some measure depression and anxiety, while others measure functioning and quality of life). These issues raise several questions:
a. is there a way to account for the dependency between groups (in addition to simply including group as a level in the model)?
b. Say I choose to specify a ?guestimated? var-cov matrix to account for the dependencies among sampling variances, is it ok to have a different number of outcome measures, and different types of outcome measures in each study? The Berkey et al., 1998 study, for example, had the same number and type of outcome measures for all studies.
2. regarding I^2 ? again, in my case (see above) I have both multilevel and multivariate data so I am not sure if the approaches mentioned in the link may be appropriate for my data. In general, I am interested in a general I^2 across outcome measures, and not necessarily by measure.
Thanks in advance,
Yogev
--
Yogev Kivity, Ph.D.
Postdoctoral Fellow
Department of Psychology
The Pennsylvania State University
Bruce V. Moore Building
University Park, PA 16802
Office Phone: (814) 867-2330
On Saturday, September 23, 2017, 5:51:56 PM EDT, Viechtbauer Wolfgang (SP) <wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
1) You do have what I would call 'multivariate' data -- that is, multiple outcomes for the same sample. Since you say that you do not have the information needed in order to compute the covariances between multiple outcomes for the same sample, a possible strategy to account for the dependency is the use cluster robust inferences. This has been discussed at quite some length in previous posts on this mailing list, so I would encourage you to look through the archives.
2) Similarly, this has come up before. There is no general concensus on how I^2 can be extended to more complex models. I have written down some ideas here:
http://www.metafor-project.org/doku.php/tips:i2_multilevel_multivariate
I cannot tell you whether this is the best/right way.
3) A standard funnel plot isn't wrong, it just does not give any indication of what points are independent vs dependent. I am not aware of any suggestions on how a funnel plot could be drawn that does provide such an indication (maybe connecting dependent estimates with lines?).
Best,
Wolfgang
-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On Behalf Of Yogev Kivity
Sent: Wednesday, 20 September, 2017 13:42
To: r-sig-meta-analysis at r-project.org
Subject: [R-meta] complex data structure for a meta-analysis
Hi everyone,I am running a meta-analysis using 'metafor' and I came across several questions that I could not find answers
for in 'metafor's documentation.
In short, were are examining psychotherapy data, and how a specific measure collected at the beginning of treatment (attachment style in relationships) predicts outcome of therapy as measured at post-treatment. Both measures are usually dimensional, so we are using Pearson's r which we then convert to Fisher's z.
The design of the meta-analysis is multilevel and multivariate in that each study usually includes several different treatment groups with different patients, as well as several subscales of attachment (e.g., level of anxiety in attachment and level of avoidance in attachment) and several measures of outcome at post-treatment (e.g., anxiety, depression etc.). This is complicated by the fact that studies rarely use the same attachment and outcome measures, and for the most part, we do not have data on the covariance among these measures.
I am assuming that our design is most similar to Konstantopoulos (2011), but we have an additional level of effect sizes repeated within groups, so basically we have multiple effect size per treatment arm, nested within treatment arm, which in are turn nested within study. Would that be correct?
My main questions are:
1. what would be the best approach for modeling all of these levels of analyses, while taking into account the fact that the effect sizes within treatment arm are likely no independent. My understanding is that usually multivariate is interpreted to mean multiple outcome measures, but in our case we have multiple outcome as well as multiple predictors.
2. How should I squared be calculated for such models?
3. is there an extension of funnel plots to multi-level models that could reliably represent the data? I guess that using the standard funnel plot ignores the mutilevel structure of the data, is that correct?
Best,
Yogev
------------------------------
Subject: Digest Footer
_______________________________________________
R-sig-meta-analysis mailing list
R-sig-meta-analysis at r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
------------------------------
End of R-sig-meta-analysis Digest, Vol 5, Issue 32
More information about the R-sig-meta-analysis
mailing list