[R-meta] Meta-analysis of linear and non-linear associations: R2 as the “effect size”?

Tue Mar 16 12:48:01 CET 2021

p.p.s. to be absolutely clear, what I mean with meta-analyzing across models is: include models that model a linear relationship (a straight line), a "nonlinear" relationship (a curve made up of several straight lines, as in a polynomial) and nonlinear relationships modelled by other "curves" (for instance, a power function, even though I doubt that anyone has actually done so).

Hope this addition makes it clearer what I mean!

-----Oorspronkelijk bericht-----
Van: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> Namens Werner, M.A. (Marlene)
Verzonden: dinsdag 16 maart 2021 12:36
Aan: Michael Dewey <lists using dewey.myzen.co.uk>; r-sig-meta-analysis using r-project.org
Onderwerp: Re: [R-meta] Meta-analysis of linear and non-linear associations: R2 as the “effect size”?

Dear Prof. Dewey, dear all,

Thank you for your time, help, and thoughtful response.

Nonlinear versus linear: I indeed know that "non-linear" models are still linear models in essence, or actually "in part"; good that the quotation marks seem to communicate that correctly!

R^2 for nonlinear models: I read on several tutorial sites that R^2 should not be interpreted for "nonlinear" models, for instance: https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstatisticsbyjim.com%2Fregression%2Fr-squared-invalid-nonlinear-regression%2F&data=04%7C01%7Cm.a.werner%40amsterdamumc.nl%7C77ee5a216b97497fd75a08d8e86fcaaa%7C68dfab1a11bb4cc6beb528d756984fb6%7C0%7C0%7C637514914127685801%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=P7SdOkezVB1k73Vd7fqXrRw%2BR4GWhmDLjBj0O4d5KHk%3D&reserved=0
To be honest, I was a bit confused by that, because I have also read that one compares nonlinear to linear models within a sample by means of the R^2.

Use in meta-analysis: "R^2 is something which could arise in many ways in the polynomial models " - I see! A high R^2 could be resulting from good linear fit at particular points ("linear trendlines") in the data. Thank you! I have not thought of that possibility. It also makes me understand better why R^2 might be problematic for comparing linear vs nonlinear within a dataset.

Do you have any idea whether there is any way to meta analyze across these models - or do I have to abandon this idea?

Thank you!

All the best,

Marlene

p.s. I will update the stackoverflow page with the suggestions made here and give credit to the mailing list (or you personally if you like)?

-----Oorspronkelijk bericht-----
Van: Michael Dewey <lists using dewey.myzen.co.uk>
Verzonden: zondag 14 maart 2021 14:31
Aan: Werner, M.A. (Marlene) <m.a.werner using amsterdamumc.nl>; r-sig-meta-analysis using r-project.org
Onderwerp: Re: [R-meta] Meta-analysis of linear and non-linear associations: R2 as the “effect size”?

Some comments in-line

On 12/03/2021 09:02, Werner, M.A. (Marlene) wrote:
> Dear all,
> 
> I have a question regarding meta-analysis of effect sizes of associations.
> I was wondering whether it is possible to somehow quantitatively summarize results across studies that modelled the association between two variables "linearly" (e.g. using simple linear regression) vs. others that modelled the association "non-linearly" (e.g. polynomials or splines). One variable is continuous (hormone concentration) and the other is ordinal (self-report of well-being).

Note that if you include, say, a quadratic for hormone concentration it is still a linear model. I suspect you know that as you put "non-linearly" in quotes.

> For instance, would it be possible to meta-analyze the (adjusted) R2 as the "effect size"? If so, how would one go about the meta-analytic steps?
> I figured that meta-analysis of proportions might be applicable, but then I would need the counts on which the proportion is based, which is not directly available for this  proportion , especially not in the non-linear case.
> 
> I started thinking about meta-analyzing "explained variance" in the first place because I cannot summarize the "regression coefficients" across these models;  non-linear models  include several coefficients, whereas  linear models  only provide one.
> Then again, R2 is not really applicable as a goodness of fit measure for non-linear models.

I do not think that is true for the reason stated above.

  So which other "summary statistic" to use that is available and comparable for these models, even when comparing them in a non-quantitative way? Would the Standard Error of the Regression be an option? But how to standardize it to make it comparable across studies?
> 
> I would be very grateful for any ideas or pointers! Thanks in advance!

The main problem I see here is that R^2 is something which could arise in many ways in the polynomial models. It just tells you how much variance in well being is explained by knowing hormone concentration but that might be either a strong linear effect, a strong quadratic, both, or something else.

> 
> (Note that I have posted the same q on stackexchange: 
> https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat
> s.stackexchange.com%2Fquestions%2F513164%2Fmeta-analysis-of-linear-and
> -non-linear-associations-r2-as-the-effect-size&data=04%7C01%7Cm.a.
> werner%40amsterdamumc.nl%7C77ac4fa0b47a4809660708d8e6ed5d94%7C68dfab1a
> 11bb4cc6beb528d756984fb6%7C0%7C0%7C637513254456249306%7CUnknown%7CTWFp
> bGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn
> 0%3D%7C1000&sdata=LnkB57gopZ6zRfmhiD2ef8ywUQK5izvZ8M6TYJF2zUk%3D&a
> mp;reserved=0?; I hope that is ok!)
> 

As long as you tell us cross-posting is OK although there is a danger that someone might answer on CV not knowing about this post.

Michael

> All the best,
> 
> Marlene Werner
> 
> Drs. M.A. (Marlene) Werner  |  PhD candidate Department of Gynaecology 
> and Obstetrics / Sexology and Psychosomatic Gynaecology Location AMC |
> H4-236 | Meibergdreef 9, 1105 AZ Amsterdam
> E: m.a.werner using amsterdamumc.nl<mailto:m.a.werner using amsterdamumc.nl>
> http://www.amsterdamumc.nl/<http://www.amsterdamumc.nl/>
> 
> ______________________________________________________
> VUmc disclaimer : http://www.vumc.nl/disclaimer AMC disclaimer : 
> http://www.amc.nl/disclaimer
> 
> 	[[alternative HTML version deleted]]
> 
> 
> 
> 
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis using r-project.org
> https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat
> .ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-meta-analysis&data=04%7C01%7
> Cm.a.werner%40amsterdamumc.nl%7C77ac4fa0b47a4809660708d8e6ed5d94%7C68d
> fab1a11bb4cc6beb528d756984fb6%7C0%7C0%7C637513254456249306%7CUnknown%7
> CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXV
> CI6Mn0%3D%7C1000&sdata=EBo%2F58n4GUXh19R2u9p7BpctBMt5tEyJAEief0%2B
> yGlE%3D&reserved=0
> 

--
Michael
https://eur04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.dewey.myzen.co.uk%2Fhome.html&data=04%7C01%7Cm.a.werner%40amsterdamumc.nl%7C77ee5a216b97497fd75a08d8e86fcaaa%7C68dfab1a11bb4cc6beb528d756984fb6%7C0%7C0%7C637514914127685801%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=vQkW9rh1WBquwrGV%2FEE2mMRFlK8Fb7Da2Sdi3VyRoBU%3D&reserved=0
_______________________________________________
R-sig-meta-analysis mailing list
R-sig-meta-analysis using r-project.org
https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-meta-analysis&data=04%7C01%7Cm.a.werner%40amsterdamumc.nl%7C77ee5a216b97497fd75a08d8e86fcaaa%7C68dfab1a11bb4cc6beb528d756984fb6%7C0%7C0%7C637514914127685801%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=M85mbqXNA4jsgKMNwxok7C6mw56%2FPZZRNDtlwuxM63A%3D&reserved=0