[R-meta] [EXT] Re: Interpreting meta-regression results for dummy-coded variables

[Michael Dewey]

Please keep the list in all correspondence (I have added it) as someone 
else on the list may understand your reply, which, sadly, I do not and 
be able to answer.

Michael

On 20/06/2022 08:42, Acar, Selcuk wrote:
> Michael,
> 
> Thanks for your response--I am glad to have it corrected.
> 
> I do not do univariate analyses when I do meta-regression, but with this 
> perspective, they could help me figure if a moderator is significant or 
> not rather than a specific pair of categories is significantly from each 
> other. I do not do it because meta-regression provides more stringent 
> evidence than univariate analyses and when they conflict, I would still 
> go with the meta-regression results for interpretation. So, I question 
> the usefulness of such univariate analyses beyond checking if a 
> moderator is significant whose sub-categories turned out significant in 
> meta-regression.
> 
> Am I correct in my thinking/understanding of it?
> 
> Selcuk Acar, Ph.D.
> Associate Professor
> Department of Educational Psychology
> University of North Texas
> ------------------------------------------------------------------------
> *From:* Michael Dewey <lists using dewey.myzen.co.uk>
> *Sent:* Sunday, June 19, 2022 7:30 AM
> *To:* Acar, Selcuk <Selcuk.Acar using unt.edu>; 
> r-sig-meta-analysis using r-project.org <r-sig-meta-analysis using r-project.org>
> *Subject:* [EXT] Re: [R-meta] Interpreting meta-regression results for 
> dummy-coded variables
> Coments in-line
> 
> On 18/06/2022 23:14, Acar, Selcuk wrote:
>> Hi,
>> 
>> I ran a meta-regression in metafor package with both continuous and dummy-coded moderators. In some of the moderators, when we had only one dummy-code significant, we interpreted this as this moderator with several categories being significant. For example,  we had "participant group" moderator consisting of  "elementary" 
> "middle" "high" and "undergraduate" categories, and used 
> "undergraduates" as the reference group. We thought this moderator would 
> be significant even when only one of dummy codes "undergraduates vs 
> elementary" is significant without a separate test (linear hypothesis 
> testing).
>> 
>> One of the reviewers provided the following feedback:
>> 
>> "Because of the dummy-coding these coefficients are differences in Fisher-z-transformed correlations between the coded category and the reference category. Hence, this reporting could be more accurately reflect this. In addition, these tests of coefficients  are not a substitute for an overall test of the moderator. In other 
> words, the fact that one coefficient related to a moderator is 
> significant, does not imply that the moderator is significant. For 
> example, an overall test for Index of Creativity can be non-significant 
> even when a single coefficient such as the one for flexibility vs. 
> fluency is significant. Overall, moderator tests could be done by means 
> of linear hypothesis testing, for example."
>> 
> 
> That is correct. an overall test of the moderator is needed.
> 
>> In my opinion, running separate tests for each moderator kills the point of a meta-regression, and meta-regression should be the basis of the interpretation including these dummy-coded variables. I thought a categorical moderator would be significant even  when one of the dummy codes turn out significant.
>> 
> 
> I am afraid your thought is a mis-conception, albeit a common one.
> 
> Michael
> 
>> Who is correct here? Is there a good source that I could cite\?
>> 
>> I would appreciate input on this.
>> 
>> Selcuk Acar, Ph.D.
>> Associate Professor
>> Department of Educational Psychology
>> University of North Texas
>> 
>>        [[alternative HTML version deleted]]
>> 
>> _______________________________________________
>> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
>> To manage your subscription to this mailing list, go to:
>> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-meta-analysis&data=05%7C01%7CSelcuk.Acar%40unt.edu%7C322f70b684c046ff8ae108da51ef8f4a%7C70de199207c6480fa318a1afcba03983%7C0%7C0%7C637912386722870865%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UPv%2B7wy%2Bpfu5%2B2pGC0iF8Dn6lvi5qDSRE1EBxN7iNpU%3D&reserved=0 
> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-meta-analysis&data=05%7C01%7CSelcuk.Acar%40unt.edu%7C322f70b684c046ff8ae108da51ef8f4a%7C70de199207c6480fa318a1afcba03983%7C0%7C0%7C637912386722870865%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UPv%2B7wy%2Bpfu5%2B2pGC0iF8Dn6lvi5qDSRE1EBxN7iNpU%3D&reserved=0>
>> 
> 
> -- 
> Michael
> https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.dewey.myzen.co.uk%2Fhome.html&data=05%7C01%7CSelcuk.Acar%40unt.edu%7C322f70b684c046ff8ae108da51ef8f4a%7C70de199207c6480fa318a1afcba03983%7C0%7C0%7C637912386722870865%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=gYh189T0Z1gkDf%2FnovOR3FDro5dGysGQr5M5qXk68fc%3D&reserved=0 
> <https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.dewey.myzen.co.uk%2Fhome.html&data=05%7C01%7CSelcuk.Acar%40unt.edu%7C322f70b684c046ff8ae108da51ef8f4a%7C70de199207c6480fa318a1afcba03983%7C0%7C0%7C637912386722870865%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=gYh189T0Z1gkDf%2FnovOR3FDro5dGysGQr5M5qXk68fc%3D&reserved=0>
> 
> 
> ------------------------------------------------------------------------
> AVG logo <https://www.avg.com/internet-security> 	
> 
> This email has been checked for viruses by AVG antivirus software.
> www.avg.com <https://www.avg.com/internet-security>
> 
> 
> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

-- 
Michael
http://www.dewey.myzen.co.uk/home.html