[R-meta] [External] Re: Do transformations of omnibus statistical tests show the correct direction of effect in the effect size values?

[Mills, Emma (Student)]

Dear James,

Thanks so much for your response.  I have done exactly that as I have been working.  It is a relief to me to know that this is a safe way to proceed.

Many thanks.

Emma

-----------------------------------------------------------------------------------------------

Emma Mills
PhD Student
Psychology Department, Lancaster University, Lancaster, LA1 4YF
Email: e.mills using lancaster.ac.uk; Telephone: +44 794 353 5855
twitter: EmmaMil26429683

Email too brief? Here's why! http://emailcharter.org
________________________________
From: James Pustejovsky <jepusto using gmail.com>
Sent: 12 January 2021 15:13
To: Mills, Emma (Student) <e.mills using lancaster.ac.uk>
Cc: r-sig-meta-analysis using r-project.org <r-sig-meta-analysis using r-project.org>
Subject: [External] Re: [R-meta] Do transformations of omnibus statistical tests show the correct direction of effect in the effect size values?


This email originated outside the University. Check before clicking links or attachments.

Hi Emma,

I would advise to carefully inspect your data and computed effect sizes to determine whether you need to make a further sign-transformation. For my own projects, I do this by creating tables (in Rmarkdown or by exporting to Excel) containing the raw summary statistics, direction of effect, and computed effect size, organized in a way to make it easy to check (e.g., by sorting the studies on direction of effect). Then I read through the table, looking for potential inconsistencies.

Even when using very well-validated software, I think this is a good general habit to build.

Kind Regards,
James

On Tue, Jan 12, 2021 at 8:31 AM Mills, Emma (Student) <e.mills using lancaster.ac.uk<mailto:e.mills using lancaster.ac.uk>> wrote:
Dear Meta-Analysis Forum

I should be very grateful if you could give me any guidance re the question below.
I've looked back in the archives over the past year but do not see anything that looks like this question.
I am a first time poster, so any other advice for making my posts more effective also greatly appreciated.

The data I have extracted for my meta-analysis includes F-ratio values, p-values, t-test values - where the original papers have omitted means and / or sd values.  Along with those values, I have collected the direction of the effect from the report narrative.

I have used esc​ / compute.es<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fcompute.es%2F&data=04%7C01%7Ce.mills%40lancaster.ac.uk%7Cd06a4c137df849ab42cd08d8b70ca9c2%7C9c9bcd11977a4e9ca9a0bc734090164a%7C0%7C0%7C637460612301083433%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=vq5EVW5SZysXE4SbKlRnzROnx2ulptoQUc4sAk%2Fy9Y4%3D&reserved=0>​ R packages to transform the collected data to either Hedges' g, log(odds) or fisher's z.

What I am not confident of is that the transformation of omnibus statistic values includes the appropriate direction of the effect, in the way that deriving the effect sizes from means so clearly does.

Should I be confident of the transformation? Or should I multiply the new value by -1 / 1 to ensure the direction of the effect is captured in the individual effect size and then continue with the meta-analysis?

Many thanks in anticipation of your response.  I hope you remain well during these strange times.

Emma Mills






-----------------------------------------------------------------------------------------------



Emma Mills
PhD Student
Psychology Department, Lancaster University, Lancaster, LA1
 4YF
Email: e.mills using lancaster.ac.uk<mailto:e.mills using lancaster.ac.uk>; Telephone: +44 794
 353 5855
twitter: EmmaMil26429683



Email too brief? Here's why! http://emailcharter.org<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Femailcharter.org%2F&data=04%7C01%7Ce.mills%40lancaster.ac.uk%7Cd06a4c137df849ab42cd08d8b70ca9c2%7C9c9bcd11977a4e9ca9a0bc734090164a%7C0%7C0%7C637460612301083433%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=y8u5AXnkXyTzcq1ywpoCcviY8b8Fgw8c5PJq3kuyMoE%3D&reserved=0>



_______________________________________________
R-sig-meta-analysis mailing list
R-sig-meta-analysis using r-project.org<mailto:R-sig-meta-analysis using r-project.org>
https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-meta-analysis&data=04%7C01%7Ce.mills%40lancaster.ac.uk%7Cd06a4c137df849ab42cd08d8b70ca9c2%7C9c9bcd11977a4e9ca9a0bc734090164a%7C0%7C0%7C637460612301093423%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=FIJP05dbpbFd%2B8MfgqBoZhR5fQ3ve%2FgyREMN421eKco%3D&reserved=0>

	[[alternative HTML version deleted]]