[R-meta] Change in results

[Viechtbauer, Wolfgang (SP)]

Dear Tina,

No idea why that happened and I have not heard about such cases before. I use a pretty extensive automated test suite to check that results are backwards compatible across updates. For certain functionality, I make use of some other packages, but most things are pretty self-contained within the metafor package and simply installing other packages should have no effect on metafor itself.

Best,
Wolfgang

-----Original Message-----
From: Tina Dudenhöffer [mailto:tina.dudenhoeffer using gmail.com] 
Sent: Friday, 15 November, 2019 17:38
To: Viechtbauer, Wolfgang (SP)
Cc: r-sig-meta-analysis using r-project.org
Subject: Change in results

Dear Wolfgang,

I have been working on a meta-analysis with the metafor package for several months now. 
Several weeks ago I sent my dataset include R script to my co-author to double check the calculations. The results were marginally different (probably due to the estimator). In the meantime I installed several other packages (unrelated to this project) and at one point I had to upgrade my R version. When I picked the Meta project up again I all of the sudden had completely different results (previously significant results were not significant anymore, very different estimates etc) than the first time (also very different from my co-author). Then I reinstalled R again and also installed the metafor package again and now my results are exactly in line with my co-author. So the results went from being marginally different to someone who ran the same script under the same package to being very different to being perfectly in line with the other person.

I was just curious as to why that is and what could have caused this changed in results (so I know what to avoid in the future). Unfortunately, I don’t remember exactly which packages I installed or what my R version was before I upgraded or re-installed and I know that this might make it difficult for you to answer my question.

Either way, maybe you have heard/experienced something like this before and could point me towards some causes.

Many thanks and kind regards,
Tina