[R-meta] Pairwise comparisons between the factor levels and compact letter display (cld)

[Jan Kardela (PGR)]

Thanks for the message Wolfgang and for the explanation of the error - it actually helped me to solve the problem.

I managed to get the code running after a a bit of 'reverse-engineering'. Basically, as the cld function requires a model object I had to change original code so it could accept a list with data necessary to run the main algorithm. I am happy to present the running code but it's a bit lengthy, so maybe it's better if I email it directly to interested users (?).

Kind regards,
Jan

-----Original Message-----
From: Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer using maastrichtuniversity.nl> 
Sent: Thursday, August 16, 2018 9:07 AM
To: Jan Kardela (PGR) <J.H.Kardela2 using newcastle.ac.uk>; r-sig-meta-analysis using r-project.org
Subject: RE: [R-meta] Pairwise comparisons between the factor levels and compact letter display (cld)

Thanks. I actually get a different error:

Error in terms.default(formula, data = data) : 
  no terms component nor attribute

Make sure you have the 'devel' version of metafor installed:

https://github.com/wviechtb/metafor#installation

Not that this fixes this problem, but it is the more up-to-date version.

Indeed, 'rma' objects do not contain a 'terms' component (for reasons that are a bit difficult to explain here). Since cld() apparently needs this, it looks like cld() cannot currently be used in combination with results from metafor.

Best,
Wolfgang

-----Original Message-----
From: Jan Kardela (PGR) [mailto:J.H.Kardela2 using newcastle.ac.uk] 
Sent: Wednesday, 15 August, 2018 18:37
To: Viechtbauer, Wolfgang (SP); r-sig-meta-analysis using r-project.org
Subject: RE: [R-meta] Pairwise comparisons between the factor levels and compact letter display (cld)

Dear Wolfgang,
Thank you for your message. 

Below I provide the snippet of my data together with the code I had been using to calculate all the statistics. 

Maybe just a little explanation of my data set as it might seem strange at first. I have run 5 independent settlement studies each using different populations of larvae. In each study there are 4 different samples (A (c), B, C, D), each consisting of few replicates (n1). Every single settlement experiment was conducted using different set of "participants" i.e. larvae. As a control group for each experiment I selected sample A (c) from appropriate Study. In case of sample C (experiments 11,12,13,14) there was no settlement so I decided to put the 0.01 values as for mean and stddev.    

In order to merge the results from my studies I conducted meta-regression analysis with the Sample as a categorical moderator and used random switch to allow the amount of residual heterogeneity to be different in each Sample subgroup.

As I mentioned earlier I wanted to run some sort of Tukey post-hoc test on the meta-regression model to determine differences between the Sample levels. In order to do that I ran glht() which gave me all pairwise comparisons, but with so many pairs (my full data set consists of 11 different samples/levels not 4 as in snippet) I wanted to use compact letter display notification to more clearly present the differences between the samples. Obviously, I could use the plot function to graphically display all pairs of groups but in my case some of the groups overlap with others making the story not very clear. 

Thank you again for your time and expertise!

Best regards,
Jan

#### ==== BEGINNING OF CODE ==== ####

library(metafor) 
library(multcomp)

df<-data.frame(Study=c(1,2,3,4,5,1,2,3,4,5,1,2,3,4,5,1,2,3,4,5),
               Sample=c("A (c)", "A (c)", "A (c)","A (c)", "A (c)", "B","B","B","B","B","C","C","C","C","C","D","D","D","D","D"),
               Experiment=c(1:20),
               mean1=c(19.87,59.91,75.73,16.78,83.1,4.92,5.37,23.68,23.68,9.68,0.01,0.01,0.01,0.01,8,12.11,86.67,63.45,37.87,67),
               stddev1=c(15.03,13.8,2.89,6.05,11.66,5,5.56,11.16,3.72,8.75,0.01,0.01,0.01,0.01,9.23,13.28,7.64,2.19,1.42,23.86),
               n1=c(3,3,2,2,8,3,3,2,2,8,3,3,2,2,8,3,3,2,2,8),
               mean2=c(19.87,59.91,75.73,16.78,83.1,19.87,59.91,75.73,16.78,83.1,19.87,59.91,75.73,16.78,83.1,19.87,59.91,75.73,16.78,83.1),
               stddev2=c(15.03,13.8,2.89,6.05,11.66,15.03,13.8,2.89,6.05,11.66,15.03,13.8,2.89,6.05,11.66,15.03,13.8,2.89,6.05,11.66),
               n2=c(3,3,2,2,8,3,3,2,2,8,3,3,2,2,8,3,3,2,2,8),
               Type=c("NI","NI","NI","NI","NI","I","I","I","I","I","I","I","I","I","I","NI","NI","NI","NI","NI"))

df

escalc.dat<-escalc(m1i=mean1,m2i=mean2,n1i=n1,n2i=n2,sd1i=stddev1,sd2i=stddev2,measure="MD",method="REML",data=df) 
escalc.dat

rma.meta<-rma.mv(yi,vi,mods=~factor(Sample)-1,random=~Sample|Experiment,struct="DIAG",data=escalc.dat,method="REML",digits=3)
rma.meta 

tukey_glht<-glht(rma.meta, linfct=cbind(contrMat(c("A (c)"=1,"B"=1,"C"=1,"D"=1),type="Tukey")),test=adjusted("Bonferroni"))
summary(tukey_glht)
plot(tukey_glht)
cld(tukey_glht) ## That's where I get the error: "Error in formula.default(object, env = baseenv()) : invalid formula"

#### ====END OF CODE==== ####

-----Original Message-----
From: Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer using maastrichtuniversity.nl> 
Sent: Tuesday, August 14, 2018 9:09 PM
To: Jan Kardela (PGR) <J.H.Kardela2 using newcastle.ac.uk>; r-sig-meta-analysis using r-project.org
Subject: RE: [R-meta] Pairwise comparisons between the factor levels and compact letter display (cld)

Hi Jan,

Can you provide a small reproducible example that yields this error? I have never tried to see how well metafor objects play together with cld().

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On Behalf Of Jan Kardela (PGR)
Sent: Monday, 13 August, 2018 12:01
To: r-sig-meta-analysis using r-project.org
Subject: [R-meta] Pairwise comparisons between the factor levels and compact letter display (cld)

Hello.
I am doing a multilevel meta-analysis in metafor and I am running a glht function from multcomp package to determine all-pairwise comparisons. I use contrMat function with Tukey switch to obtain the matrix contrast. As my categorical factor has 11 levels I would like to have some sort of compact letter display notification (cld) to more efficiently report the differences between all pairs. When I try to feed the glht model to the cld function I get an error "Error in formula.default(object, env = baseenv()) : invalid formula". Could someone please point me in the right direction and provide some advice?
Thank you in advance for all the help!
Best regards,
Jan