[R-meta] Extracting values from a list of confint() objects
Will Hopkins
w|||thek|w| @end|ng |rom gm@||@com
Tue Mar 12 21:36:54 CET 2024
Thanks for all the work, Tobias. I tried your program for tau2, and it worked. I had already adapted my (and ChatGPT's) program to do also the coverage of the fixed effects, which is also working. Your program seems to be more elegant, in that it produces list objects separately with rma, selmodel and confint, rather than using the for loop. I get lost more with your program than mine, but I will see if I can come to terms with it and implement it for the fixed effects.
Yesterday, Michael Dewey made the point that "I doubt whether the standard error of tau2 is useful for much and certainly not for establishing a confidence interval. I think Wolfgang alluded to that in a previous e-mail." Yes, but SAS is happy enough to create confidence limits for random effects by assuming normality of the sampling distribution, and over the years I have done many simulations to show that it works well. I have implemented it here with the tau2 and its SE provided by selmodel, and it works practically perfectly. See below for the results for the fixed and random effects from my program so far, with a dataset generated by SAS with publication bias simulated by selecting all significant study estimates and 20% of non-significant study estimates. In the 2500 simulations, on average there were 12 significant and 2 non-significant female study estimates, and 13 significant and 14 non-significant male study estimates, for on average a total of 40 study-estimates (range 24-55) of mean sample size 13 (range 10-30) per simulation. The true means for females and males were 3 and 1, and the true heterogeneity SD was 0.5 (as shown below). We have implemented the PEESE approach in SAS (adjusting for publication bias with an SE^2 moderator), and for this simulation it's not quite as good as the "3PSM" implemented with selmodel: it adjusts perfectly for the males, but it over-adjusts for the females and for the heterogeneity.
Will
Here are the results. I have also attached as an easy-to-read text file generated by R using the sink("results.txt")… sink() approach suggested by ChatGPT (is there a better way?).
Female mean provided by selmodel:
SexFemale lb ub TrueMean Coverage
3.11 2.53 3.69 3.00 87.96
Non-missing values for the above:
SexFemale lb ub TrueMean Coverage
2500 2483 2483 2500 2483
Male mean provided by selmodel:
SexMale lb ub TrueMean Coverage
1.18 0.63 1.73 1.00 78.70
Non-missing values for the above:
SexMale lb ub TrueMean Coverage
2500 2483 2483 2500 2483
Tau^2 with CLs provided by confint applied to selmodel-adjusted estimates:
estimate ci.lb ci.ub TrueSD2 Coverage
0.46 0.13 1.16 0.25 76.88
Number of simulations with non-missing values for the above:
estimate ci.lb ci.ub TrueSD2 Coverage
2500 2500 2500 2500 2500
Tau with CLs provided by confint applied to selmodel-adjusted estimates:
estimate ci.lb ci.ub TrueSD Coverage
0.61 0.27 1.04 0.50 76.88
Number of simulations with non-missing values for the above:
estimate ci.lb ci.ub TrueSD Coverage
2500 2500 2500 2500 2500
Tau^2 and Tau with CLs via Tau2 and its SE provided by selmodel:
Tau2 Tau2SE Tau2LCL Tau2UCL TrueSD2 Coverage Tau TauLCL TauUCL TrueSD
0.46 0.32 -0.05 1.00 0.25 91.43 0.61 -0.09 0.96 0.50
Non-missing values for the above:
Tau2 Tau2SE Tau2LCL Tau2UCL TrueSD2 Coverage Tau TauLCL TauUCL TrueSD
2500 2391 2391 2391 2500 2391 2500 2391 2391 2500
From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On Behalf Of Tobias Saueressig via R-sig-meta-analysis
Sent: Monday, March 11, 2024 8:08 PM
To: r-sig-meta-analysis using r-project.org
Cc: Tobias Saueressig <t.saueressig using gmx.de>; r-sig-meta-analysis using r-project.org
Subject: Re: [R-meta] Extracting values from a list of confint() objects
Sorry Will, two more mistakes had crept in. It's probably too early in the morning after all. So here is the script again.
library(metafor)
dat <- dat.baskerville2012
dat
#create a list of dfs with 10 replications of dat
dat_list <- replicate(10, list(dat), simplify = FALSE)
# fit random-effects model for each data set in list
# Define a function to apply metafor::rma()
apply_rma <- function(dat) {
metafor::rma(smd, se^2, data = dat, method = "ML", digits = 3)
}
# Apply the function to each element of dat_list using lapply
res <- lapply(dat_list, apply_rma)
#run selection model for each list
# Define a function to apply metafor::selmodel()
apply_selmodel <- function(x) {
metafor::selmodel(x, type = "beta")
}
# Apply the function to each element of res using lapply
sel <- lapply(res, apply_selmodel)
# Define a function to calculate the confidence interval for tau^2
calculate_tau2_ci <- function(model) {
confint(model, tau2 = TRUE)
}
# Apply the function to each element of sel using lapply
tau2_ci <- lapply(sel, calculate_tau2_ci)
# Define a function to extract df from a given model
extract_tau <- function(model) {
df <- model$random
return(df)
}
# Define a function to extract all tau2 from all lists
extract_all_df <- function(tau2_ci) {
all_df <- lapply(tau2_ci, extract_tau)
return(all_df)
}
all_tau2 <- extract_all_df(tau2_ci)
all_tau2
c <- as.data.frame(all_tau2)
c
Gesendet: Montag, 11. März 2024 um 07:54 Uhr
Von: "Tobias Saueressig via R-sig-meta-analysis" <r-sig-meta-analysis using r-project.org <mailto:r-sig-meta-analysis using r-project.org> >
An: r-sig-meta-analysis using r-project.org <mailto:r-sig-meta-analysis using r-project.org>
Cc: "Tobias Saueressig" <t.saueressig using gmx.de <mailto:t.saueressig using gmx.de> >, "'R Special Interest Group for Meta-Analysis'" <r-sig-meta-analysis using r-project.org <mailto:r-sig-meta-analysis using r-project.org> >
Betreff: Re: [R-meta] Extracting values from a list of confint() objects
Dear Will,
I misread and did not know that you were working with a list (!) of models. Sorry! So I changed my example a bit.
library(metafor)
dat <- dat.baskerville2012
dat
#create a list of dfs with 10 replications of dat
dat_list <- replicate(10, list(dat), simplify = FALSE)
# fit random-effects model for each data set in list
# Define a function to apply metafor::rma()
apply_rma <- function(dat) {
metafor::rma(dat$smd, dat$se^2, data = dat, method = "ML", digits = 3)
}
# Apply the function to each element of dat_list using lapply
res <- lapply(dat_list, apply_rma)
#run selection model for each list
# Define a function to apply metafor::selmodel()
apply_selmodel <- function(x) {
metafor::selmodel(x, type = "beta")
}
# Apply the function to each element of res using lapply
sel <- lapply(res, apply_selmodel)
# Define a function to calculate the confidence interval for tau^2
calculate_tau2_ci <- function(model) {
confint(model, tau2 = TRUE)
}
# Apply the function to each element of sel using lapply
tau2_ci <- lapply(sel, calculate_tau2_ci)
# Define a function to extract df from a given model
extract_tau <- function(model) {
df <- model$random
return(df)
}
# Define a function to extract all tau2 from all lists
extract_all_df <- function(tau2_ci) {
all_df <- lapply(tau2_ci, extract_df)
return(all_df)
}
all_tau2 <- extract_all_df(tau2_ci)
all_tau2
#put results in a df
c <- as.data.frame(all_tau2)
Best Regards,
Tobias
Gesendet: Sonntag, 10. März 2024 um 23:40 Uhr
Von: "Will Hopkins via R-sig-meta-analysis" <r-sig-meta-analysis using r-project.org <mailto:r-sig-meta-analysis using r-project.org> >
An: "'R Special Interest Group for Meta-Analysis'" <r-sig-meta-analysis using r-project.org <mailto:r-sig-meta-analysis using r-project.org> >
Cc: "Will Hopkins" <willthekiwi using gmail.com <mailto:willthekiwi using gmail.com> >
Betreff: Re: [R-meta] Extracting values from a list of confint() objects
Tobias, that dataset appears to be for a single meta, not a set of many (simulated) metas for checking bias and coverage of the fixed effects and tau2. Nevertheless I tried the last few steps on my list of confint objects (adjusted_confint).
> #get an overview of the list, so that you can see what you need to extract
> str(adjusted_confint)
List of 2500
$ :List of 2
..$ fixed : num [1:2, 1:3] 1.5 1.74 1.05 1.1 1.95 ...
.. ..- attr(*, "dimnames")=List of 2
.. .. ..$ : chr [1:2] "intrcpt" "mods"
.. .. ..$ : chr [1:3] "estimate" "ci.lb" "ci.ub"
..$ digits: Named num [1:9] 4 4 4 4 4 4 4 4 4
.. ..- attr(*, "names")= chr [1:9] "est" "se" "test" "pval" ...
..- attr(*, "class")= chr "confint.rma"
$ :List of 2
..$ fixed : num [1:2, 1:3] 1.665 1.191 1.148 0.563 2.183 ...
.. ..- attr(*, "dimnames")=List of 2
.. .. ..$ : chr [1:2] "intrcpt" "mods"
.. .. ..$ : chr [1:3] "estimate" "ci.lb" "ci.ub"
..$ digits: Named num [1:9] 4 4 4 4 4 4 4 4 4
.. ..- attr(*, "names")= chr [1:9] "est" "se" "test" "pval" ...
..- attr(*, "class")= chr "confint.rma"
etc., etc.
I find this hard to understand, but it looks like the stats for tau2 may not be in there.
The next step didn't work:
> #extract tau2 and tau from list
> adjsel <- as.data.frame(adjusted_confint[["random"]])
> adjsel
data frame with 0 columns and 0 rows
Nor did the alternative version you suggested:
> adjsel <- as.data.frame(adjusted_confint$random)
> adjsel
data frame with 0 columns and 0 rows
However, my adjust_conflim was generated with this line of code within a function:
adjusted_confint <- confint(xxx, level=90, fixed=TRUE, random=TRUE, type="PL")
So I made it:
adjusted_confint <- confint(xxx, level=90, fixed=FALSE, tau2=TRUE, type="PL")
Now str(adjusted_confint) gave this:
$ :List of 5
..$ random : num [1:2, 1:3] 0.909 0.954 0.303 0.55 2.108 ...
.. ..- attr(*, "dimnames")=List of 2
.. .. ..$ : chr [1:2] "tau^2" "tau"
.. .. ..$ : chr [1:3] "estimate" "ci.lb" "ci.ub"
..$ digits : Named num [1:9] 4 4 4 4 4 4 4 4 4
.. ..- attr(*, "names")= chr [1:9] "est" "se" "test" "pval" ...
..$ ci.null: logi FALSE
..$ lb.sign: chr ""
..$ ub.sign: chr ""
..- attr(*, "class")= chr "confint.rma"
$ :List of 5
etc., etc.
So we're nearly there. I don’t know how to translate the above information into code that will extract the values of tau^2, tau, and their confidence limits. Your previous suggestions, adjsel <- as.data.frame(adjusted_confint[["random"]]) and adjsel <- as.data.frame(adjusted_confint$random), still gave:
data frame with 0 columns and 0 rows
And my attempts to use sapply() still don't work. I hope you can suggest something else, or point out any silly mistake I might be making.
Thanks
Will
From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org <mailto:r-sig-meta-analysis-bounces using r-project.org> > On Behalf Of Tobias Saueressig via R-sig-meta-analysis
Sent: Monday, March 11, 2024 12:42 AM
To: r-sig-meta-analysis using r-project.org <mailto:r-sig-meta-analysis using r-project.org>
Cc: Tobias Saueressig <t.saueressig using gmx.de <mailto:t.saueressig using gmx.de> >
Subject: Re: [R-meta] Extracting values from a list of confint() objects
Dear Will,
you can do the following to get the values:
install.packages("metafor")
library(metafor)
dat <- dat.baskerville2012
dat
# fit random-effects model
res <- rma(smd, se^2, data=dat, method="ML", digits=3)
res
sel <- selmodel(res, type="beta")
sel
#calculate confidence interval for tau2
a <- confint(sel,tau2=TRUE)
#get an overview of the list, so that you can see what you need to extract
str(a)
#extract tau2 and tau from list
c <- as.data.frame(a[["random"]])
c
#you can also use $ to get access
d <- as.data.frame(a$random)
d
#now you can export the data e.g. to excel
install.packages("openxlsx")
library(openxlsx)
# Export data to an Excel file
write.xlsx(d, "tau2_data.xlsx", rowNames = FALSE)
Best Regards,
Tobias
Gesendet: Sonntag, 10. März 2024 um 05:58 Uhr
Von: "Will Hopkins via R-sig-meta-analysis" < <mailto:r-sig-meta-analysis using r-project.org> r-sig-meta-analysis using r-project.org <mailto:r-sig-meta-analysis using r-project.org> >
An: "'R Special Interest Group for Meta-Analysis'" < <mailto:r-sig-meta-analysis using r-project.org> r-sig-meta-analysis using r-project.org <mailto:r-sig-meta-analysis using r-project.org> >
Cc: "Will Hopkins" < <mailto:willthekiwi using gmail.com> willthekiwi using gmail.com <mailto:willthekiwi using gmail.com> >
Betreff: [R-meta] Extracting values from a list of confint() objects
With the help of ChatGPT, I have managed to extract values from a list of
rma() objects and values from a list of selmodel() objects derived from the
list of rma() objects. However, I cannot extract values from a list of
confint() objects derived from the selmodel() objects. I hope I am using the
right jargon here. If not, please be gentle, as I am still on the steep part
of the learning curve.
For example, this code (mysteriously) extracts tau2 from meta_results, which
is a list of rma objects:
tau2 <- sapply(meta_results, function(x) x$tau2).
And this code (equally mysteriously) extracts the SE of tau2 from
adjusted_meta, which is a list of selmodel objects derived by applying
selmodel() to meta_results:
seltau2se <- sapply(adjusted_meta, function(x) x$se.tau2).
But I have a list I have called adjusted_confint, which is a list of confint
objects derived by applying confint to the list adjusted_meta. I don't know
how to use sapply on this list to extract the confidence limits for tau2
that are contained within this list. I have "copied blindly" the syntax of
the other sapply statements without success. Basically I don't know what to
put for the x$..., and everything I have tried fails. ChatGPT was not
helpful here. I hope a real maven can help here. Thanks.
Will
[[alternative HTML version deleted]]
_______________________________________________
R-sig-meta-analysis mailing list @ <mailto:R-sig-meta-analysis using r-project.org> R-sig-meta-analysis using r-project.org <mailto:R-sig-meta-analysis using r-project.org>
To manage your subscription to this mailing list, go to:
<https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
[[alternative HTML version deleted]]
_______________________________________________
R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org <mailto:R-sig-meta-analysis using r-project.org>
To manage your subscription to this mailing list, go to:
https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
_______________________________________________ R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org <mailto:R-sig-meta-analysis using r-project.org> To manage your subscription to this mailing list, go to: https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20240313/390ec077/attachment-0001.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: results.txt
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20240313/390ec077/attachment-0001.txt>
More information about the R-sig-meta-analysis
mailing list