[R-meta] Question about function reporter()

[Viechtbauer, Wolfgang (NP)]

There is no universal definition of what a random-effects model is, but take a look at:

https://www.metafor-project.org/doku.php/analyses:crede2010

and especially the section "A Common Mistake when Fitting the Multilevel Model". So, if your data have such a multilevel structure, then typically we would add a random effect for articles and a random effect for the observations within articles. Beyond this, you could do '~ 1 | alloc' although it is still not clear to me what you think this is modelling.

Best,
Wolfgang

>-----Original Message-----
>From: Gabriel Cotlier [mailto:gabiklm01 using gmail.com]
>Sent: Friday, 16 June, 2023 11:45
>To: Viechtbauer, Wolfgang (NP)
>Cc: R Special Interest Group for Meta-Analysis
>Subject: Re: [R-meta] Question about function reporter()
>
>Hello Wolfgang,
>
>As far as I understand from your explanation, by setting random variables to
>observe the potential influence of whether the correlations ( yi ) are
>positive or negative would not be a random model, is this correct?
>
>But, what if I keep moderator for grouping by Type (categorical:  corresponding
>to the method employed by the studies either "CS", "R", "E")  and would include
>in random another "Id" that represent whether the correlations come from either
>the same or a different study by means of the variable named "Article"
>(numerical: with same value for same article ) as follows:
>
>res3a <- rma.mv(yi = yi,
>               V = vi,
>               mods = ~Type - 1,
>               random = list(~1 | alloc, ~1 | Article),
>               data = dat)
>
>Could now be considered in this case a random effect model?
>
>Thanks a lot.
>Kind regards,
>Gabriel
>
>On Fri, Jun 16, 2023 at 11:45 AM Viechtbauer, Wolfgang (NP)
><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>As for your actual question: That's a bit too broad. Essentially, you need to
>familiarize yourself with what people typically report for standard RE models (as
>reporter() does), then familiarize yourself with what people report meta-
>regression models (since you have a meta-regression model), and then consider how
>the random effects structure of your model differs from a standard RE model and
>how this might affect what to report. I am not aware of any papers that will walk
>you through all of that. People typically learn reporting practices from what
>other people have done, so reading lots of existing meta-analyses will give you a
>sense of how this is done.
>
>This aside, I am not sure what you hope to accomplish by adding a random effect
>for whether the outcome is positive or negative. Also, your model does not
>capture any heterogeneity beyond this, so it isn't really a random-effects model
>(or something akin to it).
>
>Best,
>Wolfgang
>
>>-----Original Message-----
>>From: Gabriel Cotlier [mailto:gabiklm01 using gmail.com]
>>Sent: Friday, 16 June, 2023 9:08
>>To: Yefeng Yang
>>Cc: R Special Interest Group for Meta-Analysis; Viechtbauer, Wolfgang (NP)
>>Subject: Re: [R-meta] Question about function reporter()
>>
>>Hello,
>>
>>Wolfgang : very helpful and nourishing knowledge and guidance about the
>>function rma.mv() potential and complexity, thanks a lot for the explanation!
>>Yes indeed, maybe in the "longue durée" --or not so much-- future time we will
>be
>>more worried about bigger threats AI technology can pose to us ...
>>
>>Anyway, It become an interesting participative discussion afterwards.
>>
>>Maybe, I should apologize due to the lack of specificity or rugurosity in my
>>question, since I was only thinking --but did not write it--in my particular
>>case, mainly of an rma.rm() function with the simplest the possible "random"
>>arguments rather than the more the complex nested random effect models, such as
>>for instance:
>>
>>data_1 <- rma.mv(yi = yi,
>>                     V = vi,
>>                     mods = ~Type - 1,
>>                     random = ~1 | alloc,
>>                     data = dat)
>>
>>where the "alloc" variable refers just whether the correlation ( yi ) has either
>>positive or negative sign (numerical variable  taking either -1 or 1) and "Type"
>>is a just categorical variable (of char class) composed of 3 clases Types namly
>>"R", "E", and "CS".
>>
>>Maybe there is instead of a reporter() function output results some bibliography
>>that can lead me to a nice manner in which I can report the results of my simple
>>application of rma.mv() function output as an example to support the bases of
>the
>>writing of my report.
>>
>>Thanks a lot.
>>Kind regards,
>>Gabriel
>>
>>On Thu, Jun 15, 2023 at 12:05 PM Yefeng Yang <yefeng.yang1 using unsw.edu.au> wrote:
>>Actually, I have a different opinion about reporter() function. Just for an open
>>discussion and no offensive. Instead of automatically generating an analysis
>>report based on the fitted model (via rma()), it is probably more useful to have
>>a helper function to automatically generate a "publication-ready" table that
>>shows the quantities recommended by various PRISMA-related reporting guidelines,
>>such as the name of the moderator, tau, I2, point estimate, CIs, k, t, p, et al.
>>The analysis report is something that should be (and must be at some point) done
>>by the analysts themselves and they are responsible for the proper
>>interpretation.
>>
>>Regards,
>>Yefeng
>>
>>________________________________________
>>From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> on behalf
>>of Viechtbauer, Wolfgang (NP) via R-sig-meta-analysis <r-sig-meta-analysis using r-
>>project.org>
>>Sent: Thursday, 15 June 2023 18:38
>>To: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
>>project.org>
>>Cc: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl>;
>>Gabriel Cotlier <gabiklm01 using gmail.com>
>>Subject: Re: [R-meta] Question about function reporter()
>>
>>Dear Gabriel,
>>
>>You can't, since reporter() doesn't currently work for 'rma.mv' objects.
>>
>>Models that can be fitted with rma() (same as rma.uni()) are relatively simple
>>and the number of possibilities that need to be covered for translating the
>>results from such a model into text are managable. Although even here, there are
>>currently restrictions. For example, reporter() currently only works for
>>'intercept-only models' (i.e., models without moderators), it doesn't work when
>>robust() was used on the model, and it doesn't work for location-scale models.
>>Allowing reporter() to work with meta-regression models is on my radar, but not
>>sure when I will get to this.
>>
>>Models that can be fitted with rma.mv() are an entirely different beast. This
>>function allows users to fit multilevel models (with essentially no limits on
>the
>>number of levels), multivariate models (with multiple correlated random
>effects),
>>network meta-analyses, phylogenetic meta-analyses, spatio-temporal models,
>models
>>with random slopes, models with crossed random effects, and combinations thereof
>>(e.g., multivariate network meta-analysis). Such models will also typically
>>involve one or multiple moderators (e.g., to distinguish different outcomes,
>>treatments, time points, etc.). Depending on the type of model, different
>aspects
>>of the results are also more or less relevant (e.g., in a phylogenetic MA, there
>>would be a lot of focus on the random effects for species, while in a network
>MA,
>>focus would be more on contrasting the different treatments with each other).
>>There is essentially no way in hell that one could write reporter()-like
>>functionality for 'rma.mv' type models that covers all these
>>aspects/possibilities in a sensible way.
>>
>>Of course, one could consider writing a version that only covers a few special
>>cases; for example, models of the form rma.mv(yi, V, random = ~ 1 |
>>level1/level2/level3/...) or rma.mv(yi, V, random = ~ var1 | var2) although the
>>latter type of model would often be used when var1 corresponds to different
>>outcomes in which case the model would probably involve moderators and be of the
>>form rma.mv(yi, V, mods = ~ outcome, random = ~ outcome | study), but in the
>end,
>>the reporter() function cannot read the users mind as to what the goal and focus
>>of their analysis was.
>>
>>Alternatively, one could generate very generic text that does cover many
>>possibilities, but this would add essentially nothing to just reading the output
>>directly.
>>
>>Maybe if we wait another 20-30 years, ChatGPT (or Skynet or whatever it will be
>>called then) will be able to do something like this automatically. However, we
>>might be too busy fighting off the Terminators at that point to worry about
>>rma.mv() models ...
>>
>>Best,
>>Wolfgang
>>
>>>-----Original Message-----
>>>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>>>Behalf Of Gabriel Cotlier via R-sig-meta-analysis
>>>Sent: Thursday, 15 June, 2023 6:37
>>>To: R Special Interest Group for Meta-Analysis
>>>Cc: Gabriel Cotlier
>>>Subject: [R-meta] Question about function reporter()
>>>
>>>Hello all,
>>>
>>>I am using an object of class "rma.mv" "rma" as :
>>>
>>>class(data)
>>>[1] "rma.mv" "rma"
>>>
>>>and would like to use the function reporter().
>>>How could this possibly be done either directly in metafor in R or maybe in
>>>JAMOVI or in other software where the metafor package is included?
>>>
>>>Thanks a lot.
>>>Kind regards,
>>>Gabriel