[R-meta] escale ROM or SMD

[Lukas Dylewski]

Dear James,
thank you for quick response !

Here is my response:
*1.  Measurement properties: what is proline concentration? Is it measured
on a ratio scale, such that ROM is a sensible way to describe change (over
time or as a result of intervention)? If you provide a bit more detail
about what the outcome is (for us social scientists on the listserv), then
perhaps others will be able to weigh in also. *
Response: Proline concentration is measure as some value not ratio (e.g.
mmol g-1 DM). For example in the control group and experimental group
(drought stress) I have some proline value in plant tissue.  In this
research, we want to check the overall effect of drought stress on proline
concentration, and how the duration of drought (in days) affected proline
concentration. Moreover, proline concentration, not a significant change in
well water treatment during the time. So in the publication, they give the
value not the ratio of change proline during the duration of drought. So,
when in one publication authors provide results for proline concentration
during the duration of drought (e.g. control, 1-day drought, 7 days
drought and 14 days drought, the effect size I calculate based on control
group compared with the drought day group, the effect size for 1 day is
calculated based on control (mean/SD/n) vs. 1 day (mean/SD/n); effect size
for 7 days is control vs. 7 days; effect size for 14 days is control vs. 14
days.

2. *Theory: Is there any relevant botanical theory that would indicate how
drought stress should be related to proline concentration? *
Response: The phenomenon of proline accumulation in plant tissue is known
to occur under environmental stress water deficit, salinity, low
temperature, etc. So the effect should be also positive on proline
concentration activate by some stressor. However, there are no studies
showing how strong this effect is for e.g. the type of plants, duration of
drought, or seed size. We know that in most studies water stress has a
positive effect on proline concentration.

3. *Empirical features: It can be helpful to create scatterplots showing
the relationship between the M and the SD of the outcome in each group and
between the Ms in different comparison groups. If the outcomes are on
drastically different scales, then the plots can be created within
subgroups that use the same or similar measurement instruments. If the SD
of the outcome is strongly related to the M, then I would take this as an
indication that ROM might be more appropriate than SMD.*
Response: I attach the graph, I hope I understood correctly. Fig. 1 is a
scatter plot of mean and SD for each group control and experiment. Fig. 2
is a scatter plot for effect size (yi) to vi in both cases ROM and SMD.

*4) Heterogeneity: Which model has less unexplained variability (as
measured by I^2, for instance)? All else equal, I would prefer the effect
size metric where the meta-analytic model has greater explanatory power.
>From what you've said, it sounds like the ROM would win out here due to
fewer outliers and no apparent funnel plot asymmetry.*
Response:
res0 <- rma(yi, vi,method="REML",data=hedges) - ROM
For ROM - I^2 equal 99.76%; effect size 0.6715

res0 <- rma(yi, vi,method="REML",data=hedges2) - SMD
For SMD - I^2 equal 99.23%; effect size 7.6561

Thank you for help !

Best

Lukasz

pon., 12 lip 2021 o 16:10 James Pustejovsky <jepusto using gmail.com> napisał(a):

> Hi Lukas,
>
> I think there are (at least) four relevant considerations here:
> 1) Measurement properties: what is proline concentration? Is it measured
> on a ratio scale, such that ROM is a sensible way to describe change (over
> time or as a result of intervention)? If you provide a bit more detail
> about what the outcome is (for us social scientists on the listserv), then
> perhaps others will be able to weigh in also.
> 2) Theory: Is there any relevant botanical theory that would indicate how
> drought stress should be related to proline concentration?
> 3) Empirical features: It can be helpful to create scatterplots showing
> the relationship between the M and the SD of the outcome in each group and
> between the Ms in different comparison groups. If the outcomes are on
> drastically different scales, then the plots can be created within
> subgroups that use the same or similar measurement instruments. If the SD
> of the outcome is strongly related to the M, then I would take this as an
> indication that ROM might be more appropriate than SMD.
> 4) Heterogeneity: Which model has less unexplained variability (as
> measured by I^2, for instance)? All else equal, I would prefer the effect
> size metric where the meta-analytic model has greater explanatory power.
> From what you've said, it sounds like the ROM would win out here due to
> fewer outliers and no apparent funnel plot asymmetry.
>
> I've listed these considerations in what I would consider to be decreasing
> order of priority (1st being essential, 2nd being important, 3rd and 4th
> being matters of judgement). Others might have different perspectives,
> though.
>
> Kind Regards,
> James
>
> On Mon, Jul 12, 2021 at 5:43 AM Lukas Dylewski <dylewski91 using gmail.com>
> wrote:
>
>> Dear All,
>>
>> I conducted a mixed meta-regression model to check the effect of drought
>> stress on proline concentration in leaves. In the model, I include the
>> following moderators: duration of drought (in the day), seed mass, and
>> plant type (coniferous vs. deciduous). The duration of the drought affects
>> the proline content, therefore it was included in the model.
>>
>> When I calculate Hedge g (measure=SMD) values I got very big values for
>> some records (e.g. 100, 50, etc.) and publication bias. However, when I
>> calculate effect size using ROM I got very nice numbers without publication
>> bias.
>>
>> My first question: Can I use the ROM method for my dataset (I attach the
>> database: proline) or should I use the SMD?
>>
>> This is my code (with ROM measure)
>>
>> z<-proline
>> z$logmass<-log(z$seed.mass.mg+1)
>>
>> hedges<-escalc(measure="ROM",data=z,append=T,m1i=experiment,n1i=n.experiment,sd1i=sd.experiment,m2i=control,n2i=n.control,sd2i=sd.control)
>> hedges
>>
>> res1 <- rma.mv(yi, vi, mods = ~ logmass + I(logmass^2)+
>> factor(plant.type) + drougth.day + I(drougth.day^2),
>> random=~1|references,data=hedges, method="REML")
>>
>> After the model summary, I would like to calculate the mean effect size
>> for the plant type (coniferous and deciduous separately). Does this code
>> correct?
>>
>> m <- mean(hedges$logmass)
>> n <- mean(hedges$drougth.day)
>>
>> For conifeorus:
>> predict(res1, newmods = c(m, m^2, 0,n,n^2))
>>
>> For deciduous:
>> predict(res1, newmods = c(m, m^2, 1,n,n^2))
>>
>> --
>> Łukasz Dylewski, PhD.
>>
>> Institute of Dendrology,
>>
>> Polish Academy of Sciences,
>>
>> Parkowa 5, 62-035 Kórnik, Poland
>> _______________________________________________
>> R-sig-meta-analysis mailing list
>> R-sig-meta-analysis using r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>
>

-- 
Łukasz Dylewski, PhD.

Institute of Dendrology,

Polish Academy of Sciences,

Parkowa 5, 62-035 Kórnik, Poland

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20210712/3d92678e/attachment-0001.html>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: Fig. 2.pdf
Type: application/pdf
Size: 15237 bytes
Desc: not available
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20210712/3d92678e/attachment-0002.pdf>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: Fig.1.pdf
Type: application/pdf
Size: 14034 bytes
Desc: not available
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20210712/3d92678e/attachment-0003.pdf>