[R-meta] escale ROM or SMD

[Lukas Dylewski]

Hi James,

thank you very much for your help!

I checked and I think that proline is measure as a ratio scale. The proline
unit is given as μmoles proline / g of the fresh or dry weight of plant
material.

I have another technical question
I would like to calculate the mean effect size for one of the category
moderator: plant type (coniferous and deciduous separately) based on the
full model including all fixed effects. Does this code correct?

m <- mean(hedges$logmass)
n <- mean(hedges$drougth.day)

For conifeorus:
predict(res1, newmods = c(m, m^2, 0,n,n^2))

For deciduous:
predict(res1, newmods = c(m, m^2, 1,n,n^2))

Once again thank you for your help I  appreciate it.

Best
Lukasz


wt., 13 lip 2021, 23:25 użytkownik James Pustejovsky <jepusto using gmail.com>
napisał:

> HI Lukasz,
>
> Regarding question 1 about measurement, I was wondering about whether
> proline concentration is on a ratio scale, meaning a scale where ratio
> comparisons are meaningful (see eg
> https://en.wikipedia.org/wiki/Level_of_measurement#Ratio_scale). Does a
> proline concentration of zero mean that there is no proline accumulation at
> all? What are the *units* of proline concentration measurements?
>
> Regarding question 3 about empirical features, I see from Figure 1 that
> there seems to be a relationship between the M and the SD---particularly in
> control conditions. That would suggest that ROM might be more appropriate
> than SMD.
>
> Regarding question 4, heterogeneity, I see in Figure 2 that there is a
> very strong relationship between the ES and its sampling variance when
> using the SMD metric. That relationship is artificial (see
> https://doi.org/10.1002/jrsm.1332). I also see that there is a really
> extreme degree of variation in the SMD effect size estimates. Values of 20+
> for a SMD are just non-sensical, in my view. In contrast, the ROM estimates
> vary over a wide range, but not an absurd one, and their sampling variances
> are much more homogeneous (with the exception of the one outlier). All of
> this further supports the use of ROM, I think.
>
> James
>
> On Mon, Jul 12, 2021 at 12:45 PM Lukas Dylewski <dylewski91 using gmail.com>
> wrote:
>
>> Dear James,
>> thank you for quick response !
>>
>> Here is my response:
>> *1.  Measurement properties: what is proline concentration? Is it
>> measured on a ratio scale, such that ROM is a sensible way to describe
>> change (over time or as a result of intervention)? If you provide a bit
>> more detail about what the outcome is (for us social scientists on the
>> listserv), then perhaps others will be able to weigh in also. *
>> Response: Proline concentration is measure as some value not ratio (e.g.
>> mmol g-1 DM). For example in the control group and experimental group
>> (drought stress) I have some proline value in plant tissue.  In this
>> research, we want to check the overall effect of drought stress on proline
>> concentration, and how the duration of drought (in days) affected proline
>> concentration. Moreover, proline concentration, not a significant change in
>> well water treatment during the time. So in the publication, they give the
>> value not the ratio of change proline during the duration of drought. So,
>> when in one publication authors provide results for proline concentration
>> during the duration of drought (e.g. control, 1-day drought, 7 days
>> drought and 14 days drought, the effect size I calculate based on control
>> group compared with the drought day group, the effect size for 1 day is
>> calculated based on control (mean/SD/n) vs. 1 day (mean/SD/n); effect size
>> for 7 days is control vs. 7 days; effect size for 14 days is control vs. 14
>> days.
>>
>> 2. *Theory: Is there any relevant botanical theory that would indicate
>> how drought stress should be related to proline concentration? *
>> Response: The phenomenon of proline accumulation in plant tissue is known
>> to occur under environmental stress water deficit, salinity, low
>> temperature, etc. So the effect should be also positive on proline
>> concentration activate by some stressor. However, there are no studies
>> showing how strong this effect is for e.g. the type of plants, duration of
>> drought, or seed size. We know that in most studies water stress has a
>> positive effect on proline concentration.
>>
>> 3. *Empirical features: It can be helpful to create scatterplots showing
>> the relationship between the M and the SD of the outcome in each group and
>> between the Ms in different comparison groups. If the outcomes are on
>> drastically different scales, then the plots can be created within
>> subgroups that use the same or similar measurement instruments. If the SD
>> of the outcome is strongly related to the M, then I would take this as an
>> indication that ROM might be more appropriate than SMD.*
>> Response: I attach the graph, I hope I understood correctly. Fig. 1 is a
>> scatter plot of mean and SD for each group control and experiment. Fig. 2
>> is a scatter plot for effect size (yi) to vi in both cases ROM and SMD.
>>
>> *4) Heterogeneity: Which model has less unexplained variability (as
>> measured by I^2, for instance)? All else equal, I would prefer the effect
>> size metric where the meta-analytic model has greater explanatory power.
>> From what you've said, it sounds like the ROM would win out here due to
>> fewer outliers and no apparent funnel plot asymmetry.*
>> Response:
>> res0 <- rma(yi, vi,method="REML",data=hedges) - ROM
>> For ROM - I^2 equal 99.76%; effect size 0.6715
>>
>> res0 <- rma(yi, vi,method="REML",data=hedges2) - SMD
>> For SMD - I^2 equal 99.23%; effect size 7.6561
>>
>> Thank you for help !
>>
>> Best
>>
>> Lukasz
>>
>> pon., 12 lip 2021 o 16:10 James Pustejovsky <jepusto using gmail.com>
>> napisał(a):
>>
>>> Hi Lukas,
>>>
>>> I think there are (at least) four relevant considerations here:
>>> 1) Measurement properties: what is proline concentration? Is it measured
>>> on a ratio scale, such that ROM is a sensible way to describe change (over
>>> time or as a result of intervention)? If you provide a bit more detail
>>> about what the outcome is (for us social scientists on the listserv), then
>>> perhaps others will be able to weigh in also.
>>> 2) Theory: Is there any relevant botanical theory that would indicate
>>> how drought stress should be related to proline concentration?
>>> 3) Empirical features: It can be helpful to create scatterplots showing
>>> the relationship between the M and the SD of the outcome in each group and
>>> between the Ms in different comparison groups. If the outcomes are on
>>> drastically different scales, then the plots can be created within
>>> subgroups that use the same or similar measurement instruments. If the SD
>>> of the outcome is strongly related to the M, then I would take this as an
>>> indication that ROM might be more appropriate than SMD.
>>> 4) Heterogeneity: Which model has less unexplained variability (as
>>> measured by I^2, for instance)? All else equal, I would prefer the effect
>>> size metric where the meta-analytic model has greater explanatory power.
>>> From what you've said, it sounds like the ROM would win out here due to
>>> fewer outliers and no apparent funnel plot asymmetry.
>>>
>>> I've listed these considerations in what I would consider to be
>>> decreasing order of priority (1st being essential, 2nd being important, 3rd
>>> and 4th being matters of judgement). Others might have different
>>> perspectives, though.
>>>
>>> Kind Regards,
>>> James
>>>
>>> On Mon, Jul 12, 2021 at 5:43 AM Lukas Dylewski <dylewski91 using gmail.com>
>>> wrote:
>>>
>>>> Dear All,
>>>>
>>>> I conducted a mixed meta-regression model to check the effect of
>>>> drought stress on proline concentration in leaves. In the model, I include
>>>> the following moderators: duration of drought (in the day), seed mass, and
>>>> plant type (coniferous vs. deciduous). The duration of the drought affects
>>>> the proline content, therefore it was included in the model.
>>>>
>>>> When I calculate Hedge g (measure=SMD) values I got very big values for
>>>> some records (e.g. 100, 50, etc.) and publication bias. However, when I
>>>> calculate effect size using ROM I got very nice numbers without publication
>>>> bias.
>>>>
>>>> My first question: Can I use the ROM method for my dataset (I attach
>>>> the database: proline) or should I use the SMD?
>>>>
>>>> This is my code (with ROM measure)
>>>>
>>>> z<-proline
>>>> z$logmass<-log(z$seed.mass.mg+1)
>>>>
>>>> hedges<-escalc(measure="ROM",data=z,append=T,m1i=experiment,n1i=n.experiment,sd1i=sd.experiment,m2i=control,n2i=n.control,sd2i=sd.control)
>>>> hedges
>>>>
>>>> res1 <- rma.mv(yi, vi, mods = ~ logmass + I(logmass^2)+
>>>> factor(plant.type) + drougth.day + I(drougth.day^2),
>>>> random=~1|references,data=hedges, method="REML")
>>>>
>>>> After the model summary, I would like to calculate the mean effect size
>>>> for the plant type (coniferous and deciduous separately). Does this code
>>>> correct?
>>>>
>>>> m <- mean(hedges$logmass)
>>>> n <- mean(hedges$drougth.day)
>>>>
>>>> For conifeorus:
>>>> predict(res1, newmods = c(m, m^2, 0,n,n^2))
>>>>
>>>> For deciduous:
>>>> predict(res1, newmods = c(m, m^2, 1,n,n^2))
>>>>
>>>> --
>>>> Łukasz Dylewski, PhD.
>>>>
>>>> Institute of Dendrology,
>>>>
>>>> Polish Academy of Sciences,
>>>>
>>>> Parkowa 5, 62-035 Kórnik, Poland
>>>> _______________________________________________
>>>> R-sig-meta-analysis mailing list
>>>> R-sig-meta-analysis using r-project.org
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>>>
>>>
>>
>> --
>> Łukasz Dylewski, PhD.
>>
>> Institute of Dendrology,
>>
>> Polish Academy of Sciences,
>>
>> Parkowa 5, 62-035 Kórnik, Poland
>>
>

	[[alternative HTML version deleted]]