[R-meta] mean-variance relationships introduces additional heterogeneity, how?

[Luke Martinez]

Many thanks James!

Just to make sure, do pi_Ai and pi_Bi represent (in my participants
taking a test context) the proportion of correct responses in each
group (treatment vs. control) respectively?

We're saying that even though these proportions (pi_Ai and pi_Bi) have
a constant and linear relation via the multiplicative coefficient
lambda in and across all the studies such that the treatment effect is
the same across all the studies, we still get varying SMDs across
these studies and that's due to simply the natural mean-sd
relationship underlying the data-generating process for the raw scores
(counts of some sort).

If yes, I guess the difficult part (for me) becomes providing
empirical support that at least there is evidence of mean-sd relation
in my data whereby to provide evidence for the assumed the
data-generating process. In my case (with longitudinal primary
studies), for example, the smallest study gives the following results.

Can I plot anything here (or for any other study) to empirically show
mean-sd relation?

Many thanks,
Luke

#----------------------------------------------
d="
Mt             SDt  nt       Mc     SDc nc time
0.0799 0.0367 21 0.0763 0.0389 26 0
0.113   0.0472 21 0.1095 0.0537 26 1"

dat <- read.table(text=d, h=T)
#----------------------------------------------

On Sun, Oct 24, 2021 at 1:42 PM James Pustejovsky <jepusto using gmail.com> wrote:
>
> Hi Luke,
>
> My original response to your question was not specifically about mean-variance relationships within a study across multiple time points, but rather was more generally about mean-variance relationships in summary statistics. The simplest case here is where there's just one effect size per study and where the studies vary in the mean level of the outcome in a given group. I'll give you an example of the sort of data-generating process that I had in mind. It's important to bear in mind, though, that this is all entirely speculative---I'm *not* asserting that this model is appropriate for your data specifically, only that the example is one plausible situation where mean-variance relationships arise.
>
> Suppose that we have k studies, each involving a two-group comparison, with groups of equal size. In study i, the outcomes in group A follow a poisson distribution with mean m_Ai, so that the variance of the outcomes in group A is also m_Ai, for i = 1,...,k. The outcomes in group B follow a poisson distribution with mean m_Bi, so the variance is also m_Bi. Now, suppose that there is a fixed, proportional relationship between pi_Bi and pi_Ai, so that pi_Bi = lambda pi_Ai for some lambda > 0. In other words, the treatment contrast is *constant* on the scale of the response ratio. However, the means in group A vary from study to study, according to some distribution, say a gamma distribution with shape parameter alpha and rate parameter beta.  What does this model imply about the distribution of standardized mean differences across this set of studies?
>
> The SMD parameter for study i (call it delta_i) is the ratio of the mean difference to the square root of the pooled variance. So:
>
> delta_i = (m_Bi - m_Ai) / sqrt[(m_Bi + m_Ai) / 2]
> = (lambda - 1) m_Ai / sqrt[(1 + lambda) m_Ai / 2]
> = sqrt(m_Ai) * (lambda - 1) * sqrt[2 * / (lambda + 1)]
>
> The second and third term in the above expression are constants that only depend on the size of the response ratio. The first term is random because we have assumed that the group A means vary from study to study. It will therefore create heterogeneity in the SMD parameters---the greater the variance of the m_Ai's, the greater the heterogeneity in delta_i.
>
> I've used poisson distributions for outcomes and the gamma distribution for the m_Ai's only for sake of simplicity. Something very similar would hold if we considered outcomes that were binomially distributed because the binomial distribution has variance that is strongly related to its mean. If the mean proportions vary from study to study, this implies that the variances will also vary from study-to-study in a non-linear way, which will induce heterogeneity in the SMD parameters.
>
> James
>
> On Sat, Oct 23, 2021 at 11:53 PM Luke Martinez <martinezlukerm using gmail.com> wrote:
>>
>> Hello All,
>>
>> I wanted to follow up on an answer
>> (https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2021-October/003354.html)
>> on the list that, in a nutshell, says:
>>
>> Existence of a relationship between the Mean and SD for a given study
>> group (e.g., Treatment), over a couple of time points, can potentially
>> introduce additional heterogeneity in the SMD effect sizes across the
>> studies.
>>
>> I was wondering how exactly this heterogeneity comes about?
>>
>> A part of me says that such a Mean-SD relationship for a given study
>> group over time is indicative of what Hedges' (1981;
>> https://doi.org/10.3102/10769986006002107) refers to as
>> "subject-treatment interaction"?
>>
>> Another part of me says, no, "subject-treatment interaction" is
>> controllable by adding a random effect for individuals, and thus it is
>> different.
>>
>> I highly appreciate your insights regarding HOW Mean-SD relationship
>> for study groups over time can potentially introduce additional
>> heterogeneity in SMDs across studies?
>>
>> Luke
>>
>> PS:
>> Here are the relation I see between mean and sd of a study group
>> across 3 time points:
>> group1_Ms = c(.39, .18, .13)
>> group1_SDs = c(.25, .16, .13)
>> plot(group1_Ms, group1_SDs, type="l")
>>
>> _______________________________________________
>> R-sig-meta-analysis mailing list
>> R-sig-meta-analysis using r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis