[R-meta] I have any problems with meta-analysis of proportions

Lukasz Stasielowicz |uk@@z@@t@@|e|ow|cz @end|ng |rom un|-o@n@brueck@de
Fri Apr 9 21:29:54 CEST 2021


Apparently my mailing list reply got lost. Following Wolfgang's advice I 
submit it again as the issue with I-squared might be of interest to some 
people:

Dear Martin,
you write: "I am having too much heterogeneity I2 97%."
Please keep in mind that I2 cannot be interpreted as the absolute amount 
of heterogeneity.

A couple of months ago there was a discussion about heterogeneity on 
Researchgate so I will slightly adjust my answer:
The problem with I2 is that it only indicates the RELATIVE amount of 
true heterogeneity (as compared to the error variance). The threshold of 
75% is often interpreted as high heterogeneity. However, it is not 
recommended to evaluate the heterogeneity solely using the I2 value. 
Why? I'll illustrate it with a short example.

  * Consider a meta-analysis where the true heterogeneity (tau) equals
    .01 and the error heterogeneity equals .005. In this case I2 = 80%.
    [.01^2/(.01^2+.005^2)]
  * Now consider a meta-analysis with tau = .10 and error = .05. We get
    the same I value I2 = 80%. [.10^2/(.10^2+.05^2)]
  * Can we really say that we have a high amount of true heterogeneity
    in both scenarios? No. We have a high RELATIVE amount (relative to
    the error heterogeneity). In order to assess the absolute amount of
    heterogeneity it's good to interpret the tau value (true
    heterogeneity) on it's own.
  * You're estimating the proportion in your meta-analysis, so a tau
    value of .01 would mean that it's normal that the proportion
    estimates vary by 1% between studies or within studies. In many
    cases it's not a big deal. A prevalence of 74%, 75% or 76% is huge
    and a difference of 1% doesn't seem to be noteworthy so many people
    would argue that there is no considerable heterogeneity (it depends
    on the context!). However, consider the 2nd scenario. If the
    prevalence estimates vary by 10% (e.g., 75%, 65%, 85%) then most
    people would agree that there is a lot of heterogeneity.
  * Some statistical packages report the true heterogeneity as variance
    (tau2). In this case it is recommended to take the square root in
    order to interpret the heterogeneity on the scale of the chosen
    effect size (e.g., prevalence).
  * One of the researchers that popularized I2 (Julian Higgins) has
    coauthored an article dealing with the wrong interpretation of I2:
    DOI: 10.1002/jrsm.1230


Summing up, please do not rely solely on I2 when assessing 
heterogeneity. It can lead to wrong conclusions. Consider interpreting 
the value of true heterogeneity (tau) too. In addition, you can assess 
the range of the effect sizes (e.g., "the proportion estimates vary from 
.24 to .37 between the studies").

Michael Borenstein recently published a video about I2 and it could be 
of interest to you: https://youtu.be/38wRNJIcqe0


Best wishes,
Lukasz
-- 
Lukasz Stasielowicz
Osnabrück University
Institute for Psychology
Research methods, psychological assessment, and evaluation
Seminarstraße 20
49074 Osnabrück (Germany)

Am 06.04.2021 um 14:30 schrieb Lukasz Stasielowicz:
>
> Dear Martin,
>
> you write: "I am having too much heterogeneity I2 97%."
> Please keep in mind that I2 cannot be interpreted as the absolute 
> amount of heterogeneity.
>
> A couple of months ago there was a discussion about heterogeneity on 
> Researchgate so I will slightly adjust my answer:
> The problem with I2 is that it only indicates the RELATIVE amount of 
> true heterogeneity (as compared to the error variance). The threshold 
> of 75% is often interpreted as high heterogeneity. However, it is not 
> recommended to evaluate the heterogeneity solely using the I2 value. 
> Why? I'll illustrate it with a short example.
>
>   * Consider a meta-analysis where the true heterogeneity (tau) equals
>     .01 and the error heterogeneity equals .005. In this case I2 =
>     80%. [.01^2/(.01^2+.005^2)]
>   * Now consider a meta-analysis with tau = .10 and error = .05. We
>     get the same I value I2 = 80%. [.10^2/(.10^2+.05^2)]
>   * Can we really say that we have a high amount of true heterogeneity
>     in both scenarios? No. We have a high RELATIVE amount (relative to
>     the error heterogeneity). In order to assess the absolute amount
>     of heterogeneity it's good to interpret the tau value (true
>     heterogeneity) on it's own.
>   * You're estimating the proportion in your meta-analysis, so a tau
>     value of .01 would mean that it's normal that the proportion
>     estimates vary by 1% between studies or within studies. In many
>     cases it's not a big deal. A prevalence of 74%, 75% or 76% is huge
>     and a difference of 1% doesn't seem to be noteworthy so many
>     people would argue that there is no considerable heterogeneity (it
>     depends on the context!). However, consider the 2nd scenario. If
>     the prevalence estimates vary by 10% (e.g., 75%, 65%, 85%) then
>     most people would agree that there is a lot of heterogeneity.
>   * Some statistical packages report the true heterogeneity as
>     variance (tau2). In this case it is recommended to take the square
>     root in order to interpret the heterogeneity on the scale of the
>     chosen effect size (e.g., prevalence).
>   * One of the researchers that popularized I2 (Julian Higgins) has
>     coauthored an article dealing with the wrong interpretation of I2:
>     DOI: 10.1002/jrsm.1230
>
>
> Summing up, please do not rely solely on I2 when assessing 
> heterogeneity. It can lead to wrong conclusions. Consider interpreting 
> the value of true heterogeneity (tau) too. In addition, you can assess 
> the range of the effect sizes (e.g., "the proportion estimates vary 
> from .24 to .37 between the studies").
>
> Michael Borenstein recently published a video about I2 and it could be 
> of interest to you: https://youtu.be/38wRNJIcqe0
>
>
> Best wishes,
> Lukasz
> -- 
> Lukasz Stasielowicz
> Osnabrück University
> Institute for Psychology
> Research methods, psychological assessment, and evaluation
> Seminarstraße 20
> 49074 Osnabrück (Germany)
>
> Am 06.04.2021 um 11:34 schrieb r-sig-meta-analysis-request using r-project.org:
>> Send R-sig-meta-analysis mailing list submissions to
>> 	r-sig-meta-analysis using r-project.org
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>> 	https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>> or, via email, send a message with subject or body 'help' to
>> 	r-sig-meta-analysis-request using r-project.org
>>
>> You can reach the person managing the list at
>> 	r-sig-meta-analysis-owner using r-project.org
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of R-sig-meta-analysis digest..."
>>
>>
>> Today's Topics:
>>
>>     1. I have any problems with meta-analysis of proportions
>>        (Martin Lobo)
>>     2. Re: I have any problems with meta-analysis of proportions
>>        (Michael Dewey)
>>     3. Re: I have any problems with meta-analysis of proportions
>>        (Nicky Welton)
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Mon, 5 Apr 2021 16:32:29 +0000
>> From: Martin Lobo<mlobo4370 using hotmail.com>
>> To:"r-sig-meta-analysis using r-project.org"
>> 	<r-sig-meta-analysis using r-project.org>
>> Subject: [R-meta] I have any problems with meta-analysis of
>> 	proportions
>> Message-ID:
>> 	<MN2PR20MB28146287CBF76410B0080F79AC779 using MN2PR20MB2814.namprd20.prod.outlook.com>
>> 	
>> Content-Type: text/plain; charset="utf-8"
>>
>> Hi everyone,
>>
>> I performed  a systematic review on the persistence of some drugs.
>> I found 30 randomized clinical trials and 10 observational studies.
>> Although I understand that they should not be meta analyzed together (I should stratify them or analyze them separately), Actually, of the RCTs, I only use the active drug arm, so I think that by breaking the branching, maybe I could take all the data as observational.
>> Is this correct ?
>> If so, how do I describe the methodological part, what guidelines and quality scales should I use (PRISMA, STROBE, COCHRANE, NOS, JADAD?).
>>
>> On the other hand, I have never performed meta-analysis of proportions, and I am having too much heterogeneity I2 97%. How could I control this? The studies are of good quality. I use the metaprop function.
>>
>> Than's
>>
>>
>>
>>
>> Lorenzo Mart�n Lobo MTSAC, FACC, FESC
>> Especialista Jerarquizado en Cardiolog�a
>> Jefe de Dpto Enf. Cardiovasculares y Cardiometabolismo Hospital Militar Campo de Mayo.
>> Jefe de Cardiolog�a Hospital Militar Campo de Mayo
>> Ex Jefe de Unidad Coronaria Hospital Militar Campo de Mayo
>> Miembro Titular de la Sociedad Argentina de Cardiolog�a
>> Fellow American College of Cardiology
>> Fellow European Society of Cardiology
>> Ex Miembro del Area de Investigaci�n de la SAC
>> Ex Director del Consejo de Aterosclerosis y Trombosis de la SAC
>> Miembro Asesor del Consejo de Aterosclerosis y Trombosis de la SAC
>> Ex Director del Consejo de Epidemiolog�a y Prevenci�n Cardiovascular de la SAC
>>
>> Miembro Asesor del Consejo de Epidemiolog�a y Prevenci�n Cardiovascular de la SAC
>>
>>
>> Experto en Lipidos de la Sociedad Argentina de Lipidos.
>> Miembro de la Sociedad Argentina de Lipidos.
>> Instructor de ACLS de la American Heart Association
>>
>>
>> ________________________________
>> De: R-sig-meta-analysis<r-sig-meta-analysis-bounces using r-project.org>  en nombre der-sig-meta-analysis-request using r-project.org  <r-sig-meta-analysis-request using r-project.org>
>> Enviado: s�bado, 31 de octubre de 2020 08:05
>> Para:r-sig-meta-analysis using r-project.org  <r-sig-meta-analysis using r-project.org>
>> Asunto: R-sig-meta-analysis Digest, Vol 41, Issue 19
>>
>> Send R-sig-meta-analysis mailing list submissions to
>>          r-sig-meta-analysis using r-project.org
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>>          https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-meta-analysis&data=04%7C01%7C%7C5d85acadc431479e885408d87d8cfe6b%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637397391813549456%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=S1XTEg5LOY7sPYJeZEHUQlCaT9Th2woJRJzMedoGQp4%3D&reserved=0
>> or, via email, send a message with subject or body 'help' to
>>          r-sig-meta-analysis-request using r-project.org
>>
>> You can reach the person managing the list at
>>          r-sig-meta-analysis-owner using r-project.org
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of R-sig-meta-analysis digest..."
>>
>>
>> Today's Topics:
>>
>>     1. Re: GOSH plots for multi-level meta (rma.mv) (Hellen Mirr)
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Fri, 30 Oct 2020 13:03:34 +0000
>> From: Hellen Mirr<hellenmir554 using gmail.com>
>> To: "Viechtbauer, Wolfgang (SP)"
>>          <wolfgang.viechtbauer using maastrichtuniversity.nl>
>> Cc:"r-sig-meta-analysis using r-project.org"
>>          <r-sig-meta-analysis using r-project.org>
>> Subject: Re: [R-meta] GOSH plots for multi-level meta (rma.mv)
>> Message-ID:
>>          <CAF6nRJqkgQ4_vkF0sdf=_anW2Etp2snB14F-v0ot8rZ9JFaXGQ using mail.gmail.com>
>> Content-Type: text/plain; charset="utf-8"
>>
>> Dear Wolfgang,
>>
>> Thank you very much for your clear explanation.
>>
>> Best,
>> Hellen
>>
>> On Fri, Oct 30, 2020 at 11:48 AM Viechtbauer, Wolfgang (SP) <
>> wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>>
>>> Dear Hellen,
>>>
>>> This is currently not implemented in metafor. In principle, the idea of a
>>> GOSH plot does generalize to more complex models although one needs to
>>> think about whether one would want to create subsets based on the indiviual
>>> estimates or based on some higher-level grouping variable. For example,
>>> suppose we have a multilevel structure such as:
>>>
>>> study  esid  yi vi
>>> ------------------
>>> 1      1     .  .
>>> 1      2     .  .
>>> 2      1     .  .
>>> 3      1     .  .
>>> 3      2     .  .
>>> 3      3     .  .
>>> 4      1     .  .
>>>
>>> So, what are, for example, then the subsets of size 2? Are they based just
>>> on the rows? Then the estimates in row 1 and 2 would be one such subset. Or
>>> does one base the subsets on the studies? Then rows 1, 2, 3 (i.e., studies
>>> 1 and 2) would be such a subset.
>>>
>>> This could all be implemented (just like cooks.distance() and rstudent()
>>> allow for the optional specification of a clustering variable), but I
>>> haven't done this.
>>>
>>> Aside from that: Fitting rma.mv models can take a bit of time. Doing this
>>> 1000's of times (based on all possible subsets) could take a LONG time.
>>>
>>> Best,
>>> Wolfgang
>>>
>>>> -----Original Message-----
>>>> From: R-sig-meta-analysis [mailto:
>>> r-sig-meta-analysis-bounces using r-project.org]
>>>> On Behalf Of Hellen Mirr
>>>> Sent: Friday, 30 October, 2020 12:12
>>>> To:r-sig-meta-analysis using r-project.org
>>>> Subject: [R-meta] GOSH plots for multi-level meta (rma.mv)
>>>>
>>>> Hello,
>>>>
>>>> Apologies if this has already been answered, as I could not find any
>>>> threads on it.
>>>> I was wondering whether it is possible to create a GOSH plot for a
>>>> multi-level meta analysis that is an rma.mv object, and how I would go
>>>> about that.
>>>>
>> [[elided Hotmail spam]]
>>>> Best wishes
>>>> Hellen
>>          [[alternative HTML version deleted]]
>>
>>
>>
>>
>> ------------------------------
>>
>> Subject: Digest Footer
>>
>> _______________________________________________
>> R-sig-meta-analysis mailing list
>> R-sig-meta-analysis using r-project.org
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-meta-analysis&data=04%7C01%7C%7C5d85acadc431479e885408d87d8cfe6b%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637397391813549456%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=S1XTEg5LOY7sPYJeZEHUQlCaT9Th2woJRJzMedoGQp4%3D&reserved=0
>>
>>
>> ------------------------------
>>
>> End of R-sig-meta-analysis Digest, Vol 41, Issue 19
>> ***************************************************
>>
>> 	[[alternative HTML version deleted]]
>>
>>
>>
>>
>> ------------------------------
>>
>> Message: 2
>> Date: Tue, 6 Apr 2021 09:48:25 +0100
>> From: Michael Dewey<lists using dewey.myzen.co.uk>
>> To: Martin Lobo<mlobo4370 using hotmail.com>,
>> 	"r-sig-meta-analysis using r-project.org"
>> 	<r-sig-meta-analysis using r-project.org>
>> Subject: Re: [R-meta] I have any problems with meta-analysis of
>> 	proportions
>> Message-ID:<e92a5ee6-0055-81a0-2b74-77b8036d8913 using dewey.myzen.co.uk>
>> Content-Type: text/plain; charset="utf-8"; Format="flowed"
>>
>> Comments in-line
>>
>> On 05/04/2021 17:32, Martin Lobo wrote:
>>> Hi everyone,
>>>
>>> I performed  a systematic review on the persistence of some drugs.
>>> I found 30 randomized clinical trials and 10 observational studies.
>>> Although I understand that they should not be meta analyzed together (I should stratify them or analyze them separately), Actually, of the RCTs, I only use the active drug arm, so I think that by breaking the branching, maybe I could take all the data as observational.
>> Yes, you now have a set of observational studies if you only take one
>> arm from the trials.
>>
>>> Is this correct ?
>>> If so, how do I describe the methodological part, what guidelines and quality scales should I use (PRISMA, STROBE, COCHRANE, NOS, JADAD?).
>>>
>> It is still a meta-analysis so use PRISMA
>>
>>> On the other hand, I have never performed meta-analysis of proportions, and I am having too much heterogeneity I2 97%. How could I control this? The studies are of good quality. I use the metaprop function.
>>>
>> In a meta-analysis of observational studies high heterogeneity is almost
>> ineitable.
>>
>>> Than's
>>>
>>>
>>>
>>>
>>> Lorenzo Mart�n Lobo MTSAC, FACC, FESC
>>> Especialista Jerarquizado en Cardiolog�a
>>> Jefe de Dpto Enf. Cardiovasculares y Cardiometabolismo Hospital Militar Campo de Mayo.
>>> Jefe de Cardiolog�a Hospital Militar Campo de Mayo
>>> Ex Jefe de Unidad Coronaria Hospital Militar Campo de Mayo
>>> Miembro Titular de la Sociedad Argentina de Cardiolog�a
>>> Fellow American College of Cardiology
>>> Fellow European Society of Cardiology
>>> Ex Miembro del Area de Investigaci�n de la SAC
>>> Ex Director del Consejo de Aterosclerosis y Trombosis de la SAC
>>> Miembro Asesor del Consejo de Aterosclerosis y Trombosis de la SAC
>>> Ex Director del Consejo de Epidemiolog�a y Prevenci�n Cardiovascular de la SAC
>>>
>>> Miembro Asesor del Consejo de Epidemiolog�a y Prevenci�n Cardiovascular de la SAC
>>>
>>>
>>> Experto en Lipidos de la Sociedad Argentina de Lipidos.
>>> Miembro de la Sociedad Argentina de Lipidos.
>>> Instructor de ACLS de la American Heart Association
>>>
>>>
>>> ________________________________
>>> De: R-sig-meta-analysis<r-sig-meta-analysis-bounces using r-project.org>  en nombre der-sig-meta-analysis-request using r-project.org  <r-sig-meta-analysis-request using r-project.org>
>>> Enviado: s�bado, 31 de octubre de 2020 08:05
>>> Para:r-sig-meta-analysis using r-project.org  <r-sig-meta-analysis using r-project.org>
>>> Asunto: R-sig-meta-analysis Digest, Vol 41, Issue 19
>>>
>>> Send R-sig-meta-analysis mailing list submissions to
>>>           r-sig-meta-analysis using r-project.org
>>>
>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>           https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-meta-analysis&data=04%7C01%7C%7C5d85acadc431479e885408d87d8cfe6b%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637397391813549456%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=S1XTEg5LOY7sPYJeZEHUQlCaT9Th2woJRJzMedoGQp4%3D&reserved=0
>>> or, via email, send a message with subject or body 'help' to
>>>           r-sig-meta-analysis-request using r-project.org
>>>
>>> You can reach the person managing the list at
>>>           r-sig-meta-analysis-owner using r-project.org
>>>
>>> When replying, please edit your Subject line so it is more specific
>>> than "Re: Contents of R-sig-meta-analysis digest..."
>>>
>>>
>>> Today's Topics:
>>>
>>>      1. Re: GOSH plots for multi-level meta (rma.mv) (Hellen Mirr)
>>>
>>> ----------------------------------------------------------------------
>>>
>>> Message: 1
>>> Date: Fri, 30 Oct 2020 13:03:34 +0000
>>> From: Hellen Mirr<hellenmir554 using gmail.com>
>>> To: "Viechtbauer, Wolfgang (SP)"
>>>           <wolfgang.viechtbauer using maastrichtuniversity.nl>
>>> Cc:"r-sig-meta-analysis using r-project.org"
>>>           <r-sig-meta-analysis using r-project.org>
>>> Subject: Re: [R-meta] GOSH plots for multi-level meta (rma.mv)
>>> Message-ID:
>>>           <CAF6nRJqkgQ4_vkF0sdf=_anW2Etp2snB14F-v0ot8rZ9JFaXGQ using mail.gmail.com>
>>> Content-Type: text/plain; charset="utf-8"
>>>
>>> Dear Wolfgang,
>>>
>>> Thank you very much for your clear explanation.
>>>
>>> Best,
>>> Hellen
>>>
>>> On Fri, Oct 30, 2020 at 11:48 AM Viechtbauer, Wolfgang (SP) <
>>> wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>>>
>>>> Dear Hellen,
>>>>
>>>> This is currently not implemented in metafor. In principle, the idea of a
>>>> GOSH plot does generalize to more complex models although one needs to
>>>> think about whether one would want to create subsets based on the indiviual
>>>> estimates or based on some higher-level grouping variable. For example,
>>>> suppose we have a multilevel structure such as:
>>>>
>>>> study  esid  yi vi
>>>> ------------------
>>>> 1      1     .  .
>>>> 1      2     .  .
>>>> 2      1     .  .
>>>> 3      1     .  .
>>>> 3      2     .  .
>>>> 3      3     .  .
>>>> 4      1     .  .
>>>>
>>>> So, what are, for example, then the subsets of size 2? Are they based just
>>>> on the rows? Then the estimates in row 1 and 2 would be one such subset. Or
>>>> does one base the subsets on the studies? Then rows 1, 2, 3 (i.e., studies
>>>> 1 and 2) would be such a subset.
>>>>
>>>> This could all be implemented (just like cooks.distance() and rstudent()
>>>> allow for the optional specification of a clustering variable), but I
>>>> haven't done this.
>>>>
>>>> Aside from that: Fitting rma.mv models can take a bit of time. Doing this
>>>> 1000's of times (based on all possible subsets) could take a LONG time.
>>>>
>>>> Best,
>>>> Wolfgang
>>>>
>>>>> -----Original Message-----
>>>>> From: R-sig-meta-analysis [mailto:
>>>> r-sig-meta-analysis-bounces using r-project.org]
>>>>> On Behalf Of Hellen Mirr
>>>>> Sent: Friday, 30 October, 2020 12:12
>>>>> To:r-sig-meta-analysis using r-project.org
>>>>> Subject: [R-meta] GOSH plots for multi-level meta (rma.mv)
>>>>>
>>>>> Hello,
>>>>>
>>>>> Apologies if this has already been answered, as I could not find any
>>>>> threads on it.
>>>>> I was wondering whether it is possible to create a GOSH plot for a
>>>>> multi-level meta analysis that is an rma.mv object, and how I would go
>>>>> about that.
>>>>>
>>> [[elided Hotmail spam]]
>>>>> Best wishes
>>>>> Hellen
>>>           [[alternative HTML version deleted]]
>>>
>>>
>>>
>>>
>>> ------------------------------
>>>
>>> Subject: Digest Footer
>>>
>>> _______________________________________________
>>> R-sig-meta-analysis mailing list
>>> R-sig-meta-analysis using r-project.org
>>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-meta-analysis&data=04%7C01%7C%7C5d85acadc431479e885408d87d8cfe6b%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637397391813549456%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=S1XTEg5LOY7sPYJeZEHUQlCaT9Th2woJRJzMedoGQp4%3D&reserved=0
>>>
>>>
>>> ------------------------------
>>>
>>> End of R-sig-meta-analysis Digest, Vol 41, Issue 19
>>> ***************************************************
>>>
>>> 	[[alternative HTML version deleted]]
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> R-sig-meta-analysis mailing list
>>> R-sig-meta-analysis using r-project.org
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>>

	[[alternative HTML version deleted]]



More information about the R-sig-meta-analysis mailing list