[R] Waaaayy off topic...Statistical methods, pub bias, scientific validity

[John Kane]

--- On Fri, 1/7/11, Peter Langfelder <peter.langfelder at gmail.com> wrote:

> From: Peter Langfelder <peter.langfelder at gmail.com>
> Subject: Re: [R] Waaaayy off topic...Statistical methods, pub bias, scientific validity
> To: "r-help at r-project.org" <r-help at r-project.org>
> Received: Friday, January 7, 2011, 2:06 AM
> >From a purely statistical and
> maybe somewhat naive point of view,
> published p-values should be corrected for the multiple
> testing that
> is effectively happening because of the large number of
> published
> studies. My experience is also that people will often try
> several
> statistical methods to get the most significant p-value but
> neglect to
> share that fact with the audience and/or at least attempt
> to correct
> the p-values for the selection bias.
> 
> That being said, it would seem that biomedical sciences do
> make
> progress, so some of the published results are presumably
> correct :)
> 


Totally a placebo effect :)

> Peter
> 
> On Thu, Jan 6, 2011 at 9:13 PM, Spencer Graves
> <spencer.graves at structuremonitoring.com>
> wrote:
> >      Part of the phenomenon can be explained by the
> natural censorship in
> > what is accepted for publication:  Stronger results
> tend to have less
> > difficulty getting published.  Therefore, given that
> a result is published,
> > it is evident that the estimated magnitude of the
> effect is in average
> > larger than it is in reality, just by the fact that
> weaker results are less
> > likely to be published.  A study of the literature on
> this subject might
> > yield an interesting and valuable estimate of the
> magnitude of this
> > selection bias.
> >
> >
> >      A more insidious problem, that may not affect
> the work of Jonah Lehrer,
> > is political corruption in the way research is funded,
> with less public and
> > more private funding of research
> > (http://portal.unesco.org/education/en/ev.php-URL_ID=21052&URL_DO=DO_TOPIC&URL_SECTION=201.html).
> >  For example, I've heard claims (which I cannot
> substantiate right now) that
> > cell phone companies allegedly lobbied successfully to
> block funding for
> > researchers they thought were likely to document
> health problems with their
> > products.  Related claims have been made by
> scientists in the US Food and
> > Drug Administration that certain therapies were
> approved on political
> > grounds in spite of substantive questions about the
> validity of the research
> > backing the request for approval (e.g.,
> > www.naturalnews.com/025298_the_FDA_scientists.html).
>  Some of these
> > accusations of political corruption may be groundless.
>  However, as private
> > funding replaces tax money for basic science, we must
> expect an increase in
> > research results that match the needs of the funding
> agency while degrading
> > the quality of published research.  This produces
> more research that can not
> > be replicated -- effects that get smaller upon
> replication.  (My wife and I
> > routinely avoid certain therapies recommended by
> physicians, because the
> > physicians get much of their information on recent
> drugs from the
> > pharmaceuticals, who have a vested interest in
> presenting their products in
> > the most positive light.)
> >
> >
> >      Spencer
> >
> >
> > On 1/6/2011 2:39 PM, Carl Witthoft wrote:
> >>
> >> The next week's New Yorker has some decent
> rebuttal letters.  The case is
> >> hardly as clear-cut as the author would like to
> believe.
> >>
> >> Carl
> 
> ______________________________________________
> R-help at r-project.org
> mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained,
> reproducible code.
>