[BioC] Harsh results using limma!
A.J. Rossini
rossini at blindglobe.net
Fri Aug 13 15:38:54 CEST 2004
The OTHER explanation could be technician error or inadvertent
cross-hybridization due to processing; the catch is that you've also
got a batch effect, it seems.
The more critical issue is that you've got to use statistics that
describe what you want. Obviously, those that standardize location
(mean/median) by some variability (std error/interquartile range) are
going to give you p-values (non-sig) which reflect that
standardization.
If you want a "consistency" / magnitude statistic (say, a sign test
augmented in some manner with the magnitude), at this point you'd have
to be creative. But having been creative, you still could get a
distribution via simulation or resampling to work from to obtain
p-values.
The only problem will be trying to convince reviewers (or folks
playing devil's advocate) that your "statistic" is reasonable for
differential expression.
best,
-tony
"michael watson (IAH-C)" <michael.watson at bbsrc.ac.uk> writes:
> Hi Gordon
>
> Yes you're right. I didn't really mean to compare limma to a t-test.
> It's just that the results are very consistent within technical
> replicates (the dye-swaps), just not consistent between biological
> replicates. But this is the situation we expect - technical replicates
> highly correlated and biological replicates much less so. Clearly
> differences of 0.2 could be noise, but my due-swaps BOTH came up with
> 0.2. If I had ten replicate dye-swaps, all with 0.2 as the log(ratio)
> would we still call this noise? Given that the other replicate
> experiments were also highly reproducible, I can't help but think this
> gene is differentially expressed.
>
> I know why limma and t-test disregard this gene, I just still think it
> is a little harsh and that I am "throwing the baby away with the
> bathwater", as it were.
>
> Mick
>
> -----Original Message-----
> From: Gordon Smyth [mailto:smyth at wehi.edu.au]
> Sent: 13 August 2004 12:56
> To: michael watson (IAH-C)
> Cc: bioconductor at stat.math.ethz.ch
> Subject: Re: [BioC] Harsh results using limma!
>
>
> At 09:14 PM 13/08/2004, michael watson (IAH-C) wrote:
>>Hi
>>
>>Firstly, I think limma is excellent and use it a lot, but some recent
>>results are a bit, erm, disappointing and I wondered if someone could
>>explain them.
>>
>>Basic set up was a double dye-swap experiment (4 arrays) involving
>>different animals, one infected with one type of bacterium and the
>>other a different bacterium, compared to one another directly. I used
>>limma to analyse this and got a list of genes differentially regulated
>>- great!
>>
>>THEN another replicate experiment was performed (so now I have 6
>>arrays, 3 dye-swaps), and I re-did the analysis and my set of genes was
>
>>completely different - but that's fine, we can put that down to
>>biological variation. We know limma likes genes which show consistent
>>results across arrays, and when I looked at my data, I found that the
>>genes in my original list were not consistent across all six arrays.
>>So I am reasonably happy about this.
>>
>>My question comes from looking at the top gene from my old list in the
>>context of all six arrays. Here are the normalised log ratios across
>>all six arrays (ds indicates the dye-swap):
>>
>>Gene1
>>Exp1 -5.27
>>Exp1ds 6.29
>>Exp2 -4.61
>>Exp2ds 5.54
>>Exp3 -0.2
>>Exp3ds 0.2
>
> Changes of +-0.2 are tiny and look like pure noise. So, you can have a
> gene
> for which only 2/3 of your mice show a difference. Statistical methods
> based on means and standard deviations will always judge this situation
> harshly. If you try an ordinary t-test rather than the limma method,
> you'll
> find that this gene would be judged much more harshly again.
>
> Gordon
>
>>Not suprisingly, limma put this as the top gene when looking at the
>>first four arrays. However, when looking across all six arrays, limma
>>places it at 230 in the list with a p-value of 0.11 (previously the
>>p-value was 0.0004).
>>
>>So finally we get to my point/question - does this gene really
>>"deserve" a p-value of 0.11 (ie not significant)? In every case the
>>dye-flips are the correct way round, it is only the magnitude of the
>>log(ratio) which differs - and as we are talking about BIOLOGICAL
>>variation here, don't we expect the magnitude to change? If we are
>>taking into account biological variation, surely we can't realistically
> expect consistent
>>ratios across all replicate experiments?? Isn't limma being a little
>>harsh here? After all the average log ratio is -3.7 (taking into
>>account the dye-flips) - and to me, experiment 3's results still
>>support the idea of the gene being differentially expressed, and are
>>even consistent within that biological replicate.
>>
>>Clearly I am looking at this data from a biologists point of view and
>>not a statisticians. But we are studying biology, not statistics, and
>>I can't help feel I am missing out on something important here if I
>>disregard this gene as not significantly differentially expressed (NB
>>this is just the first example, there are many others).
>>
>>I should also add that there appears nothing strange about the arrays
>>for Experiment 3 - the distribution of log(ratio) for those arrays is
>>pretty much the same as the other four, so this is not an array-effect,
>
>>it is an effect due to natural biological variation.
>>
>>Comments, questions, criticisms all welcome :-)
>>
>>Mick
>
> _______________________________________________
> Bioconductor mailing list
> Bioconductor at stat.math.ethz.ch
> https://stat.ethz.ch/mailman/listinfo/bioconductor
>
--
Anthony Rossini Research Associate Professor
rossini at u.washington.edu http://www.analytics.washington.edu/
Biomedical and Health Informatics University of Washington
Biostatistics, SCHARP/HVTN Fred Hutchinson Cancer Research Center
UW (Tu/Th/F): 206-616-7630 FAX=206-543-3461 | Voicemail is unreliable
FHCRC (M/W): 206-667-7025 FAX=206-667-4812 | use Email
CONFIDENTIALITY NOTICE: This e-mail message and any attachme...{{dropped}}
More information about the Bioconductor
mailing list