[BioC] Cutoff to use for IQR filtering in genefilter

[Mark Cowley]

Hi Fraser,
that's exactly right, using the median IQR as the filter will remove  
50% of your data every time.
Other alternatives could be the 20th percentile of the IQR's as your  
filter to remove the least variable 20%.

Since all of the IQR's make up a distribution of numbers, there will  
always be a median of that distribution. I think that the question  
you're asking is: what if the median IQR is still not variable enough  
in a biological context, or in a system with large changes, perhaps a  
median IQR filter would remove too many genes that have large  
variability.
That would be where plotting the data, perhaps against the t-tests as  
you have suggested would be a good means of choosing the best filter.  
perhaps IQR vs average expression level, or IQR vs standard deviation  
might also help?

Incidentally, I rarely use a variability filter, I rely on the  
statistics with FDR < 5%, and accept that some of these will be due to  
genes with small, but consistent differences.

cheers,
Mark

On 24/06/2008, at 3:06 AM, Sim, Fraser wrote:

> Hi Mark,
>
> Am I right in the interpretation that using the median cutoff of the
> distribution of IQRs would remove 50% of the genes in every analysis.
>
> As below:
>
> eset <- readAffy()
> IQRs <- esApply(eset,1,IQR)
> f1 <- function(x) ( IQR(x) > median(IQRs) )
> selected <- genefilter(eset, f1)
>
> What happens if more than 50% of genes are variable or for that matter
> less than 50%? Should one plot the IQRs against some value of  
> interest,
> e.g. t-test statistic and determine the IQR cut-off on that basis?
>
> Thanks, Fraser
>
> -----Original Message-----
> From: bioconductor-bounces at stat.math.ethz.ch
> [mailto:bioconductor-bounces at stat.math.ethz.ch] On Behalf Of Mark  
> Cowley
> Sent: Sunday, June 22, 2008 7:32 PM
> To: swhwang10 at yahoo.com
> Cc: bioconductor at stat.math.ethz.ch
> Subject: Re: [BioC] Cutoff to use for IQR filtering in genefilter
>
> Hi Seungwoo,
> The range/IQR/SE/SD of your data is dependent on a number of factors,
> including biological variability, and other sources of technical
> variability, which can include the type of normalisation algorithm
> (think RMA vs MAS5).
> Basically, applying a filter on IQR of 0.1 in my study might remove
> half the genes, whereas in your study it may remove 10% of them.
> Suggestions such as Robert's are useful because they use the IQR of
> YOUR data in order to set that cutoff.
>
> I suggest caculating the IQR's for all of your genes, and then either
> plotting them plot(density(IQRs)) or just try summary( IQRs ) which
> will give you a good feel for just how variable your data is.
>
> If you need help calculating the IQR's and/or variances of your genes,
> please post back to the list.
>
> cheers,
> Mark
>
> On 22/06/2008, at 9:05 PM, Seungwoo Hwang wrote:
>
>> I am wondering what cutoff value I should use for IQR filtering in
>> genefilter. I did some literature search. It varies from paper to
>> paper. I have read two papers so far. One used 0.5, the other used
>> 0.18. affylmGUI has an option of 0.5, 0.25, and 0.1.
>>
>> I also searched Bioconductor archive and read that Dr. Robert
>> Gentleman suggested to filter out the genes whose IQR below median,
>> not for some fixed value.
>>
>> I have two questions on this vein.
>>
>> (1) How small is a gene's variance (in terms of number) if its IQR
>> is some value, say, 0.5 or 0.1? Can I calculate it?
>> (2) When median is used instead of fixed number, wouldn't it be too
>> large, since median of a gene's expression intensities across
>> samples can be anything?
>>
>> Thanks,
>>
>> Seungwoo
>> ------------------------------------
>> Seungwoo Hwang, Ph.D.
>> Senior Research Scientist
>> Korean Bioinformation Center
>>
>> _______________________________________________
>> Bioconductor mailing list
>> Bioconductor at stat.math.ethz.ch
>> https://stat.ethz.ch/mailman/listinfo/bioconductor
>> Search the archives:
> http://news.gmane.org/gmane.science.biology.informatics.conductor
>
> ----------------------------------------------------------------------
> Mark Cowley, BSc (Bioinformatics)(Hons)
>
> Peter Wills Bioinformatics Centre
> Garvan Institute of Medical Research
>
> _______________________________________________
> Bioconductor mailing list
> Bioconductor at stat.math.ethz.ch
> https://stat.ethz.ch/mailman/listinfo/bioconductor
> Search the archives:
> http://news.gmane.org/gmane.science.biology.informatics.conductor