[BioC] P calls (VSN and RMA)

[Petra B Ross-MacDonald]

Okay, I spent today checking how much these "non-hybridizing" probesets are
costing me. I found that a conservative filter does not change my results much
if I use a Bonferroni or Dunn-Sidak multiple test correction, but it matters a
lot for a False Discovery Rate correction. Details below.

I used VSN data and did a filtering step based on the Affymetrix bacterial
probes on the U133A as follows:

Non-hybridizing probesets were removed from the analysis using the following
criteria: Twelve probesets derived from the Bacillis subtilis genome are
present on the U133A chip. These probesets should not hybridize to human cDNA,
and are thus a control for selectivity. In the 16 samples analyzed, these
twelve probesets showed intensity values with a median of 6.8 and a maximum of
7.9. The E. coli BioB transcript is spiked into the labeling reaction at a
concentration of 1.5pM, providing a control for sensitivity. Affymetrix
specifies that this concentration is at the lower boundary for detection.  In
the 16 samples analyzed, intensity values for the six BioB probesets had median
of 8 and a minimum value of 7.4. Thus, the transition between sensitivity and
selectivity occurs at intensity values between 7.4 and 7.9. Probesets for which
any two or more of the 16 samples showed intensity values of 7.5 or greater
were included in the statistical analysis. 4,444 probesets did not meet this
criteria and were excluded..

Data was imported into Partek. A mixed model ANOVA, using treatment as a fixed
effect and treatment block as a random effect, was used to identify markers
that showed a treatment effect. Using the Partek False Discovery Rate (FDR)
tool,  a step-up FDR analysis was performed on the p values, allowing us to
identify p-value cutoffs for certain levels of significance.

Using the mixed ANOVA without filtering (22284 probesets), I identified no
probesets that made an FDR cutoff of 0.05, and 11 that made a cutoff of 0.1.  A
list of the top 300 probesets for this ANOVA had a FDR of between 40 and 50%.

Using the filtered set (17840 probesets), I identified 4 probesets that made an
FDR cutoff of 0.05, and 300 that made a cutoff of 0.1. These 300 included 272
that were in the top 300 of the mixed ANOVA without filtering ie 28 were lost
in the filter step.

So filtering doesn't really change the probesets or rankings that come out of
the ANOVA, but do you agree it gives the False Discovery Rate more meaning?
It's a lot better if I can give the biologist back a list of 300 genes and tell
them 10% are false positives, than a list of 11 with one false positive. Maybe
the 28 markers have to be looked at seperately, to avoid the false negatives
that Naomi was worried about.

Based on what I saw today, I thnk it is valid to make a "P calling" function
for VSN data based on the intensity values for non-human probes on the arrays.
Isaac?

cheers, Petra


"Rafael A. Irizarry" wrote:

> i agree with wolfgang. to reiterate, what is very important is that you
> remember that due to uncertainity in measurement (not to mention Naef's
> observation), there is also
> uncertainty in P/A calls and any other filtering operation. what is
> lacking is a rigorous assssment of this uncertainty.
> thus, by filtering genes before looking
> at other stats (such as fold change) you will likely introduce false
> negatives but wont have a clue of how much. i havent seen evidence
> suggesting that, when using RMA, this
> sacrifice in sensitivity is worth the gain in specificity. although, for
> MAS 5.0 it certailiny helps.
>
> another issue is how do you define P/A when you get differing calls in
> technical (or biological) replicates. i have seen many arbitrary choices
> used such as requiring 80% presence... but again, i havent seen evidence
> suggesting this helps, when using RMA.
>