[BioC] Odd contrast; does it make statistical sense?

[Gordon K Smyth]

On Fri, 24 Jan 2014, Aaron Mackey wrote:

> On Thu, Jan 23, 2014 at 6:48 PM, Gordon K Smyth <smyth at wehi.edu.au> wrote:
>
>> My worry is that with this contrast, I'm effectively just testing two
>>> groups against each other, and by having 4 groups in the design I will be
>>> estimating dispersions that are not appropriate for the test that I'm
>>> doing, and hence I will overstate my confidence.
>>>
>>
>> The dispersions remain unchanged regardless of the contrast you test. The
>> dispersions have been estimated after removing all differences between the
>> four groups, i.e., without bias.
>>
>
> But had he more simply coded the samples as belonging to two groups, AB 
> and CD, then the dispersions could be larger, and the AB vs. CD mean 
> differences could be less significant than in his four-subgroup design. 
> I think that was the intent of Ryan's question.

Coding that doesn't reflect the true experimental design is likely to 
perform badly, and give less significance.  That doesn't make it more 
correct.

> Is it fair to stratify along a priori expected subgroupings to minimize 
> variance and then ask group-level questions?

You are not asking a well posed question.  For one thing, "fair" and 
"unfair" are unhelpful concepts.  The only consideration is whether the 
statistical test that is done answers the scientific question being 
answered.  You haven't explained what scientific question you want to 
answer, so there is no basis for choosing a scientific test.

Fitting a model that matches the experimental conditions and then making 
comparisons between groups has been the anova method since anova was first 
invented.  It answers what it answers, as I explained in my response to 
Ryan.

Gordon

> As you later say, if there is "lots of DE" between
> A vs. B and/or C vs. D (read: larger group-wise variance) then the test may
> need to be "carefully interpreted".
>
> Thanks for your insight,
> -Aaron
>

______________________________________________________________________
The information in this email is confidential and intend...{{dropped:4}}