[R-sig-ME] F test vs. mcmcpvalue

[Ken Beath]

On 08/07/2008, at 3:16 AM, Ben Bolker wrote:

>
> Hank Stevens wrote:
> d> HI Ben and Spencer,
> | Thank you very much for your help.
> |
> | 1. The QQ plots look normal, but highlight the lack of balance  
> (from one
> | to dozens of reps per treatment combo).
> | 2. The MCMC sample traces look (in my limited experience) without
> | peculiarities, and the densityplots are all quite symmetrical and
> | normal-ish.
> | 3. Simulations (lmer::simulate) of the null hypothesis indicate that
> | F-stats as large (or larger) than my observed F-stats are VERY  
> unlikely,
> | under the null hypothesis.
> |
> | As I learn anything else useful, I will be happy to share.
> | Cheers,
> | Hank
> |
>
> ~  #3 pretty much seals it for me -- since that is really what
> the F test is trying to test.
>
> ~  It's a little hard to reconcile #2 and #3, though ... I would
> think you could move on at this point, but just for laughs --
> are you using mcmcpvalue on a single contrast, or multiple
> parameters?  If the former, does it seem to agree with the results of
> HPDinterval() or quantile()?  If the latter, is there something
> about the _combinations_ of parameters that is wonky?
>

If an MCMC isn't traversing the parameter space properly the traces  
will probably still look OK until it shifts into a new region which  
may take a while.

Also it was mentioned that there were 500 observations. For clustered  
data it is the number of clusters that is more important.

Ken