[R] statistical significance test for cluster agreement

[Liaw, Andy]

[Apology to the list for the off-topic rant...]

As it turned out, I also have a problem with LOF/GOL/etc. tests:  I'd bet
most of the time when such a test is carried out, it is _not_ the only test
being done, but the p-values in the downstream analysis are almost never
adjusted for this.  How valid would the p-values be?

IMHO, it's bad enough that users of statistical methods do things like this,
but it's quite something else that statisticians do just the same, or even
promote such tests.  It's not a crime to do analysis like that, but to treat
the p-values as if they actually are meaningful probably ought to be
outlawed.

OK, I better run for cover now...

Andy

> From: Alexander Sirotkin [at Yahoo] [mailto:alex_s_42 at yahoo.com] 
> 
> Like you said, such kind of test will not give me
> anything that Rand index does not, except for p-value.
> 
> The null hypothesis, in my case, is that clustering
> results does not match a different clustering, that
> someone alse did on the same data.
> 
> And I do believe that this hypothesis is valid.
> Basicly, it's not that different from chi-squared
> goodness of fit test which check whether or not my 
> data comes from particular distribution. With an 
> exception that I don't know how to do chi-squared test
> in this case :)
> 
> 
> 
> --- "Liaw, Andy" <andy_liaw at merck.com> wrote:
> > But what would such a test do that the rand index
> > does not?  Would you
> > interpret the p-value from such a test, if exists,
> > to have the meaning that
> > a real test of hypothesis has?  AFAIK you basically
> > need to have the
> > hypotheses pinned down even before you see any data,
> > for the inference to be
> > valid.  Is that possible with clustering?
> > 
> > Just my $0.02...
> > Andy
> > 
> > > From: Alexander Sirotkin [at Yahoo]
> > > 
> > > I was wondering, whether there is a way to have
> > > statistical significance test for cluster
> > agreement.
> > > 
> > > I know that I can use classAgreement() function to
> > get
> > > Rand index, which will give me some indication
> > whether
> > > the clusters agree or not, but it would be
> > interesting
> > > to have a formal test.
> > > 
> > > Thanks.
> > > 
> > > ______________________________________________
> > > R-help at stat.math.ethz.ch mailing list
> > >
> >
> https://www.stat.math.ethz.ch/mailman/listinfo/r-help
> > > PLEASE do read the posting guide! 
> > > http://www.R-project.org/posting-guide.html
> > > 
> > > 
> > 
> > 
> >
> --------------------------------------------------------------
> ----------------
> > Notice:  This e-mail message, together with any
> > attachments, contains
> > information of Merck & Co., Inc. (One Merck Drive,
> > Whitehouse Station, New
> > Jersey, USA 08889), and/or its affiliates (which may
> > be known outside the
> > United States as Merck Frosst, Merck Sharp & Dohme
> > or MSD and in Japan as
> > Banyu) that may be confidential, proprietary
> > copyrighted and/or legally
> > privileged. It is intended solely for the use of the
> > individual or entity
> > named on this message.  If you are not the intended
> > recipient, and have
> > received this message in error, please notify us
> > immediately by reply e-mail
> > and then delete it from your system.
> >
> --------------------------------------------------------------
> ----------------
> 
> 
> __________________________________
>