[R-sig-eco] Testing "order" on predicted data

[Aitor Gastón]

As your observed data are not binary AUC (operating characteristic curve 
area) won't work properly.

Any rank correlation coefficient may be useful, check the example, if 
predictions order the sites as the observations the correlation (rho) is 1. 
Changing the last two predictions causes a reduction of rho (0.8 in the 
example).

observed<-c(0.3,0.4,0.42,0.99)
predicted1<-c(2.4,2.8,4.0,32)
predicted2<-c(2.4,2.8,32,4.0)

rho1<-cor(observed,predicted1,method = "spearman")
rho2<-cor(observed,predicted2,method = "spearman")
rho1
rho2

Aitor

----- Original Message ----- 
From: "Corrado" <ct529 at york.ac.uk>
To: <r-sig-ecology at r-project.org>
Sent: Tuesday, November 03, 2009 4:40 PM
Subject: Re: [R-sig-eco] Testing "order" on predicted data


> Dear friends,
>
> the data are ordered real numbers between 0 and 1. something of that sort:
>
> observations
> site4 0.3
> site1 0.4
> site5 0.42
> .
> .
> .
> siten 0.999
>
> after being sorted. I would like the models to order the sites in the same
> order, even if the values are not right.
>
> That is ideally:
>
> model1
> site4 2.4
> site1 2.8
> site5 4.0
> .
> .
> .
> siten 32
>
> As you see, the values are different, but the order is still the same, 
> that is
> if I sort the values, I still get the same sorting order.
>
> 1) I do not understand how to use AIC, ROC or Spearmann in this 
> situation!?
> Could you pleas explain?
>
> 2) I had thought about using Kendal or Wilcoxon. What do you think?
>
> Regards
>
> On Tuesday 03 November 2009 14:49:32 Aitor Gastón wrote:
>> You can compute Spearman's rank correlation coefficient using cor() 
>> between
>> predicted and observed values for each model and test differences.
>>
>>
>>
>> If observed data are binary and the predictions probabilities, you may 
>> use
>>  a discrimination statistic like Somers' Dxy Rank Correlation or receiver
>>  operating characteristic curve area using somers2() in Hmisc package.
>>
>>
>>
>> Hope this helps,
>>
>>
>>
>> Aitor
>>
>>
>>
>> ----- Original Message -----
>> From: "Corrado" <ct529 at york.ac.uk>
>> To: <r-sig-ecology at r-project.org>
>> Sent: Tuesday, November 03, 2009 1:36 PM
>> Subject: [R-sig-eco] Testing "order" on predicted data
>>
>> > Dear all,
>> >
>> > I have a strange situation:
>> >
>> > 1) I have some data that are associated with "sites"
>> > 2) I have two models that predict the data on the "sites"
>> > 3) I would like to understand which of the models predicts the order of
>> > the
>> > data better. In other words, I am not interested in the models 
>> > predicting
>> > the
>> > values exactly, but only in predicting values that are in the same 
>> > order
>> > (smaller to bigger).
>> >
>> > What is the best test?
>> >
>> > PS: Does that make sense?
>> >
>> > Best,
>> >
>> > Global Climate Change & Biodiversity Indicators
>> > Area 18,Department of Biology
>> > University of York, York, YO10 5YW, UK
>> > Phone: + 44 (0) 1904 328645, E-mail: ct529 at york.ac.uk
>> >
>> > _______________________________________________
>> > R-sig-ecology mailing list
>> > R-sig-ecology at r-project.org
>> > https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
>>
>
>
>
> -- 
> Corrado Topi
>
> Global Climate Change & Biodiversity Indicators
> Area 18,Department of Biology
> University of York, York, YO10 5YW, UK
> Phone: + 44 (0) 1904 328645, E-mail: ct529 at york.ac.uk
>
> _______________________________________________
> R-sig-ecology mailing list
> R-sig-ecology at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
>