[R-sig-Geo] Calculating alternative measures of model performance for species distribution models

[Agus Camacho]

Dear all,

Im now struggling with the problem of demonstrating the performance of
species distribution models that I created (SDMs).

The area under the curve has been largely criticised:

http://onlinelibrary.wiley.com/doi/10.1111/j.1466-8238.2007.00358.x/full
http://www.riceanalytics.com/db3/00232/riceanalytics.com/_download/Is%20the%20AUC%20the%20Best%20Measure.pdf
http://www.mssanz.org.au/modsim09/J1/liu_c_J1b.pdf

And Robert has gently provided a workaround in R:
http://cran.r-project.org/web/packages/dismo/vignettes/sdm.pdf

Still, I'd like to compute a " straight classification error rate and
average squared error", as proposed by Rice, or then, the proportion of
explained deviance D2 or the , the coefficient of determination,as proposed
by Liu. However, I was not able to implement that in R.

Would anybody have a suggestion?

Here goes a reproducible example (it takes a little to run):

set.seed(64)
require(raster)
require(dismo)
x=data.frame(Longitude=rnorm(30,mean=-50,
s=3),Latitude=rnorm(30,mean=-10,s=3))

# calculate extent square for x
ext = extent( min(x$Longitude), max(x$Longitude), min(x$Latitude),
max(x$Latitude))

# dowload envir from here:
https://www.dropbox.com/s/3ctpfbij04kgcq7/LA.environment.grd?dl=0

envir = stack("C:/Users/Agus/Dropbox/Distributional range
size/rasters/LA.environment.grd")

# special division of presence and test data for removing spatial sorting
bias for calculation of AUC (Hijmans sdm in R 2014 and Hijmans 2012)
nr <- nrow(x)
s <- sample(nr, 0.25 * nr)
pres_train <- x[-s,]
pres_test <- x[s, ]

nr <- nrow(backgr)
s <- sample(nr, 0.25 * nr)

backgr <- randomPoints(envir, n=100)
back_train <- backgr[-s, ]
back_test <- backgr[s, ]
sb <- ssb(pres_test, back_test, pres_train)


xm <- maxent(envir,
            pres_train,
            factors=c(
"HWSD_ADD_PROP",
"HWSD_AWC_CLASS",
"HWSD_DRAINAGE",
"HWSD_T_CEC_CLAY",
"HWSD_T_CEC_SOIL",
#"HWSD_T_ECE",
"HWSD_T_ESP",
#"HWSD_T_OC",
"HWSD_T_PH_H2O",
"HWSD_T_TEXTURE",
"HWSD_T_USDA_TEX_CLASS"
                    ),
             removeDuplicates=TRUE)

# evaluate model
e = try(evaluate(pres_test, back_test, xm, envir),silent=T)


Thanks in advance.
-- 
Agustín Camacho Guerrero.
Doutor em Zoologia.
Laboratório de Herpetologia, Departamento de Zoologia, Instituto de
Biociências, USP.
Rua do Matão, trav. 14, nº 321, Cidade Universitária,
São Paulo - SP, CEP: 05508-090, Brasil.

	[[alternative HTML version deleted]]