[R-sig-Geo] Evaluating maxent model in dismo package

[Robert J. Hijmans]

Manuel,

You are making two maxent models "me" (with all data) and "me2" (with
partitioned data) (I realize you copied this this error in the help
file..., sorry about that). You could test (with some caveats) me2
with the data withheld (see below). But I think you are testing me
instead (using data it had to fit the model). Even so, 1 is a high AUC
score. Perhaps you have something like a spatially clustered clump
presence points and absence points that are far away (i.e. a somewhat
trivial problem).

# predict to testing points
testp <- predict(me2, pvtest)
head(testp)
testa <- predict(me2, avtest)



Here is a fixed example that I have put in the help file


# get predictor variables
predictors <- stack(list.files(path=paste(system.file(package="dismo"),
'/ex', sep=''), pattern='grd', full.names=TRUE ))
plot(predictors)

# file with presence points
occurence <- paste(system.file(package="dismo"), '/ex/bradypus.csv', sep='')
occ <- read.table(occurence, header=TRUE, sep=',')[,-1]

# witholding a 20% sample for testing
fold <- kfold(occ, k=5)
occtest <- occ[fold == 1, ]
occtrain <- occ[fold != 1, ]

# fit model, biome is a categorical variable
me <- maxent(predictors, occtrain, factors='biome')
# see the maxent results in a browser:
# me

# use "args"
# me2 <- maxent(predictors, occtrain, factors='biome', args=c("-J", "-P"))

# plot showing importance of each variable
plot(me)

# predict to entire dataset
r <- predict(me, predictors, progress='text')
# r <- predict(me, predictors, progress='text', args=c("outputformat=raw"))

plot(r)
points(occ)

#testing
# background data
bg <- randomPoints(predictors, 1000)

#simplest way to use 'evaluate'
e1 = evaluate(me, p=occtest, a=bg, x=predictors)

# alternative 1
# extract values
pvtest <- data.frame(xyValues(predictors, occtest))
avtest <- data.frame(xyValues(predictors, bg))

e2 = evaluate(me, p=pvtest, a=avtest)

# alternative 2
# predict to testing points
testp <- predict(me, pvtest)
head(testp)
testa <- predict(me, avtest)

e3 = evaluate(p=testp, a=testa)
e3
plot(e3, 'ROC')


On Thu, Oct 21, 2010 at 3:51 AM, Manuel Spínola <mspinola10 at gmail.com> wrote:
> Dear list member,
>
> I am running a maxent model in dismo package and I got AUC = 1, but I am
> not sure if this is right.  Is there a way to find out if my evaluation
> is correct?
> Here is my code:
>
> # fit model
> me <- maxent(predictores, dendro)
>
> # plot showing importance of each variable
> plot(me)
>
> # predict to entire dataset
> r <- predict(me, predictores, progress='text')
> # r <- predict(me, predictors, progress='text', args=c("outputformat=raw"))
>
> plot(r)
> points(dendro)
>
>
> # The below is equivalent (except that we use
> # k-fold partitioning of the data for testing purposes
>
> # extract values
> pv <- data.frame(xyValues(predictores, dendro))
> fp <- kfold(pv, k=5)
> pvtrain <- pv[fp != 1, ]
> pvtest <- pv[fp == 1, ]
> head(pvtest)
>
> # background data
> bg <- randomPoints(predictores, 210)
> av <- data.frame(xyValues(predictores, bg))
> fb <- kfold(av, k=5)
> avtrain <- av[fb != 1, ]
> avtest <- av[fb == 1, ]
>
> # combine train presence and background data
> train <- rbind(pvtrain, avtrain)
> # vector indicating presence or absence
> p <- c(rep(1, nrow(pvtrain)), rep(0, nrow(avtrain)))
>
> # fit maxent model
> me2 <- maxent(x=train, p=p)
>
> # predict to testing points
> testp <- predict(me, pvtest)
> head(testp)
> testa <- predict(me, avtest)
>
> #evaluate
> e = evaluate(p=testp, a=testa)
> e
> plot(e, 'ROC')
>
> RESULTADOS:
>  > e
> class            : ModelEvaluation
> n presences      : 14
> n absences       : 42
> AUC              : 1
> cor              : 0.9428767
> TPR+TNR threshold: 0.04
>
>
> Thank you very much in advance.
> Best,
>
> Manuel
>
>
> --
> *Manuel Spínola, Ph.D.*
> Instituto Internacional en Conservación y Manejo de Vida Silvestre
> Universidad Nacional
> Apartado 1350-3000
> Heredia
> COSTA RICA
> mspinola at una.ac.cr
> mspinola10 at gmail.com
> Teléfono: (506) 2277-3598
> Fax: (506) 2237-7036
> Personal website: Lobito de río
> <https://sites.google.com/site/lobitoderio/>
> Institutional website: ICOMVIS <http://www.icomvis.una.ac.cr/>
>
>        [[alternative HTML version deleted]]
>
>
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at stat.math.ethz.ch
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
>