[R-sig-eco] Meaning and intrpretation of TSS and ROC in the biomod2 package in R (Giacomo May)

[Howard, Tim G (DEC)]

Giancomo,

TSS is "True Skill Statistic" and calculated as: sensitivity + specificity - 1. It is from this paper: 

Allouche, O., A. Tsoar, and R. Kadmon. 2006. Assessing the accuracy of species distribution models: prevalence, kappa and the true skill statistic (TSS). Journal of Applied Ecology 43:1223-1232.

Values can range from -1 to 1 with 0 being your model is no different than random. Models closer to 1 are better at discerning your presence and absence points given the cutoff supplied. 

A ROC plot evaluates over all cutoffs and usually the metric from the ROC plot is AUC, or area under the curve. Models that have higher AUC values are better at discerning presence and absence throughout the range of cutoffs.  Wikipedia does a pretty good job with ROC and AUC:

https://en.wikipedia.org/wiki/Receiver_operating_characteristic


To repeat, TSS applies to a specific threshold/cutoff (and thus to a specific confusion matrix) while ROC applies to the full range of thresholds. One use of a ROC plot is to find an appropriate threshold (threshold where curve is closest to upper left corner). 

Cheers, 
Tim

> Date: Thu, 21 Jul 2016 19:56:58 +0200
> From: "Giacomo May" <Giacomo_May94 at gmx.de>
> To: r-sig-ecology at r-project.org
> Subject: [R-sig-eco] Meaning and intrpretation of TSS and ROC in the
>         biomod2 package in R
> Message-ID:
>         <trinity-9edcd9ca-2b11-421c-908a-283fd3471eac-1469123818711 at 3capp-
> gmx-bs24>
> 
> Content-Type: text/plain; charset=UTF-8
> 
> 
> Dear R users,
> This is probably a pretty basic question, but since I am new to species
> distribution modeling? I was working through the explanatory vignette for
> the biomod2 package when I stumbled across it. So apparently TSS and ROC
> scores play a rather big role in the evaluation of the model but unfortunately
> no real explanation of their meaning or how they are to be interpreted is
> provided. I've been looking on the web as well but everything I found wasn't
> very satisfying so i was hoping to get some quick answers here. Can
> somebody maybe provide a short explanation of what is expressed through
> the TSS and ROC scores and what I can take from them ? Any help is greatly
> appreciated.
> Best,
> Giacomo
> 
> 
> 
> ------------------------------
> 
> Subject: Digest Footer
> 
> _______________________________________________
> R-sig-ecology mailing list
> R-sig-ecology at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
> 
> ------------------------------
> 
> End of R-sig-ecology Digest, Vol 100, Issue 6
> *********************************************