[R-sig-Geo] ROC or TSS?

[Wilfried Thuiller]

Dear Maurizio,

Both measures give somehow similar results. A AUC close to 0.5 means your models are not better than random predictions. A TSS close to 0 means the same.`
In other words, your models are generally not very good, excepted MARS and Random Forest, using both evaluation metrics.

HIH,
Wilfried

 


Le 8 oct. 2014 à 13:13, Maurizio Marchi <mauriziomarchi85 at gmail.com> a écrit :

> Dear List Members,
> I have some troubles with models evaluation. I'm using biomod2 package and
> I'm comparing 7 models with an Evaluation dataset. I obtain very different
> results between ROC and TSS as shown here:
> 
>> myBiomodModelEval["ROC","Evaluating.data",,,]
>               RUN1  RUN2  RUN3  Full
> GLM        0.642 0.651 0.673 0.654
> GAM        0.522 0.493 0.495 0.509
> GBM        0.582 0.576 0.580 0.581
> ANN         0.760 0.613 0.671 0.801
> MARS      0.902 0.892 0.898 0.902
> RF            0.814 0.809 0.770 0.800
> MAXENT 0.638 0.616 0.606 0.614
> 
>> myBiomodModelEval["TSS","Evaluating.data",,,]
>                RUN1   RUN2   RUN3   Full
> GLM        0.023  0.014  0.000  0.007
> GAM       -0.093 -0.095 -0.095 -0.099
> GBM         0.202 -0.134  0.244  0.204
> ANN          0.386  0.311  0.257  0.469
> MARS       0.614  0.402  0.608  0.634
> RF            0.469  0.389  0.349  0.406
> MAXENT  0.219  0.326  0.259  0.280
> 
> How good are my models? Should I use ROC or TSS?
> 
> Many thanks,
> 
> -- 
> Maurizio Marchi, Ph.D. student
> Florence, Italy
> ID skype: maurizioxyz
> Ubuntu 14.04 LTS
> linux user 552742
> 
> 	[[alternative HTML version deleted]]
> 
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo

-----------------------------
Dr. Wilfried Thuiller
Laboratoire d'Ecologie Alpine, UMR CNRS 5553
Université Joseph Fourier
BP53, 38041 Grenoble cedex 9, France
tel: +33 (0)4 76 51 44 97
fax: +33 (0)4 76 51 42 79

Email: wilfried.thuiller at ujf-grenoble.fr
Personal website: http://www.will.chez-alice.fr
Team website: http://www-leca.ujf-grenoble.fr/equipes/emabio.htm

ERC Starting Grant TEEMBIO project: http://www.will.chez-alice.fr/Research.html