[R-sig-eco] Calculate AIC, DIC and BIC for models machine learning

[Jane Elith]

Hi Lara, Sarah
There is a problem in estimation of degrees of freedom so even though some people try to estimate AIC eg for Maxent models, it's not correct (the number of parameters in the model is not the effective degrees of freedom). Same for GBM, RF etc.
e.g. see Hauenstein, S., Wood, S.N. & Dormann, C.F. (2018) Computing AIC for black-box models using generalized degrees of freedom: A comparison with cross-validation. Communications in Statistics - Simulation and Computation, 47, 1382–1396. 

Jane Elith

On 22/3/19, 10:05 pm, "R-sig-ecology on behalf of r-sig-ecology-request using r-project.org" <r-sig-ecology-bounces using r-project.org on behalf of r-sig-ecology-request using r-project.org> wrote:

    
    Message: 2
    Date: Thu, 21 Mar 2019 09:55:03 -0400
    From: Sarah Goslee <sarah.goslee using gmail.com>
    To: Lara Silva <lara.sfp.silva using gmail.com>
    Cc: "r-sig-ecology using r-project.org" <r-sig-ecology using r-project.org>
    Subject: Re: [R-sig-eco]  Calculate AIC, DIC and BIC for models
    	machine learning
    Message-ID:
    	<CAM_vju=D7AVWG5a9e=P6mFNyA9R3j+76k9DExD5vhxGz+Y6www using mail.gmail.com>
    Content-Type: text/plain; charset="utf-8"
    
    Yes, of course it is.
    
    Many of the machine learning packages in R offer AIC as part of their
    model summaries.
    
    You should probably spend some time with the machine learning taskview
    to discover more about R's extensive capabilities.
    
    https://cran.r-project.org/web/views/MachineLearning.html
    
    Sarah
    
    On Thu, Mar 21, 2019 at 9:40 AM Lara Silva <lara.sfp.silva using gmail.com> wrote:
    >
    > Hello everyone!
    >
    > In R, it is possible to calculate AIC, DIC, or BIC  for models machine
    > learning, like RF, ANN, GBM, MARS?
    >
    > Are there any functions or specific packages in R?
    >
    > Any suggestion?
    >
    > Thanks
    >
    > Lara
    >
    > <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
    > Sem
    > vírus. www.avast.com
    > <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
    > <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
    >
    >         [[alternative HTML version deleted]]