[R-sig-eco] significance of GAMM terms

[Basil Iannone]

Dear Marc,

I believe that using a p-value to rank the importance of your model terms
is NOT a good idea, given that p-values communicate no information
regarding effect sizes. David Anderson and Ken Burnham have suggested an
approach to choosing the model that best fits one's data and assessing the
importance of terms using AIC values. I have attached a very useful paper
of theirs to this email and below is a book that Anderson wrote regarding
this issue. Although the paper does not address GAM (from what I recall),
the approach used can be utilized with many kinds of statistical models.
Hope this was helpful.

The book:
Model Based Inference in the Life Sciences: A Primer on Evidence  David R.
Anderson<http://www.amazon.com/s/ref=ntt_athr_dp_sr_1?_encoding=UTF8&sort=relevancerank&search-alias=books&ie=UTF8&field-author=David%20R.%20Anderson>

On Mon, Jan 30, 2012 at 3:39 AM, Marc Taylor <marchtaylor at gmail.com> wrote:

> Dear r-sig-ecology group,
>
> I have a general question about determining the significance of GAMM smooth
> terms. I have fit a GAMM model with several smooth terms, some of which
> contain an additional variance structures in the random part.
>
> My summary of the model looks something like this:
>
> Formula:
> CHLASURF ~ s(KPPmld, bs = "cr", k = -1, fx = FALSE) + s(PARmld,
>    bs = "cr", k = -1, fx = FALSE) + s(SST, bs = "cr", k = -1,
>    fx = FALSE) + s(SSS, bs = "cr", k = -1, fx = FALSE) + s(DSISURF,
>    bs = "cr", k = -1, fx = FALSE) + s(FESURF, bs = "cr", k = -1,
>    fx = FALSE)
>
> Parametric coefficients:
>             Estimate Std. Error t value Pr(>|t|)
> (Intercept) -0.001213   0.005727  -0.212    0.832
>
> Approximate significance of smooth terms:
>             edf Ref.df      F p-value
> s(KPPmld)  8.272  8.272 294.19  <2e-16 ***
> s(PARmld)  7.550  7.550 317.84  <2e-16 ***
> s(SST)     8.584  8.584 128.16  <2e-16 ***
> s(SSS)     7.909  7.909  51.73  <2e-16 ***
> s(DSISURF) 7.356  7.356  13.94  <2e-16 ***
> s(FESURF)  8.672  8.672  52.47  <2e-16 ***
> ---
> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>
> R-sq.(adj) =  0.871  Scale est. = 0.084409  n = 5478
>
>
>
> In this example KPPmld, PARmld, and DSISURF all have associated variance
> structures.
>
> Basically all terms are highly significant (due to the high n=5478), but I
> would like to be able to rank the variables in terms of their importance.
> Given that the above summary gives only "approximate significance", I
> gathered that the best test would be to drop terms and refit the model, and
> then to compare the "small" model with the full model using a Likelihood
> ratio. Unfortunately, there are some cases where the fitting of the smaller
> model doesn't converge (due to difficulties in fitting only 2 variance
> structures instead of 3 - an annoyance that I alluded to in a previous
> question but have been unable to remedy).
>
> My question is, since a Likelihood ratio can not be calculated in all
> cases, would it make sense to present the F-values above as a measure of
> each terms importance in the model? Do I have any other options for such a
> comparison? I have several geographic areas where I have fit similar models
> and would like to be able to say something about overall trends in the
> importance of the variables - something like in this graph (
> http://dl.dropbox.com/u/52403158/F_value_plot.png). My number of samples
> is
> always the same (n=5478), but the number of terms in each areas model can
> be different, and of course the degrees of freedom in the smooth terms are
> different for each area.
>
> Many thanks for your help and advice.
> Cheers,
> Marc
>
>        [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-ecology mailing list
> R-sig-ecology at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
>
>


-- 
Basil Iannone
University of Illinois at Chicago
Department of Biological Sciences (MC 066)
845 W. Taylor St.
Chicago, IL  60607-7060
Email: bianno2 at uic.edu
Phone: 312-355-3231
Fax: 312-413-2435
http://www2.uic.edu/~bianno2