[R-sig-eco] mvabund: difference between 'glm1path' and 'manyglm'

[Joanne Potts]

Ah ok thanks David! I downloaded from Github and all is making sense for me
now.

Most appreciated,

Jo




On Tue, Dec 4, 2018 at 11:51 PM David Warton <david.warton using unsw.edu.au>
wrote:

> Hi Jo,
>
> OooooK, I see your point, sorry for slowness.  Yes the residuals look a
> bit wonky on your first glm1path plot (ft2).  I’ve dug into the reason and
> found a bug in the code for residual.glm1path nbinom fits, where it was
> using the wrong parameterisation of overdispersion in computing residuals.
> Now fixed on github J.
>
> All the best
>
> David
>
>
>
>
>
>
>
> *From:* Joanne Potts <joanne using theanalyticaledge.com>
> *Sent:* Wednesday, 5 December 2018 9:49 AM
> *To:* David Warton <david.warton using unsw.edu.au>
> *Cc:* r-sig-ecology using r-project.org
> *Subject:* Re: mvabund: difference between 'glm1path' and 'manyglm'
>
>
>
> Thanks David for getting back to me.  I think I have followed your answer,
> thank you, and I get that when one specifies the theta value, all the
> ft3$phis are now constant for each lambda.
>
>
>
> Now I wonder if there is any value of ever specifying "
> negative.binomial(theta) " as I did below with ft3 (cf the ?glm1path helpfile)
> to improve the residuals, when using the LASSO? I guess I always thought
> the LASSO was a more robust way to select models but it seems the residuals
> of ft2 suggest otherwise.
>
>
>
> These questions are motivated for some over dispersed seal-fish data for a
> student in Sydney (as we've discussed off list) but I guess these questions
> are more of a theoretical nature. I over came my social phobia of posting
> on a list instead of hassling you privately(!), maybe someone else can
> value from this discussion too :)
>
>
>
> Thanks once again,
>
>
>
> Jo
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Mon, Dec 3, 2018 at 12:10 AM David Warton <david.warton using unsw.edu.au>
> wrote:
>
> Hi Jo,
>
> Thanks for the e-mail, always good to see statistical modelling questions
> on this list!
>
>
>
> In the mvabund package, you can fit trait models using different methods
> of estimation, method=”manyglm” will fit a GLM, “glm1path” will fit a GLM
> with a LASSO penalty (chosen using BIC by default but there are other
> options).  The way we coded LASSO negative binomial regression was to
> update estimates of the overdispersion parameter as the slope parameters
> update.  Because the LASSO fit gives different slope parameters, it will
> also have a different overdispersion parameter.  It probably has a larger
> overdispersion parameter, because the LASSO pushes slope parameters away
> from the best (in-sample) fit hence there is more unexplained variation in
> the LASSO model.
>
>
>
> All the best
>
> David
>
>
>
> Professor David Warton
>
> School of Mathematics and Statistics, Evolution & Ecology Research Centre,
> Centre for Ecosystem Science
>
> UNSW Sydney
>
> NSW 2052 AUSTRALIA
>
> phone +61(2) 9385 7031
>
> fax +61(2) 9385 7123
>
>
>
> http://www.eco-stats.unsw.edu.au
>
>
>
>
>
>
>
> *From:* Joanne Potts <joanne using theanalyticaledge.com>
> *Sent:* Friday, 30 November 2018 1:51 AM
> *To:* r-sig-ecology using r-project.org
> *Cc:* David Warton <david.warton using unsw.edu.au>
> *Subject:* mvabund: difference between 'glm1path' and 'manyglm'
>
>
>
>
>
> Hi David and list,
>
>
>
> Can someone please help me understand why, when changing the
> 'method=manyglm' argument to 'method=glm1path' under default settings
> (negative binomial) the estimates of theta change in the 'trait.glm'
> function?
>
>
>
> I have provided example code below us the antTraits data set. And you
> should see the plots for ft and ft3 are similar, yet ft2 is quite
> different, so I think I am missing something (no doubt, probably very
> obvious!).
>
>
>
> Advice appreciated, thank you.
>
>
>
> Jo
>
>
>
>
>
>
>
> data(antTraits)
>
>
>
>
> ft=traitglm(antTraits$abund,antTraits$env,antTraits$traits,method="manyglm")
>
> ft$phi
>
> ft$theta
>
> qqnorm(residuals(ft)); abline(c(0,1),col="red")
>
>
>
>
> ft2=traitglm(antTraits$abund,antTraits$env,antTraits$traits,method="glm1path")
>
> mean(ft2$phis)
>
> qqnorm(residuals(ft2)); abline(c(0,1),col="red")
>
>
>
>
>
> ft3=traitglm(antTraits$abund,antTraits$env,antTraits$traits,method="glm1path",
> negative.binomial(theta=1.641763))
>
> 1/mean(ft3$phis)
>
> qqnorm(residuals(ft3)); abline(c(0,1),col="red")
>
>
>
>
>
> --
>
>
>
> Kind regards,
>
>
>
> Joanne Potts
>
>
>
>
>
> ___________________
>
> Statistical Consultant
>
> theanalyticaledge.com
>
>
>
>
> --
>
>
>
> Kind regards,
>
>
>
> Joanne Potts
>
>
>
>
>
> ___________________
>
> Statistical Consultant
>
> theanalyticaledge.com
>

	[[alternative HTML version deleted]]