[R-sig-ME] Different results for between/within groups and within group regression analyses

Fri Jan 26 13:46:57 CET 2018

Dear Luca,

I´d usually use the Anova() function from John Fox´s car package for 
that. It will give you order-independent tests on the terms in your model.

Best wishes,
Christoph

Am 25.01.2018 um 23:30 schrieb Alday, Phillip:
> I don't think this set of questions was ever addressed in any of the subsequent threads on this or other forums .... and so I'll give you a quick non answer of bullet points:
>
>
> * "significance" should not be the driving force in whether or not an estimate is correct. While it does tell you something about the relative error in that estimate, but it tells you very little about how well that estimate describes the data or predicts future data.
>
>
> * Much as it is difficult and potentially problematic to interpret a main effect in the presence of interactions (after all, by definition, an interaction implies that the main effect changes conditional on another variable), it is also problematic to intentionally exclude a known interaction to better estimate the main effect.
>
>
> * However, this above advice has to be tempered by the issues related to statistical power, the bias-variance tradeoff and over- vs. underfitting -- a lot of preprints, journal articles and recent textbooks by members of this list and their collaborators have tried to address different aspects of this in the context of mixed models, but the issues are fundamentally the same for *all* statistical models. My advice is to find the parsimonious model that best describes the data (so the best model according to AIC or BIC) and look at the predictions that model makes and derive your inferences from that, instead of getting hung up on the significance of any one coefficient. The effects package can be particularly useful here, especially for plotting these things.
>
>
> Phillip
>
> On 13/01/18 23:07, Luca Danieli wrote:
> Hi Phillip,
>
> sorry if I ask you a question.
> In this moment I have a 3x4x8 matrix, where '3' is the number of groups, '8' the number of tests, and '4' the levels of the potential main effect.
>
> Following your explanation, I was before thinking that leaving interactions out of the models would give you a better approximation of the main effect. But now that I read it again, I am unsure about it.
>
> In my case, the '4' levels are nested in each set.
> If I write lmer(Score ~ pot_ME + random effects) I have no statistical significance.
> If I write lmer(Score ~ pot_ME*groups + random effects) I have statistical significance for the main effect (p<0.05) and a strong interaction (p<.001).
> If I write lmer(Score ~ pot_ME*groups*tests + random effects) I have no statistical significance nor interactions.
>
> What approach is the more correct to get information about the presence of a main effect?
>
> (My parameters are not-continuous)
>
> Best
> Luca
> ________________________________
> From: Alday, Phillip <Phillip.Alday at mpi.nl><mailto:Phillip.Alday at mpi.nl>
> Sent: 11 January 2018 15:38
> To: Luca Danieli; r-sig-mixed-models at r-project.org<mailto:r-sig-mixed-models at r-project.org>
> Subject: Re: [R-sig-ME] Different results for between/within groups and within group regression analyses
>
> I'll do it myself when I get the chance in the next day or so.  :-)
>
> Phillip
>
> ________________________________
> From: Luca Danieli <mr.lucedan at hotmail.it><mailto:mr.lucedan at hotmail.it>
> Sent: Thursday, January 11, 2018 10:26 AM
> To: Alday, Phillip; r-sig-mixed-models at r-project.org<mailto:r-sig-mixed-models at r-project.org>
> Subject: Re: [R-sig-ME] Different results for between/within groups and within group regression analyses
>
> Thank you Phillip!
>
> Can I add your answer to CrossValidated for future concerns by other users?
>
> Best
> Luca
> ________________________________
> From: Alday, Phillip <Phillip.Alday at mpi.nl><mailto:Phillip.Alday at mpi.nl>
> Sent: 11 January 2018 15:18
> To: Luca Danieli; r-sig-mixed-models at r-project.org<mailto:r-sig-mixed-models at r-project.org>
> Subject: Re: [R-sig-ME] Different results for between/within groups and within group regression analyses
>
> By only using one group, you're changing the amount of pooling going on, which affects shrinkage and the bias-variance / over- vs. underfitting tradeoff. When you fit a model to a subset, it will generally be better at describing that subset but often worse at describing the full set / other sets. In other words, your subset model better describes the subset because it doesn't have to spend "resources" describing the other data, but of course this also means that it will tend to not describe the other data as well - it's better at the small details but worse at the big picture.
>
> Best,
> Phillip
>
> Sent from my mobile, please excuse my brevity.
>
> ________________________________
> From: Luca Danieli <mr.lucedan at hotmail.it><mailto:mr.lucedan at hotmail.it>
> Sent: Thursday, January 11, 2018 10:10 AM
> To: r-sig-mixed-models at r-project.org<mailto:r-sig-mixed-models at r-project.org>
> Subject: [R-sig-ME] Different results for between/within groups and within group regression analyses
>
>
> Dear all,
>
> from CrossValidates I was suggested to repost my question to you, as it is a technical question about R and mixed models.
> Particularly, as I have a thesis to hand in in a few weeks, I hope you are able to help me understanding some problems that I cannot figure out by myself.
>
> In this case, I have used the function lmer() to look for an interaction between groups and then used the function predict() to plot the fits for each group on a graphic.
> Then I applied the lmer() to just one of those groups (same formula, technically) and used the predict() function to plot the fits for that specific group. I was thinking to obtain the same graphic for that group type and instead I obtained two different results.
>
> I explained the process, models and presented the plots in this post:
> https://stats.stackexchange.com/questions/322608/different-results-for-between-within-groups-and-within-group-regression-analyses
>
> Can somebody help me understand this?
>
> Best
> Luca
>
>
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-models at r-project.org<mailto:R-sig-mixed-models at r-project.org> mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>
>
> 	[[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

-- 

*Prof. Dr. Christoph Scherber*

Institute of Landscape Ecology
Animal ecology and multitrophic interactions 
<http://www.uni-muenster.de/tieroekologie/en/index.html>
Room 536
University of Münster
Heisenbergstr. 2
48149 Münster
Germany
phone +49(0)251-83 33 996
facsimile +49 (0)251 - 83 38 338

See also my Personal website <http://www.christoph-scherber.de>

____________________________________________________

*----See the following recently published manuscripts----*

  * *Biodiversity effects on ecosystem functioning in a 15-year
    grassland experiment: patterns, mechanisms, and open questions*
    <https://doi.org/10.1016/j.baae.2017.06.002>*(2017)*
  * *Consistent drivers of plant biodiversity across managed ecosystems
    <http://dx.doi.org/10.1098/rstb.2015.0284> (2016)*
  * *Trophic and Non-Trophic Interactions in a Biodiversity Experiment
    Assessed by Next-Generation Sequencing
    <http://dx.doi.org/10.1371/journal.pone.0148781> (2016)*
  * *Plant diversity increases spatio-temporal niche complementarity in
    plant-pollinator interactions
    <http://dx.doi.org/10.1002/ece3.2026>(2016)*
  * *Corridors restore animal-mediated pollination in fragmented
    tropical forest landscapes
    <http://rspb.royalsocietypublishing.org/content/283/1823/20152347>
    (2016) *

	[[alternative HTML version deleted]]