[R-sig-ME] Question regarding random factor accounting for scores scaled by grade
Phillip Alday
ph||||p@@|d@y @end|ng |rom mp|@n|
Mon Dec 23 13:47:28 CET 2019
Hi,
I wouldn't treat grade as a blocking/grouping variable here, because it
doing so doesn't reflect the systematic differences between grades.
Grades aren't just random offsets from some ideal mean value and you
don't want to make statements about some abstract set of grades for
which you've only observed a few concrete realizations. Instead, you
want to make statements about the grades you've observed and which show
a systematic structure. On the numerical end of things, six levels is
right on the lower edge of what you need to be able to estimate random
effects.
When moving grades into the fixed effects, you then have a choice. You
can treat grade as a numerical, linear effect, but I suspect linearity
is a rather strong assumption here. You could also try using a smoother
or a polynomial fit. What you could do instead is treat grade as a
categorical fixed effect. Using sequential difference coding (e.g. with
MASS::contr.sdif), you would even get the individual estimates for each
step increase in grade level.
It sounds like the SBAC score is scaled by grade level, i.e. that the
raw score is converted to an abstract scale that is a statement relative
to a given grade level. If this is the case, then the statements your
model makes will be statements about change relative to the grade level
and not about absolute changes in test score, even without having grade
as a predictor in the model. I would still keep grade in the model,
however, because it gives you some indication about whether students are
able to improve beyond the scaling inherent in SBAC. For example, do
students who start off behind relative to their current grade level fall
further behind with each grade advancement or are they able to catch up?
I see you have school as a fixed effect. Unless you have a very small
number of schools or only interested in making statements about the
particular schools you observed, I would consider treating schools as a
blocking variable.
Best,
Phillip
On 23/10/19 6:33 am, Ades, James wrote:
> Hi all:
>
> I’m looking at SBAC standardized test scores (math in one model and English in the other) for middle schoolers (as the dependent variable) and then executive function task scores and demographic factors as explanatory variables.
>
> So in a simple model looking at the relationship between a stroop task and the SBAC math variable I’d have the model:
>
> model <- lmer(math.score ~ School + Ethnicity + Language.Fluency*attendance + Parent.Ed.Lvl +SpEd + t4.minus + eff.rt.stroop + (t4.minus|pid) + (1|grade) + (1|Teacher), data = ace, na.action = 'na.exclude', control = lmerControl(optimizer = "nloptwrap", calc.derivs = FALSE), REML = FALSE)
>
> where t4.minus is the time in between timepoints (there were four, and they varied from participant to participant), pid is the participant, and eff.rt.stroop is the efficiency stroop score.
>
> Since there are four timepoints over two years, there are ultimately six grades: 3rd graders who then become 4th graders, fifth graders who then become sixth graders, and seventh graders who then become eighth graders.
>
> My question is whether this random factor of grade would not only account for natural variance between grades (with the random intercept) but would also, in a principled and valid way, account for the fact that SBAC score falls on a continuous scale (between 2000-3000) that increases with each grade, such that an equivalent score of 2381 in 3rd grade would be 2411 in 4th grade. Explained (better?) here: http://www.smarterbalanced.org/assessments/scores/.
>
> Thanks much,
>
> James
>
>
> Have uploaded 10 rows here: https://drive.google.com/open?id=17FJjeln3Ipg0-uq0wQL9tWkJRnHTqdt6
>
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-models using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>
More information about the R-sig-mixed-models
mailing list