[R-sig-ME] Question regarding random factor accounting for scores scaled by grade

[Phillip Alday]

Hi,

I wouldn't treat grade as a blocking/grouping variable here, because it 
doing so doesn't reflect the systematic differences between grades. 
Grades aren't just random offsets from some ideal mean value and you 
don't want to make statements about some abstract set of grades for 
which you've only observed a few concrete realizations. Instead, you 
want to make statements about the grades you've observed and which show 
a systematic structure. On the numerical end of things, six levels is 
right on the lower edge of what you need to be able to estimate random 
effects.

When moving grades into the fixed effects, you then have a choice. You 
can treat grade as a numerical, linear effect, but I suspect linearity 
is a rather strong assumption here. You could also try using a smoother 
or a polynomial fit. What you could do instead is treat grade as a 
categorical fixed effect. Using sequential difference coding (e.g. with 
MASS::contr.sdif), you would even get the individual estimates for each 
step increase in grade level.

It sounds like the SBAC score is scaled by grade level, i.e. that the 
raw score is converted to an abstract scale that is a statement relative 
to a given grade level. If this is the case, then the statements your 
model makes will be statements about change relative to the grade level 
and not about absolute changes in test score, even without having grade 
as a predictor in the model. I would still keep grade in the model, 
however, because it gives you some indication about whether students are 
able to improve beyond the scaling inherent in SBAC. For example, do 
students who start off behind relative to their current grade level fall 
further behind with each grade advancement or are they able to catch up?

I see you have school as a fixed effect. Unless you have a very small 
number of schools or only interested in making statements about the 
particular schools you observed, I would consider treating schools as a 
blocking variable.

Best,
Phillip

On 23/10/19 6:33 am, Ades, James wrote:
> Hi all:
> 
> I’m looking at SBAC standardized test scores (math in one model and English in the other) for middle schoolers (as the dependent variable) and then executive function task scores and demographic factors as explanatory variables.
> 
> So in a simple model looking at the relationship between a stroop task and the SBAC math variable I’d have the model:
> 
> model <- lmer(math.score ~ School + Ethnicity + Language.Fluency*attendance + Parent.Ed.Lvl +SpEd + t4.minus + eff.rt.stroop + (t4.minus|pid) + (1|grade) + (1|Teacher), data = ace, na.action = 'na.exclude', control = lmerControl(optimizer = "nloptwrap", calc.derivs = FALSE), REML = FALSE)
> 
> where t4.minus is the time in between timepoints (there were four, and they varied from participant to participant), pid is the participant, and eff.rt.stroop is the efficiency stroop score.
> 
> Since there are four timepoints over two years, there are ultimately six grades: 3rd graders who then become 4th graders,  fifth graders who then become sixth graders, and seventh graders who then become eighth graders.
> 
> My question is whether this random factor of grade would not only account for natural variance between grades (with the random intercept) but would also, in a principled and valid way, account for the fact that SBAC score falls on a continuous scale (between 2000-3000) that increases with each grade, such that an equivalent score of 2381 in 3rd grade would be 2411 in 4th grade. Explained (better?) here: http://www.smarterbalanced.org/assessments/scores/.
> 
> Thanks much,
> 
> James
> 
> 
> Have uploaded 10 rows here: https://drive.google.com/open?id=17FJjeln3Ipg0-uq0wQL9tWkJRnHTqdt6
> 
> 
> 	[[alternative HTML version deleted]]
> 
> _______________________________________________
> R-sig-mixed-models using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>