[R] Re : using predict() or fitted() from a model with offset; unsolved, included reproducible code

[sam_oi at yahoo.fr]

Thanks for your reply Mark, 
but no, using predict on the new data.frame does not help here.
 
I had first thought that the probelm was due the explanatory variable (age) and the offset one (date) being very similar (highly correlated, I am trying to tease their effect apart, and hoped offset would help in this since I know the relationship with age already). But this appears not to be the case. Simply, the predicted  (or fitted) values for the offset model always return predicted values based on the effect of the variable within offset(), completely ignoring the explanatory variable and that it is supposed to offset the effect in the first place:
such as, i get the same predicted values for the 2 very different models below. The summary table and coefficents remain perfectly valid though (and very different). 
 
lmAO<-glm(MassChange24h~T1+offset(-2*AGE),  family=gaussian,na.action=na.exclude)

lmAO<-glm(MassChange24h~AGE,  family=gaussian,na.action=na.exclude) 

Has anyone got any experience in predicting from models that include an offset term? Am I not specifying the offset term correctly in the model? Please get back to me if you have the slightest idea of what is going on. Or if you would know of another way than offset for my purposes


I include below reproducible code with dummy data. Models do not fit, but they work. 

Thank you

Samuel Riou

##########
AGE<- c(1:10) 
MassChange24h<-c(10,8,6,4,2,0,-2,-4,-6,-8)
T1<-c(10,11,12,13,14,15,16,17,18,19)  ### variable for which I want the effect, taking into acount the known effect of AGE
T<-c("A","B","A","B","A","B","A","B","A","B") ## added for testing 
T<-c(1,2,3,4,5,6,5,4,3,2) ## added for testing 

#no offset 
lmA<-glm(MassChange24h~T1, na.action=na.exclude, family=gaussian)
summary(lmA)
fitted(lmA)
#linear offset
lmAO<-glm(MassChange24h~T1+offset(-2*AGE),  family=gaussian,na.action=na.exclude) ### model 
lmAO<-glm(MassChange24h~T1+offset(AGE),  family=gaussian,na.action=na.exclude)
lmAO<-glm(MassChange24h~AGE,  family=gaussian,na.action=na.exclude) ###the fitted values from the offset model are the same as from this one!
summary(lmAO) ## table is fine, shows the effect of T1, taking into account the offset
fitted(lmAO)  ## Problem : getting same values as for model lmA
nd1<-expand.grid(T1=c(10,11,12,13,14,15,16,17,18,19))
Pred<-predict(lmA, nd1, type="response")
nd1<-expand.grid(T1=c(10,11,12,13,14,15,16,17,18,19))
Pred<-predict(lmAO, nd1, type="response") #### get same values as for model lmA , and changing T variable in the offset model, again i get the same predicted values...very strange
#################


----- Message d'origine ----
De : "markleeds at verizon.net" <markleeds at verizon.net>
À : sam_oi at yahoo.fr
Envoyé le : Dimanche, 12 Octobre 2008, 20h16mn 36s
Objet : RE: [R] using predict() or fitted() from a model with offset

hi: I haven't use fitted much but when you used predict, did you send in 
the new dataframe ? the code below says that you didn't
but i don't know if that would fix it anyway.



On Sun, Oct 12, 2008 at  6:36 AM, sam_oi at yahoo.fr wrote:

> Dear R-users,
>
> I have come across some difficulties using the offset argument in a 
> model. There is not much information in ?offset, ?lm, ?glm and 
> therefore have resorted to post here.
> Offset appears to work in the models I run becuase I get the expected 
> coefficients when comparing offset and non-offset models . BUT the 
> fitted values obtained from fitted() are identical in both models!! 
> Why is this? Is there an argument to add to fitted() so that it takes 
> the offset into accout? Note that I have included the offset in the 
> formula, as seen below in the code.
> I have also tried to use predict, with exactly the same result: the 
> offset is ignored. This applies to both lms and glms.
>
> Am I missing something here?
> Thank you
> Samuel Riou
>
> CODE
> #no offset lmA<-lm(MassChange24h~DATEN1, subset(Chicks1, Year==2007  & 
> AGE>10), na.action=na.exclude)
> summary(lmA)
>
> #linear offset
> lmAO<-lm(MassChange24h~DATEN1+offset(-0.37356*AGE), subset(Chicks1, 
> Year==2007  & AGE>10), na.action=na.exclude)
> summary(lmAO)
>
>
> print(Chicks$DATEN1[Year==2007  & AGE>10])
> print(t(fitted(lmA)))
> NEW<-cbind(as.vector(t(fitted(lmA))), Chicks$DATEN1[Year==2007  & 
> AGE>10])
> NEW<-as.data.frame(NEW)
> m1<-aggregate(NEW[1],NEW[2],mean, na.rm=TRUE)
> plot(m1$V1~m1$V2, pch=20, col="black") Pred<-predict(lmA)
>
> print(Chicks$DATEN1[Year==2007  & AGE>10])
> print(t(fitted(lmAO)))
> NEW<-cbind(as.vector(t(fitted(lmAO))), Chicks$DATEN1[Year==2007  & 
> AGE>10])
> NEW<-as.data.frame(NEW)
> m1<-aggregate(NEW[1],NEW[2],mean, na.rm=TRUE)
> points(m1$V1~m1$V2, pch=20, col="red") ###but the fitted values dont 
> seem to take into account the offset
> Pred<-predict(lmAO)
>
>
>
>
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide 
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.