[R] How to combine two learned regression models?

[David Winsemius]

On Aug 25, 2011, at 12:38 AM, Andra Isan wrote:

> Hi All,
>
> I have a set of features of size p and I would like to separate my  
> feature space into two sets so that p = p1 + p2, p1 is a set of  
> features and p2 is another set of features and I want to fit a glm  
> model for each sets of features separately.

You will be extracting the parameters and creating a summary variable  
for each model? That sounds pretty straight-forward. That's not the  
point you have questions about, right? Let's call teh functions that  
you create p1pred and p2pred.

> Then I want to combine the results of two glm models with a  
> parameter beta. For example, beta * F(p1) + (1-beta) * F(p2) where  
> F(p1) is a learned model for feature set p1 and F(p2) is the learned  
> model for feature set p2. Is there any way to do that in R?

In any GLM fitting program that provides an offset term (as does R's  
glm(.) ), you can construct:

y = beta1*(p1pred  -p2pred) + offset(p2pred)

# you would create the difference score first or wrap the difference  
of predictions in the I() function.

This has the same fitted values as would:

y = beta1*p1pred + (1-beta2)*p2pred

I'm not sure about the inferential statistics. Seems to me that they  
would be acceptable, but I am not a statistician.  Have you looked at  
the 'flexmix' package? I suspect its got that all mapped out in the  
vignette. If it can model difference components with varying  
distributions, it should be able to model "parceled features". Too bad  
I don't understand the theoretical presentation.

http://finzi.psych.upenn.edu/R/library/flexmix/doc/regression-examples.pdf

>
> There is a package called mixtools which can fit a mixture of two  
> regression models but it does not separate the features. I would  
> also like to separate features and fit a model for each feature set  
> and then combine them.
>
> Thanks,
> Andra

--

David Winsemius, MD
West Hartford, CT