[R] Comparing non nested models with correlation coefficients (inspired from Lorch and Myers )

[Greg Snow]

If you are interested in the fits, then I would just plot the fits.  Plot the fitted/predicted values from the 1st model as the x-values and the fitted/predicted values from the second model as the y-values.  It is best to plot on a square plotting region and use asp=1, probably also doing abline(0,1) after.  This gives a nice visual of how the predictions from the models compare.  You could take it a step further by taking the 2 sets of predictions and doing a Tukey mean-difference plot, or a Bland-Altman plot.

I am not sure that the test you suggest would be meaningful.  What null hypothesis is it testing and what does it mean if it is not rejected? (what is the power to reject?).  Do you really need a formal test?  If not then the plot is probably more meaningful.

-- 
Gregory (Greg) L. Snow Ph.D.
Statistical Data Center
Intermountain Healthcare
greg.snow at imail.org
801.408.8111


> -----Original Message-----
> From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-
> project.org] On Behalf Of Boris New
> Sent: Wednesday, March 23, 2011 7:32 AM
> To: r-help at r-project.org
> Subject: [R] Comparing non nested models with correlation coefficients
> (inspired from Lorch and Myers )
> 
> Hi,
> 
> I would like to compare two models in R with the same dependant
> variable but
> different predictors (two different types of frequency and reaction
> times as
> RT).
> An editor told me to have a look at Lorch and Myers 1990.
> 
> Lorch and Myers use the following technique:
> 1) they perform regressions on individual subjects' data
> 2) they extract the beta weights
> 3) they run t-test on these beta weights.
> 
> The point is that I don't want to compare the "size effect" from the
> different models but how well the two models fit. So my idea was to
> extract
> the correlation coefficients instead of betas and doing t-tests on
> these.
> I checked and my correlation coefficients are normally distributed...
> 
> Is it ok to do that?
> 
> Best regards,
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-
> guide.html
> and provide commented, minimal, self-contained, reproducible code.