As a follow-up to Greg's suggested graphical presentation, it seems like 
the Vuong test is sometimes used to compare fits of non nested models.

Brian

Brian S. Cade, PhD

U. S. Geological Survey
Fort Collins Science Center
2150 Centre Ave., Bldg. C
Fort Collins, CO  80526-8818

email:  brian_cade@usgs.gov
tel:  970 226-9326



From:
Greg Snow <Greg.Snow@imail.org>
To:
Boris New <new.boris@gmail.com>, "r-help@r-project.org" 
<r-help@r-project.org>
Date:
03/23/2011 01:14 PM
Subject:
Re: [R] Comparing non nested models with correlation coefficients 
(inspired from Lorch and Myers )
Sent by:
r-help-bounces@r-project.org



If you are interested in the fits, then I would just plot the fits.  Plot 
the fitted/predicted values from the 1st model as the x-values and the 
fitted/predicted values from the second model as the y-values.  It is best 
to plot on a square plotting region and use asp=1, probably also doing 
abline(0,1) after.  This gives a nice visual of how the predictions from 
the models compare.  You could take it a step further by taking the 2 sets 
of predictions and doing a Tukey mean-difference plot, or a Bland-Altman 
plot.

I am not sure that the test you suggest would be meaningful.  What null 
hypothesis is it testing and what does it mean if it is not rejected? 
(what is the power to reject?).  Do you really need a formal test?  If not 
then the plot is probably more meaningful.

-- 
Gregory (Greg) L. Snow Ph.D.
Statistical Data Center
Intermountain Healthcare
greg.snow@imail.org
801.408.8111


> -----Original Message-----
> From: r-help-bounces@r-project.org [mailto:r-help-bounces@r-
> project.org] On Behalf Of Boris New
> Sent: Wednesday, March 23, 2011 7:32 AM
> To: r-help@r-project.org
> Subject: [R] Comparing non nested models with correlation coefficients
> (inspired from Lorch and Myers )
> 
> Hi,
> 
> I would like to compare two models in R with the same dependant
> variable but
> different predictors (two different types of frequency and reaction
> times as
> RT).
> An editor told me to have a look at Lorch and Myers 1990.
> 
> Lorch and Myers use the following technique:
> 1) they perform regressions on individual subjects' data
> 2) they extract the beta weights
> 3) they run t-test on these beta weights.
> 
> The point is that I don't want to compare the "size effect" from the
> different models but how well the two models fit. So my idea was to
> extract
> the correlation coefficients instead of betas and doing t-tests on
> these.
> I checked and my correlation coefficients are normally distributed...
> 
> Is it ok to do that?
> 
> Best regards,
> 
>                [[alternative HTML version deleted]]
> 
> ______________________________________________
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-
> guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide 
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.



	[[alternative HTML version deleted]]

