Dear Harry,
 
You get different results because your model specification is
different!. The specificication of your first model seems wrong to me.
Having an intercept for each level is non-sense. You probably defined
the random effects as (MH_D |TAZ) + (APT_D|TAZ) + (ResOth_D|TAZ) +
(NonRes_D|TAZ) + (Vacant_noimp_D| TAZ ) + (Vacant_imp_D|TAZ). 
You should either use 
    (1|TAZ)  + (MH_D -1 |TAZ) + (APT_D - 1|TAZ) + (ResOth_D - 1|TAZ) +
(NonRes_D - 1|TAZ) + (Vacant_noimp_D - 1| TAZ ) + (Vacant_imp_D - 1|TAZ)

or
    (MH_D  + APT_D + ResOth_D + NonRes_D + Vacant_noimp_D +
Vacant_imp_D|TAZ)
 
The last model is equivalent with (HousingType|TAZ)
 
The difference between both models is the specication of the random
effects The first model assumes that the levels of Housingtype are
independent. The last model allows for correlation between those levels.
 
HTH,
 
Thierry

------------------------------------------------------------------------
----
ir. Thierry Onkelinx
Instituut voor natuur- en bosonderzoek / Research Institute for Nature
and Forest
Cel biometrie, methodologie en kwaliteitszorg / Section biometrics,
methodology and quality assurance
Gaverstraat 4
9500 Geraardsbergen
Belgium
tel. + 32 54/436 185
Thierry.Onkelinx@inbo.be
www.inbo.be

To call in the statistician after the experiment is done may be no more
than asking him to perform a post-mortem examination: he may be able to
say what the experiment died of.
~ Sir Ronald Aylmer Fisher

The plural of anecdote is not data.
~ Roger Brinner

The combination of some data and an aching desire for an answer does not
ensure that a reasonable answer can be extracted from a given body of
data.
~ John Tukey
  



________________________________

Van: Hongwei Dong [mailto:pdxdong@gmail.com] 
Verzonden: woensdag 5 augustus 2009 21:12
Aan: ONKELINX, Thierry
CC: r-help@r-project.org
Onderwerp: Re: [R] lme funcion in R


Thanks, Thierry and other R users.  

I estimate the model using the factor rather than the dummy variables I
used previously. It still takes forever for the function "lme" to run.
But "lmer" is much better with my large data size (about 60,000
observations). The interesting part is that the results from the model
using factor are slightly different from what I got from the model using
dummy variables, especially for the variables at random level.

The estimated random effects by using dummy variable are like this (each
dummy got one intercept):

 Random effects:
 Groups   Name           Variance Std.Dev. Corr   
 TAZ      (Intercept)    0.059160 0.24323         
          MH_D           0.215210 0.46391  -0.583 
 TAZ      (Intercept)    0.212061 0.46050         
          APT_D          0.205028 0.45280  -0.992 
 TAZ      (Intercept)    0.086223 0.29364         
          ResOth_D       0.305678 0.55288  0.665  
 TAZ      (Intercept)    0.161892 0.40236         
          NonRes_D       0.537284 0.73300  -0.874 
 TAZ      (Intercept)    0.088684 0.29780         
          Vacant_noimp_D 0.501495 0.70816  -0.570 
 TAZ      (Intercept)    0.136630 0.36964         
          Vacant_imp_D   0.368722 0.60722  -0.850 
 Residual                0.382439 0.61842         
Number of obs: 55762, groups: TAZ, 739

The estimated random effects by using factor are like this (one
intercept for all):

Random effects:
 Groups   Name                       Variance Std.Dev. Corr

 TAZ      (Intercept)                0.83894  0.91594

          HousingType1MH_D           0.23214  0.48181  -0.375

          HousingType1APT_D          0.28850  0.53712  -0.827  0.630

          HousingType1ResOth_D       0.29392  0.54214   0.156 -0.251
-0.165                      
          HousingType1NonRes_D       0.58169  0.76269  -0.572  0.155
0.656 -0.030               
          HousingType1Vacant_imp_D   0.45349  0.67342  -0.522  0.203
0.265  0.101  0.611        
          HousingType1Vacant_noimp_D 0.54146  0.73584  -0.286  0.251
0.265  0.390  0.313  0.475 
 Residual                            0.38228  0.61829

Number of obs: 55762, groups: TAZ, 739

The fixed coefficients for each group are also slightly different. I'm
wondering which one makes more sense.


Thanks.

Harry




Harry  



R still report error

On Wed, Aug 5, 2009 at 1:22 AM, ONKELINX, Thierry
<Thierry.ONKELINX@inbo.be> wrote:


	Harry,
	
	I you use dummy variables, then you can only use (n-1) dummy
variables
	if your variable has n levels. Otherwise you introduce
	multicollinearity! If you use n dummy variable then you can
express one
	dummy variable as a linear combination of the others.
	
	Make use of a factor variable. That is much easier to work with
that
	dummy variables. The model itself will create the necessary
dummy
	variables.
	
	lusdrdata$HousingType <- factor(lusdrdata$HousingType, levels =
1:6,
	labels = c("Reference", "MH_D", "APT_D", "ResOth_D", "NonRes_D",
	"Vacant_D"))
	lme(fixed = LN_unitlandval ~ HousingType +
	access_emp1+pct_vacant+transit_D +park_dum,data=lusdrdata,
random = ~
	HousingType | TAZ)
	

	HTH,
	
	Thierry
	
	
------------------------------------------------------------------------
	----
	ir. Thierry Onkelinx
	Instituut voor natuur- en bosonderzoek / Research Institute for
Nature
	and Forest
	Cel biometrie, methodologie en kwaliteitszorg / Section
biometrics,
	methodology and quality assurance
	Gaverstraat 4
	9500 Geraardsbergen
	Belgium
	tel. + 32 54/436 185
	Thierry.Onkelinx@inbo.be
	www.inbo.be
	
	To call in the statistician after the experiment is done may be
no more
	than asking him to perform a post-mortem examination: he may be
able to
	say what the experiment died of.
	~ Sir Ronald Aylmer Fisher
	
	The plural of anecdote is not data.
	~ Roger Brinner
	
	The combination of some data and an aching desire for an answer
does not
	ensure that a reasonable answer can be extracted from a given
body of
	data.
	~ John Tukey
	
	-----Oorspronkelijk bericht-----
	Van: r-help-bounces@r-project.org
[mailto:r-help-bounces@r-project.org]
	Namens Hongwei Dong
	
	Verzonden: woensdag 5 augustus 2009 1:49
	
	Aan: r-help@r-project.org
	Onderwerp: Re: [R] lme funcion in R
	
	Yeah, I have a very large sample size, about 60,000
observations.
	Multicollinearity should not be a problem here. The weird thing
is that
	SPSS can converge very quickly and gives out reasonable results.
	The only problem I can think of is that, my first level (random)
	variables are dummy variables: 6 housing types, and I used five
dummies
	in model and one as the reference. I also tried to combine them
into two
	groups and use only dummy at random level, but it does not work
either.
	
	is there any one here has similar experience with the LME
function in R?
	
	Thanks.
	
	Harry
	
	
	
	On Tue, Aug 4, 2009 at 1:28 AM, ONKELINX, Thierry
	<Thierry.ONKELINX@inbo.be>wrote:
	
	> Dear Harry,
	>
	> Your model seems rather complex. Do you have enough data to
support
	it?
	> Did you check for multicollinearity between the variables?
	>
	> HTH,
	>
	> Thierry
	>
	>
	>
	>
----------------------------------------------------------------------
	> --
	> ----
	> ir. Thierry Onkelinx
	> Instituut voor natuur- en bosonderzoek / Research Institute
for Nature
	
	> and Forest Cel biometrie, methodologie en kwaliteitszorg /
Section
	> biometrics, methodology and quality assurance Gaverstraat 4
9500
	> Geraardsbergen Belgium tel. + 32 54/436 185
Thierry.Onkelinx@inbo.be
	> www.inbo.be
	>
	> To call in the statistician after the experiment is done may
be no
	> more than asking him to perform a post-mortem examination: he
may be
	> able to say what the experiment died of.
	> ~ Sir Ronald Aylmer Fisher
	>
	> The plural of anecdote is not data.
	> ~ Roger Brinner
	>
	> The combination of some data and an aching desire for an
answer does
	> not ensure that a reasonable answer can be extracted from a
given body
	
	> of data.
	> ~ John Tukey
	>
	> -----Oorspronkelijk bericht-----
	> Van: r-help-bounces@r-project.org
	> [mailto:r-help-bounces@r-project.org]
	> Namens Hongwei Dong
	> Verzonden: maandag 3 augustus 2009 19:45
	> Aan: r-help@r-project.org
	> Onderwerp: Re: [R] lme funcion in R
	>
	> Thanks for the replies above. Here are my script and data
structure:
	> library(nlme)
	> tlevel<-lme(fixed = LN_unitlandval ~
	>
MH_D+APT_D+ResOth_D+NonRes_D+Vacant_D+access_emp1+pct_vacant+transit_D
	> +p
	> ark_dum,data=lusdrdata,random
	> = ~MH_D+APT_D+ResOth_D+NonRes_D+Vacant_D | TAZ)
	>
	> str:
	>
	> $ TAZ : int 100 100 100 100 100 100 100 100 100 100 ...
	> $ MH_D : num 0 0 0 0 0 0 0 0 0 0 ...
	> $ APT_D : num 0 0 0 0 0 0 0 0 0 0 ... $ ResOth_D : num 0 0 0 0
0 0 0 0
	
	> 0 0 ... $ NonRes_D : num 0 0 0 0 0 0 0 0 0 1 ...
	> $ Vacant_D : num 1 1 1 0 0 1 1 1 1 0 ...
	> $ access_emp1 : num 45.8 45.8 45.8 45.8 45.8 ...
	> $ pct_vacant : num 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 ...
$
	> transit_D :
	> num 0 0 0 0 0 0 0 0 0 0 ... $ park_dum : num 0 0 0 0 0 0 0 0 0
0 ...
	>
	>
	> Thanks.
	>
	> Harry
	>
	>
	>
	> On Mon, Aug 3, 2009 at 10:36 AM, Jason Morgan
<jwm-r-help@skepsi.net>
	> wrote:
	>
	> > On 2009.08.03 10:15:46, Hongwei Dong wrote:
	> > > Hi, R users,
	> > >   I'm using the "lme" function in R to estimate a 2 level
mixed
	> > > effects model, in which the size of the subject groups are
	> > > different. It turned
	> > out
	> > > that It takes forever for R to converge. I also tried the
same
	> > > thing
	>
	> > > in
	> > SPSS
	> > > and SPSS can give the results out within 20 minutes.
Anyone can
	> > > give
	>
	> > > me
	> > some
	> > > advice on the lme function in R, especially why R does not
	converge?
	> > Thanks.
	> > >
	> > > Harry
	> >
	> > Hello Harry,
	> >
	> > As Chuck mentions, providing some more information on the
model and
	> > the data you are using would be helpful. Also, be sure to
compare
	> > the optimization methods used in SPSS to that used in R. You
can
	> > change the optimization method in R if the default seems to
be
	> > causing issues. See help(lmeControl) for numerous setting
options.
	> >
	> > ~Jason
	> >
	> > --
	> > Jason W. Morgan
	> > Graduate Student
	> > Department of Political Science
	> > *The Ohio State University*
	> > 154 North Oval Mall
	> > Columbus, Ohio 43210
	> >
	> >
	>
	>         [[alternative HTML version deleted]]
	>
	> ______________________________________________
	> R-help@r-project.org mailing list
	> https://stat.ethz.ch/mailman/listinfo/r-help
	> PLEASE do read the posting guide
	> http://www.R-project.org/posting-guide.html
	> and provide commented, minimal, self-contained, reproducible
code.
	>
	> Dit bericht en eventuele bijlagen geven enkel de visie van de
	> schrijver weer en binden het INBO onder geen enkel beding,
zolang dit
	> bericht niet bevestigd is door een geldig ondertekend
document. The
	> views expressed in  this message and any annex are purely
those of the
	
	> writer and may not be regarded as stating an official position
of
	> INBO, as long as the message is not confirmed by a duly signed
	> document.
	>
	
	       [[alternative HTML version deleted]]
	
	______________________________________________
	R-help@r-project.org mailing list
	https://stat.ethz.ch/mailman/listinfo/r-help
	PLEASE do read the posting guide
	http://www.R-project.org/posting-guide.html
	and provide commented, minimal, self-contained, reproducible
code.
	
	Dit bericht en eventuele bijlagen geven enkel de visie van de
schrijver weer
	en binden het INBO onder geen enkel beding, zolang dit bericht
niet bevestigd is
	door een geldig ondertekend document. The views expressed in
this message
	and any annex are purely those of the writer and may not be
regarded as stating
	an official position of INBO, as long as the message is not
confirmed by a duly
	signed document.
	



Dit bericht en eventuele bijlagen geven enkel de visie van de schrijver weer 
en binden het INBO onder geen enkel beding, zolang dit bericht niet bevestigd is
door een geldig ondertekend document. The views expressed in  this message
and any annex are purely those of the writer and may not be regarded as stating 
an official position of INBO, as long as the message is not confirmed by a duly 
signed document.

	[[alternative HTML version deleted]]


