
> On 01/07/2008, at 5:07 AM, Antonio P. Ramos wrote:
>
>> Thanks for the comments.
>>
>> What I'm doing is very simple: I'm running an one dimensional item
>> response model, similar to the ones use in psychology and educational
>> testing data via Markov Chain Monte Carlo Methods.
>>
>>
>> model_m12<- ideal(rollcall_m2,  maxiter = 500 000 000, thin = 1000,
>> burnin = 5000,
>> 		   store.item = TRUE,  normalize=T,
>> priors=list(xp=1e-12,xpv=1e-12,bp=1e-12,bpv=1e-12), verbose=T)
>>
>> # my data matrix is provided by the rollcall object, but it has just
>> 155*17 dimensions: rollcall_m2
>>
>> # the number of interactions is maxiter/thin= 500,000
>>
>> # store.item=true is the main source of the problem: it's store the
>> discrimination
>> # parameter, which consumer a large amount of memory.  
>> Unfortunately, I
>> need this information.
>>
>>
>> So, if R can access up to 3.5 Gb how can I fix the problem. I'm sure
>> lot's of mac users will be also interested in increasing its memory
>> allocation capabilities in R.
>>
>>
>
> I would wonder about the need for 500000 MCMC samples.
>
> Also I assume 17 discriminant parameters so 500000*17*8=68M or 136M
> with the difficulty, which seems much less than 1.2G.
>
> Ken

Hi

I am the author of the pscl package in which the ideal() function  
resides.

I agree that 5e8 iterations seems like an awful lot, even thinned by  
1e3 down to 5e5 stored iterations.  I'd try thinning that out some  
more, if you really need that many iterations.  I've seen some of  
these rollcall data that do require many iterations to get the  
standard MCMC convergence diagnostics to look reasonable: e.g., the  
item parameters for very lop-sided rollcalls, or the latent trait for  
a legislator/test-taker with an "extreme" voting history/response  
profile.  But even there I'd say "many" \approx 1e6, and there is  
certainly no need to save them all.

also, do note that there is an option in the ideal() function to dump  
the output of the MCMC algorithm to file, rather than trying to store  
it all in memory: see ?ideal and the file argument.  If you are  
generating a massive amount of MCMC output this option might help you  
manage it in a more sane way (say, summarizing it in chunks, either of  
the parameters or of the saved iterations, etc).

and not a R-sig-mac point, but alternative parameterizations can help  
with these very slow mixing analyses: but at that point you're looking  
at BUGS/JAGS and friends.

Regards

-- Simon Jackman

Professor Simon Jackman
Dept of Political Science, and (by courtesy) Statistics,
Director, Political Science Computational Laboratory,
Director, Methods of Analysis Program in the Social Sciences,
Stanford University, Stanford CA 94305-6044
http://jackman.stanford.edu




	[[alternative HTML version deleted]]

