[R] bitwise addition

[Nameeta Lobo]

Hi all

I am actually trying to create a permuatation matrix whose number of columns
equals the number of subjects I have. This matrix is actually supposed to have
-1s and 1s in a unique combination and hence the expand grid question before.
The reason I need this was because I was going to multiply the conditions by
this matrix and calculate a t-test or any other stat and I needed unique
simulations.(these simulations were being generated for further calculations to
find significant data). As the number of subjects grew more, we decided to
switch signs for just half the number of subjects so as to reduce the number of
simulations and get more reliable output. Besides for e.g. 1 1 1 -1 and -1 -1 -1
1 would give me the same output being mirror images and so I could reduce the
number of simulations even more. 

I had written all this is a for loop with the 1s and -1s and then I wasnt really
sure if I tried bitwise addition whether it would be faster. I thought I could
then just replace the 0s in the matrix by -1s. I knew that computer time as well
as RAM required would be a major problem but was still unsure. I have 11
subjects in one study and so I followed the 2^11 with the expand grid and with
switching half the signs just change it for 6 of the 11 but just wanted to know
how I would manage with a higher number of subjects.

Sorry about all this. I am new to R and I was just trying to speed up the
process without killing the machine and just wanted to be aware of what was out
there.

thanks a million for all your extremely prompt responses. This is really
appreciated.

Nameeta








Quoting Marc Schwartz <MSchwartz at mn.rr.com>:

> On Tue, 2006-05-16 at 09:45 +0200, Uwe Ligges wrote:
> > Nameeta Lobo wrote:
> > 
> > > Hello all
> > > 
> > > thank you very much for all your suggestions. I actually need binary
> > > representations. I tried all the methods that Marc,Jim and Charles have
> > > suggested and they ran fine(thanks a lot). I tried doing it then with 26
> and 13
> > > and that's when the computer gave way. I just got a message with all
> three
> > > methods that a vector of .....Kb cannot be allocated. guess I will have
> to
> > > change the environment to allow for huge vector size allocation. How do I
> do that?
> > 
> > 
> > You should have *at least* 512Mb in your machine for the solution given 
> > by Charles C. Berry with the numbers given above, better a machine with
> 1Gb.
> > 
> > Uwe Ligges
> 
> In addition to Uwe's comment, there are some practical issues that will
> apply here shortly if Nameeta continues to increase the size of the
> source vector:
> 
> 1. R has a limitation of 2^32 - 1 elements in a vector. This is the same
> for both 32 and 64 bit platforms. Thus, if Nameeta is planning to
> continue to expand the upper limit of the range, you will hit this
> fairly quickly. You would then need to consider some form of a
> partitioning approach if you go beyond that limit.
> 
> 2. The RAM requirements to simply apply Charles' solution will continue
> to expand as the upper limit increases, so Uwe's figure is but one
> number that solves the indicated example of 2^26, but will be
> insufficient beyond that.
> 
> 3. This still does not address Nameeta's now explicitly stated desire
> for the binary character representations, which requires additional
> memory beyond that required for the initial step of identifying the
> numbers that meet the 'bit requirements' alone.
> 
> >From my prior post over the weekend, to store the character matrix of
> binary representations for 2^25 with 9 bits, which contained 2,042,975
> values, it required approximately 128 Mb for the final paste()'d
> versions of the numbers. 
> 
> That is AFTER doing the initial conversion using digitsBase(), which
> required 400 Mb to store the intermediate integer matrix result.  One
> could certainly do that in a partitioned or loop based approach to
> conserve memory, but it still will hit practical limits in short order.
> 
> Those figures too will expand dramatically as the upper limit increases.
> 
> For example, going from 2^24 with 12 bits to 2^26 with 13 bits, results
> in going from 2,704,156 values in the result to 10,400,600 in the
> result. That's a 3.8 fold increase in the result vector size. It does
> not take long to figure out how much memory will be required for these
> operations as the upper range increases.
> 
> Depending upon what Nameeta is planning to do with the final resultant
> character vectors, one could consider a loop based print method/function
> that takes the values in the initial 'dec.index' vector and simply
> cat()'s them to some output. However, you would not be able to actually
> store them as a single matrix given the memory requirements.
> 
> Perhaps Nameeta can indicate what the primary problem is here, which
> might in turn allow someone to offer an alternative approach that is
> more resource sparing.
> 
> HTH,
> 
> Marc Schwartz
> 
> 
> 
> 
> 





-------------------------------------------------
This email is intended only for the use of the individual or...{{dropped}}