[R] Averaging Out many rows from a column AND funtion to string

Wed Mar 27 20:37:35 CET 2013

This completes in about a second on my system and uses the actual matrix dimensions you quote:

  nr <- 153899
  nc <- 3415
  keep <- rnorm(nr * nc, 80, 20)
  dim(keep) <- c(nr, nc)
  shrink.to <- 1000
  system.time(
    {
      idx <- rep(1:shrink.to, length.out = nr)
      plot.me <- sweep(rowsum(keep, idx), 1, tabulate(idx), , FUN = '/')
    })
   user  system elapsed 
   0.92    0.00    0.9

Jason Law
Statistician
City of Portland
Bureau of Environmental Services
Water Pollution Control Laboratory
6543 N Burlington Avenue
Portland, OR 97203-5452
503-823-1038
jason.law at portlandoregon.gov

-----Original Message-----
From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On Behalf Of Alaios
Sent: Wednesday, March 27, 2013 9:05 AM
To: PIKAL Petr; R help
Subject: Re: [R] Averaging Out many rows from a column AND funtion to string

Well my true matrix is 

of

num [1:153899, 1:3415]

that I want to convert to something like

num [1:1000, 1:3415] (keeping column number the same).

I only give subsets here to allow others to run the code at their computer

Thanks a lot

Regards
Alex

________________________________
 From: PIKAL Petr <petr.pikal at precheza.cz>

Sent: Wednesday, March 27, 2013 4:45 PM
Subject: RE: [R] Averaging Out many rows from a column AND funtion to string

Hi

> system.time({
+ keep<-matrix(data=rnorm(9000000,80,20),nrow=300000,ncol=30)
+ ShrinkTo<-500
+ idx<-cut(1:nrow(keep), ShrinkTo)
+ keep.ag<-aggregate(keep, list(idx), mean)
+ PlotMe<-as.matrix(keep.ag[,-1])
+ })
   user  system elapsed
  29.23    0.14   29.37 
>  

It takes 30 seconds when using 300000 rows. It is not enough time to get a cup of tee, which I do not consider ages. Maybe split lapply approach or data.matrix or  **ply could be quicker but I do not consider worth spending hours to elaborate some solution which will spare 20 sec computing time.

Regards
Petr 

Sent: Wednesday, March 27, 2013 4:12 PM
To: PIKAL Petr; R help
Subject: Re: [R] Averaging Out many rows from a column AND funtion to string

I have fixed it like that:

keep<-matrix(data=rnorm(900,80,20),nrow=30,ncol=30)
ShrinkTo<-500
idx<-cut(1:nrow(keep), ShrinkTo)
keep.ag<-aggregate(keep, list(idx), mean)
PlotMe<-data.matrix(keep.ag[2:ShrinkTo])

The only problem is that takes ages to finish..... Would it be possible to convert it to something like lapply?

Regards
Alex

________________________________

From:PIKAL Petr <petr.pikal at precheza.cz>

Sent: Wednesday, March 27, 2013 4:02 PM
Subject: RE: [R] Averaging Out many rows from a column AND funtion to string

Hi

> str(as.matrix(keep.ag[,-1]) )  # does look like numeric
num [1:10, 1:30] 75.1 93 79 81.7 76.3 ...
- attr(*, "dimnames")=List of 2
  ..$ : NULL
  ..$ : chr [1:30] "V1" "V2" "V3" "V4" ...

Please read and follow what was recommended.

quote

use as.matrix(data.frame) on numeric part
                                                          ^^^^^^^^^^^

aggregate produces data frame with its first column being your idx variable, which is factor. Trying to convert it whole to matrix results in character matrix. You need to exclude first column from conversion

And please can you explain how mean(rnorm(whatever)) shall be integer?

Regards
Petr

Sent: Wednesday, March 27, 2013 3:01 PM
To: PIKAL Petr; R help
Subject: Re: [R] Averaging Out many rows from a column AND funtion to string

see inline

________________________________

From:PIKAL Petr <petr.pikal at precheza.cz>

Sent: Wednesday, March 27, 2013 1:50 PM
Subject: RE: [R] Averaging Out many rows from a column AND funtion to string

Hi

Sent: Wednesday, March 27, 2013 11:46 AM
To: PIKAL Petr; R help
Subject: Re: [R] Averaging Out many rows from a column AND funtion to string

see inline

________________________________

From:PIKAL Petr <petr.pikal at precheza.cz>

Sent: Wednesday, March 27, 2013 11:24 AM
Subject: RE: [R] Averaging Out many rows from a column AND funtion to string

Hi

> -----Original Message-----
> From: r-help-bounces at r-project.org [mailto:r-help-bounces at r- 
> project.org] On Behalf Of Alaios
> Sent: Wednesday, March 27, 2013 9:13 AM
> To: R help
> Subject: [R] Averaging Out many rows from a column AND funtion to 
> string
> 
> Dear all,
> 1) I have a very large matrix of
> str(keep)
>  num [1:153899, 1:3415] -98.6 -95.8 -96.4 -95.8 -98 ...
> that  I would like to reduce its size to something like
> 
> str(keep)
>  num [1:1000, 1:3415] -98.6 -95.8 -96.4 -95.8 -98 ... or anything 
> similar in size as this is a matrix that needs plotting (so is ok if 
> it is 1000 row, 995, or 1123)
> 
> I think what I need here is a way of selecting multiple rows and 
> averaging per column (notice that the column number should stay the
> same)

Make an index variable and aggregate values according it

Something like
idx<-cut(1:153899, 153)
keep.ag<-aggregate(keep, list(idx), mean)

1) Thanks that returned a data frame.. How I can have a matrix at the end?
use as.matrix(data.frame) on numeric part

a bit of my code that you can re run. I convert a 30,30 matrix to a 10,30.  but it looks at the end that I do not get the Data part of the data.fram correctly: 
Data<-matrix(data=rnorm(900,80,20),nrow=30,ncol=30)
idx<-cut(1:30, 10)
keep.ag<-aggregate(Data, list(idx), mean)
str(as.matrix(keep.ag) ) # it does not look like integers

2) I want to have a string that can be used also for calling a function with the same name. Think of using "mean" to call mean. I need to have R interpret the string in different ways.
To call "mean" of what? I am sure I do not understand what is your intention.
LogFunction<- function(){}
FunctionIndex<- rbind (c(1,"LogFunction"),
               c(2,"TakeFunction"))
print(sprintf('Using the function %s',FunctionIndex[1,1])) This does not contain much clue.
Regards
Petr
PS. Do not use HTML mail messages. 
Regards
Alex

> 
> b. I would like to be able to convert strings that are function names 
> to real function calls. For example I have something like
> 
> LogFunction<- function(){}
> FunctionIndex<- rbind (c(1,"LogFunction"),
>                c(2,"TakeFunction")
>               )
> print(sprintf('Using the function %s',FunctionIndex[1,1])) # call the 
> FunctionIndex[1,1] somehow

I am not sure if I understand correctly. 

Is this what you want?
myf <-function(fun="mean", arg) eval(call(fun, arg))

Regards
Petr

> 
> 
> 
> I would like to thank you in advance for your help
> 
> Regards
> Alex
> 
>     [[alternative HTML version deleted]]
	[[alternative HTML version deleted]]