[R] very slow code execution

[Jeff Newmiller]

This seems like an odd analysis to me, but I don't have time to look 
closer at it. It certainly doesn't look simple to me... I hope you have 
some good theoretical guidance in tackling this (which this mailing list 
is not).

One obvious thing is that z1 only depends on i, so you should only have to 
compute z1 3 times... but you are computing it 177*182*3 times. Make three 
lists of fourier results using three separate preparation loops before 
diving into your triple loop. Then just retrieve the relevant z1, z2 and 
z3 from these lists when you run the auto.arima calculation.

The easy way to make lists is to use lapply rather than a for loop:

demand7 <- ts( demand, frequency = 7 ) # don't repeat the ts conversion
z1List <- lapply( 1:3, function(i) { fourier( demand7, K = i ) } )

then you can retrieve whichever z1 you want with

z1 <- z1List[[ i ]]

rather than recalculating it.

Also, data.table is implemented by a contributed package. It is a special 
type of data frame. It is not at all clear that you need its special 
properties here, but most people who create a matrix and then immediately 
converts the matrix to a (type of) data frame aren't using the data frame 
properly anyway. Here is one way to assemble your results:

AICc <- expand.grid( k = 1:181, j = 1:177, i = 1:3 )
AICc$fit <- NA
idx <- 1
for ( i in 1:3 ) {
   for ( j in 1:177 ) {
     for ( k in 1:181 ) {
       # compute fit
       AICc$fit[ idx ] <- fit$aicc
       idx <- idx + 1
     }
   }
}

On Wed, 6 Feb 2019, salah maadawy wrote:

> i am a beginner regarding R but i am trying to do a simple thing, but it is
> taking too much time and i am asking if there is any way to achieve what i
> need, i have a time series data set with 730 data points, i detected 7, 354
> and 365 seasonality periods. i am trying to use Fourier terms for
> seasonality and for loop to get the K value for each while minimizing AICc,
> my code is
>
>    AICc<- data.table(matrix(nrow = 96642, ncol = 4))for (i in 1:3) {
>  for (j in 1:177) {
>    for (k in 182) {                     #i,j and k values are choosen
> with regad that K cannot exceed seasonality period/2
>      z1 <- fourier(ts(demand,frequency = 7), K=i)
>      z2 <- fourier(ts(demand,frequency=354), K=j)
>      z3 <- fourier(ts(demand,frequency = 365),K=k)
>      fit <- auto.arima(demand, xreg =cbind(z1,z2,z3),
>         seasonal = FALSE)
>      fit$aicc
>      AICc[,1]<-i
>      AICc[,2]<-j
>      AICc[,3]<-k
>      AICc[,4]<-fit$aicc
>    }
>
>  }
> }
>  AICc
>
> i have created a data table to store AICc values from all possible i,j,k
> combinations so that i can find later the minimum AICc value. the problem
> now is that it is taking forever to do so not only to iterate all
> combinations but also due to the large K values.
>
> , is there any possible solution for this? thank you in advance
>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

---------------------------------------------------------------------------
Jeff Newmiller                        The     .....       .....  Go Live...
DCN:<jdnewmil using dcn.davis.ca.us>        Basics: ##.#.       ##.#.  Live Go...
                                       Live:   OO#.. Dead: OO#..  Playing
Research Engineer (Solar/Batteries            O.O#.       #.O#.  with
/Software/Embedded Controllers)               .OO#.       .OO#.  rocks...1k