[R] very slow code execution

[salah maadawy]

thank you all for your suggestions and feedback, with further search and
experimentation, the problem is with the auto.arima function with large k
values, it takes 4 min to compute one model with k=30 and the time
increases with K, so i used your suggestions for collecting the output but
i limited my loop (i=3,j=25 and k=25) , it took around 17 hours to finish
already (1875 models). thank you again.

On Thu, Feb 7, 2019 at 4:10 PM Bob O'Hara <rni.boh using gmail.com> wrote:

> With 96k model fits it's going to be slow, so you might want to think
> first about whether you need to do them all. Beyond that, I think this
> is more in the R style, so might be quicker (I don't know how much the
> loops are slowing you down), and even if not it should be easier to
> adapt.
>
> The other thing to think about is parallelising the code - the
> parallel package should help.
>
> FitModel <- function(K, data) {
>   z1 <- fourier(ts(data,frequency = 7), K=K["i"])
>   z2 <- fourier(ts(data,frequency=354), K=K["j"])
>   z3 <- fourier(ts(data,,frequency = 365),K=K["k"])
>   fit <- auto.arima(data,, xreg =cbind(z1,z2,z3), seasonal = FALSE)
>   fit$aicc
> }
>
> # smaller MaxOrders used so if you run it like this, it won't take 5 hours
> MaxOrders <- expand.grid(i = 1:3, j=1=7, k=1:8)
> AICc <- apply(MaxOrders, FitModel, data=demand)
>
> Bob
>
>       AICc<- data.table(matrix(nrow = 96642, ncol = 4))for (i in 1:3) {
> > >    for (j in 1:177) {
> > >      for (k in 182) {                     #i,j and k values are choosen
> > > with regad that K cannot exceed seasonality period/2
> > >        z1 <- fourier(ts(demand,frequency = 7), K=i)
> > >        z2 <- fourier(ts(demand,frequency=354), K=j)
> > >        z3 <- fourier(ts(demand,frequency = 365),K=k)
> > >        fit <- auto.arima(demand, xreg =cbind(z1,z2,z3),
> > >           seasonal = FALSE)
> > >        fit$aicc
> > >        AICc[,1]<-i
> > >        AICc[,2]<-j
> > >        AICc[,3]<-k
> > >        AICc[,4]<-fit$aicc
> > >      }
> > >change
> > >    }
> > > }
> > >    AICc
>
> On Thu, 7 Feb 2019 at 13:44, salah maadawy <salahmaadawy using gmail.com> wrote:
> >
> > Hi Micheal, Maybe there is a simple way but i wanted to get the lowest
> aicc
> > ana i could not find a way to do so, that's why i created the  table to
> > store all possible outcomes and then i can easily get the minimum value
> and
> > the values of (i,j and k) used for that minimum value. The first column
> in
> > the table is AICc[,1] to store i and second column for j and so on.
> Maybe i
> > am mistaken and this won't give me what i want, the code been running
> for 5
> > hours now. So i am waiting
> >
> > On Wed, Feb 6, 2019 at 4:59 PM Michael Dewey <lists using dewey.myzen.co.uk>
> > wrote:
> >
> > > This is not an answer to your speed problem but are your assignments to
> > > AICc[,1] and so on doing what you hope they are doing?
> > >
> > > Michael
> > >
> > > On 06/02/2019 12:03, salah maadawy wrote:
> > > > i am a beginner regarding R but i am trying to do a simple thing,
> but it
> > > is
> > > > taking too much time and i am asking if there is any way to achieve
> what
> > > i
> > > > need, i have a time series data set with 730 data points, i detected
> 7,
> > > 354
> > > > and 365 seasonality periods. i am trying to use Fourier terms for
> > > > seasonality and for loop to get the K value for each while minimizing
> > > AICc,
> > > > my code is
> > > >
> > > >      AICc<- data.table(matrix(nrow = 96642, ncol = 4))for (i in 1:3)
> {
> > > >    for (j in 1:177) {
> > > >      for (k in 182) {                     #i,j and k values are
> choosen
> > > > with regad that K cannot exceed seasonality period/2
> > > >        z1 <- fourier(ts(demand,frequency = 7), K=i)
> > > >        z2 <- fourier(ts(demand,frequency=354), K=j)
> > > >        z3 <- fourier(ts(demand,frequency = 365),K=k)
> > > >        fit <- auto.arima(demand, xreg =cbind(z1,z2,z3),
> > > >           seasonal = FALSE)
> > > >        fit$aicc
> > > >        AICc[,1]<-i
> > > >        AICc[,2]<-j
> > > >        AICc[,3]<-k
> > > >        AICc[,4]<-fit$aicc
> > > >      }
> > > >
> > > >    }
> > > > }
> > > >    AICc
> > > >
> > > > i have created a data table to store AICc values from all possible
> i,j,k
> > > > combinations so that i can find later the minimum AICc value. the
> problem
> > > > now is that it is taking forever to do so not only to iterate all
> > > > combinations but also due to the large K values.
> > > >
> > > > , is there any possible solution for this? thank you in advance
> > > >
> > > >       [[alternative HTML version deleted]]
> > > >
> > > > ______________________________________________
> > > > R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > > > https://stat.ethz.ch/mailman/listinfo/r-help
> > > > PLEASE do read the posting guide
> > > http://www.R-project.org/posting-guide.html
> > > > and provide commented, minimal, self-contained, reproducible code.
> > > >
> > >
> > > --
> > > Michael
> > > http://www.dewey.myzen.co.uk/home.html
> > >
> >
> >         [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>
>
>
> --
> Bob O'Hara
> Institutt for matematiske fag
> NTNU
> 7491 Trondheim
> Norway
>
> Mobile: +47 915 54 416
> Journal of Negative Results - EEB: www.jnr-eeb.org
>

	[[alternative HTML version deleted]]