[R-SIG-Finance] timout using "evalWithTimeout" in looped rugarch estimation
alexios ghalanos
alexios at 4dscape.com
Sat May 10 21:22:57 CEST 2014
On 10/05/2014 18:55, Johannes Moser wrote:
> Once again many thanks, Alexios!
>
> It was not clear to me that the solvers are contained in separate
> packages with separate manuals.
> I still have got three questions
>
> QUESTION 1: I`ve experimented a bit with the outer.iter and inner.iter
> values. Can you recommend any minimal values for them? Should they be
> kept in a certain relationship when being modified (default is
> outer.iter=400 and inner.iter=800)?
>
No. Depends on your problem. Obviously, if it does not converge in N
iterations then it will certainly NOT converge in less than N.
> QUESTION 2: Moreover, there seems to be NO direct relationship between
> lowering the values and shortening the estimation time (until abortion).
> Is this as expected or is it a bug?
> E.g. holding outer.iter fixed and lowering the inner.iter parameter from
> 130 to 115 decreases time until "no convergence"-message, but lowering
> it from 130 to only 120 strongly INCREASES time (actually I`m not sure
> if the solver in this case would finish at all one time or if it`s
> somehow stuck).
> I tried 3 times. The results are:
>
> #################
> require('rugarch')
> ptm <- proc.time()
> spec <- ugarchspec(
> variance.model =
> list(model = "eGARCH", garchOrder = c(0,7), submodel = NULL
> , external.regressors = NULL, variance.targeting = FALSE),
> mean.model = list(armaOrder = c(0,0), external.regressors = NULL),
> distribution.model = "sged")
> tempgarch <- ugarchfit(spec=spec, data=tempdata , solver="solnp" ,
> solver.control=list( outer.iter = 70, inner.iter=130 ) )
> tempgarch
> (proc.time()-ptm)[3]
>
> # trial 1:
> # outer.iter = 70, inner.iter=115 elapsed 79.87 ugarchfit-->warning:
> solver FAILER to converge. (there`s a typo in the error message)
> # outer.iter = 70, inner.iter=120 +++stopped manually after 15 minutes+++
> # outer.iter = 70, inner.iter=130 elapsed 135.8 ugarchfit-->warning:
> solver failer to converge.
>
> # trial 2:
> # outer.iter = 70, inner.iter=115 elapsed 79.2 ugarchfit-->warning:
> solver failer to converge.
> # outer.iter = 70, inner.iter=120 +++stopped manually after 15 minutes+++
> # outer.iter = 70, inner.iter=130 elapsed 135.09 ugarchfit-->warning:
> solver failer to converge.
>
> # trial 3:
> # outer.iter = 70, inner.iter=115 elapsed 79.3 ugarchfit-->warning:
> solver failer to converge.
> # outer.iter = 70, inner.iter=120 +++stopped manually after 15 minutes+++
> # outer.iter = 70, inner.iter=130 elapsed 136.13 ugarchfit-->warning:
> solver failer to converge.
> #################
There are many parameters and many options to consider (fit.control and
solver.control). See the FAQ in the vignette on model convergence. I'll
have nothing more to say on this, particularly for a GARCH(0,7) model,
whatever that means.
>
> QUESTION 3: I`m a bit confused about the parallel computing
> implementation in ugarchfit.
> In the vignette it says (on p. 44) "Since version 1.0-14, rugarch makes
> exclusive use of the parallel package for all parallel computations."
> But in the manual on p. 70 only the packages "snowfall" or "multicore"
> seem to be supported for the solver used in ugarchfit.
> So either EXCLUSIVELY parallel or not?
> As I could not find any example about how to set up snowfall in this
> context I simply tried this way (without speed improvement):
Parallel is a wrapper for multicore and snowfall. However, for making
use of the parallel functionality of ANOTHER package (Rsolnp) the
instructions are quite clear on what to pass (see Details of ?ugarchfit).
But where exactly do you expect the parallel functionality to play a
role in the univariate estimation? For ugarchfit it is only used when
the solver is gosolnp (and you have set n.restarts>1), which is again
stated in the manual. Parallel functionality otherwise is used for the
simulation routines (multiple runs), rolling estimation i.e. where it
makes sense to use parallel resources.
>
> ##################
> require('rugarch')
> require('snowfall')
> spec <- ugarchspec(
> variance.model=list( model = "eGARCH", garchOrder=c(1,3),
> submodel=NULL, external.regressors=NULL, variance.targeting=FALSE ),
> mean.model=list( armaOrder=c(0,0), external.regressors=NULL ),
> distribution.model = "std")
> ptm <- proc.time()
> tempgarch <- ugarchfit( spec=spec, data=tempdata ,solver="solnp" ,
> solver.control=list( parallel=TRUE, pkg="snowfall", cores=4 ) )
> (proc.time()-ptm)[3]
> tempgarch
> ##################
>
> What am I doing wrong? Do I have to wrap the ugarchfit function into
> something like the following?
>
> ##################
> sfInit( parallel=TRUE, cpus=4 )
> ...
> sfStop()
> ##################
>
> (I`d expect a "no" as an answer assuming that the solver itself will do
> something alike if the relevant solver.control options are set)
>
> Best, Johannes
>
>
Regards,
Alexios
>
>
> Am 10.05.2014 13:25, schrieb alexios ghalanos:
>> Hi Johannes,
>>
>> On 10 May 2014, at 12:19, Johannes Moser <jzmoser at gmail.com> wrote:
>>
>>> Many thanks Alexios!!
>>>
>>> 1. In my TGARCH setup nlminb doesn`t converge even at smaller GARCH
>>> order. So I will stick to solnp then.
>>>
>>> 2. Unfortunately I couldn`t find the correct command which limits the
>>> number of iterations via the solver.control options.
>>> The details from the manual mention n.sim and n.restarts, but these
>>> seem to control other parameters.
>>> For the nloptr solver the option maxeval is mentioned. But I don`t
>>> work with this solver and trial-and-error-implementation of this
>>> option to sonnp leaded to no success.
>>> Other packages inspired me to try "maxiter" , "iter.max" , "n.iter" ,
>>> but they didn`t work either.
>>>
>>> E.g. ugarchfit( spec=spec, data=tempdata ,
>>> solver="solnp", solver.control=list( maxeval=20, rseed=9876 ) )
>> See the documentation for solnp (?solnp). The ‘inner.iter’ and
>> ‘outer.iter’ are what you are likely looking for. The n.sim and
>> n.restarts if for the gosolnp solver (multi-start solnp).
>>> 3. You`re surely right. The whole study should actually investigate
>>> this issue empirically.
>>> E.g. in one case there was a surprising result in a sample of size 1200:
>>> An ARMA(0,0) eGARCH(5,5) model with a skewed normal for the
>>> innovations yielded very good results.
>>> No sign biases, nice gof, no autocorrelation in standardized and
>>> squared standardized residuals up to order p+q+10, nice AIC and BIC
>>> as well as only highly significant coefficients (6 out of 18 were not
>>> significant as to the robust SE, though). I will compare this model
>>> to a more parsimonious one and also investigate parameter uncertainty.
>> Yes, in-sample of course….but I was referring to forecast performance
>> (out of sample).
>>> Best, Johannes
>>>
>> Regards,
>> Alexios
>>>
>>> Am 10.05.2014 11:34, schrieb alexios ghalanos:
>>>> Johannes,
>>>>
>>>> I suggest the following:
>>>>
>>>> 1. Don't use hybrid, use instead solnp or nlminb.
>>>>
>>>> 2. You can control a number of solver convergence criteria (e.g. number
>>>> of iterations) using the solver.control argument.
>>>>
>>>> 3. Before running the code, do consider a little more how reasonable it
>>>> is to be modelling a TGARCH(7,8) model. Investigate the model first
>>>> (don't just return the AIC or BIC). Are any of the higher order
>>>> ARCH/GARCH parameters different from zero or even significant? I have
>>>> not seen a single study which shows that such very high order GARCH
>>>> models have better performance than more parsimonious alternatives.
>>>>
>>>> 4. At the best of times it takes a considerable amount of data to
>>>> estimate the GARCH persistence. Try running a simulation exercise using
>>>> for example the ugarchdistribution function to obtain some insight into
>>>> higher order GARCH models.
>>>>
>>>> 5. Finally, as mentioned numerous times on this forum, the fGARCH model
>>>> is a highly parameterized omnibus model. Imposing stationarity during
>>>> the optimization, particularly for non-symmetric distributions such as
>>>> the ged, is a costly exercise. Consider using the GJR instead and a
>>>> distribution which is a little faster to evaluate such as the JSU.
>>>> Alternatively consider using the normal distribution to estimate the
>>>> GARCH parameters for the purpose of model comparison.
>>>>
>>>> -Alexios
>>>>
>>>> On 10/05/2014 08:23, Johannes Moser wrote:
>>>>> I guess that the problem is due to the processing in C as part of the
>>>>> ugarchfit routine.
>>>>>
>>>>> Is there any way to timeout a ugarchfit command or to constrain the
>>>>> number if iterations?
>>>>>
>>>>> At one time the loop seems to be stuck completely.
>>>>> I waited for several hours for a single ugarchfit step which just
>>>>> didn`t
>>>>> complete. Then I manually stopped the process.
>>>>>
>>>>> Separate calculation of the respective model also seems to be "stuck"
>>>>> (CPU is still working, the "hybrid" algorithms seem to find no
>>>>> solution
>>>>> though and just keep running).
>>>>>
>>>>> As I want to set up a GARCH model-preselection battery there hopefully
>>>>> is a way to handle such problems?
>>>>>
>>>>> Best, Johannes
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Am 09.05.2014 13:58, schrieb Johannes Moser:
>>>>>> Dear all,
>>>>>>
>>>>>> I`ve set up a double loop which loops through different GARCH-orders
>>>>>> and ARMA-orders in a rugarch estimation (of several models and error
>>>>>> distributions) and each time writes the AIC and other information
>>>>>> into
>>>>>> a data frame.
>>>>>> The resulting data frame should be used for the pre-selection of a
>>>>>> model, which then will be examined manually.
>>>>>>
>>>>>> A small part of the model estimation steps using "ugarchfit" take
>>>>>> very
>>>>>> long time. So I implemented a timeout function using
>>>>>> "evalWithTimeout"
>>>>>> which stops the current estimation step and proceeds with the next
>>>>>> step in the loop and estimates the next model.
>>>>>>
>>>>>> The timeout function is wrapped into a "tryCatch" function which
>>>>>> assures thet the loop keeps running after e.g. convergence problems.
>>>>>>
>>>>>> A small toy model works fine:
>>>>>>
>>>>>>
>>>>>> #######################################################################
>>>>>>
>>>>>> require('R.utils')
>>>>>> abc <- matrix(NA,10,3)
>>>>>>
>>>>>> foo <- function() {
>>>>>> print("Tic");
>>>>>> for (kk in 1:50) {
>>>>>> print(kk);
>>>>>> Sys.sleep(0.1);
>>>>>> }
>>>>>> print("Tac");
>>>>>> }
>>>>>>
>>>>>>
>>>>>> for (i in 1:10){
>>>>>> ptm <- proc.time()
>>>>>> tryCatch( { abc[i,1] <- evalWithTimeout({foo()} ,timeout=(4+i*0.2)
>>>>>> ,onTimeout="silent" )
>>>>>> abc[i,2] <- 1
>>>>>> }
>>>>>> , error = function(x) x)
>>>>>> tt<- proc.time() - ptm
>>>>>> abc[i,3]<-tt[3]
>>>>>> }
>>>>>>
>>>>>> abc
>>>>>> #####################################################################
>>>>>>
>>>>>>
>>>>>> However, in the rugarch setup the "evalWithTimeout" doesn't seem to
>>>>>> stop the "ugarchfit" estimation reliably. E.g. in one instance the
>>>>>> recorded time for a step was 1388.03 seconds even though the limit
>>>>>> was
>>>>>> set to be 300 seconds. The next example illustrates my setup in a
>>>>>> simplified version (unfortunately my results depend on the data I
>>>>>> have
>>>>>> used, so you will not be able to reproduce them):
>>>>>>
>>>>>>
>>>>>> #####################################################################
>>>>>> require('rugarch')
>>>>>> quiet1 <- read.table( "dax_quiet1.txt" , header=T)
>>>>>> tempdata <- quiet1$logreturns
>>>>>>
>>>>>> g_order <- matrix(NA,5,2)
>>>>>> g_order[1,]<-c(1,1)
>>>>>> g_order[2,]<-c(1,8)
>>>>>> g_order[3,]<-c(9,6)
>>>>>> g_order[4,]<-c(9,8)
>>>>>> g_order[5,]<-c(3,10)
>>>>>>
>>>>>> overview <- data.frame(matrix(NA,5,2))
>>>>>>
>>>>>> for(i in 1:5){
>>>>>> ptm <- proc.time()
>>>>>>
>>>>>> spec <- ugarchspec(
>>>>>> variance.model = list(model = "fGARCH", garchOrder =
>>>>>> g_order[i,], submodel = "TGARCH", external.regressors = NULL,
>>>>>> variance.targeting = FALSE),
>>>>>> mean.model = list(armaOrder = c(0,0),
>>>>>> external.regressors =
>>>>>> NULL), distribution.model = "sged")
>>>>>>
>>>>>> tryCatch( {tempgarch <- evalWithTimeout({ugarchfit(spec=spec,
>>>>>> data=tempdata ,solver="hybrid")} ,timeout=20 ,onTimeout="silent" )
>>>>>> overview[i,1]<-infocriteria(tempgarch)[1]
>>>>>> }
>>>>>> , error = function(x) x)
>>>>>>
>>>>>> tt<- proc.time() - ptm
>>>>>> overview[i,2]<-tt[3]
>>>>>> }
>>>>>>
>>>>>> overview
>>>>>>
>>>>>> # If the timeout is set set to 20, this setup leads to:
>>>>>> # 2.87 sec.
>>>>>> # 6.95 sec.
>>>>>> # 125 sec. ... here, tryCatch interrupted the process
>>>>>> # 51.73 sec.
>>>>>> # 27.11 sec.
>>>>>> # for the 5 different estimation steps.
>>>>>>
>>>>>> # timeout set to 300:
>>>>>> # 2.81 sec.
>>>>>> # 6.85 sec.
>>>>>> # 743.58 sec.
>>>>>> # 41.70 sec.
>>>>>> # 26.85 sec.
>>>>>> # no process was interrupted by tryCatch
>>>>>> #######################################################################
>>>>>>
>>>>>>
>>>>>>
>>>>>> As can be seen even from this simplified example, when the timeout
>>>>>> was
>>>>>> set to be 20 there still was a process that took 125 seconds
>>>>>> (which is
>>>>>> more than 5 times longer!).
>>>>>> I would be very thankful for any ideas or comments!
>>>>>>
>>>>>> Best, Johannes
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> --
>>>>>
>>>>> _______________________________________________
>>>>> R-SIG-Finance at r-project.org mailing list
>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>>>> -- Subscriber-posting only. If you want to post, subscribe first.
>>>>> -- Also note that this is not the r-help list where general R
>>>>> questions
>>>>> should go.
>>>>>
>>>>>
>>> --
>>>
>>>
>
More information about the R-SIG-Finance
mailing list