[R-SIG-Finance] timout using "evalWithTimeout" in looped rugarch estimation
Johannes Moser
jzmoser at gmail.com
Sat May 10 22:22:50 CEST 2014
In the context of question2 I was actually looking for a model that does
NOT converge.
But in being so focussed on the ways to handle long convergence times I
indeed picked a totally meaningless model which is a bit embarassing.
I`ll keep playing around with numbers and have another look at the manuals.
Thank you very much for the hints and your patience, Alexios!
Am 10.05.2014 21:22, schrieb alexios ghalanos:
> On 10/05/2014 18:55, Johannes Moser wrote:
>> Once again many thanks, Alexios!
>>
>> It was not clear to me that the solvers are contained in separate
>> packages with separate manuals.
>> I still have got three questions
>>
>> QUESTION 1: I`ve experimented a bit with the outer.iter and inner.iter
>> values. Can you recommend any minimal values for them? Should they be
>> kept in a certain relationship when being modified (default is
>> outer.iter=400 and inner.iter=800)?
>>
> No. Depends on your problem. Obviously, if it does not converge in N
> iterations then it will certainly NOT converge in less than N.
>
>> QUESTION 2: Moreover, there seems to be NO direct relationship between
>> lowering the values and shortening the estimation time (until abortion).
>> Is this as expected or is it a bug?
>> E.g. holding outer.iter fixed and lowering the inner.iter parameter from
>> 130 to 115 decreases time until "no convergence"-message, but lowering
>> it from 130 to only 120 strongly INCREASES time (actually I`m not sure
>> if the solver in this case would finish at all one time or if it`s
>> somehow stuck).
>> I tried 3 times. The results are:
>>
>> #################
>> require('rugarch')
>> ptm <- proc.time()
>> spec <- ugarchspec(
>> variance.model =
>> list(model = "eGARCH", garchOrder = c(0,7), submodel = NULL
>> , external.regressors = NULL, variance.targeting = FALSE),
>> mean.model = list(armaOrder = c(0,0), external.regressors = NULL),
>> distribution.model = "sged")
>> tempgarch <- ugarchfit(spec=spec, data=tempdata , solver="solnp" ,
>> solver.control=list( outer.iter = 70, inner.iter=130 ) )
>> tempgarch
>> (proc.time()-ptm)[3]
>>
>> # trial 1:
>> # outer.iter = 70, inner.iter=115 elapsed 79.87 ugarchfit-->warning:
>> solver FAILER to converge. (there`s a typo in the error message)
>> # outer.iter = 70, inner.iter=120 +++stopped manually after 15 minutes+++
>> # outer.iter = 70, inner.iter=130 elapsed 135.8 ugarchfit-->warning:
>> solver failer to converge.
>>
>> # trial 2:
>> # outer.iter = 70, inner.iter=115 elapsed 79.2 ugarchfit-->warning:
>> solver failer to converge.
>> # outer.iter = 70, inner.iter=120 +++stopped manually after 15 minutes+++
>> # outer.iter = 70, inner.iter=130 elapsed 135.09 ugarchfit-->warning:
>> solver failer to converge.
>>
>> # trial 3:
>> # outer.iter = 70, inner.iter=115 elapsed 79.3 ugarchfit-->warning:
>> solver failer to converge.
>> # outer.iter = 70, inner.iter=120 +++stopped manually after 15 minutes+++
>> # outer.iter = 70, inner.iter=130 elapsed 136.13 ugarchfit-->warning:
>> solver failer to converge.
>> #################
> There are many parameters and many options to consider (fit.control and
> solver.control). See the FAQ in the vignette on model convergence. I'll
> have nothing more to say on this, particularly for a GARCH(0,7) model,
> whatever that means.
>
>> QUESTION 3: I`m a bit confused about the parallel computing
>> implementation in ugarchfit.
>> In the vignette it says (on p. 44) "Since version 1.0-14, rugarch makes
>> exclusive use of the parallel package for all parallel computations."
>> But in the manual on p. 70 only the packages "snowfall" or "multicore"
>> seem to be supported for the solver used in ugarchfit.
>> So either EXCLUSIVELY parallel or not?
>> As I could not find any example about how to set up snowfall in this
>> context I simply tried this way (without speed improvement):
> Parallel is a wrapper for multicore and snowfall. However, for making
> use of the parallel functionality of ANOTHER package (Rsolnp) the
> instructions are quite clear on what to pass (see Details of ?ugarchfit).
> But where exactly do you expect the parallel functionality to play a
> role in the univariate estimation? For ugarchfit it is only used when
> the solver is gosolnp (and you have set n.restarts>1), which is again
> stated in the manual. Parallel functionality otherwise is used for the
> simulation routines (multiple runs), rolling estimation i.e. where it
> makes sense to use parallel resources.
>> ##################
>> require('rugarch')
>> require('snowfall')
>> spec <- ugarchspec(
>> variance.model=list( model = "eGARCH", garchOrder=c(1,3),
>> submodel=NULL, external.regressors=NULL, variance.targeting=FALSE ),
>> mean.model=list( armaOrder=c(0,0), external.regressors=NULL ),
>> distribution.model = "std")
>> ptm <- proc.time()
>> tempgarch <- ugarchfit( spec=spec, data=tempdata ,solver="solnp" ,
>> solver.control=list( parallel=TRUE, pkg="snowfall", cores=4 ) )
>> (proc.time()-ptm)[3]
>> tempgarch
>> ##################
>>
>> What am I doing wrong? Do I have to wrap the ugarchfit function into
>> something like the following?
>>
>> ##################
>> sfInit( parallel=TRUE, cpus=4 )
>> ...
>> sfStop()
>> ##################
>>
>> (I`d expect a "no" as an answer assuming that the solver itself will do
>> something alike if the relevant solver.control options are set)
>>
>> Best, Johannes
>>
>>
> Regards,
>
> Alexios
>
>>
>> Am 10.05.2014 13:25, schrieb alexios ghalanos:
>>> Hi Johannes,
>>>
>>> On 10 May 2014, at 12:19, Johannes Moser <jzmoser at gmail.com> wrote:
>>>
>>>> Many thanks Alexios!!
>>>>
>>>> 1. In my TGARCH setup nlminb doesn`t converge even at smaller GARCH
>>>> order. So I will stick to solnp then.
>>>>
>>>> 2. Unfortunately I couldn`t find the correct command which limits the
>>>> number of iterations via the solver.control options.
>>>> The details from the manual mention n.sim and n.restarts, but these
>>>> seem to control other parameters.
>>>> For the nloptr solver the option maxeval is mentioned. But I don`t
>>>> work with this solver and trial-and-error-implementation of this
>>>> option to sonnp leaded to no success.
>>>> Other packages inspired me to try "maxiter" , "iter.max" , "n.iter" ,
>>>> but they didn`t work either.
>>>>
>>>> E.g. ugarchfit( spec=spec, data=tempdata ,
>>>> solver="solnp", solver.control=list( maxeval=20, rseed=9876 ) )
>>> See the documentation for solnp (?solnp). The ‘inner.iter’ and
>>> ‘outer.iter’ are what you are likely looking for. The n.sim and
>>> n.restarts if for the gosolnp solver (multi-start solnp).
>>>> 3. You`re surely right. The whole study should actually investigate
>>>> this issue empirically.
>>>> E.g. in one case there was a surprising result in a sample of size 1200:
>>>> An ARMA(0,0) eGARCH(5,5) model with a skewed normal for the
>>>> innovations yielded very good results.
>>>> No sign biases, nice gof, no autocorrelation in standardized and
>>>> squared standardized residuals up to order p+q+10, nice AIC and BIC
>>>> as well as only highly significant coefficients (6 out of 18 were not
>>>> significant as to the robust SE, though). I will compare this model
>>>> to a more parsimonious one and also investigate parameter uncertainty.
>>> Yes, in-sample of course….but I was referring to forecast performance
>>> (out of sample).
>>>> Best, Johannes
>>>>
>>> Regards,
>>> Alexios
>>>> Am 10.05.2014 11:34, schrieb alexios ghalanos:
>>>>> Johannes,
>>>>>
>>>>> I suggest the following:
>>>>>
>>>>> 1. Don't use hybrid, use instead solnp or nlminb.
>>>>>
>>>>> 2. You can control a number of solver convergence criteria (e.g. number
>>>>> of iterations) using the solver.control argument.
>>>>>
>>>>> 3. Before running the code, do consider a little more how reasonable it
>>>>> is to be modelling a TGARCH(7,8) model. Investigate the model first
>>>>> (don't just return the AIC or BIC). Are any of the higher order
>>>>> ARCH/GARCH parameters different from zero or even significant? I have
>>>>> not seen a single study which shows that such very high order GARCH
>>>>> models have better performance than more parsimonious alternatives.
>>>>>
>>>>> 4. At the best of times it takes a considerable amount of data to
>>>>> estimate the GARCH persistence. Try running a simulation exercise using
>>>>> for example the ugarchdistribution function to obtain some insight into
>>>>> higher order GARCH models.
>>>>>
>>>>> 5. Finally, as mentioned numerous times on this forum, the fGARCH model
>>>>> is a highly parameterized omnibus model. Imposing stationarity during
>>>>> the optimization, particularly for non-symmetric distributions such as
>>>>> the ged, is a costly exercise. Consider using the GJR instead and a
>>>>> distribution which is a little faster to evaluate such as the JSU.
>>>>> Alternatively consider using the normal distribution to estimate the
>>>>> GARCH parameters for the purpose of model comparison.
>>>>>
>>>>> -Alexios
>>>>>
>>>>> On 10/05/2014 08:23, Johannes Moser wrote:
>>>>>> I guess that the problem is due to the processing in C as part of the
>>>>>> ugarchfit routine.
>>>>>>
>>>>>> Is there any way to timeout a ugarchfit command or to constrain the
>>>>>> number if iterations?
>>>>>>
>>>>>> At one time the loop seems to be stuck completely.
>>>>>> I waited for several hours for a single ugarchfit step which just
>>>>>> didn`t
>>>>>> complete. Then I manually stopped the process.
>>>>>>
>>>>>> Separate calculation of the respective model also seems to be "stuck"
>>>>>> (CPU is still working, the "hybrid" algorithms seem to find no
>>>>>> solution
>>>>>> though and just keep running).
>>>>>>
>>>>>> As I want to set up a GARCH model-preselection battery there hopefully
>>>>>> is a way to handle such problems?
>>>>>>
>>>>>> Best, Johannes
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Am 09.05.2014 13:58, schrieb Johannes Moser:
>>>>>>> Dear all,
>>>>>>>
>>>>>>> I`ve set up a double loop which loops through different GARCH-orders
>>>>>>> and ARMA-orders in a rugarch estimation (of several models and error
>>>>>>> distributions) and each time writes the AIC and other information
>>>>>>> into
>>>>>>> a data frame.
>>>>>>> The resulting data frame should be used for the pre-selection of a
>>>>>>> model, which then will be examined manually.
>>>>>>>
>>>>>>> A small part of the model estimation steps using "ugarchfit" take
>>>>>>> very
>>>>>>> long time. So I implemented a timeout function using
>>>>>>> "evalWithTimeout"
>>>>>>> which stops the current estimation step and proceeds with the next
>>>>>>> step in the loop and estimates the next model.
>>>>>>>
>>>>>>> The timeout function is wrapped into a "tryCatch" function which
>>>>>>> assures thet the loop keeps running after e.g. convergence problems.
>>>>>>>
>>>>>>> A small toy model works fine:
>>>>>>>
>>>>>>>
>>>>>>> #######################################################################
>>>>>>>
>>>>>>> require('R.utils')
>>>>>>> abc <- matrix(NA,10,3)
>>>>>>>
>>>>>>> foo <- function() {
>>>>>>> print("Tic");
>>>>>>> for (kk in 1:50) {
>>>>>>> print(kk);
>>>>>>> Sys.sleep(0.1);
>>>>>>> }
>>>>>>> print("Tac");
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>> for (i in 1:10){
>>>>>>> ptm <- proc.time()
>>>>>>> tryCatch( { abc[i,1] <- evalWithTimeout({foo()} ,timeout=(4+i*0.2)
>>>>>>> ,onTimeout="silent" )
>>>>>>> abc[i,2] <- 1
>>>>>>> }
>>>>>>> , error = function(x) x)
>>>>>>> tt<- proc.time() - ptm
>>>>>>> abc[i,3]<-tt[3]
>>>>>>> }
>>>>>>>
>>>>>>> abc
>>>>>>> #####################################################################
>>>>>>>
>>>>>>>
>>>>>>> However, in the rugarch setup the "evalWithTimeout" doesn't seem to
>>>>>>> stop the "ugarchfit" estimation reliably. E.g. in one instance the
>>>>>>> recorded time for a step was 1388.03 seconds even though the limit
>>>>>>> was
>>>>>>> set to be 300 seconds. The next example illustrates my setup in a
>>>>>>> simplified version (unfortunately my results depend on the data I
>>>>>>> have
>>>>>>> used, so you will not be able to reproduce them):
>>>>>>>
>>>>>>>
>>>>>>> #####################################################################
>>>>>>> require('rugarch')
>>>>>>> quiet1 <- read.table( "dax_quiet1.txt" , header=T)
>>>>>>> tempdata <- quiet1$logreturns
>>>>>>>
>>>>>>> g_order <- matrix(NA,5,2)
>>>>>>> g_order[1,]<-c(1,1)
>>>>>>> g_order[2,]<-c(1,8)
>>>>>>> g_order[3,]<-c(9,6)
>>>>>>> g_order[4,]<-c(9,8)
>>>>>>> g_order[5,]<-c(3,10)
>>>>>>>
>>>>>>> overview <- data.frame(matrix(NA,5,2))
>>>>>>>
>>>>>>> for(i in 1:5){
>>>>>>> ptm <- proc.time()
>>>>>>>
>>>>>>> spec <- ugarchspec(
>>>>>>> variance.model = list(model = "fGARCH", garchOrder =
>>>>>>> g_order[i,], submodel = "TGARCH", external.regressors = NULL,
>>>>>>> variance.targeting = FALSE),
>>>>>>> mean.model = list(armaOrder = c(0,0),
>>>>>>> external.regressors =
>>>>>>> NULL), distribution.model = "sged")
>>>>>>>
>>>>>>> tryCatch( {tempgarch <- evalWithTimeout({ugarchfit(spec=spec,
>>>>>>> data=tempdata ,solver="hybrid")} ,timeout=20 ,onTimeout="silent" )
>>>>>>> overview[i,1]<-infocriteria(tempgarch)[1]
>>>>>>> }
>>>>>>> , error = function(x) x)
>>>>>>>
>>>>>>> tt<- proc.time() - ptm
>>>>>>> overview[i,2]<-tt[3]
>>>>>>> }
>>>>>>>
>>>>>>> overview
>>>>>>>
>>>>>>> # If the timeout is set set to 20, this setup leads to:
>>>>>>> # 2.87 sec.
>>>>>>> # 6.95 sec.
>>>>>>> # 125 sec. ... here, tryCatch interrupted the process
>>>>>>> # 51.73 sec.
>>>>>>> # 27.11 sec.
>>>>>>> # for the 5 different estimation steps.
>>>>>>>
>>>>>>> # timeout set to 300:
>>>>>>> # 2.81 sec.
>>>>>>> # 6.85 sec.
>>>>>>> # 743.58 sec.
>>>>>>> # 41.70 sec.
>>>>>>> # 26.85 sec.
>>>>>>> # no process was interrupted by tryCatch
>>>>>>> #######################################################################
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> As can be seen even from this simplified example, when the timeout
>>>>>>> was
>>>>>>> set to be 20 there still was a process that took 125 seconds
>>>>>>> (which is
>>>>>>> more than 5 times longer!).
>>>>>>> I would be very thankful for any ideas or comments!
>>>>>>>
>>>>>>> Best, Johannes
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> --
>>>>>>
>>>>>> _______________________________________________
>>>>>> R-SIG-Finance at r-project.org mailing list
>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>>>>> -- Subscriber-posting only. If you want to post, subscribe first.
>>>>>> -- Also note that this is not the r-help list where general R
>>>>>> questions
>>>>>> should go.
>>>>>>
>>>>>>
>>>> --
>>>>
>>>>
--
More information about the R-SIG-Finance
mailing list