[R-SIG-Finance] rugarch gives two different results based on the same model…how is that even possible?

GALIB KHAN ghk18 @ending from @c@rletm@il@rutger@@edu
Mon Aug 20 14:29:03 CEST 2018


Alexios,

I changed the solver to "hybrid" but kept the scaling that you provided and
got different results when switching the positions for the external
regressors.

ugarchfit(spec = spec, data = as.matrix(temp$y),solver =
"hybrid",fit.control=list(scale=1))

So it looks like you have to change the solver to "nlminb" and update
fit.control with the scaling that you provided.

Galib


On Sun, Aug 19, 2018 at 11:41 PM, GALIB KHAN <ghk18 using scarletmail.rutgers.edu>
wrote:

> Alexios,
>
>
> Veryyyy interesting!!!!! No I cannot see any differences at all lol.
>
> I updated the code and indeed you are correct sir. Thank you for your time
> in investigating this.
>
> I will update my stack exchange post to reflect your answer in the morning.
>
> Again thank you for all your help!!!!
>
> Best,
> Galib Khan
>
>
> On Sun, Aug 19, 2018 at 11:19 PM, alexios galanos <alexios using 4dscape.com>
> wrote:
>
>> I did use the seed you provided.
>>
>> Use the following code for estimation:
>>
>> fit <- ugarchfit(spec = spec, data = as.matrix(temp$y),solver = "nlminb",
>> fit.control=list(scale=1))
>>
>> model_maker(var1)
>>          Estimate  Std. Error     t value     Pr(>|t|)
>> mu     -7.3998577  0.69086641 -10.7109821 0.0000000000
>> ar1     0.3387323  0.08280162   4.0908900 0.0000429721
>> ar2    -0.8834201  0.06569477 -13.4473414 0.0000000000
>> ma1    -0.2902069  0.08598589  -3.3750525 0.0007380161
>> ma2     0.8660807  0.06778418  12.7770320 0.0000000000
>> mxreg1  1.6782992  0.12769644  13.1428825 0.0000000000
>> mxreg2  2.5225382  0.04292728  58.7630625 0.0000000000
>> omega  12.0047145  0.82986864  14.4658010 0.0000000000
>> alpha1  0.0000000  0.07358520   0.0000000 1.0000000000
>> shape  63.0103309 98.49188643   0.6397515 0.5223341761
>>
>> model_maker(var2)
>>          Estimate  Std. Error     t value     Pr(>|t|)
>> mu     -7.3998549  0.69086651 -10.7109764 0.000000e+00
>> ar1     0.3387334  0.08280150   4.0909088 4.296861e-05
>> ar2    -0.8834206  0.06569433 -13.4474406 0.000000e+00
>> ma1    -0.2902081  0.08598562  -3.3750776 7.379487e-04
>> ma2     0.8660811  0.06778412  12.7770487 0.000000e+00
>> mxreg1  2.5225383  0.04292728  58.7630642 0.000000e+00
>> mxreg2  1.6782987  0.12769640  13.1428817 0.000000e+00
>> omega  12.0047142  0.82992363  14.4648419 0.000000e+00
>> alpha1  0.0000000  0.07359329   0.0000000 1.000000e+00
>> shape  63.0105962 98.49368444   0.6397425 5.223400e-01
>>
>>
>> I can’t see any “significant” differences, can you?
>> It’s completely related to the optimization/starting parameters. The
>> “scale” is documented and not on by default (perhaps it should be).
>>
>> Alexios
>>
>>
>> > On Aug 19, 2018, at 9:02 PM, GALIB KHAN <ghk18 using scarletmail.rutgers.edu>
>> wrote:
>> >
>> > Sorry for sending this again, I didn't include r-sig-finance in the
>> email address. I'm still adjusting in how to respond.
>> >
>> > Alexios,
>> >
>> > Did you set the set the seed to 1, because I'm looking at your results
>> and the numbers do not match with the numbers that I have provided.
>> >
>> > I understand why the coefficients' estimates are similar but it doesn't
>> explain why other columns such as the t-value and pr are off by a large
>> margin. Also estimates for mu, ar*, ma*, omega, alpha1, and shape may have
>> large differences.
>> >
>> > Take mu as an example:
>> > -7.538187e+00 - (-7.877120e+00) = 0.338933, isn't that considered a
>> large difference to the point where it's safe to say that these two values
>> are not similar?
>> >
>> > Another example is the t-values for x1 and x2:
>> > x1 = 8.799994e+01   -  5.509361e+02 = -462.9362
>> > x2 = 8.508606e+01   -  5.287634e+02 = -443.6773
>> >
>> > An more alarming case that unfortunately I cannot share due to the data
>> being sensitive is that when the x variables' positions are switched, the
>> p-values are not the same. The p-value for a particular external regressor
>> went from 0 to 0.4385.
>> >
>> > I will attempt to re-create a separate generic dataset that is similar
>> to the sensitive data that I am using.
>> >
>> >
>> > Galib Khan
>> >
>> >
>> > On Sun, Aug 19, 2018 at 10:06 PM, alexios galanos <alexios using 4dscape.com>
>> wrote:
>> > I run the code you provided and obtain the following results related to
>> the external parameters:
>> >
>> >
>> > Case 1 (x1,x2)
>> > # x2 is second
>> >
>> >             Estimate   Std. Error       t value  Pr(>|t|)
>> > mxreg1  1.6724148 1.203377e-01  1.389767e+01 0.0000000
>> > mxreg2  2.5310286 1.878833e-02  1.347128e+02 0.0000000
>> >
>> > Case 2 (x2,x1)
>> > # i.e. x2 is now first
>> >
>> > mxreg1  2.5225382  0.04292725  58.7631024 0.000000e+00
>> > mxreg2  1.6782986  0.12769622  13.1428990 0.000000e+00
>> >
>> > Small differences in the coefficients are the result of the optimizer.
>> There may be an issues in the
>> > way starting parameters are being generated based on some recent input
>> from Josh Ulrich (still to investigate)
>> > and related to arima0 (used to generate start parameters), but
>> otherwise don’t see a large problem at first glance.
>> >
>> > Alexios
>> >
>> > > On Aug 19, 2018, at 5:46 PM, GALIB KHAN <
>> ghk18 using scarletmail.rutgers.edu> wrote:
>> > >
>> > > Recently I have discovered a problem with a package called rugarch
>> that
>> > > creates arma-garch models. The issue is that if you literally change
>> the
>> > > positions of the x variables (external regressors) then you get two
>> > > completely different results.
>> > >
>> > > In other words:
>> > >
>> > >   - model1 = (arma(2,2) + garch(1,0) + x1 + x2)
>> > >   - model2 = (arma(2,2) + garch(1,0) + x2 + x1)
>> > >   - rugarch's output is essentially saying that model1 != model2
>> > >   - When the correct result should be model1 == model2
>> > >
>> > > I may not know a lot of statistics but I know for a fact that if you
>> move
>> > > the x variables around, the output should still be the same.
>> > >
>> > > Am I wrong on this?
>> > >
>> > > Here's my stack exchange post that shows a generic R script proving my
>> > > point: Should the positioning of the external regressors change the
>> output
>> > > of arma-garch? (Possible rugarch bug/error)
>> > > <https://stackoverflow.com/questions/51900177/should-the-pos
>> itioning-of-the-external-regressors-change-the-output-of-arma-garc>
>> > >
>> > > Any feedback is welcomed.
>> > >
>> > > Thanks
>> > >
>> > >       [[alternative HTML version deleted]]
>> > >
>> > > _______________________________________________
>> > > R-SIG-Finance using r-project.org mailing list
>> > > https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>> > > -- Subscriber-posting only. If you want to post, subscribe first.
>> > > -- Also note that this is not the r-help list where general R
>> questions should go.
>> > >
>> >
>> >
>>
>>
>

Galib Khan


On Sun, Aug 19, 2018 at 11:41 PM, GALIB KHAN <ghk18 using scarletmail.rutgers.edu>
wrote:

> Alexios,
>
>
> Veryyyy interesting!!!!! No I cannot see any differences at all lol.
>
> I updated the code and indeed you are correct sir. Thank you for your time
> in investigating this.
>
> I will update my stack exchange post to reflect your answer in the morning.
>
> Again thank you for all your help!!!!
>
> Best,
> Galib Khan
>
>
> On Sun, Aug 19, 2018 at 11:19 PM, alexios galanos <alexios using 4dscape.com>
> wrote:
>
>> I did use the seed you provided.
>>
>> Use the following code for estimation:
>>
>> fit <- ugarchfit(spec = spec, data = as.matrix(temp$y),solver = "nlminb",
>> fit.control=list(scale=1))
>>
>> model_maker(var1)
>>          Estimate  Std. Error     t value     Pr(>|t|)
>> mu     -7.3998577  0.69086641 -10.7109821 0.0000000000
>> ar1     0.3387323  0.08280162   4.0908900 0.0000429721
>> ar2    -0.8834201  0.06569477 -13.4473414 0.0000000000
>> ma1    -0.2902069  0.08598589  -3.3750525 0.0007380161
>> ma2     0.8660807  0.06778418  12.7770320 0.0000000000
>> mxreg1  1.6782992  0.12769644  13.1428825 0.0000000000
>> mxreg2  2.5225382  0.04292728  58.7630625 0.0000000000
>> omega  12.0047145  0.82986864  14.4658010 0.0000000000
>> alpha1  0.0000000  0.07358520   0.0000000 1.0000000000
>> shape  63.0103309 98.49188643   0.6397515 0.5223341761
>>
>> model_maker(var2)
>>          Estimate  Std. Error     t value     Pr(>|t|)
>> mu     -7.3998549  0.69086651 -10.7109764 0.000000e+00
>> ar1     0.3387334  0.08280150   4.0909088 4.296861e-05
>> ar2    -0.8834206  0.06569433 -13.4474406 0.000000e+00
>> ma1    -0.2902081  0.08598562  -3.3750776 7.379487e-04
>> ma2     0.8660811  0.06778412  12.7770487 0.000000e+00
>> mxreg1  2.5225383  0.04292728  58.7630642 0.000000e+00
>> mxreg2  1.6782987  0.12769640  13.1428817 0.000000e+00
>> omega  12.0047142  0.82992363  14.4648419 0.000000e+00
>> alpha1  0.0000000  0.07359329   0.0000000 1.000000e+00
>> shape  63.0105962 98.49368444   0.6397425 5.223400e-01
>>
>>
>> I can’t see any “significant” differences, can you?
>> It’s completely related to the optimization/starting parameters. The
>> “scale” is documented and not on by default (perhaps it should be).
>>
>> Alexios
>>
>>
>> > On Aug 19, 2018, at 9:02 PM, GALIB KHAN <ghk18 using scarletmail.rutgers.edu>
>> wrote:
>> >
>> > Sorry for sending this again, I didn't include r-sig-finance in the
>> email address. I'm still adjusting in how to respond.
>> >
>> > Alexios,
>> >
>> > Did you set the set the seed to 1, because I'm looking at your results
>> and the numbers do not match with the numbers that I have provided.
>> >
>> > I understand why the coefficients' estimates are similar but it doesn't
>> explain why other columns such as the t-value and pr are off by a large
>> margin. Also estimates for mu, ar*, ma*, omega, alpha1, and shape may have
>> large differences.
>> >
>> > Take mu as an example:
>> > -7.538187e+00 - (-7.877120e+00) = 0.338933, isn't that considered a
>> large difference to the point where it's safe to say that these two values
>> are not similar?
>> >
>> > Another example is the t-values for x1 and x2:
>> > x1 = 8.799994e+01   -  5.509361e+02 = -462.9362
>> > x2 = 8.508606e+01   -  5.287634e+02 = -443.6773
>> >
>> > An more alarming case that unfortunately I cannot share due to the data
>> being sensitive is that when the x variables' positions are switched, the
>> p-values are not the same. The p-value for a particular external regressor
>> went from 0 to 0.4385.
>> >
>> > I will attempt to re-create a separate generic dataset that is similar
>> to the sensitive data that I am using.
>> >
>> >
>> > Galib Khan
>> >
>> >
>> > On Sun, Aug 19, 2018 at 10:06 PM, alexios galanos <alexios using 4dscape.com>
>> wrote:
>> > I run the code you provided and obtain the following results related to
>> the external parameters:
>> >
>> >
>> > Case 1 (x1,x2)
>> > # x2 is second
>> >
>> >             Estimate   Std. Error       t value  Pr(>|t|)
>> > mxreg1  1.6724148 1.203377e-01  1.389767e+01 0.0000000
>> > mxreg2  2.5310286 1.878833e-02  1.347128e+02 0.0000000
>> >
>> > Case 2 (x2,x1)
>> > # i.e. x2 is now first
>> >
>> > mxreg1  2.5225382  0.04292725  58.7631024 0.000000e+00
>> > mxreg2  1.6782986  0.12769622  13.1428990 0.000000e+00
>> >
>> > Small differences in the coefficients are the result of the optimizer.
>> There may be an issues in the
>> > way starting parameters are being generated based on some recent input
>> from Josh Ulrich (still to investigate)
>> > and related to arima0 (used to generate start parameters), but
>> otherwise don’t see a large problem at first glance.
>> >
>> > Alexios
>> >
>> > > On Aug 19, 2018, at 5:46 PM, GALIB KHAN <
>> ghk18 using scarletmail.rutgers.edu> wrote:
>> > >
>> > > Recently I have discovered a problem with a package called rugarch
>> that
>> > > creates arma-garch models. The issue is that if you literally change
>> the
>> > > positions of the x variables (external regressors) then you get two
>> > > completely different results.
>> > >
>> > > In other words:
>> > >
>> > >   - model1 = (arma(2,2) + garch(1,0) + x1 + x2)
>> > >   - model2 = (arma(2,2) + garch(1,0) + x2 + x1)
>> > >   - rugarch's output is essentially saying that model1 != model2
>> > >   - When the correct result should be model1 == model2
>> > >
>> > > I may not know a lot of statistics but I know for a fact that if you
>> move
>> > > the x variables around, the output should still be the same.
>> > >
>> > > Am I wrong on this?
>> > >
>> > > Here's my stack exchange post that shows a generic R script proving my
>> > > point: Should the positioning of the external regressors change the
>> output
>> > > of arma-garch? (Possible rugarch bug/error)
>> > > <https://stackoverflow.com/questions/51900177/should-the-pos
>> itioning-of-the-external-regressors-change-the-output-of-arma-garc>
>> > >
>> > > Any feedback is welcomed.
>> > >
>> > > Thanks
>> > >
>> > >       [[alternative HTML version deleted]]
>> > >
>> > > _______________________________________________
>> > > R-SIG-Finance using r-project.org mailing list
>> > > https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>> > > -- Subscriber-posting only. If you want to post, subscribe first.
>> > > -- Also note that this is not the r-help list where general R
>> questions should go.
>> > >
>> >
>> >
>>
>>
>

	[[alternative HTML version deleted]]



More information about the R-SIG-Finance mailing list