[R-SIG-Finance] Preparing data for Superior predictive ability (SPA) test
Karthik Raju
commons.r at karthik.in
Fri Dec 5 09:17:27 CET 2014
Sorry Brian, I misunderstood your last reply.
Intent: Evaluate forecasting accuracy of autoregressive models
Approach: Apply Superior Predictive Ability (SPA) test in the “ttrTests”
package to assess the forecasting accuracy
Version: R ver. 3.0.2 (2013-09-25) - "Frisbee Sailing"
Editor: R Studio ver. 0.98.1091
rugarch package: 1.3-4
ttrTests package: 1.7
Code: http://dropbox.unl.edu/uploads/20141219/e61f31203e15e90d/sample.zip
I am lost and looking for some direction on how to use a loss function
series to evaluate the efficacy of volatility forecasts between models
fm1 through fm5 using the SPA test in the "ttrTests" package.
My attempt is as follows:
spa <- dataSnoop(MeanSquareError,bSamples=1000,test="SPA")
# The structure of "MeanSquareError" loss function series is as follows:
Day msefm1 msefm2 .. msefm5
T+1 0.0000464 0.0000411 .. 0.0000407
T+2 0.0000461 0.0000421 .. 0.0000428
...
T+49 0.0000174 0.0000653 .. 0.0001126
T+50 0.0000174 0.0000657 .. 0.0001138
My code is structured as follows:
# To load the sample data in the "rugarch" package
data(sp500ret)
# To specify the desired autoregressive models
model1 = ugarchspec(variance.model = list(model = "sGARCH",garchOrder =
c(1, 1)), distribution.model = "std")
model2 = ugarchspec(variance.model = list(model = "gjrGARCH",garchOrder
= c(1, 1)), distribution.model = "std")
model3 = ugarchspec(variance.model = list(model = "sGARCH",garchOrder =
c(1, 1)), distribution.model = "std")
model4 = ugarchspec(variance.model = list(model = "csGARCH",garchOrder =
c(1, 1)), distribution.model = "std")
model5 = ugarchspec(variance.model = list(model = "apARCH",garchOrder =
c(1, 1)), distribution.model = "std")
# To estimate the autoregressive models specified
m1 = ugarchfit(sp500ret, spec = model1)
m2 = ugarchfit(sp500ret, spec = model2)
m3 = ugarchfit(sp500ret, spec = model3)
m4 = ugarchfit(sp500ret, spec = model4)
m5 = ugarchfit(sp500ret, spec = model5)
# To forecast the volatility for next 50 days with the parameters
estimated above
fm1 = ugarchforecast(m1,data=NULL,n.ahead=50, n.roll= 0, out.sample = 0)
fm2 = ugarchforecast(m2,data=NULL,n.ahead=50, n.roll= 0, out.sample = 0)
fm3 = ugarchforecast(m3,data=NULL,n.ahead=50, n.roll= 0, out.sample = 0)
fm4 = ugarchforecast(m4,data=NULL,n.ahead=50, n.roll= 0, out.sample = 0)
fm5 = ugarchforecast(m5,data=NULL,n.ahead=50, n.roll= 0, out.sample = 0)
I used the extractor function "sigma" in the "rugarch" package to
extract the conditional standard deviation of the forecasts fm1 through
fm5, which is then fed to a spreadsheet to estimate the Mean Square
Error (MSE) loss function series for the 50 days.
# The MSE loss function series is then imported into R.
MeanSquareError <- read.csv("F:/MeanSquareError.csv")
Now, how do I go about using this “MeanSquareError” loss function series
such as this: spa<-dataSnoop(MeanSquareError, bSamples=1000,
test="SPA")? Please note that I am not looking to convert
“MeanSquareError.csv” into R-data, rather I am searching for a way to
apply the SPA test.
Thank you for your time.
Karthik
Brian G. Peterson wrote on 2014-11-29 03:51 PM +0530:
> I asked you a reproducible example, and instead, you're asking the list
> to do a bunch of work to replicate your code for you.
>
> Please respect the time of the list members, and provide a few lines of
> code, so that we can *run* your example code and then *help you* with
> your problem.
>
> Brian
>
> On 11/28/2014 09:44 PM, Karthik Raju wrote:
>> Hello Brian,
>>
>> Thank you for your response.
>>
>> I will explain my work with an example.
>>
>> - I have logarithmic returns computed from daily closing values of a
>> stock market index for a period of 500 days.
>>
>> - The logarithmic returns are used in the "rugarch" package to fit the
>> following models,
>> Model-1 - GARCH (1,1)
>> Model-2 - GJR-GARCH (1,1)
>> Model-3 - EGARCH (1,1)
>> Model-4 - IGARCH (1,1)
>> Model-5 - APARCH (1,1)
>>
>> - Then, using "rugarch", forecasted one step ahead volatility with each
>> of the above model parameters for an out-of-sample period of 100 days.
>>
>> - The forecasted 100 days "sigma" values of each model is extracted from
>> R to a worksheet and used against true volatility proxy to compute the
>> loss functions.
>>
>> - The structure of loss function data for each model is an univariate
>> series for the forecasted period (100 days).
>>
>> - For example, the structure of Mean Square Error (MSE) loss function
>> looks like,
>>
>> Forecasts MSE_GARCH MSE_GJR . . MSE_APARCH
>> Day 1 0.008548 0.005509 . . 0.00412
>> Day 2 0.008655 0.005498 . . 0.00414
>> Day 3 0.008759 0.005488 . . 0.004156
>> Day 4 0.008861 0.005478 . . 0.004169
>> . . . . . .
>> . . . . . .
>> . . . . . .
>> Day 99 0.008759 0.005488 . . 0.004156
>> Day 100 0.008861 0.005478 . . 0.004169
>>
>> - Similarly, the loss functions like Mean Absolute Error, Mean Absolute
>> Percentage Error etc. are in the same structure, which are then imported
>> into R.
>>
>> - Now, by using each of this loss function data series, one at a time,
>> in the "ttrTests" package, I want to know how to compute the SPA test
>> for the following combinations?
>>
>> Benchmark model GARCH (1,1) Vs rest of 4 models
>> Benchmark model GJR GARCH (1,1) Vs rest of 4 models
>> Benchmark model EGARCH (1,1) Vs rest of 4 models
>> Benchmark model IGARCH (1,1) Vs rest of 4 models
>> Benchmark model APARCH (1,1) Vs rest of 4 models
>>
>> Best,
>> Karthik
>>
>>> Please follow the posting guide
>>>
>>> http://www.r-project.org/posting-guide.html
>>>
>>> and provide a reproducible example. For instance, you could repeat your
>>> 'n' samples using some of the test data from rugarch.
>>>
>>> It is very difficult for others to help you if we don't know the format
>>> your data is in *now*.
>>>
>>> Regards,
>>>
>>> Brian
>>>> On 11/27/2014 11:24 PM, Karthik Raju wrote:
>>>>
>>>> Hello,
>>>>
>>>> Can anyone please let me know how to prepare data to compute Superior
>>>> predictive ability (SPA) test in R?
>>>>
>>>> I am working on forecasting volatility in stock markets, the context is,
>>>>
>>>> 1) I used "rugarch" package to forecast the volatility using "n"
>>>> autoregressive models.
>>>>
>>>> 3) Extracted "sigma" values from the forecasts of those "n" models and
>>>> used it in a worksheet against realized volatility to calculate daily
>>>> loss functions for each model, say, MSE, MAE, RMSE etc.
>>>>
>>>> 3) I want to compute the SPA test statistics using "ttrTests" package
>>>> and find the SPA test p-values for a benchmark model against "n"
>>>> alternatives. How to prepare data for this purpose?
>>>>
>>>> Thanks,
>>>> Karthik
>>
>> _______________________________________________
>> R-SIG-Finance at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>> -- Subscriber-posting only. If you want to post, subscribe first.
>> -- Also note that this is not the r-help list where general R questions
>> should go.
More information about the R-SIG-Finance
mailing list