Hello Wind,
I was wondering if you made any progress on the R implementation of the
model, and if you could/would be willing to share what you got so far?

Regards,

-Mark-

2009/7/24 Wind <windspeedo99@gmail.com>

> Since I am not good at coding or statistics, genetic algo has been used
> instead of tabu search.    Package rgenoud is great and idiot proof.
>
> Attached is the latest analysis on SSEC index, the equity market index of
> mainland China.  genoud() has been run for 10 times, generating 10 fits.
> The residuals are all stationary according to ADF test from package urca.
> But the time window setting is subjective and maybe some minor problems on
> parameter conditions such as omega and C.  So,  just for fun.
>
> Thanks for all the encouragement and help from the list.    And thanks for
> the detailed instructions on statistics issues by one of the author on LPPL,
> Dr. Lin.
>
> wind
>
> On Mon, Jul 20, 2009 at 9:22 PM, Brian G. Peterson <brian@braverock.com>wrote:
>
>> Glad to see you're making progress on this problem.
>>
>> This paper
>> http://www.diegm.uniud.it/satt/papers/DiSc06b.pdf
>>
>> implements a tabu search in R, though they didn't publish their code.  You
>> might want to contact them for the implementation and permission to share
>> their tabu search algorithm/code with the R community.
>>
>> They also reference an R package called RACE by this gentleman:
>> http://iridia.ulb.ac.be/~mbiro/
>> that they use for evaluating the solution.
>>
>> More generally, from my limited understanding, tabu search is an extension
>> and refinement of simulated annealing approaches easily implemented in R.
>>  In brief the approach is to take your best 'n' solutions from a random
>> space search, and then search "near" those solutions.  Simulated annealing
>> and its close cousins have a lot of benefits in finance, where a single true
>> optima from a closed form problem is not likely to be available.  I
>> personally have rarely found 'optim' to be usable for my problem space, and
>> have had to use other solvers for practical problems in finance.
>>
>> Regards,
>>
>>  - Brian
>>
>>
>> Wind wrote:
>>
>>> Some progress.   The LPPL curve could be plotted with the following
>>> codes.
>>> The problem now is how to get the best fit parameters.
>>> Some researchers  use python or matlab for LPPL calibrating.     It
>>> seems that some of them prefer tabu search for optimums locating.   It
>>> seems that there's still no general function for tabu search in R.
>>> At the end of codes, I give the possible parameter combinations to be
>>> searched in, maybe there are other functions for optimum searching in
>>> R.
>>> Any suggestion would be appreciated.
>>>
>>>
>>> ## Financial Bubbles, Real Estate bubbles, Derivative Bubbles, and the
>>> Financial and Economic Crisis
>>> ## http://arxiv.org/abs/0905.0220
>>> ## Fig. 23 S&P500 index (in logarithmic scale)in  Page 39
>>>
>>>
>>> library(quantmod)
>>>
>>> LPPL1<-function(p,dtc=20,alpha=0.35,omega=0.1,phi=1)
>>> {
>>>        #function in page 26 of http://arxiv.org/abs/0905.0220
>>>        #the basic form of LPPL
>>>        dtc=abs(floor(dtc))
>>>        tc<-length(p)+dtc
>>>        dt<-abs(tc-(1:length(p)))
>>>
>>>        x1<-dt^alpha
>>>        x2<-(dt^alpha)*cos(omega*log(dt)+phi)
>>>
>>>        f<-lm(log(p) ~ x1+x2)
>>>
>>>
>>>  f$para<-list(recno=dim(f$model)[1],dtc=dtc,alpha=alpha,omega=omega,phi=phi,sigma=summary(f)$sigma)
>>>        return(f)
>>> }
>>>
>>> opt.lppl<-function(x)
>>> {
>>>        #derived function for optim
>>>
>>>  return(LPPL1(p,dtc=x[1],alpha=x[2],omega=x[3],phi=x[4])$para$sigma)
>>> }
>>>
>>> LPPL1.x<-function(lpplf,pt=100)
>>> {
>>>        #x axis for predicting
>>>        dt<-abs((lpplf$recno+lpplf$dtc)-(1:(lpplf$recno+lpplf$dtc+pt)))
>>>        dt[dt==0]<-0.5
>>>
>>>        x1<-dt^lpplf$alpha
>>>        x2<-(dt^lpplf$alpha)*cos(lpplf$omega*log(dt)+lpplf$phi)
>>>        return(list(x1=x1,x2=x2))
>>>
>>> }
>>>
>>> #get the SP500 index
>>>
>>> pr<-getSymbols("^GSPC",auto.assign=FALSE,from="2003-10-1",to="2007-05-16")[,4]
>>> p<-as.numeric(pr)
>>>
>>>
>>> plot(p,type="l",log="y",xlim=c(0,length(p)+100),ylim=c(min(p),max(p)*1.2))
>>> abline(v=length(p),col="green")
>>>
>>> #something like the Fig. 23 in  Page 39 of
>>> http://arxiv.org/abs/0905.0220
>>> #but obviously the result is not calibrated well
>>> #using optim like this can not calibrate the LPPL model
>>> opts<-sapply(seq(1,50,10),function(x){
>>>                        o<-optim(c(x,0.6,20,1),opt.lppl)
>>>
>>>  f3<-LPPL1(p,dtc=o$par[1],alpha=o$par[2],omega=o$par[3],phi=o$par[4])
>>>                        xp<-LPPL1.x(f3$para,200)
>>>                        f3p<-predict(f3,data.frame(x1=xp$x1,x2=xp$x2))
>>>                        lines(exp(f3p),col="blue")
>>>                        lines(exp(fitted(f3)),col="red")
>>>                        f3$para
>>>                })
>>>
>>>
>>>
>>> ##crash point after dtc days
>>> dtc<-seq(1,100,1)
>>>
>>> ##appropraite range of the parametes of LPPL
>>> ##according to Dr. W.X. Zhou's new book which is in Chinese
>>> ##the increments of the sequences are added according to my own judement
>>> alpha<-seq(0.01,1.2,0.1)
>>> omega<-seq(0,40,1)
>>> phi<-seq(0,7,0.1)
>>>
>>> ##millions possible combinations
>>> #complete test would be difficult
>>> para<-expand.grid(dtc=dtc,alpha=alpha,omega=omega,phi=phi)
>>> dim(para)
>>>
>>> system.time(sigs<-apply(para[1:100,],1,function(x){LPPL1(p,dtc=x[1],alpha=x[2],omega=x[3],phi=x[4])$para$sigma}))
>>>
>>> ##methods for minimum sigma searching within the parameter combinations
>>> ##not implemented yet
>>>
>>>
>>> wind
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Jul 16, 2009 at 9:25 PM, Brian G. Peterson<brian@braverock.com>
>>> wrote:
>>>
>>>
>>>> So first, using your real name and ideally your professional identity,
>>>> ask
>>>> for the python code.  Better yet, get an academic buddy to do it.
>>>> Usually
>>>> getting access to the code isn't too tough.  Mention things like
>>>> "repeatable
>>>> research" and "collaboration" in your email.  Two of the authors publish
>>>> their email addresses in one of the papers you reference, so contacting
>>>> them
>>>> should be easy.
>>>>
>>>> Next port the python code to R.
>>>>
>>>> If you can't do that, then replicate the model in R "from scratch".  A
>>>> trivial scan of the paper in question lends several techniques that are
>>>> well
>>>> covered in R: AR, GARCH, power laws, linear regression, stochastic
>>>> discount
>>>> factor, Ornstein-Uhlenbeck, etc.
>>>> There are volumes of information available on these topics from within
>>>> R, in
>>>> numerous books, and in the archives of this mailing list and r-help.
>>>>
>>>> You're going to have to do your replication in pieces, probably starting
>>>> with their implementation of the log periodic power law (LPPL), for
>>>> which I
>>>> do not believe there is an existing direct analogue in R though all the
>>>> component parts necessary to replicate it should be readily available.
>>>>
>>>> As you work on each step of the replication, share your code with this
>>>> list
>>>> and the problems you are having with a particular step.  Ask specific,
>>>> directed questions with code to back them up.  Someone will likely help
>>>> you
>>>> solve the specific problem.
>>>>
>>>> In R generally, it is not necessary that you be able to *do* the math
>>>> (think
>>>> pencil and paper), but if you plan to replicate published work, it will
>>>> be
>>>> necessary to *understand* at least some of how the math works, and to be
>>>> able to pick out the names of techniques that you can search for an
>>>> utilize.
>>>>
>>>> Basically, I'm recommending that you (specifically) and others (more
>>>> generally) should share the process of replicating a technique like
>>>> this, as
>>>> well as the final product, to give all the rest of us who are likely to
>>>> be
>>>> helping "you" get all this done. quid pro quo.
>>>>
>>>> Cheers,
>>>>
>>>>  - Brian
>>>>
>>>>
>>>> --
>>>> Brian G. Peterson
>>>> http://braverock.com/brian/
>>>> Ph: 773-459-4973
>>>> IM: bgpbraverock
>>>>
>>>>
>>>>
>>>> Wind wrote:
>>>>
>>>>
>>>>> Prof. Sornette has spent years forcasting bubble burst with
>>>>> "log-periodic power law".    The latest paper  gives "a
>>>>> self-consistent model for explosive financial bubbles, which combines
>>>>> a mean-reverting volatility process and a stochastic conditional
>>>>> return which reflects nonlinear positive feedbacks and continuous
>>>>> updates of the investors' beliefs and sentiments."
>>>>>
>>>>> And his  latest  predicting is the burst of Chinese equity bubble at
>>>>> the end of July.     http://arxiv.org/abs/0907.1827
>>>>>
>>>>> While waiting to see the result, I wonder whether it is possible to
>>>>> replicate the forcast with R.  The model is in the page 10 of the "A
>>>>> Consistent Model of `Explosive' Financial Bubbles With Mean-Reversing
>>>>> Residuals",  http://arxiv.org/abs/0905.0128  .   The output chart is
>>>>> in the page 3 of "The Chinese Equity Bubble: Ready to Burst",
>>>>> http://arxiv.org/abs/0907.1827 .   I guess the authors of the latter
>>>>> paper use the same model as described in the first paper.
>>>>>
>>>>> Because statistics is still challenging for me though I could use R
>>>>> for  basic data manipulations,  I wonder which package or function
>>>>> would be necessary to implement the model in the paper.  The model
>>>>> seems more complicated than the models in the R tutorials for me.
>>>>> By the way, the author of the paper used Python and the codes are
>>>>> private.
>>>>>
>>>>> Any suggestion would be highly appreciated.
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>
>> --
>> Brian G. Peterson
>> http://braverock.com/brian/
>> Ph: 773-459-4973
>> IM: bgpbraverock
>>
>>
>
> _______________________________________________
> R-SIG-Finance@stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only.
> -- If you want to post, subscribe first.
>

	[[alternative HTML version deleted]]

