[R-SIG-Finance] portfolio.optim and error in solve.QP: matrix D not positive definite

Sat Jan 29 22:32:26 CET 2011

The problem could be due to few reasons the papers by Higham &  
Rebonato are a good read as to what the recipe does. The simplest  
recipe is to set the negative eigenvalues to a small positive number  
and rescale the matrix.

There are a few causes for this if you create corr matrices from  
bivariate estimation and slap them together that may not be PD. In  
some cases  in derivative pricing the matrix is hand glued together  
from implied correlations between two assets. Now a matrix of such  
implied corrs does not have to be PD and fails.

Sample corr matrices are by definition PD however often times if you  
have a lot of missing data(illiquid names in your universe?) and if  
you do a  na.locf (creating a const asset) this could do it as well.

It might be useful to do multivariate imputation with some o the R  
packages rather than deletion or carrying fwd on your data.

There are other reasons besides all of the above for your corr matrix  
being non PD. I think if you are working with such a large universe it  
might be easier to have factor corr matrices using PCA or ICA. This  
way if you manage to label the factors then you can see what bets your  
optimizer is taking.

HTH

Best
Krishna

On Jan 29, 2011, at 3:52 PM, "Lui ##" <lui.r.project at googlemail.com>  
wrote:

> Hello everybody,
>
> sorry for my delayed "thanks" note - I was travelling.
>
> @Arun: Debugging the underlying code is a little bit difficult since
> the optimizer was written in FORTRAN. I think going for the nearest PD
> (as Krishna also suggested) might be the best way. However, I honestly
> don't understand why it is not PD... Does anybody have an explanation
> for that?
> @Guy: The weird thing is that I got the error code without the t(x) in
> the first place. t(x) solved the problem (for some assets) and the
> result indicated that it took a look at the assets and not the
> observations... I am going to give it a try with the covariance matrix
> again and let you know if it worked out... strange though.
> @Krishna: Thanks for your link! I think that really helps... I am
> going to try it out! Do you have an explanation why this is a common
> problem with a large number of assets?
>
> Thank you! Have a nice weekend!
>
> Lui
>
> On Fri, Jan 28, 2011 at 3:51 PM, krishna <kriskumar at earthlink.net>  
> wrote:
>> Also the link below might help with a large number of assets this  
>> is a
>> common problem.
>>
>> https://stat.ethz.ch/pipermail/r-sig-finance/2008q3/002854.html
>>
>>
>> Cheers
>> Krishna
>>
>>
>> On Jan 27, 2011, at 12:03 PM, Guy Yollin <gyollin at r- 
>> programming.org> wrote:
>>
>>> Hi Lui,
>>>
>>> Without seeing the data this is just speculation but...
>>>
>>> Are you sure you want t(x)? If you're mixing up your observations  
>>> versus
>>> your assets this may explain the error.
>>>
>>> The first parameter of portfolio.optim (in the tseries package) is a
>>> returns matrix, one column for each asset and one row for each day  
>>> (assuming
>>> daily returns).  If you have this wrong then for your small  
>>> datasets you'd
>>> have more columns than rows and this could produce that error.
>>>
>>> Also, you don't have to pass the entire returns matrix to  
>>> portfolio.optim,
>>> you could pass just the covariance matrix you calculate yourself  
>>> and a
>>> vector (1-row matrix) of mean returns as follows:
>>>
>>>
>>> library(tseries)
>>> set.seed(2)
>>> R <- matrix(rnorm(100*10),nrow=100,ncol=10) # 10 assets, 100  
>>> observations
>>> averet <- matrix(apply(R,2,mean),nrow=1)
>>> rcov <- cov(R)
>>> current_er <- 0.05
>>> (op <- portfolio.optim(x=averet,pm=current_er,covmat=rcov,riskless =
>>> FALSE,shorts = FALSE, rf = 0.0))
>>>
>>> Hope this helps.
>>>
>>> Best,
>>>
>>> Guy
>>>
>>>
>>> On 1/26/2011 7:51 PM, Lui ## wrote:
>>>>
>>>> Dear Group,
>>>>
>>>> I have  a large set of stocks and want to determine the efficient
>>>> frontier. The data set covers approx. 1.5 years and S&P 500  
>>>> companies
>>>> (nothing weird). portfolio.optim from the PerformanceAnalytics  
>>>> package
>>>> works very well and fast. However, whenever I decrease the number  
>>>> of
>>>> stocks in the portfolio (to 10 or 400), I receive an error message:
>>>>
>>>> "solve.QP(Dmat, dvec, Amat, bvec = b0, meq = 2) :
>>>>  matrix D in quadratic function is not positive definite!"
>>>>
>>>> My command settings for portfolio.optim were:
>>>>
>>>> seed<- portfolio.optim(t(x), pm = current_er, riskless = FALSE,
>>>> shorts = FALSE, rf = 0.0)
>>>>
>>>> Even when I tried it with shorts = TRUE the error would still  
>>>> remain.
>>>> x is the set of stocks (stocks in columns, time in rows),  
>>>> current_er
>>>> is the target return (lies between the minimal mean and the maximum
>>>> mean of a long only portfolio).
>>>> I can not post the stock data here - so maybe you have some general
>>>> suggestions for me of what could have gone wrong... The covariance
>>>> matrix is positive definite. What could cause the problem? It works
>>>> fine with the large data set but does not work at all with the  
>>>> small
>>>> one...
>>>> Thanks a lot for your suggestions!
>>>>
>>>> Lui
>>>>
>>>> _______________________________________________
>>>> R-SIG-Finance at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>>> -- Subscriber-posting only. If you want to post, subscribe first.
>>>> -- Also note that this is not the r-help list where general R  
>>>> questions
>>>> should go.
>>>>
>>>
>>> _______________________________________________
>>> R-SIG-Finance at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>> -- Subscriber-posting only. If you want to post, subscribe first.
>>> -- Also note that this is not the r-help list where general R  
>>> questions
>>> should go.
>>
>> _______________________________________________
>> R-SIG-Finance at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>> -- Subscriber-posting only. If you want to post, subscribe first.
>> -- Also note that this is not the r-help list where general R  
>> questions
>> should go.
>>
>
> _______________________________________________
> R-SIG-Finance at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R  
> questions should go.