[R-SIG-Finance] comparison of trading rules

[Murali.MENON at fortisinvestments.com]

Hi Robert, Mark, Matthias,

Thanks for your responses.

I agree that measures such as IR are not robust, but for whatever
reason, practitioners seem to stick to these. They look at year-to-date
and 1yr and 3yr horizons, and seem to decide that one performance was
better than the other across these horizons. In such a comparison, they
would take daily returns, say, irrespective of whether the strategy was
active or not, and then do a straight (mean - benchmark_mean) / std dev.
Which is why I asked the question if this comparison was fair.

Regarding leverage: if both strategies are scaled to the same risk
budget and invest in the same underlying assets, they should have
roughly similar levels of leverage, I guess? 

Are you saying that over-leverage affects performance because of
increased transaction costs? Or is there some other reason for the
parabolic function?

Cheers,
Murali


-----Original Message-----
From: Robert Sams [mailto:robert at sanctumfi.com] 
Sent: 16 July 2009 12:35
To: MENON Murali; r-sig-finance at stat.math.ethz.ch
Subject: RE: [R-SIG-Finance] comparison of trading rules

Hi Murali,

Trading system performance is a subtle and multi-dimensional subject.
There is no simple metric or suite of metrics that can answer the
question "does system X have a better return-for-risk than system Y"? A
couple of key concepts to consider are:

1) Robustness. Widely used measures like sharpe ratio, max drawdown, etc
are NOT robust. They are very sensitive to the sample period used and
comparisons based on such metrics are not reliable. So, I think
pondering the subject of robust estimators is fundamental to trading
system evaluation.

2) Leverage. Cumulative return on a trading system is a geometric
process, so return and risk are not linear functions of leverage (as is
sometimes thought) but parabolic. Return increases with increased
leverage up to a point and then declines thereafter. Even a system with
a great positive edge can have negative overall performance if it is
over-leveraged. The leveraging assumptions behind two trading systems
must be normalised somehow for a comparison of returns to be meaningful.
Leverage is a ramified subject and raises many important questions in
trading system evaluation. In the literature it sometimes goes under the
title 'money management.'

As for your specific example of two systems with different percentages
of time out of the market, I'm not really sure what you're after. You
mentioned standard deviation. Presumably you would exclude those
observations where the system is out of the market, right? If we're
working with the assumption that you can dynamically invest (withdrawal)
capital from a strategy as it enters (exits) the market, then we
probably don't care about those zero-return periods in annualising risk
and return metrics. We might care about what happens when the system
isn't in the market from the perspective of understanding what
contribution the system's exit rules make to its overall performance,
but that's another matter. 

In short, you need to have a clear idea about what you're trying to
'compare', what questions you are trying to answer.

I hope this helps.

Robert


> -----Original Message-----
> From: r-sig-finance-bounces at stat.math.ethz.ch
> [mailto:r-sig-finance-bounces at stat.math.ethz.ch] On Behalf Of 
> Murali.MENON at fortisinvestments.com
> Sent: 15 July 2009 10:55
> To: r-sig-finance at stat.math.ethz.ch
> Subject: [R-SIG-Finance] comparison of trading rules
> 
> Folks,
>  
> As far as I can tell, the comparison of competing trading rules seems 
> to be done on the basis of such metrics as Sharpe ratio, drawdowns, 
> hit ratios and the like.
>  
> I wonder if these are entirely fair comparisons.
>  
> For example, if one strategy results in active positions 90% of the 
> time, while the other is active only 50% of the time, the standard 
> deviation of the latter strategy might be lower than the former 
> because of the longer stretches of zeroes in the returns. Should a 
> further adjustment be made for the inactive periods? Would this be 
> something as simple as dividing by the square root of the number of 
> inactive zeroes?
>  
> Are there any other ways of comparing the performance of strategies 
> that you can suggest? Any related literature?
>  
> Thanks,
>  
> Murali
> 
> 	[[alternative HTML version deleted]]
> 
> _______________________________________________
> R-SIG-Finance at stat.math.ethz.ch mailing list 
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only.
> -- If you want to post, subscribe first.
>