[R-SIG-Finance] How to properly compare a trading signal to a random strategy.

[Brian G. Peterson]

Harry Georgakopoulos wrote:
> 
>  Set up ^^^^
>  Let's say i have an evenly spaced discrete time-series of bid-ask prices.  Let's also say that the total number of such bid-ask pairs is N for a given day. Given a signal that generates a buy opportunity on "n" such times (where n << N), how can I  reliably say that these n signals have a mean profit that is statistically significant?
>  For example, assume I get 35 buy signals throughout the day where I buy the offer, wait 5  minutes and then sell the bid.  This will generate a vector of 35 price-differences.  These price  differences will have a particular distribution. 
>  Thoughts  ^^^^^ 
>  1.  I can compare the distribution of the 35 price-differences generated from the signal against      the distribution of 35 randomly chosen entry points throughout the day. (maybe some kind        of t-test on the difference of the means of these distributions)
>  2.  I can compare the distribution of the 35 price-differences to a rolling window of all possible  buys throughout the day and selling after 5 mins.  (more data-points to compare against)
>  3.  I can compare the distribution of the 35 price-differences against an absolute value of 0.
> Any ideas on quantifying the significance of such a signal would be appreciated.  Is one method preferred over another?  Am I inadvertently introducing bias in the analysis?  I realize that the distribution of the price-differences might not be normally distributed.  This might make any analysis based on a t-test invalid.  
> Thank you in advance.
> H.

Pat Burns has a paper on this topic on his website.

Regards,

     - Brian

-- 
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock