[R-SIG-Finance] IBrokers - reqHistory results in missing random data

[algotr8der]

Has anyone come across situations where the export of historical data from
IB's API has missing data points that occur randomly? 

I did the following to download data for the Select SPDR etfs:

> tws <- twsConnect()
> contract <- twsEquity('XHB','SMART','ISLAND')
> reqHistory(tws, Contract=contract) -> XHB

By default this is set to retrieve 1 years worth of minute data. I received
all of the data for XLE but noticed that XHB has fewer data points. I pulled
up the chart of XHB to examine whether those missing data points showed up
on the chart but all was well on the TWS Chart. As per IB, the backfilling
on the TWS Chart uses the same data export framework as that used by
reqHistoricalData. So I decided to re-download XHB. This time the missing
data points were different from the previous download. 

I used IB's TswDde Excel file to cross verify and I noticed that the data is
present using the Excel API. It could be that the problem did not surface
because TwsDde limits the export of 1 minute data to 2 days worth of data.
I'm speculating here but I do know that downloading via reqHistory produces
data with missing data points that occur randomly.

The other thing I noticed was that the data pulled by reqHistory begins at
9:30:00 and ends at 15:59:00 while the same using TswDde begins at 9:31:00
and ends at 16:00:00. 

2010-06-18 15:58:00    15.85    15.85   15.84     15.84       1569  15.843          
0       337
2010-06-18 15:59:00    15.84    15.85   15.81     15.81       3518  15.828          
0       527
2010-06-21 09:30:00    16.04    16.10   16.03     16.09        240  16.047          
0        47
2010-06-21 09:31:00    16.09    16.09   16.07     16.08        226  16.081          
0       119
2010-05-17 09:31:00    18.00    18.03   18.00     18.02        115  18.020          
0        39

--
View this message in context: http://r.789695.n4.nabble.com/IBrokers-reqHistory-results-in-missing-random-data-tp3533694p3533694.html
Sent from the Rmetrics mailing list archive at Nabble.com.