[R-SIG-Finance] quantmod::saveSymbols not working properly for me

Jeff Ryan jeff.a.ryan at gmail.com
Sun Jul 3 20:04:06 CEST 2011 

Take a look at attachSymbols in quantmod. 

It uses lazy loading and some additional R coolness to give you access to all symbols automatically. The example is using data sourced from yahoo on demand, but is easy enough to extend to files on disk as well. 

I'll try and post a nice example when I am not replying from my phone. 

Jeff

Jeffrey Ryan    |    Founder    |    jeffrey.ryan at lemnica.com

www.lemnica.com

On Jul 3, 2011, at 11:30 AM, Samo Pahor <samo.pahor at gmail.com> wrote:

> Hello experts,
> 
> I am trying to get rid of the database while managing my data. I am testing
> saving xts objects on disk in binanry form in .RData files. I encountered
> some strange issues.
> 
> I am downloading data from yahoo (approx 600 symbols - I have omitted some
> below to make exmaple more clear) and when I use saveSymbols to save them to
> disk some of them are not saved properly although I can manipulate them
> using R. If I save particular symbol "manually" using save it works OK. I am
> using two R sessions to see if it works correctley.
> 
> Reproducible code(session that is downloading data from yahoo and saving to
> disk):
> require(quantmod)
> dwhFolderRaw = "D:/Temp/symbols/yahoo_hist/raw/"
> #download from
> http://www.standardandpoors.com/prot/spf/docs/indices/SPUSA-500-USDUF--P-US-L--Constituents.xls
> SnP500ConstituentsFileName = "SPUSA-500-USDUF--P-US-L--Constituents.csv"
> urlSnP500 <- paste(eligibleFIsFolder, SnP500ConstituentsFileName, sep="")
> # Read in CSV from S&P website
> SnP500 <- read.csv(urlSnP500)
> symbols <- paste(trim(SnP500[,'Symbol']),collapse=";")
> # BRK needs to be "BRK_B" or "BRK-A", actually, change all "/" to "-"
> symbols <- gsub("/", "-", symbols)
> getSymbols(symbols, from='1900-01-01')
> showSymbols()
> saveSymbols(file.path=dwhFolderRaw)
> head(F)
> tail(F)
>> head(F)
>           F.Open F.High F.Low F.Close F.Volume F.Adjusted
> 1977-01-03  61.00  61.63 61.00   61.25   896300       0.50
> 1977-01-04  61.25  61.25 59.00   59.50  1504600       0.49
> 1977-01-05  59.50  59.88 58.50   58.75  1481600       0.48
> 1977-01-06  59.00  60.38 59.00   59.75  1670300       0.49
> 1977-01-07  59.75  60.50 59.63   60.13   940400       0.49
> 1977-01-10  60.13  60.63 60.13   60.63  1334000       0.50
>> tail(F)
>           F.Open F.High F.Low F.Close F.Volume F.Adjusted
> 2011-06-24  13.52  13.60 13.14   13.24 82707400      13.24
> 2011-06-27  13.35  13.55 13.09   13.46 49987600      13.46
> 2011-06-28  13.55  13.55 13.25   13.33 54013200      13.33
> 2011-06-29  13.41  13.51 13.32   13.42 54105500      13.42
> 2011-06-30  13.46  13.97 13.36   13.79 74076900      13.79
> 2011-07-01  13.92  14.13 13.73   14.02 66304800      14.02
> 
> When I do this F (and some other symbols also) only has 1K size of a file on
> disk.
> 
> Reproducible code with error (R session that tries to read/read data):
> require(quantmod)
> library(xts)
> getSymbols("BRF", src='RData', extension="RData",
> dir="D:/Temp/symbols/yahoo_hist/raw/")
> head(BRF)
> tail(BRF)
> getSymbols("F", src='RData', extension="RData",
> dir="D:/Temp/symbols/yahoo_hist/raw/")
> head(F)
> tail(F)
> 
>> require(quantmod)
> Loading required package: quantmod
> Loading required package: Defaults
> Loading required package: xts
> Loading required package: zoo
> Loading required package: TTR
>> getSymbols("BRF", src='RData', extension="RData",
> dir="D:/Temp/symbols/yahoo_hist/raw/")
> [1] "BRF"
>> 
>> head(BRF)
>           BRF.Open BRF.High BRF.Low BRF.Close BRF.Volume BRF.Adjusted
> 2009-05-14    23.68    24.49   23.68     24.45      31400        22.71
> 2009-05-15    24.67    24.84   24.25     24.35      28200        22.62
> 2009-05-18    25.00    26.85   24.85     25.89      53000        24.05
> 2009-05-19    25.99    26.89   25.69     26.31      97400        24.44
> 2009-05-20    26.93    27.29   26.10     26.20     133100        24.34
> 2009-05-21    25.92    25.93   25.30     25.50      71800        23.69
>> tail(BRF)
>           BRF.Open BRF.High BRF.Low BRF.Close BRF.Volume BRF.Adjusted
> 2011-06-24    55.36    59.33   54.89     54.99     136200        54.99
> 2011-06-27    55.05    55.62   54.86     55.40     324800        55.40
> 2011-06-28    55.50    56.47   55.36     56.33     218700        56.33
> 2011-06-29    56.38    56.71   56.11     56.17     472800        56.17
> 2011-06-30    56.61    57.14   56.39     56.74     360000        56.74
> 2011-07-01    57.10    57.76   56.83     57.58     269100        57.58
>> getSymbols("F", src='RData', extension="RData",
> dir="D:/Temp/symbols/yahoo_hist/raw")
> Error in fr[, -1] : incorrect number of dimensions
>> 
>> 
>> head(F)
> [1] FALSE
>> tail(F)
> [1] FALSE
> For example, F is not saved correctley. On disk it size is only 1 KB as
> output from saveSymbols(). When save from base is used everything is OK and
> it gets the size of 141 KB. Which is strange since saveSymbols is using
> save... Maybe it has something to do with outputing/saving big number of
> symbols/objects at once... What am I doing wrong?
> 
> If I save F in fors R session using
> save(F, file=paste(dwhFolderRaw, "F.RData", sep=""))
> 
> then F is saved to disk correctley (has size of 141K), in the second session
> everything works as it should:
>> getSymbols("F", src='RData', extension="RData",
> dir="D:/Temp/symbols/yahoo_hist/raw/")
> [1] "F"
>> 
>> head(F)
>           F.Open F.High F.Low F.Close F.Volume F.Adjusted
> 1977-01-03  61.00  61.63 61.00   61.25   896300       0.50
> 1977-01-04  61.25  61.25 59.00   59.50  1504600       0.49
> 1977-01-05  59.50  59.88 58.50   58.75  1481600       0.48
> 1977-01-06  59.00  60.38 59.00   59.75  1670300       0.49
> 1977-01-07  59.75  60.50 59.63   60.13   940400       0.49
> 1977-01-10  60.13  60.63 60.13   60.63  1334000       0.50
>> tail(F)
>           F.Open F.High F.Low F.Close F.Volume F.Adjusted
> 2011-06-24  13.52  13.60 13.14   13.24 82707400      13.24
> 2011-06-27  13.35  13.55 13.09   13.46 49987600      13.46
> 2011-06-28  13.55  13.55 13.25   13.33 54013200      13.33
> 2011-06-29  13.41  13.51 13.32   13.42 54105500      13.42
> 2011-06-30  13.46  13.97 13.36   13.79 74076900      13.79
> 2011-07-01  13.92  14.13 13.73   14.02 66304800      14.02
> 
> BTW: is there some sort of automatic way to get a list of
> constituents/symbols of S&P 500? I tried using code in the comments posted
> by Joshua Ulrich on this site http://www.maxdama.com/?p=148 and here
> http://www.mathworks.com/matlabcentral/fileexchange/23569-get-trading-data-from-yahoo-in-your-matlab-workpace-or-excel
> :
> require(quantmod)
> url <- "
> http://www2.standardandpoors.com/servlet/Satellite?pagename=spcom/page/download&sectorid=%20%3E%20%2700%27&itemname=%3E=%20%271%27&dt=07-MAY-2009&indexcode=500
> "
> # Read in CSV from S&P website
> x <- read.csv(url)
> # Pull the first 10 symbols from Yahoo
> symbols <- paste(x[1:10,'Symbol'],collapse=";")
> getSymbols(symbols, env=.GlobalEnv)
> 
> It is not working for dates beyond 2010. Any alternative ways?
> 
> Thanks in advance for any ideas what might be wrong.
> 
> Kind regards,
> Samo.
> 
>    [[alternative HTML version deleted]]
> 
> _______________________________________________
> R-SIG-Finance at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions should go.

More information about the R-SIG-Finance mailing list