[R-SIG-Finance] Yahoo Finance Stock data (FTSEMIB.MI) Issue

[Daniel Cegiełka]

> On 18 Jun 2020, at 19:48, Rasmus Liland <jral using posteo.no> wrote:
> 
> On 2020-06-18 18:48 +0200, Daniel Cegiełka wrote:
>> On 18 Jun 2020, at 16:58, Christopher Kromm wrote:
>>> 
>>> If I cannot get the data from Yahoo 
>>> Finance, are there alternative suggested 
>>> sources for an R user to get this data?
>> 
>> curl 'https://charts.borsaitaliana.it/charts/services/ChartWService.asmx/GetPrices' -H 'Connection: keep-alive' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'DNT: 1' -H 'X-Requested-With: XMLHttpRequest' -H 'Content-Type: application/json; charset=UTF-8' -H 'Origin: https://charts.borsaitaliana.it' -H 'Sec-Fetch-Site: same-origin' -H 'Sec-Fetch-Mode: cors' -H 'Referer: https://charts.borsaitaliana.it/charts/Bit/SummaryChart.aspx?code=CB.FTSEMIB&lang=en' -H 'Accept-Encoding: gzip, deflate, br' --data-binary '{"request":{"SampleTime":"1d","TimeFrame":"5y","RequestedDataSetType":"ohlc","ChartPriceType":"price","Key":"FTSEMIB.IDX","OffSet":0,"FromDate":null,"ToDate":null,"UseDelay":true,"KeyType":"Topic","KeyType2":"Topic","Language":"en-US"}}' --compressed > FTSEMIB.json
> 
> Dear Daniel Cegiełka,

Daniel :)

> that's a great answer!  Are you (or others) 
> able to elaborate more on how you determined 
> the headers to send?
> 

Let's start that web scraping is probably legal now (in US):

https://towardsdatascience.com/web-scraping-is-now-legal-6bf0e5730a78 <https://towardsdatascience.com/web-scraping-is-now-legal-6bf0e5730a78>

Scraping requires some experience and intuition. There is no other way than web debugging - and each one is different. However, current web browsers are very powerful and have many useful add-ons. Sometimes it takes a minute, and sometimes it requires a lot of work. A good source for learning is getSymbols:

https://github.com/joshuaulrich/quantmod/blob/master/R/getSymbols.R <https://github.com/joshuaulrich/quantmod/blob/master/R/getSymbols.R>

And

https://stackoverflow.com/questions/44030983/yahoo-finance-url-not-working <https://stackoverflow.com/questions/44030983/yahoo-finance-url-not-working>

Best


> Best,
> Rasmus


	[[alternative HTML version deleted]]