[R] read .csv from web from password protected site

[Mike Pfeiff]

Yes, I meant to reply to all (sorry still new at asking for help)

1. No, I am not able to open the file when I insert "userID:password@"  between "http://" and "www"

	http://userid:password@www.frontierweather.com/degreedays/L15N15PowerRegionAverages_10weeks.txt
	(replacing userid and password with my actual information that is known to work)

2. Yes, the webpage where the data is stored does require me to my userid and password and gives me the option to remember password for future use which I have selected.  Subsequent visits to the website do not require me to reenter info.  



-----Original Message-----
From: Sarah Goslee [mailto:sarah.goslee at gmail.com] 
Sent: Monday, October 03, 2011 3:07 PM
To: Mike Pfeiff; r-help
Subject: Re: [R] read .csv from web from password protected site

Hi,

I've assumed that you meant to send this to the R-help list, and not just me.

On Mon, Oct 3, 2011 at 3:54 PM, Mike Pfeiff <MikeP at kfoc.net> wrote:
> Sarah,  Thanks for the suggestion.  Although, read.table("http://userid:password@my.url/file.csv") did not work as it returned the following:
>
>        Error in file(file, "rt") : cannot open the connection
>        In addition: Warning message:
>        In file(file, "rt") : unable to resolve 'userid'
>
> (where 'usesid is my actual userid)
>
> I've tried the following RCurl commands...
>
>        myURL   ="http://www.frontierweather.com/degreedays/L15N15PowerRegionAverages_10weeks.txt"
>        h=getURL(myURL, userpw = "userid:passwod", followlocation=TRUE)
>        test=read.table(h,header=TRUE,sep=",")
>
> ..and I can't get the data to read and get the following errors:
>
>        Error in file(file, "rt") : cannot open the connection
>        In addition: Warning message:
>        In file(file, "rt") : unable to resolve 'userid'
>
> I'm at a total loss.  Any assistance anyone could provide would be greatly appreciated.

You can get to the file using your web browser and that userid/password combo, right?

Do you have to go through a dialog box? Press a button to login? Any of those have the potential to complicate the task.

If so, you'll need to work through the Forms section at http://www.omegahat.org/RCurl/philosophy.html

Sarah

>
>
> -----Original Message-----
> From: Sarah Goslee [mailto:sarah.goslee at gmail.com]
> Sent: Monday, October 03, 2011 2:26 PM
> To: Mike Pfeiff
> Cc: r-help at r-project.org
> Subject: Re: [R] read .csv from web from password protected site
>
> Hi Mike,
>
> On Mon, Oct 3, 2011 at 12:31 PM, Mike Pfeiff <MikeP at kfoc.net> wrote:
>> I am very new to R and have been struggling trying to read a basic ".csv" file from a password protected site with the following code:
>>
>> myURL  ="http://www.frontierweather.com/degreedays/L15N15PowerRegionAverages_10weeks.txt"
>> test2=read.table(url(myURL),header=TRUE,sep=",")
>
>
>> A 'data.frame' is returned into the workspace, however it is not the data contained in the ".csv" file.   I think this occurs because the website where I am trying to retrieve the data is password protected.
>>
>> Is there a way to specify the username and password?
>
>
> I'd try first
> read.table("http://userid:password@my.url/file.csv"), which is the standard way to do it (hint: try that form in your web browser and see whether you can access the data), and if that doesn't work look into the RCurl package. The list archives have a fair bit of information on this topic.
>
> Sarah
>
>
>> Any guidance would be greatly appreciated.
>>
>> Sincerely,
>>
>> Mike
>>
>

--
Sarah Goslee
http://www.functionaldiversity.org