Thank you for your help Duncan,

I have been trying what you suggested however  I am getting an error when
trying to create the function fun<- createFunction(forms[[1]])
it says Error in isHidden I hasDefault :
operations are possible only for numeric, logical or complex types

On Wed, Sep 19, 2012 at 12:15 AM, Duncan Temple Lang <
dtemplelang@ucdavis.edu> wrote:

> Hi ?
>
> The key is that you want to use the same curl handle
> for both the postForm() and for getting the data document.
>
> site = u =
> "
> http://www.wateroffice.ec.gc.ca/graph/graph_e.html?mode=text&stn=05ND012&prm1=3&syr=2012&smo=09&sday=15&eyr=2012&emo=09&eday=18
> "
>
> library(RCurl)
> curl = getCurlHandle(cookiefile = "", verbose = TRUE)
>
> postForm(site, disclaimer_action="I Agree")
>
> Now we have the cookie in the curl handle so we can use that same curl
> handle
> to request the data document:
>
> txt = getURLContent(u, curl = curl)
>
> Now we can use readHTMLTable() on the local document content:
>
> library(XML)
> tt = readHTMLTable(txt, asText = TRUE, which = 1, stringsAsFactors = FALSE)
>
>
>
> Rather than knowing how to post the form, I like to read
> the form programmatically and generate an R function to do the submission
> for me. The RHTMLForms package can do this.
>
> library(RHTMLForms)
> forms = getHTMLFormDescription(u, FALSE)
> fun = createFunction(forms[[1]])
>
> Then we can use
>
>  fun(.curl = curl)
>
> instead of
>
>   postForm(site, disclaimer_action="I Agree")
>
> This helps to abstract the details of the form.
>
>   D.
>
> On 9/18/12 5:57 PM, CPV wrote:
> > Hi, I am starting coding in r and one of the things that i want to do is
> to
> > scrape some data from the web.
> > The problem that I am having is that I cannot get passed the disclaimer
> > page (which produces a session cookie). I have been able to collect some
> > ideas and combine them in the code below but I dont get passed the
> > disclaimer page.
> > I am trying to agree the disclaimer with the postForm and write the
> cookie
> > to a file, but I cannot do it succesfully....
> > The webpage cookies are written to the file but the value is FALSE... So
> > any ideas of what I should do or what I am doing wrong with?
> > Thank you for your help,
> >
> > library(RCurl)
> > library(XML)
> >
> > site <- "
> >
> http://www.wateroffice.ec.gc.ca/graph/graph_e.html?mode=text&stn=05ND012&prm1=3&syr=2012&smo=09&sday=15&eyr=2012&emo=09&eday=18
> "
> >
> > postForm(site, disclaimer_action="I Agree")
> >
> > cf <- "cookies.txt"
> >
> > no_cookie <- function() {
> >         curlHandle <- getCurlHandle(cookiefile=cf, cookiejar=cf)
> >         getURL(site, curl=curlHandle)
> >
> >         rm(curlHandle)
> >         gc()
> > }
> >
> > if ( file.exists(cf) == TRUE ) {
> >         file.create(cf)
> >         no_cookie()
> > }
> > allTables <- readHTMLTable(site)
> > allTables
> >
> >       [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > R-help@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> >
> >
>
> ______________________________________________
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

	[[alternative HTML version deleted]]


