[R] print and lapply....

[Rui Barradas]

Às 14:47 de 08/11/2022, akshay kulkarni escreveu:
> Dear Rui,
>                    The replies from you, Bert, Tim and solved my problem. My last question: what if I put print inside the body of the function passed on to lapply, instead of separately in the function argument of apply? Is this what you insinuated in your reply?
> 
> THanking you,
> yours sincerely,
> AKSHAY M KULKARNI
> ________________________________
> From: Rui Barradas <ruipbarradas using sapo.pt>
> Sent: Tuesday, November 8, 2022 2:20 AM
> To: akshay kulkarni <akshay_e4 using hotmail.com>; R help Mailing list <r-help using r-project.org>
> Subject: Re: [R] print and lapply....
> 
> Às 19:22 de 07/11/2022, akshay kulkarni escreveu:
>> Dear Rui,
>>                    THanks for your reply...The point is the loop is a scraping code, and in your examples you have assumed that the body acts on i, the loop variable. Can you adapt your code to JUST PRINT the loop variable i ?
>>
>> By the by, I think I have stumbled upon the answer: The lapply() caches the result, and prints the output of the function in question  immediately after printing the final i. The i's get printed serially, as the function progresses....
>>
>>> lapply(1:4,function(x){print(x);Sys.sleep(x^2);x^2})
>> [1] 1
>> [1] 2
>> [1] 3
>> [1] 4
>> [[1]]
>> [1] 1
>>
>> [[2]]
>> [1] 4
>>
>> [[3]]
>> [1] 9
>>
>> [[4]]
>> [1] 16
>>
>> Here x^2 's print only after 4 is printed on the console....
>>
>> tHanks anyways for your reply....
>>
>> THanking you,
>> Yours sincerely,
>> AKSHAY M KULKARNI
>> ________________________________
>> From: Rui Barradas <ruipbarradas using sapo.pt>
>> Sent: Tuesday, November 8, 2022 12:24 AM
>> To: akshay kulkarni <akshay_e4 using hotmail.com>; R help Mailing list <r-help using r-project.org>
>> Subject: Re: [R] print and lapply....
>>
>> Às 18:33 de 07/11/2022, akshay kulkarni escreveu:
>>> Dear Rui,
>>>                       Actually, I am replacing a big for loop by the lapply() function, and report the progress:
>>>
>>> lapply(TP, function(i) { BODY; print(i)})
>>>
>>> Can you please adjust your solution in this light?
>>>
>>> THanking you,
>>> Yours sincerely,
>>> AKSHAY M KULKARNI
>>> ________________________________
>>> From: Rui Barradas <ruipbarradas using sapo.pt>
>>> Sent: Monday, November 7, 2022 11:59 PM
>>> To: akshay kulkarni <akshay_e4 using hotmail.com>; R help Mailing list <r-help using r-project.org>
>>> Subject: Re: [R] print and lapply....
>>>
>>> Às 17:17 de 07/11/2022, akshay kulkarni escreveu:
>>>> Dear members,
>>>>                                  I have the following code and output:
>>>>
>>>>> TP <- 1:4
>>>>> lapply(TP,function(x){print(x);x^2})
>>>> [1] 1
>>>> [1] 2
>>>> [1] 3
>>>> [1] 4
>>>> [[1]]
>>>> [1] 1
>>>>
>>>> [[2]]
>>>> [1] 4
>>>>
>>>> [[3]]
>>>> [1] 9
>>>>
>>>> [[4]]
>>>> [1] 16
>>>>
>>>> How do I make the print function output x along with x^2, i.e not at the beginning but before each of x^2?
>>>>
>>>> Many thanks in advance....
>>>>
>>>> THanking you,
>>>> Yours sincerely
>>>> AKSHAY M KULKARNI
>>>>
>>>>           [[alternative HTML version deleted]]
>>>>
>>>> ______________________________________________
>>>> R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
>>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>>>> and provide commented, minimal, self-contained, reproducible code.
>>> Hello,
>>>
>>> Here are two options, with ?cat and with ?message.
>>>
>>>
>>> TP <- 1:4
>>> lapply(TP, function(x){
>>>       cat("x =", x, "x^2 =", x^2, "\n")
>>> })
>>>
>>> lapply(TP, function(x){
>>>       msg <- paste("x =", x, "x^2 =", x^2)
>>>       message(msg)
>>> })
>>>
>>>
>>> Hope this helps,
>>>
>>> Rui Barradas
>>>
>>>
>>>
>> Hello,
>>
>>
>> What do you want the lapply loop to return? If you have a BODY doing
>> computations, do you want the lapply to return those values and report
>> the progress?
>>
>> I have chosen cat or message over print because
>>
>>     - cat returns invisible(NULL),
>>     - message returns invisible()
>>     - print returns a value, what it prints.
>>
>> Can you adapt the code below to your use case?
>>
>>
>>
>> TP <- 1:4
>> lapply(TP, function(x, verbose = TRUE){
>>      # BODY
>>      y <- rnorm(100, mean = x)
>>
>>      # show progress
>>      if(verbose)
>>        cat("x =", x, "x^2 =", x^2, "\n")
>>
>>      #return value
>>      c(x = x, mean = mean(y))
>> })
>>
>> lapply(TP, function(x, verbose = TRUE){
>>      # BODY
>>      y <- rnorm(100, mean = x)
>>
>>      # show progress
>>      if(verbose) {
>>        msg <- paste("x =", x, "x^2 =", x^2)
>>        message(msg)
>>      }
>>
>>      #return value
>>      c(x = x, mean = mean(y))
>> })
>>
>>
>>
>> Hope this helps,
>>
>> Rui Barradas
>>
>>
> Hello,
> 
> No, the x^2 are not printed after the i's. The x^2 are the function's
> return values. The function prints the i's, then returns x^2.
> 
> As for your problem, it is now more clerar.
> I would write a function accepting a url to take care of scraping and
> call it in the lapply loop. The progress report can be in the loop, like
> below.
> 
> This is a complete working example, scraping the Wikipedia list of
> countries by GDP. The urls are in a list (it's always the same, I'm not
> complicating things) and in a real scraping function I would wrap
> tryCatch around it, just in case.
> 
> First the function, then the urls list, then the lapply loop.
> 
> 
> 
> library(rvest)
> 
> scrape <- function(url) {
>     page <- read_html(url)
>     gdp <- page |>
>       html_element(".wikitable") |>
>       html_table() |>
>       as.data.frame()
>     names(gdp) <- unlist(gdp[1,, drop = TRUE])
>     gdp <- gdp[-1,]
>     row.names(gdp) <- NULL
> 
>     #return value
>     gdp
> }
> 
> wiki_gdp_url <-
> "https://en.wikipedia.org/wiki/List_of_countries_by_GDP_(nominal)"
> urls_list <- list(wiki_gdp_url, wiki_gdp_url)
> TP <- seq_along(urls_list)
> 
> TP
> # [1] 1 2
> 
> df_list <- lapply(TP, \(i) {
>     URL <- urls_list[[i]]
>     data <- scrape(URL)
>     # show progress
>     message("iteration: ", i)
>     #return value
>     data
> })
> 
> str(df_list)
> 
> 
> Hope this helps,
> 
> Rui Barradas
> 
> 
Hello,

Yes, you can put it in the function. I have separated the function from 
printing the progress because I thought it might make things more clear, 
that's all.

Hope this helps,

Rui Barradas