[Bioc-devel] Time-out error on Ubuntu 20.04 build

Selles Vidal, Lara |@r@@@e||e@12 @end|ng |rom |mper|@|@@c@uk
Wed May 5 15:38:05 CEST 2021


Hi Hervé,

Thanks a lot for looking with detail into this. I completely agree that a fix for this would be very beneficial.

I pushed yesterday an update to the devel version of ncRNAtools, which hopefully should fix the issue.

The change seems not to be propagated yet, but hopefully it will be reflected in the build report soon!

Best wishes,

Lara


Lara Selles
lara.selles12 using imperial.ac.uk







________________________________
De: Hervé Pagès <hpages.on.github using gmail.com>
Enviado: martes, 4 de mayo de 2021 4:24
Para: Selles Vidal, Lara <lara.selles12 using imperial.ac.uk>; Mike Smith <grimbough using gmail.com>
Cc: bioc-devel using r-project.org <bioc-devel using r-project.org>
Asunto: Re: [Bioc-devel] Time-out error on Ubuntu 20.04 build

Hi Lara,

I took a closer look at this and yes there seems to be something about
nebbiolo1 that causes problems: it's behind a firewall. And for some
reason this firewall causes the internal calls to httr::GET() to return
responses with the following header:

   > GET(redirectURL)$headers
   $date
   [1] "Tue, 04 May 2021 01:59:02 GMT"

   $server
   [1] "Apache    adding: CentroidFold/ (stored 0%),
CentroidFold/base-pairing-prob.txt (deflated 58%),
CentroidFold/structure.eps (deflated 58%),
CentroidFold/base-pairing-prob.png (deflated 18%),
CentroidFold/base-pairing-prob.pdf (deflated 10%),
CentroidFold/structure.pdf (deflated 19%), CentroidFold/structure.txt
(deflated 25%), CentroidFold/base-pairing-prob.eps (deflated 60%),
CentroidFold/structure.png (deflated 13%)"

   $`transfer-encoding`
   [1] "chunked"

   $`content-type`
   [1] "text/html; charset=UTF-8"

   $via
   [1] "1.1 phswsa8.partners.org:80 (Cisco-WSA/12.5.1-035)"

   $connection
   [1] "keep-alive"

   attr(,"class")
   [1] "insensitive" "list"

Two things about this header:

   1) The "via" field indicates the presence of the firewall.

   2) There's no "  adding" field. My understanding is that
ncRNAtools:::checkSecondaryStructureQuery() expects to see that field as
an indication that the initial query is no longer running. Note that the
initial query here is no longer running and the final results are
present in the header. However, instead of showing up in the "  adding"
field, they've been merged to the "server" field!

For comparison here is what a clean header looks like on my laptop or on
other build machines:

   > GET(redirectURL)$headers
   $date
   [1] "Tue, 04 May 2021 02:13:18 GMT"

   $server
   [1] "Apache"

   $`  adding`
   [1] "CentroidFold/ (stored 0%), CentroidFold/base-pairing-prob.txt
(deflated 58%), CentroidFold/structure.eps (deflated 58%),
CentroidFold/base-pairing-prob.png (deflated 18%),
CentroidFold/base-pairing-prob.pdf (deflated 10%),
CentroidFold/structure.pdf (deflated 19%), CentroidFold/structure.txt
(deflated 25%), CentroidFold/base-pairing-prob.eps (deflated 60%),
CentroidFold/structure.png (deflated 13%)"

   $connection
   [1] "close"

   $`transfer-encoding`
   [1] "chunked"

   $`content-type`
   [1] "text/html; charset=UTF-8"

   attr(,"class")
   [1] "insensitive" "list"

Unfortunately I don't have any control on how the firewall at DFCI is
configured so I'm asking if you would be willing to modify ncRNAtools to
accommodate this situation. Note that the benefit of doing this is that
it will also solve the issue for people in other institutions that have
a firewall configured in a way similar to the DFCI's firewall.

Hope this makes sense.


Best regards,
H.



On 03/05/2021 14:37, Selles Vidal, Lara wrote:
> Dear All,
>
> Since it seems the main Linux target in 3.13 is now Ubuntu 20.04, I have been recently trying to get back to fixing the TIMEOUT of my package ncRNAtools.
>
> Mike previously reported as seen below that he does not get the TIMEOUT when running R CMD build / check on an Ubuntu 20.04. I have also tried to reproduce the error on an Ubuntu 20.04, and could not manage to do so.
>
> I am unsure what to do at this stage in order to trouble shoot the problem. Does anybody have any suggestions about to go around this?
>
> Thanks a lot in advance,
>
> Best wishes,
>
> Lara
>
>
>
> Lara Selles
> lara.selles12 using imperial.ac.uk
>
>
>
>
>
>
>
> ________________________________
> De: Mike Smith <grimbough using gmail.com>
> Enviado: jueves, 15 de octubre de 2020 1:04
> Para: Pages, Herve <hpages using fredhutch.org>
> Cc: Selles Vidal, Lara <lara.selles12 using imperial.ac.uk>; bioc-devel using r-project.org <bioc-devel using r-project.org>
> Asunto: Re: [Bioc-devel] Time-out error on Ubuntu 20.04 build
>
> I think this information will probably just confuse the issue, but I've successfully checked out ncRNAtools and run R CMD build / check without issue on my Ubuntu 20.04 test machine.  It took a couple of minutes.  Rolling back to an older version prior to the SSL patch would lead to failure and the original error.
>
> I'm afraid that doesn't offer much guidance, but perhaps this is something more specific to the setup on nebbiolo1 ?
>
> Mike
>
> On Wed, 14 Oct 2020 at 07:52, Pages, Herve <hpages using fredhutch.org<mailto:hpages using fredhutch.org>> wrote:
> On 10/13/20 15:18, Selles Vidal, Lara wrote:
>> Hi Herv�,
>>
>> Thanks a lot for your reply and clarification. It is reassuring to know
>> that Ubuntu 18.04 is still the primary Linux target!
>>
>> I am not running Ubuntu 20.04 on a Docker image, but I am running it on
>> a virtual machine (VirtualBox) on RedHat 7. Do you think this could be a
>> similar case as for the Docker image?
>
> Probably.
>
>> Interestingly, my other package on Bioconductor rfaRm had the same issue
>> related to SSL on Ubuntu 20.04.
>
> Yep, I know ;-)
>
>> I was able to solve it by using the same
>> workaround, and am now getting OK on R CMD install, build and check on
>> nebbiolo1 (http://bioconductor.org/checkResults/devel/bioc-LATEST/rfaRm/
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__bioconductor.org_checkResults_devel_bioc-2DLATEST_rfaRm_&d=DwMGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=c1ETpOKFrZFp9E7pXw9m7rKkDFSAu3LXirxKvpof68A&s=7ruZcWM501J5RQVuoEFDPhJ7ZbP_7HLfezgTXJUPpKg&e=>
>> ).
>
> Glad it worked. Even though Ubuntu 20.04 is not our primary target, many
> users are probably going to use your package on Ubuntu 20.04 so it's
> worth trying to address this issue.
>
>>
>> This seems to indicate that the same workaround has different effects
>> depending on the URL being accessed possibly?
>
> Apparently. I didn't try Mike Smith's suggested workaround myself so I
> don't know for sure. Based on what you are reporting, we know that it
> works for some of the websites affected by the SSL certificate issue but
> not for all of them. Always good to know even though I have no idea why
> that would be.
>
>>
>> Interestingly, the workaround has at least led to a TIMEOUT in the case
>> of ncRNAtools, and not an ERROR, as was previously the case.  The
>> TIMEOUT seems to happen when creating the vignettes. Is it safe to
>> assume then than knitting the vignette under the same environment should
>> also get stuck at some endless/very long step? I will try to see if I
>> can get access to a native Ubuntu 20.04 environment, to try knitting the
>> vignette with different code chunks removed. Maybe we can figure out
>> some additional extra setting to pass to config() for the specific URL
>> that is leading to the TIMEOUT, that will provide a temporary workaround.
>
> Thanks for investigating. FWIW, here is what I get on nebbiolo1 (our
> Ubuntu 20.04 builder):
>
>     $ cd ncRNAtools/vignettes
>
>     $ R CMD Stangle ncRNAtools.Rmd
>     Output file:  ncRNAtools.R
>
>     $ R
>
>     R version 4.0.3 (2020-10-10) -- "Bunny-Wunnies Freak Out"
>     Copyright (C) 2020 The R Foundation for Statistical Computing
>     Platform: x86_64-pc-linux-gnu (64-bit)
>
>     R is free software and comes with ABSOLUTELY NO WARRANTY.
>     You are welcome to redistribute it under certain conditions.
>     Type 'license()' or 'licence()' for distribution details.
>
>       Natural language support but running in an English locale
>
>     R is a collaborative project with many contributors.
>     Type 'contributors()' for more information and
>     'citation()' on how to cite R or R packages in publications.
>
>     Type 'demo()' for some demos, 'help()' for on-line help, or
>     'help.start()' for an HTML browser interface to help.
>     Type 'q()' to quit R.
>
>     > source("ncRNAtools.R", echo=TRUE)
>
>     > ## ----setup,
> echo=FALSE--------------------------------------------------------
>     > knitr::opts_chunk$set(message=FALSE, fig.path='figures/')
>
>     > ## ----tidy = TRUE, eval =
> FALSE------------------------------------------------
>     > ## if (!requireNamespace("BiocManager", quietly = TRUE))
>     > ##     .... [TRUNCATED]
>
>     > library(GenomicRanges)
>
>     ...
>
>     > ## ----tidy =
> TRUE--------------------------------------------------------------
>     > tRNAfragment <- "UGCGAGAGGCACAGGGUUCGAUUCCCUGCAUCUCCA"
>
>     > centroidFoldPrediction <- predictSecondaryStructure(tRNAfragment,
> "centroidFold")
>     Secondary structure prediction is running, please wait.
>     Secondary structure prediction is running, please wait.
>     Secondary structure prediction is running, please wait.
>     Secondary structure prediction is running, please wait.
>     Secondary structure prediction is running, please wait.
>     Secondary structure prediction is running, please wait.
>     ...
>     etc.. (seems like the while loop in predictSecondaryStructure
>     never ends)
>
> Doesn't look like this has anything to do with the code being executed
> inside a knitting context.
>
> Hope this helps.
>
> Cheers,
> H.
>
>>
>> Best wishes,
>>
>> Lara
>>
>>
>>
>>> On 13 Oct 2020, at 22:49, Pages, Herve <hpages using fredhutch.org<mailto:hpages using fredhutch.org>
>>> <mailto:hpages using fredhutch.org<mailto:hpages using fredhutch.org>>> wrote:
>>>
>>>
>>> *******************
>>> This email originates from outside Imperial. Do not click on links and
>>> attachments unless you recognise the sender.
>>> If you trust the sender, add them to your safe senders list
>>> https://spam.ic.ac.uk/SpamConsole/Senders.aspx
>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__spam.ic.ac.uk_SpamConsole_Senders.aspx&d=DwMGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=c1ETpOKFrZFp9E7pXw9m7rKkDFSAu3LXirxKvpof68A&s=PPHDsEkC391NRNRUVixnJHVV-Ch8xA3-ytdD1l_h_88&e=>
>>> to disable email stamping for this address.
>>> *******************
>>> Hi Lara,
>>>
>>> Thanks for trying hard to address this problem. I can reproduce the
>>> error by running 'R CMD build ncRNAtools' interactively on nebbiolo1.
>>> Yes it seems that the timeout is related to the SSL issue on Ubuntu
>>> 20.04. Are you running Ubuntu 20.04 on a Docker image? I've read
>>> somewhere recently that some HTTPS/HTTP access issues on Ubuntu 20.04
>>> are not always reproducible on a Docker image.
>>>
>>> I see in the 'git log' that yesterday you've reverted back your recent
>>> changes and that you are now using HTTPS again together with the
>>> following workaround:
>>> https://github.com/Ensembl/ensembl-rest/issues/427#issuecomment-702044967
>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Ensembl_ensembl-2Drest_issues_427-23issuecomment-2D702044967&d=DwMGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=c1ETpOKFrZFp9E7pXw9m7rKkDFSAu3LXirxKvpof68A&s=ehZ1nEOqZCQwVZ_VvWH19bk-1HQP-HFThOmxw0bLCRU&e=>
>>>
>>> Unfortunately it didn't seem to work (we still see the TIMEOUT on
>>> today's report). Thanks for trying this anyway.
>>>
>>> At this point I don't have a good suggestion, unfortunately. Note that
>>> malbec1 (Ubuntu 18.04) is still our primary Linux builder and
>>> propagation is still decided on whether or not a package passes BUILD
>>> and CHECK there. So the TIMEOUT on nebbiolo1 won't have any practical
>>> consequences as far as the next release is concerned.
>>>
>>> At some point, we really want to make Ubuntu 20.04 our primary Linux
>>> target, and get rid of the Ubuntu 18.04 builder. But we won't do this
>>> before we have a good solution for the current SSL issue on Ubuntu 20.04.
>>>
>>> Best,
>>> H.
>>>
>>>
>>> On 10/13/20 09:07, Selles Vidal, Lara wrote:
>>>> Dear all,
>>>>
>>>> I have been seeing the same problem on all builds of ncRNAtools for a
>>>> week now. I have run the exact same R CMD build command (R CMD build
>>>> --keep-empty-dirs --no-resave-data) on an Ubuntu 20.04 environment,
>>>> using the environment settings described here
>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__bioconductor.org_checkResults_devel_bioc-2DLATEST_Renviron.bioc&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=nE6T7FcXLo61NvG8iDhxBeraWyR9mF-daYXZpIwXCkg&s=H9gZj9GQAFFoNmEPlMZCNJSvvxeErCwbVbXJw1bxOxQ&e=
>>>>   .
>>>>
>>>> After running R CMD build multiple times under such conditions, I
>>>> have not been able to reproduce the error or even come close to a
>>>> Timeout. I also have recently tried to make a new version bump to see
>>>> if that would trigger a new behaviour, but I am still getting the
>>>> same TIMEOUT.
>>>>
>>>> Since the deadline for submissions to devel branch is coming soon,
>>>> does anybody have any idea what could be possibly going on, or how I
>>>> could possibly get more information about what�s causing the error?
>>>>
>>>> Thanks a lot in advance!
>>>>
>>>> Best wishes,
>>>>
>>>> Lara
>>>>
>>>> On 6 Oct 2020, at 19:11, Selles Vidal, Lara
>>>> <lara.selles12 using imperial.ac.uk<mailto:lara.selles12 using imperial.ac.uk><mailto:lara.selles12 using imperial.ac.uk<mailto:lara.selles12 using imperial.ac.uk>%3E%3E
>>>> wrote:
>>>>
>>>> Dear all,
>>>>
>>>> I have recently pushed some changes to my ncRNAtools package to fix
>>>> an error caused by a bug in the most recent version of OpenSSL which
>>>> made requests to https addresses fail with default security settings.
>>>> The changes have fixed all errors in my local Ubuntu 20.04
>>>> environment, and the package is now installing correctly in
>>>> nebbiolo1. However, I am getting TIMEOUT at the build stage on
>>>> nebbiolo1. It seems to be stuck at creating the vignette
>>>> (https://urldefense.proofpoint.com/v2/url?u=http-3A__bioconductor.org_checkResults_devel_bioc-2DLATEST_ncRNAtools_&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=nE6T7FcXLo61NvG8iDhxBeraWyR9mF-daYXZpIwXCkg&s=0ntfLi4_hO8Npg0WeCl_20F9KefutInFzvShPjGv6Hs&e=
>>>> ).
>>>>
>>>> I have run successfully multiple times R CMD install, build and check
>>>> in my local Ubuntu 20.04, and cannot reproduce the error. Does anyone
>>>> have any idea what I could possibly do to troubleshoot the error? I
>>>> have seen the TIMEOUT occur in at least two consecutive build
>>>> reports, so I do not think it is something spurious.
>>>>
>>>> Thanks a lot in advance!
>>>>
>>>> Best wishes,
>>>>
>>>> Lara
>>>>
>>>>
>>>> [[alternative HTML version deleted]]
>>>>
>>>> _______________________________________________
>>>> Bioc-devel using r-project.org<mailto:Bioc-devel using r-project.org> mailing list
>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=nE6T7FcXLo61NvG8iDhxBeraWyR9mF-daYXZpIwXCkg&s=_dUDRCIGHEpFPIHLxeFUUn8xWAZE6v4Ku4BIT8VL9kw&e=
>>>>
>>>
>>> --
>>> Herv� Pag�s
>>>
>>> Program in Computational Biology
>>> Division of Public Health Sciences
>>> Fred Hutchinson Cancer Research Center
>>> 1100 Fairview Ave. N, M1-B514
>>> P.O. Box 19024
>>> Seattle, WA 98109-1024
>>>
>>> E-mail: hpages using fredhutch.org<mailto:hpages using fredhutch.org>
>>> Phone:  (206) 667-5791
>>> Fax:    (206) 667-1319
>>
>
> --
> Herv� Pag�s
>
> Program in Computational Biology
> Division of Public Health Sciences
> Fred Hutchinson Cancer Research Center
> 1100 Fairview Ave. N, M1-B514
> P.O. Box 19024
> Seattle, WA 98109-1024
>
> E-mail: hpages using fredhutch.org<mailto:hpages using fredhutch.org>
> Phone:  (206) 667-5791
> Fax:    (206) 667-1319
> _______________________________________________
> Bioc-devel using r-project.org<mailto:Bioc-devel using r-project.org> mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>
>        [[alternative HTML version deleted]]
>
>
> _______________________________________________
> Bioc-devel using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>

--
Hervé Pagès

Bioconductor Core Team
hpages.on.github using gmail.com

	[[alternative HTML version deleted]]



More information about the Bioc-devel mailing list