[Bioc-devel] EXTERNAL: "invalid connection" when building vignette - Windows only

Mon Mar 18 09:54:08 CET 2024

Hi Ramos,

Thank you very much – this is really informative. Can I ask you how you got the tracback? I’m not really used at debugging this sort of issues.

I can see that the EpiMix::EpiMix function we use does use foreach (foreach::makeCluster and %dopar%). I have no experience with foreach, but I had a dig, and I can see that this function cleans up the parallel environment with stopCluster only before just one of the several returns it has. So it is possible that the environment isn’t cleaned up between consequent function calls. We will investigate.

Of course if anyone with more foreach experience wants to pinch in, they are very welcome

Thank you again

Matteo Tiberti

Danish Cancer Institute
Strandboulevarden 49
DK-2100 Copenhagen
Telephone: +45 35 25 73 07
– a part of the Danish Cancer Society

[An image showing the Danish Cancer Society's name and logo called the bow in red font. Displayed in red]<https://www.cancer.dk/?utm_source=email&utm_medium=medarbejderemail&utm_campaign=medarbejderemail&utm_content=cancerdk>

www.cancer.dk<https://www.cancer.dk/international/> | Our privacy policy<https://www.cancer.dk/om-os/privatlivspolitik/>

From: Ramos Perez, Marcel <Marcel.RamosPerez using RoswellPark.org>
Date: Friday, 15 March 2024 at 22.44
To: bioc-devel using r-project.org <bioc-devel using r-project.org>
Cc: Matteo Tiberti <tiberti using cancer.dk>
Subject: Re: EXTERNAL: [Bioc-devel] "invalid connection" when building vignette - Windows only

Hi Matteo,

I was also able to reproduce on Bioc 3.18 on Windows with a traceback (see below).

It may be that there are symbol clashes between parallel runs in the
`foreach:::.foreachGlobals` environment as described in
https://stackoverflow.com/a/67497500<https://stackoverflow.com/a/67497500>

Clearing the symbols in that environment alleviates the problem temporarily
until running the chunk in line 1124 where the same error occurs.

Someone with more experience using `foreach` and socket clusters may be able to help.

Note that the code should be running on either one or two cores by default.

Best regards,

Marcel

```

processing file: Moonlight2R.Rmd
  |.......................                            |  45% [unnamed-chunk-20]
Quitting from lines 727-746 [unnamed-chunk-20] (Moonlight2R.Rmd)

Error in `summary.connection()`:
! invalid connection
Backtrace:
  1. Moonlight2R::GMA(...)
  2. EpiMix::EpiMix(...)
       at Moonlight2R/R/GMA.R:117:3
  3. EpiMix:::MethylMix_MixtureModel(...)
  4. ... %dopar% ...
     ...
  7. parallel:::sendCall(cl[[i]], fun, list(...))
  8. parallel:::postNode(...)
 10. parallel:::sendData.SOCKnode(...)
 11. base::serialize(data, node$con)
 13. base::summary.connection(connection)
Execution halted

```
On 3/15/24 6:05 AM, Matteo Tiberti via Bioc-devel wrote:
Dear developers,

we've been getting a build fail for our Moonlight2R package on Windows only, during vignette build in CHECK, which just prints out an "invalid connection" error:

Quitting from lines 727-746 [unnamed-chunk-20] (Moonlight2R.Rmd)
Error: processing vignette 'Moonlight2R.Rmd' failed with diagnostics:
invalid connection
--- failed re-building 'Moonlight2R.Rmd'

The following snippet is the code chunk that causes the fail:

data("dataMethyl")
data("dataFilt")
data("dataPRA")
data("DEGsmatrix")
data("LUAD_sample_anno")
data("NCG")
data("EncodePromoters")
data("MetEvidenceDriver")

# Subset column names (sample names) in expression data to patient level
pattern <- "^(.{4}-.{2}-.{4}-.{2}).*"
colnames(dataFilt) <- sub(pattern, "\\1", colnames(dataFilt))

dataGMA <- GMA(dataMET = dataMethyl, dataEXP = dataFilt,
              dataPRA = dataPRA, dataDEGs = DEGsmatrix,
              sample_info = LUAD_sample_anno, met_platform = "HM450",
              prevalence_filter = NULL,
              output_dir = "./GMAresults", cores = 1, roadmap.epigenome.ids = "E096",
              roadmap.epigenome.groups = NULL)

I could reproduce this fail on my local Windows machine (on release rather than devel at the moment), so it's probably not an issue with the builders. It builds and runs successfully on Linux, both on devel and release. I have also tried running the same code by hand on the R prompt, and that completes successfully.

The error I get when trying to build the vignette manually is possibly more informative:

devtools::build_vignettes('../Downloads/Moonlight2R-devel/Moonlight2R-devel/')

ℹ Installing Moonlight2R in temporary library
ℹ Building vignettes for Moonlight2R
--- re-building 'Moonlight2R.Rmd' using rmarkdown

processing file: Moonlight2R.Rmd
 |.......................                            |  45% [unnamed-chunk-20]trying URL 'https://bioconductor.org/packages/3.18/data/experiment/src/contrib/sesameData_1.20.0.tar.gz<https://bioconductor.org/packages/3.18/data/experiment/src/contrib/sesameData_1.20.0.tar.gz>'

trying URL 'https://bioconductor.org/packages/3.18/data/experiment/src/contrib/sesameData_1.20.0.tar.gz<https://bioconductor.org/packages/3.18/data/experiment/src/contrib/sesameData_1.20.0.tar.gz>'
Content type 'application/x-gzip' length 1975798 bytes (1.9 MB)
==================================================
downloaded 1.9 MB

* installing *source* package 'sesameData' ...
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading

** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location

** testing if installed package can be loaded from final location

** testing if installed package keeps a record of temporary installation path
* DONE (sesameData)

The downloaded source packages are in
       'C:\Users\Work\AppData\Local\Temp\RtmpmsVLSL\downloaded_packages'

Quitting from lines 727-746 [unnamed-chunk-20] (Moonlight2R.Rmd)

Error: processing vignette 'Moonlight2R.Rmd' failed with diagnostics:
invalid connection
--- failed re-building 'Moonlight2R.Rmd'

SUMMARY: processing the following file failed:
 'Moonlight2R.Rmd'

so it installs the sesameData package (because of ExperimentHub::ExperimentHub()[['EH3675']]) , and then fails like that.

We have very little experience in development under Windows and are a bit stumped on what might be the problem; do you know whether this error message comes from anything in particular or do you have any further idea on how to troubleshoot this?

I assume the error message does not refer to connecting to some server on the internet, since all works fine on Linux and I can reproduce the problem from different machines/connections. But happy to hear your thoughts

Thank you and kind regards,

Matteo Tiberti

Danish Cancer Institute
Strandboulevarden 49
DK-2100 Copenhagen
Telephone: +45 35 25 73 07
– a part of the Danish Cancer Society

[An image showing the Danish Cancer Society's name and logo called the bow in red font. Displayed in red]<https://www.cancer.dk/?utm_source=email&utm_medium=medarbejderemail&utm_campaign=medarbejderemail&utm_content=cancerdk><https://www.cancer.dk/?utm_source=email&utm_medium=medarbejderemail&utm_campaign=medarbejderemail&utm_content=cancerdk>

www.cancer.dk<http://www.cancer.dk><https://www.cancer.dk/international/><https://www.cancer.dk/international/> | <https://www.cancer.dk/om-os/privatlivspolitik/><https://www.cancer.dk/om-os/privatlivspolitik/> Our privacy policy<https://www.cancer.dk/om-os/privatlivspolitik/><https://www.cancer.dk/om-os/privatlivspolitik/>

_______________________________________________
Bioc-devel using r-project.org<mailto:Bioc-devel using r-project.org> mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel<https://stat.ethz.ch/mailman/listinfo/bioc-devel>

---

Marcel Ramos

Bioconductor Core Team

Roswell Park Comprehensive Cancer Center

Dept. of Biostatistics & Bioinformatics

Elm St. & Carlton St.

Buffalo, New York 14263

This email message may contain legally privileged and/or confidential information. If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited. If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you.