[Bioc-devel] avoiding circular dependencies between a Data package and a Software package

Obenchain, Valerie V@lerie@Obench@in @ending from Ro@wellP@rk@org
Tue Oct 9 21:29:00 CEST 2018 

I'm not sure what you mean by flipping the script ...?

Looks like I was out of date with my advice on the .BBSoptions. A few months ago the BBS was modified to install all packages so the

ForceInstall: True

is no longer needed. You can remove the .BBSoptions file - sorry for the confusion.

Lori will be updating the docs (https://github.com/Bioconductor/Contributions#submitting-related-packages) with more detail on handling packages with mutual (circular) dependencies. I see your package was assigned to Herve - if you are still having problems with the build you can ask @lshep or @hpages for help on the issue.

Thanks for the pull request, it's been merged.

Val


On 10/8/18 7:23 AM, Tim Triche, Jr. wrote:
D'oh.  Does this mean that I should flip the script a second time?

I went ahead and added the .BBSoptions file as suggested and flushed out some additional irritants; it seemed that having the main package Suggest: the Data package went more smoothly (no cycles) so I had updated the ticket to reflect this. I also updated the support site question to reflect that suggestion. But now I am not sure if that's right.

It is a bit confusing. I'd send a PR except I'm not really qualified to patch it. However, there was a reference to Subversion that stuck in my head after submission, and I did track that down; see https://github.com/Bioconductor/Contributions/pull/916 for details ;-)

Thank you,



--t


On Mon, Oct 8, 2018 at 8:44 AM Shepherd, Lori <Lori.Shepherd using roswellpark.org<mailto:Lori.Shepherd using roswellpark.org>> wrote:

Hello,


The Single Package builder (SPB) does check the .BBSOptions but only for UnsupportedPlatforms.  This is because the single package builder automatically installs each package before the build and check stage as part of the process.  Each package that goes through the single package builder for new submits temporarily gets its own directory to use for installing and referencing dependencies -  This is why we document that packages with additional or supporting packages should be built under the same issue as then they will be able to find each other.  The submission process where the data package that "suggests" the software package is submitted first, followed by the software package that "depends" on the data package is correct.


Its a little confusing at first but the following will/should happen on the SPB:


  1.   The data package will be submitted and fail because the software package is not available (but it will be installed in the Rlibs directory for the issue)
  2.  The software package is submitted and fails because the data package can't be found (but also installs)
  3.  The maintainer should do an arbitrary version bump on the data package to kick off a new build that should now pass as it will find the software package
  4.  Do any additional debugging on the data package to ensure the data package has a clean build and check.
  5.  The maintainer should do an arbitrary version bump on the software package, only after the data package is building and checking cleanly, to rebuild the software package
  6.  Review process continues and packages are accepted

The same failure scheme will happen on the main builders once the package is accepted, the only difference is the maintainer will not have to version bump for the new builds since the daily builds builds and checks all packages automatically (daily for software and  Sun/Tue/Thru for data packages)- The ERRORS will clean up on there own after 3-5 days depending on when it is accepted and added to the main builders in reference to the schedule of the building of data packages (Sun/Tue/Thur).  Once the packages are both installed once, the circular dependency issue will be resolved.

Cheers,



Lori Shepherd

Bioconductor Core Team

Roswell Park Cancer Institute

Department of Biostatistics & Bioinformatics

Elm & Carlton Streets

Buffalo, New York 14263

________________________________
From: Bioc-devel <bioc-devel-bounces using r-project.org<mailto:bioc-devel-bounces using r-project.org>> on behalf of Obenchain, Valerie <Valerie.Obenchain using RoswellPark.org><mailto:Valerie.Obenchain using RoswellPark.org>
Sent: Sunday, October 7, 2018 11:54:46 AM
To: Michael Lawrence; Tim Triche, Jr.
Cc: packages using bioconductor.org<mailto:packages using bioconductor.org>; bioc-devel
Subject: Re: [Bioc-devel] avoiding circular dependencies between a Data package and a Software package

I forgot to add the .BBSoptions file should be located at the 'top' level of the package, e.g., MTseekerData/.BBSoptions. You might as well add that now instead of waiting until the packages are approved. I think the SPB was modified to be aware of the .BBSoptions file so it may require it too (Lori would know). If the SPB does require .BBSoptions we should document that in

https://github.com/Bioconductor/Contributions/blob/master/CONTRIBUTING.md

Valerie

On 10/7/18 8:39 AM, Obenchain, Valerie wrote:

Hi Tim,

As Michael said, the usual case is the data package 'Suggests' the software and the software 'Depends' or 'Imports' the data but this dependency could be reversed.

When submitting a combo of packages to the SPB for review, the package that 'Suggests' is submitted first followed by the package that 'Depends'. More on that here:

https://github.com/Bioconductor/Contributions#submitting-related-packages

One additional step taken to transition the packages to the build system after approval is to add a .BBSoptions file in the package that 'Suggests'. In this example, the .BBSoptions would be in the data package and have this one line:

ForceInstall: TRUE

Valerie


On 10/6/18 11:06 AM, Michael Lawrence wrote:

I think you're doing the right thing, suggesting the data package from the
software package. This might require some manual intervention on the build
system side to ensure that the software package is installed before passing
full check, so that the data package can be installed/checked and finally
the software package checked. I guess a note about this could be added to
the page at the cited URL.

Michael

On Sat, Oct 6, 2018 at 10:42 AM Tim Triche, Jr. <tim.triche using gmail.com<mailto:tim.triche using gmail.com>><mailto:tim.triche using gmail.com<mailto:tim.triche using gmail.com>>
wrote:



Last night I submitted MTseeker and its companion package MTseekerData,
both festooned with examples that run smoothly and pass BiocCheck.

HOWEVER! Breaking up the data into a Data package inadvertently seems to
have created a circular dependency between the Software and Data packages.
I haven't had to deal with this in the past, and mostly sidestepped it by
skipping a Depends: entry in Data.

(cf. https://support.bioconductor.org/p/113711 , which I will update with
answers from here to try and avoid having other people with the same
problem bug youse plural)

The issue seems to be that, since the MTseeker-defined MVRanges and
MAlignments classes (which, as you might imagine, subclass the VRanges and
GAlignments classes) hold the data stored in MTseekerData, the examples in
MTseekerData call the `show` method (and others) on the objects and this
create issues.  The most bizarre of these complained at installation about
(sp?) .lazyLoadCacheManager() (I'm having trouble reproducing this exact
error but it was memorable).  Does this ring a bell for anyone? Is there an
elegant workaround?

I can't imagine I'm the first person to have this issue.  I know there are
also various automagic hooks in BioC to load a package upon discovering a
data structure that belongs to it (and not before) -- should I be using
attachNamespace() somehow to side-step this problem?  (Eventually I do want
to submit BAM files to ExperimentHub for an end-to-end example and unit
tests, but the more pressing issue here was BiocCheck).

Any feedback is much appreciated. It's kind of a drag when all the examples
and vignettes pass and build, but only with a side-stepping of
dependencies. That can't possibly be the correct solution, yet I'm at a
loss to find documentation (which I'm sure exists somewhere) that explains
how to avoid this (and/or implement the hooks).

Thank you,

--t

        [[alternative HTML version deleted]]

_______________________________________________
Bioc-devel using r-project.org<mailto:Bioc-devel using r-project.org><mailto:Bioc-devel using r-project.org<mailto:Bioc-devel using r-project.org>> mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel



        [[alternative HTML version deleted]]

_______________________________________________
Bioc-devel using r-project.org<mailto:Bioc-devel using r-project.org><mailto:Bioc-devel using r-project.org<mailto:Bioc-devel using r-project.org>> mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel






This email message may contain legally privileged and/or confidential information.  If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited.  If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you.
        [[alternative HTML version deleted]]

_______________________________________________
Bioc-devel using r-project.org<mailto:Bioc-devel using r-project.org> mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

This email message may contain legally privileged and/or confidential information. If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited. If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you.



This email message may contain legally privileged and/or confidential information.  If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited.  If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you.
	[[alternative HTML version deleted]]

More information about the Bioc-devel mailing list