[Bioc-devel] avoiding circular dependencies between a Data package and a Software package

[Tim Triche, Jr.]

D'oh.  Does this mean that I should flip the script a second time?

I went ahead and added the .BBSoptions file as suggested and flushed out
some additional irritants; it seemed that having the main package Suggest:
the Data package went more smoothly (no cycles) so I had updated the ticket
to reflect this. I also updated the support site question to reflect that
suggestion. But now I am not sure if that's right.

It is a bit confusing. I'd send a PR except I'm not really qualified to
patch it. However, there was a reference to Subversion that stuck in my
head after submission, and I did track that down; see
https://github.com/Bioconductor/Contributions/pull/916 for details ;-)

Thank you,



--t


On Mon, Oct 8, 2018 at 8:44 AM Shepherd, Lori <Lori.Shepherd using roswellpark.org>
wrote:

> Hello,
>
>
> The Single Package builder (SPB) does check the .BBSOptions but only for
> UnsupportedPlatforms.  This is because the single package builder
> automatically installs each package before the build and check stage as
> part of the process.  Each package that goes through the single package
> builder for new submits temporarily gets its own directory to use for
> installing and referencing dependencies -  This is why we document that
> packages with additional or supporting packages should be built under the
> same issue as then they will be able to find each other.  The submission
> process where the data package that "suggests" the software package is
> submitted first, followed by the software package that "depends" on the
> data package is correct.
>
>
> Its a little confusing at first but the following will/should happen on
> the SPB:
>
>
>
>    1.  The data package will be submitted and fail because the software
>    package is not available (but it will be installed in the Rlibs directory
>    for the issue)
>    2. The software package is submitted and fails because the data
>    package can't be found (but also installs)
>    3. The maintainer should do an arbitrary version bump on the data
>    package to kick off a new build that should now pass as it will find the
>    software package
>    4. Do any additional debugging on the data package to ensure the data
>    package has a clean build and check.
>    5. The maintainer should do an arbitrary version bump on the software
>    package, only after the data package is building and checking cleanly, to
>    rebuild the software package
>    6. Review process continues and packages are accepted
>
>
> The same failure scheme will happen on the main builders once the package
> is accepted, the only difference is the maintainer will not have to version
> bump for the new builds since the daily builds builds and checks all
> packages automatically (daily for software and  Sun/Tue/Thru for data
> packages)- The ERRORS will clean up on there own after 3-5 days depending
> on when it is accepted and added to the main builders in reference to the
> schedule of the building of data packages (Sun/Tue/Thur).  Once the
> packages are both installed once, the circular dependency issue will be
> resolved.
>
> Cheers,
>
>
>
> Lori Shepherd
>
> Bioconductor Core Team
>
> Roswell Park Cancer Institute
>
> Department of Biostatistics & Bioinformatics
>
> Elm & Carlton Streets
>
> Buffalo, New York 14263
> ------------------------------
> *From:* Bioc-devel <bioc-devel-bounces using r-project.org> on behalf of
> Obenchain, Valerie <Valerie.Obenchain using RoswellPark.org>
> *Sent:* Sunday, October 7, 2018 11:54:46 AM
> *To:* Michael Lawrence; Tim Triche, Jr.
> *Cc:* packages using bioconductor.org; bioc-devel
> *Subject:* Re: [Bioc-devel] avoiding circular dependencies between a Data
> package and a Software package
>
> I forgot to add the .BBSoptions file should be located at the 'top' level
> of the package, e.g., MTseekerData/.BBSoptions. You might as well add that
> now instead of waiting until the packages are approved. I think the SPB was
> modified to be aware of the .BBSoptions file so it may require it too (Lori
> would know). If the SPB does require .BBSoptions we should document that in
>
> https://github.com/Bioconductor/Contributions/blob/master/CONTRIBUTING.md
>
> Valerie
>
> On 10/7/18 8:39 AM, Obenchain, Valerie wrote:
>
> Hi Tim,
>
> As Michael said, the usual case is the data package 'Suggests' the
> software and the software 'Depends' or 'Imports' the data but this
> dependency could be reversed.
>
> When submitting a combo of packages to the SPB for review, the package
> that 'Suggests' is submitted first followed by the package that 'Depends'.
> More on that here:
>
> https://github.com/Bioconductor/Contributions#submitting-related-packages
>
> One additional step taken to transition the packages to the build system
> after approval is to add a .BBSoptions file in the package that 'Suggests'.
> In this example, the .BBSoptions would be in the data package and have this
> one line:
>
> ForceInstall: TRUE
>
> Valerie
>
>
> On 10/6/18 11:06 AM, Michael Lawrence wrote:
>
> I think you're doing the right thing, suggesting the data package from the
> software package. This might require some manual intervention on the build
> system side to ensure that the software package is installed before passing
> full check, so that the data package can be installed/checked and finally
> the software package checked. I guess a note about this could be added to
> the page at the cited URL.
>
> Michael
>
> On Sat, Oct 6, 2018 at 10:42 AM Tim Triche, Jr. <tim.triche using gmail.com
> ><mailto:tim.triche using gmail.com>
> wrote:
>
>
>
> Last night I submitted MTseeker and its companion package MTseekerData,
> both festooned with examples that run smoothly and pass BiocCheck.
>
> HOWEVER! Breaking up the data into a Data package inadvertently seems to
> have created a circular dependency between the Software and Data packages.
> I haven't had to deal with this in the past, and mostly sidestepped it by
> skipping a Depends: entry in Data.
>
> (cf. https://support.bioconductor.org/p/113711 , which I will update with
> answers from here to try and avoid having other people with the same
> problem bug youse plural)
>
> The issue seems to be that, since the MTseeker-defined MVRanges and
> MAlignments classes (which, as you might imagine, subclass the VRanges and
> GAlignments classes) hold the data stored in MTseekerData, the examples in
> MTseekerData call the `show` method (and others) on the objects and this
> create issues.  The most bizarre of these complained at installation about
> (sp?) .lazyLoadCacheManager() (I'm having trouble reproducing this exact
> error but it was memorable).  Does this ring a bell for anyone? Is there an
> elegant workaround?
>
> I can't imagine I'm the first person to have this issue.  I know there are
> also various automagic hooks in BioC to load a package upon discovering a
> data structure that belongs to it (and not before) -- should I be using
> attachNamespace() somehow to side-step this problem?  (Eventually I do want
> to submit BAM files to ExperimentHub for an end-to-end example and unit
> tests, but the more pressing issue here was BiocCheck).
>
> Any feedback is much appreciated. It's kind of a drag when all the examples
> and vignettes pass and build, but only with a side-stepping of
> dependencies. That can't possibly be the correct solution, yet I'm at a
> loss to find documentation (which I'm sure exists somewhere) that explains
> how to avoid this (and/or implement the hooks).
>
> Thank you,
>
> --t
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> Bioc-devel using r-project.org<mailto:Bioc-devel using r-project.org> mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>
>
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> Bioc-devel using r-project.org<mailto:Bioc-devel using r-project.org> mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>
>
>
>
>
>
> This email message may contain legally privileged and/or confidential
> information.  If you are not the intended recipient(s), or the employee or
> agent responsible for the delivery of this message to the intended
> recipient(s), you are hereby notified that any disclosure, copying,
> distribution, or use of this email message is prohibited.  If you have
> received this message in error, please notify the sender immediately by
> e-mail and delete this email message from your computer. Thank you.
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> Bioc-devel using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>
> This email message may contain legally privileged and/or confidential
> information. If you are not the intended recipient(s), or the employee or
> agent responsible for the delivery of this message to the intended
> recipient(s), you are hereby notified that any disclosure, copying,
> distribution, or use of this email message is prohibited. If you have
> received this message in error, please notify the sender immediately by
> e-mail and delete this email message from your computer. Thank you.

	[[alternative HTML version deleted]]