[Bioc-devel] avoiding circular dependencies between a Data package and a Software package

[Tim Triche, Jr.]

awesome, thanks.

--t


On Tue, Oct 9, 2018 at 3:29 PM Obenchain, Valerie <
Valerie.Obenchain using roswellpark.org> wrote:

> I'm not sure what you mean by flipping the script ...?
>
> Looks like I was out of date with my advice on the .BBSoptions. A few
> months ago the BBS was modified to install all packages so the
>
> ForceInstall: True
>
> is no longer needed. You can remove the .BBSoptions file - sorry for the
> confusion.
>
> Lori will be updating the docs (
> https://github.com/Bioconductor/Contributions#submitting-related-packages)
> with more detail on handling packages with mutual (circular) dependencies.
> I see your package was assigned to Herve - if you are still having problems
> with the build you can ask @lshep or @hpages for help on the issue.
>
> Thanks for the pull request, it's been merged.
>
> Val
>
>
> On 10/8/18 7:23 AM, Tim Triche, Jr. wrote:
>
> D'oh.  Does this mean that I should flip the script a second time?
>
> I went ahead and added the .BBSoptions file as suggested and flushed out
> some additional irritants; it seemed that having the main package Suggest:
> the Data package went more smoothly (no cycles) so I had updated the ticket
> to reflect this. I also updated the support site question to reflect that
> suggestion. But now I am not sure if that's right.
>
> It is a bit confusing. I'd send a PR except I'm not really qualified to
> patch it. However, there was a reference to Subversion that stuck in my
> head after submission, and I did track that down; see
> https://github.com/Bioconductor/Contributions/pull/916 for details ;-)
>
> Thank you,
>
>
>
> --t
>
>
> On Mon, Oct 8, 2018 at 8:44 AM Shepherd, Lori <
> Lori.Shepherd using roswellpark.org> wrote:
>
>> Hello,
>>
>>
>> The Single Package builder (SPB) does check the .BBSOptions but only for
>> UnsupportedPlatforms.  This is because the single package builder
>> automatically installs each package before the build and check stage as
>> part of the process.  Each package that goes through the single package
>> builder for new submits temporarily gets its own directory to use for
>> installing and referencing dependencies -  This is why we document that
>> packages with additional or supporting packages should be built under the
>> same issue as then they will be able to find each other.  The submission
>> process where the data package that "suggests" the software package is
>> submitted first, followed by the software package that "depends" on the
>> data package is correct.
>>
>>
>> Its a little confusing at first but the following will/should happen on
>> the SPB:
>>
>>
>>
>>    1.  The data package will be submitted and fail because the software
>>    package is not available (but it will be installed in the Rlibs directory
>>    for the issue)
>>    2. The software package is submitted and fails because the data
>>    package can't be found (but also installs)
>>    3. The maintainer should do an arbitrary version bump on the data
>>    package to kick off a new build that should now pass as it will find the
>>    software package
>>    4. Do any additional debugging on the data package to ensure the data
>>    package has a clean build and check.
>>    5. The maintainer should do an arbitrary version bump on the software
>>    package, only after the data package is building and checking cleanly, to
>>    rebuild the software package
>>    6. Review process continues and packages are accepted
>>
>>
>> The same failure scheme will happen on the main builders once the package
>> is accepted, the only difference is the maintainer will not have to version
>> bump for the new builds since the daily builds builds and checks all
>> packages automatically (daily for software and  Sun/Tue/Thru for data
>> packages)- The ERRORS will clean up on there own after 3-5 days depending
>> on when it is accepted and added to the main builders in reference to the
>> schedule of the building of data packages (Sun/Tue/Thur).  Once the
>> packages are both installed once, the circular dependency issue will be
>> resolved.
>>
>> Cheers,
>>
>>
>>
>> Lori Shepherd
>>
>> Bioconductor Core Team
>>
>> Roswell Park Cancer Institute
>>
>> Department of Biostatistics & Bioinformatics
>>
>> Elm & Carlton Streets
>>
>> Buffalo, New York 14263
>> ------------------------------
>> *From:* Bioc-devel <bioc-devel-bounces using r-project.org> on behalf of
>> Obenchain, Valerie <Valerie.Obenchain using RoswellPark.org>
>> <Valerie.Obenchain using RoswellPark.org>
>> *Sent:* Sunday, October 7, 2018 11:54:46 AM
>> *To:* Michael Lawrence; Tim Triche, Jr.
>> *Cc:* packages using bioconductor.org; bioc-devel
>> *Subject:* Re: [Bioc-devel] avoiding circular dependencies between a
>> Data package and a Software package
>>
>> I forgot to add the .BBSoptions file should be located at the 'top' level
>> of the package, e.g., MTseekerData/.BBSoptions. You might as well add that
>> now instead of waiting until the packages are approved. I think the SPB was
>> modified to be aware of the .BBSoptions file so it may require it too (Lori
>> would know). If the SPB does require .BBSoptions we should document that in
>>
>> https://github.com/Bioconductor/Contributions/blob/master/CONTRIBUTING.md
>>
>> Valerie
>>
>> On 10/7/18 8:39 AM, Obenchain, Valerie wrote:
>>
>> Hi Tim,
>>
>> As Michael said, the usual case is the data package 'Suggests' the
>> software and the software 'Depends' or 'Imports' the data but this
>> dependency could be reversed.
>>
>> When submitting a combo of packages to the SPB for review, the package
>> that 'Suggests' is submitted first followed by the package that 'Depends'.
>> More on that here:
>>
>> https://github.com/Bioconductor/Contributions#submitting-related-packages
>>
>> One additional step taken to transition the packages to the build system
>> after approval is to add a .BBSoptions file in the package that 'Suggests'.
>> In this example, the .BBSoptions would be in the data package and have this
>> one line:
>>
>> ForceInstall: TRUE
>>
>> Valerie
>>
>>
>> On 10/6/18 11:06 AM, Michael Lawrence wrote:
>>
>> I think you're doing the right thing, suggesting the data package from the
>> software package. This might require some manual intervention on the build
>> system side to ensure that the software package is installed before
>> passing
>> full check, so that the data package can be installed/checked and finally
>> the software package checked. I guess a note about this could be added to
>> the page at the cited URL.
>>
>> Michael
>>
>> On Sat, Oct 6, 2018 at 10:42 AM Tim Triche, Jr. <tim.triche using gmail.com
>> ><mailto:tim.triche using gmail.com>
>> wrote:
>>
>>
>>
>> Last night I submitted MTseeker and its companion package MTseekerData,
>> both festooned with examples that run smoothly and pass BiocCheck.
>>
>> HOWEVER! Breaking up the data into a Data package inadvertently seems to
>> have created a circular dependency between the Software and Data packages.
>> I haven't had to deal with this in the past, and mostly sidestepped it by
>> skipping a Depends: entry in Data.
>>
>> (cf. https://support.bioconductor.org/p/113711 , which I will update with
>> answers from here to try and avoid having other people with the same
>> problem bug youse plural)
>>
>> The issue seems to be that, since the MTseeker-defined MVRanges and
>> MAlignments classes (which, as you might imagine, subclass the VRanges and
>> GAlignments classes) hold the data stored in MTseekerData, the examples in
>> MTseekerData call the `show` method (and others) on the objects and this
>> create issues.  The most bizarre of these complained at installation about
>> (sp?) .lazyLoadCacheManager() (I'm having trouble reproducing this exact
>> error but it was memorable).  Does this ring a bell for anyone? Is there
>> an
>> elegant workaround?
>>
>> I can't imagine I'm the first person to have this issue.  I know there are
>> also various automagic hooks in BioC to load a package upon discovering a
>> data structure that belongs to it (and not before) -- should I be using
>> attachNamespace() somehow to side-step this problem?  (Eventually I do
>> want
>> to submit BAM files to ExperimentHub for an end-to-end example and unit
>> tests, but the more pressing issue here was BiocCheck).
>>
>> Any feedback is much appreciated. It's kind of a drag when all the
>> examples
>> and vignettes pass and build, but only with a side-stepping of
>> dependencies. That can't possibly be the correct solution, yet I'm at a
>> loss to find documentation (which I'm sure exists somewhere) that explains
>> how to avoid this (and/or implement the hooks).
>>
>> Thank you,
>>
>> --t
>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> Bioc-devel using r-project.org<mailto:Bioc-devel using r-project.org> mailing list
>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>>
>>
>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> Bioc-devel using r-project.org<mailto:Bioc-devel using r-project.org> mailing list
>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>>
>>
>>
>>
>>
>>
>> This email message may contain legally privileged and/or confidential
>> information.  If you are not the intended recipient(s), or the employee or
>> agent responsible for the delivery of this message to the intended
>> recipient(s), you are hereby notified that any disclosure, copying,
>> distribution, or use of this email message is prohibited.  If you have
>> received this message in error, please notify the sender immediately by
>> e-mail and delete this email message from your computer. Thank you.
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> Bioc-devel using r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>>
>> This email message may contain legally privileged and/or confidential
>> information. If you are not the intended recipient(s), or the employee or
>> agent responsible for the delivery of this message to the intended
>> recipient(s), you are hereby notified that any disclosure, copying,
>> distribution, or use of this email message is prohibited. If you have
>> received this message in error, please notify the sender immediately by
>> e-mail and delete this email message from your computer. Thank you.
>
>
>
> This email message may contain legally privileged and/or confidential
> information. If you are not the intended recipient(s), or the employee or
> agent responsible for the delivery of this message to the intended
> recipient(s), you are hereby notified that any disclosure, copying,
> distribution, or use of this email message is prohibited. If you have
> received this message in error, please notify the sender immediately by
> e-mail and delete this email message from your computer. Thank you.

	[[alternative HTML version deleted]]