[Bioc-devel] Git lfs support
Anatoly Sorokin
|pto||k @end|ng |rom gm@||@com
Mon Sep 13 16:08:39 CEST 2021
Hi, sorry to bother you again, but I'm completely lost: how could I test
that my new package behaves properly?
AnnotationHub by default loads a list of all available Hub data, how can I
add my new package to that list without publishing it in AnnotationHub
first? On the other hand, I don't think that publishing untested packages
is a good idea.
Please, can anyone provide more or less extended documentation? I've read
the CreateAnAnnotationPackage vignette. It is great to understand the
internals of the Annotation package, but there is no clue how to use the
package without publishing it.
Thank you in advance,
Anatoly
On Fri, Sep 10, 2021 at 6:24 PM Kern, Lori <Lori.Shepherd using roswellpark.org>
wrote:
> The paths will be given through the AnnotationHub interface.
>
> I would like to also point out that there is a SQLite DisplatchClass that
> might be more appropriate? It would load the database automatically using
> AnnotaionDbi::loadDb
> For a Zip DispatchClass, It will get the zipped file, unzip, and provide a
> list of the file paths.
>
> make-data.R should be how you created the sqlite database. It doesn't
> need to be run again. It could be code, sudo-code, or text but should
> describte how you created the data and any relevant source information for
> the data.
>
> Hope this helps
>
> Cheers,
>
>
> Lori Shepherd
>
> Bioconductor Core Team
>
> Roswell Park Comprehensive Cancer Center
>
> Department of Biostatistics & Bioinformatics
>
> Elm & Carlton Streets
>
> Buffalo, New York 14263
> ------------------------------
> *From:* Bioc-devel <bioc-devel-bounces using r-project.org> on behalf of
> Anatoly Sorokin <lptolik using gmail.com>
> *Sent:* Friday, September 10, 2021 5:56 AM
> *To:* bioc-devel using r-project.org <bioc-devel using r-project.org>
> *Subject:* Re: [Bioc-devel] Git lfs support
>
> Hi Hervé,
>
> I've made a new package with metadata.csv, as described in
> CreateAnAnnotationPackage
> vignette.
>
> The data is located on the public service as a zipped SQLite database. So
> I've set both SourceType and DispatchClass as 'Zip'.
> From AnnotationHub::DispatchClassList() I expect that I should have an
> unzipped file path. My first question is: how do I get that path?
>
> The second question is: what should I put into make-data.R? The database is
> already created, cleaned and validated. All I need for the package is to
> download it, unzip and connect.
>
> Thank you,
> Anatoly
>
> On Thu, Aug 26, 2021 at 12:59 PM Hervé Pagès <hpages.on.github using gmail.com>
> wrote:
>
> > Hi Anatoly,
> >
> > Let's keep this conversation on the bioc-devel mailing list where it
> > started.
> >
> > On 26/08/2021 02:46, Anatoly Sorokin wrote:
> > > Hi Hervé,
> > >
> > > thank you for your answer. Does this mean that the SQLite file will be
> > > downloaded separately and kept in some cache?
> >
> > Yes.
> >
> > >
> > > And another question: does this mean that there should be two packages,
> > > one for code and another one for the database?
> >
> > Yes. One is what we call the software package and the other one the
> > accompanying data package.
> >
> > Cheers,
> > H.
> >
> > >
> > > On Wed, Aug 25, 2021 at 6:58 AM Hervé Pagès <
> hpages.on.github using gmail.com
> > > <mailto:hpages.on.github using gmail.com <hpages.on.github using gmail.com>>>
> wrote:
> > >
> > > Hi Anatoly,
> > >
> > > What kind of data is in your SQLite database? The Bioconductor
> > approach
> > > for this is to provide the data as a separate data annotation or
> data
> > > experiment package. More precisely, the data itself should go on
> > > AnnotationHub or ExperimentHub. It should be associated with a "Hub
> > > package", that is, a data package that documents it and explains
> how
> > to
> > > retrieve it from AnnotationHub or ExperimentHub. See vignettes in
> the
> > > HubPub package for more information:
> > >
> > > http://bioconductor.org/packages/HubPub
> > > <http://bioconductor.org/packages/HubPub>
> > >
> > > Best,
> > >
> > > H.
> > >
> > > On 24/08/2021 17:53, Nitesh Turaga wrote:
> > > > Hi,
> > > >
> > > > We don’t have LFS support on the Bioconductor git server.
> > > >
> > > > I would suggest finding an alternative.
> > > >
> > > > Best,
> > > >
> > > >
> > > >
> > > >
> > > > Nitesh Turaga
> > > > Scientist II, Department of Data Science,
> > > > Bioconductor Core Team Member
> > > > Dana Farber Cancer Institute
> > > >
> > > >> On Aug 24, 2021, at 6:18 PM, Anatoly Sorokin <
> lptolik using gmail.com
> > > <mailto:lptolik using gmail.com <lptolik using gmail.com>>> wrote:
> > > >>
> > > >> Hi all,
> > > >>
> > > >> we have developed a package that provides access to the data in
> > > the locally
> > > >> stored SQLite database. The GitHub Action successfully built
> the
> > > package,
> > > >> but in Bioconductor, it got an error. The only reason for the
> > > error is that
> > > >> the database itself is stored on GitHub via git-lfs, and the R
> > > code got an
> > > >> lfs placeholder instead of the actual database.
> > > >>
> > > >> Is it possible to activate lfs within Bioconductor automatic
> > > building
> > > >> server? The database is tiny (80 Mb), but I don't want to have
> > > it as a
> > > >> regular git object.
> > > >>
> > > >> Thank you,
> > > >> Anatoly
> > > >>
> > > >> [[alternative HTML version deleted]]
> > > >>
> > > >> _______________________________________________
> > > >> Bioc-devel using r-project.org <mailto:Bioc-devel using r-project.org
> <Bioc-devel using r-project.org>>
> > > mailing list
> > > >>
> https://secure-web.cisco.com/1WUW7ZRdbtpRvJ5xI0GsYHli6Fa14AGQb8WMb4rQ3UJJMaH1UjGkvTOPPE-nuN6rDlQq_KGnu0uVjf2v5v5bsuiqhHF7AUrVpF6bNQkRgCyLo8e-HiA3KLjbJ8_B4XYC58Duri0Sh4_YTCpgpvDPcP6WfBHvGC9oB-4QifRz-gUYg4C65EXHrA7TjMAzj4CHo9a6zIQN4VjeWyW_p5L1ueEaw9Jn8xa9gurW85DezyosVZrZYUNthLQcPeuePvCpK2Or4CP1tuaLXSiLtAzynJYcvyELR4tPWNyvN_qqEpYQC-ld9a0P8zTZAXtTtwbR3/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel
> > > <
> https://secure-web.cisco.com/1WUW7ZRdbtpRvJ5xI0GsYHli6Fa14AGQb8WMb4rQ3UJJMaH1UjGkvTOPPE-nuN6rDlQq_KGnu0uVjf2v5v5bsuiqhHF7AUrVpF6bNQkRgCyLo8e-HiA3KLjbJ8_B4XYC58Duri0Sh4_YTCpgpvDPcP6WfBHvGC9oB-4QifRz-gUYg4C65EXHrA7TjMAzj4CHo9a6zIQN4VjeWyW_p5L1ueEaw9Jn8xa9gurW85DezyosVZrZYUNthLQcPeuePvCpK2Or4CP1tuaLXSiLtAzynJYcvyELR4tPWNyvN_qqEpYQC-ld9a0P8zTZAXtTtwbR3/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel
> >
> > > >
> > > > _______________________________________________
> > > > Bioc-devel using r-project.org <mailto:Bioc-devel using r-project.org
> <Bioc-devel using r-project.org>>
> > > mailing list
> > > >
> https://secure-web.cisco.com/1WUW7ZRdbtpRvJ5xI0GsYHli6Fa14AGQb8WMb4rQ3UJJMaH1UjGkvTOPPE-nuN6rDlQq_KGnu0uVjf2v5v5bsuiqhHF7AUrVpF6bNQkRgCyLo8e-HiA3KLjbJ8_B4XYC58Duri0Sh4_YTCpgpvDPcP6WfBHvGC9oB-4QifRz-gUYg4C65EXHrA7TjMAzj4CHo9a6zIQN4VjeWyW_p5L1ueEaw9Jn8xa9gurW85DezyosVZrZYUNthLQcPeuePvCpK2Or4CP1tuaLXSiLtAzynJYcvyELR4tPWNyvN_qqEpYQC-ld9a0P8zTZAXtTtwbR3/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel
> > > <
> https://secure-web.cisco.com/1WUW7ZRdbtpRvJ5xI0GsYHli6Fa14AGQb8WMb4rQ3UJJMaH1UjGkvTOPPE-nuN6rDlQq_KGnu0uVjf2v5v5bsuiqhHF7AUrVpF6bNQkRgCyLo8e-HiA3KLjbJ8_B4XYC58Duri0Sh4_YTCpgpvDPcP6WfBHvGC9oB-4QifRz-gUYg4C65EXHrA7TjMAzj4CHo9a6zIQN4VjeWyW_p5L1ueEaw9Jn8xa9gurW85DezyosVZrZYUNthLQcPeuePvCpK2Or4CP1tuaLXSiLtAzynJYcvyELR4tPWNyvN_qqEpYQC-ld9a0P8zTZAXtTtwbR3/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel
> >
> > > >
> > >
> > > --
> > > Hervé Pagès
> > >
> > > Bioconductor Core Team
> > > hpages.on.github using gmail.com <mailto:hpages.on.github using gmail.com
> <hpages.on.github using gmail.com>>
> > >
> >
> > --
> > Hervé Pagès
> >
> > Bioconductor Core Team
> > hpages.on.github using gmail.com
> >
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> Bioc-devel using r-project.org mailing list
>
> https://secure-web.cisco.com/1WUW7ZRdbtpRvJ5xI0GsYHli6Fa14AGQb8WMb4rQ3UJJMaH1UjGkvTOPPE-nuN6rDlQq_KGnu0uVjf2v5v5bsuiqhHF7AUrVpF6bNQkRgCyLo8e-HiA3KLjbJ8_B4XYC58Duri0Sh4_YTCpgpvDPcP6WfBHvGC9oB-4QifRz-gUYg4C65EXHrA7TjMAzj4CHo9a6zIQN4VjeWyW_p5L1ueEaw9Jn8xa9gurW85DezyosVZrZYUNthLQcPeuePvCpK2Or4CP1tuaLXSiLtAzynJYcvyELR4tPWNyvN_qqEpYQC-ld9a0P8zTZAXtTtwbR3/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel
>
>
> This email message may contain legally privileged and/or confidential
> information. If you are not the intended recipient(s), or the employee or
> agent responsible for the delivery of this message to the intended
> recipient(s), you are hereby notified that any disclosure, copying,
> distribution, or use of this email message is prohibited. If you have
> received this message in error, please notify the sender immediately by
> e-mail and delete this email message from your computer. Thank you.
[[alternative HTML version deleted]]
More information about the Bioc-devel
mailing list