[Bioc-devel] Git lfs support
Kern, Lori
Lor|@Shepherd @end|ng |rom Ro@we||P@rk@org
Fri Sep 10 17:24:24 CEST 2021
The paths will be given through the AnnotationHub interface.
I would like to also point out that there is a SQLite DisplatchClass that might be more appropriate? It would load the database automatically using AnnotaionDbi::loadDb
For a Zip DispatchClass, It will get the zipped file, unzip, and provide a list of the file paths.
make-data.R should be how you created the sqlite database. It doesn't need to be run again. It could be code, sudo-code, or text but should describte how you created the data and any relevant source information for the data.
Hope this helps
Cheers,
Lori Shepherd
Bioconductor Core Team
Roswell Park Comprehensive Cancer Center
Department of Biostatistics & Bioinformatics
Elm & Carlton Streets
Buffalo, New York 14263
________________________________
From: Bioc-devel <bioc-devel-bounces using r-project.org> on behalf of Anatoly Sorokin <lptolik using gmail.com>
Sent: Friday, September 10, 2021 5:56 AM
To: bioc-devel using r-project.org <bioc-devel using r-project.org>
Subject: Re: [Bioc-devel] Git lfs support
Hi Herv�,
I've made a new package with metadata.csv, as described in
CreateAnAnnotationPackage
vignette.
The data is located on the public service as a zipped SQLite database. So
I've set both SourceType and DispatchClass as 'Zip'.
From AnnotationHub::DispatchClassList() I expect that I should have an
unzipped file path. My first question is: how do I get that path?
The second question is: what should I put into make-data.R? The database is
already created, cleaned and validated. All I need for the package is to
download it, unzip and connect.
Thank you,
Anatoly
On Thu, Aug 26, 2021 at 12:59 PM Herv� Pag�s <hpages.on.github using gmail.com>
wrote:
> Hi Anatoly,
>
> Let's keep this conversation on the bioc-devel mailing list where it
> started.
>
> On 26/08/2021 02:46, Anatoly Sorokin wrote:
> > Hi Herv�,
> >
> > thank you for your answer. Does this mean that the SQLite file will be
> > downloaded separately and kept in some cache?
>
> Yes.
>
> >
> > And another question: does this mean that there should be two packages,
> > one for code and another one for the database?
>
> Yes. One is what we call the software package and the other one the
> accompanying data package.
>
> Cheers,
> H.
>
> >
> > On Wed, Aug 25, 2021 at 6:58 AM Herv� Pag�s <hpages.on.github using gmail.com
> > <mailto:hpages.on.github using gmail.com>> wrote:
> >
> > Hi Anatoly,
> >
> > What kind of data is in your SQLite database? The Bioconductor
> approach
> > for this is to provide the data as a separate data annotation or data
> > experiment package. More precisely, the data itself should go on
> > AnnotationHub or ExperimentHub. It should be associated with a "Hub
> > package", that is, a data package that documents it and explains how
> to
> > retrieve it from AnnotationHub or ExperimentHub. See vignettes in the
> > HubPub package for more information:
> >
> > http://bioconductor.org/packages/HubPub
> > <http://bioconductor.org/packages/HubPub>
> >
> > Best,
> >
> > H.
> >
> > On 24/08/2021 17:53, Nitesh Turaga wrote:
> > > Hi,
> > >
> > > We don�t have LFS support on the Bioconductor git server.
> > >
> > > I would suggest finding an alternative.
> > >
> > > Best,
> > >
> > >
> > >
> > >
> > > Nitesh Turaga
> > > Scientist II, Department of Data Science,
> > > Bioconductor Core Team Member
> > > Dana Farber Cancer Institute
> > >
> > >> On Aug 24, 2021, at 6:18 PM, Anatoly Sorokin <lptolik using gmail.com
> > <mailto:lptolik using gmail.com>> wrote:
> > >>
> > >> Hi all,
> > >>
> > >> we have developed a package that provides access to the data in
> > the locally
> > >> stored SQLite database. The GitHub Action successfully built the
> > package,
> > >> but in Bioconductor, it got an error. The only reason for the
> > error is that
> > >> the database itself is stored on GitHub via git-lfs, and the R
> > code got an
> > >> lfs placeholder instead of the actual database.
> > >>
> > >> Is it possible to activate lfs within Bioconductor automatic
> > building
> > >> server? The database is tiny (80 Mb), but I don't want to have
> > it as a
> > >> regular git object.
> > >>
> > >> Thank you,
> > >> Anatoly
> > >>
> > >> [[alternative HTML version deleted]]
> > >>
> > >> _______________________________________________
> > >> Bioc-devel using r-project.org <mailto:Bioc-devel using r-project.org>
> > mailing list
> > >> https://secure-web.cisco.com/1WUW7ZRdbtpRvJ5xI0GsYHli6Fa14AGQb8WMb4rQ3UJJMaH1UjGkvTOPPE-nuN6rDlQq_KGnu0uVjf2v5v5bsuiqhHF7AUrVpF6bNQkRgCyLo8e-HiA3KLjbJ8_B4XYC58Duri0Sh4_YTCpgpvDPcP6WfBHvGC9oB-4QifRz-gUYg4C65EXHrA7TjMAzj4CHo9a6zIQN4VjeWyW_p5L1ueEaw9Jn8xa9gurW85DezyosVZrZYUNthLQcPeuePvCpK2Or4CP1tuaLXSiLtAzynJYcvyELR4tPWNyvN_qqEpYQC-ld9a0P8zTZAXtTtwbR3/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel
> > <https://secure-web.cisco.com/1WUW7ZRdbtpRvJ5xI0GsYHli6Fa14AGQb8WMb4rQ3UJJMaH1UjGkvTOPPE-nuN6rDlQq_KGnu0uVjf2v5v5bsuiqhHF7AUrVpF6bNQkRgCyLo8e-HiA3KLjbJ8_B4XYC58Duri0Sh4_YTCpgpvDPcP6WfBHvGC9oB-4QifRz-gUYg4C65EXHrA7TjMAzj4CHo9a6zIQN4VjeWyW_p5L1ueEaw9Jn8xa9gurW85DezyosVZrZYUNthLQcPeuePvCpK2Or4CP1tuaLXSiLtAzynJYcvyELR4tPWNyvN_qqEpYQC-ld9a0P8zTZAXtTtwbR3/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel>
> > >
> > > _______________________________________________
> > > Bioc-devel using r-project.org <mailto:Bioc-devel using r-project.org>
> > mailing list
> > > https://secure-web.cisco.com/1WUW7ZRdbtpRvJ5xI0GsYHli6Fa14AGQb8WMb4rQ3UJJMaH1UjGkvTOPPE-nuN6rDlQq_KGnu0uVjf2v5v5bsuiqhHF7AUrVpF6bNQkRgCyLo8e-HiA3KLjbJ8_B4XYC58Duri0Sh4_YTCpgpvDPcP6WfBHvGC9oB-4QifRz-gUYg4C65EXHrA7TjMAzj4CHo9a6zIQN4VjeWyW_p5L1ueEaw9Jn8xa9gurW85DezyosVZrZYUNthLQcPeuePvCpK2Or4CP1tuaLXSiLtAzynJYcvyELR4tPWNyvN_qqEpYQC-ld9a0P8zTZAXtTtwbR3/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel
> > <https://secure-web.cisco.com/1WUW7ZRdbtpRvJ5xI0GsYHli6Fa14AGQb8WMb4rQ3UJJMaH1UjGkvTOPPE-nuN6rDlQq_KGnu0uVjf2v5v5bsuiqhHF7AUrVpF6bNQkRgCyLo8e-HiA3KLjbJ8_B4XYC58Duri0Sh4_YTCpgpvDPcP6WfBHvGC9oB-4QifRz-gUYg4C65EXHrA7TjMAzj4CHo9a6zIQN4VjeWyW_p5L1ueEaw9Jn8xa9gurW85DezyosVZrZYUNthLQcPeuePvCpK2Or4CP1tuaLXSiLtAzynJYcvyELR4tPWNyvN_qqEpYQC-ld9a0P8zTZAXtTtwbR3/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel>
> > >
> >
> > --
> > Herv� Pag�s
> >
> > Bioconductor Core Team
> > hpages.on.github using gmail.com <mailto:hpages.on.github using gmail.com>
> >
>
> --
> Herv� Pag�s
>
> Bioconductor Core Team
> hpages.on.github using gmail.com
>
[[alternative HTML version deleted]]
_______________________________________________
Bioc-devel using r-project.org mailing list
https://secure-web.cisco.com/1WUW7ZRdbtpRvJ5xI0GsYHli6Fa14AGQb8WMb4rQ3UJJMaH1UjGkvTOPPE-nuN6rDlQq_KGnu0uVjf2v5v5bsuiqhHF7AUrVpF6bNQkRgCyLo8e-HiA3KLjbJ8_B4XYC58Duri0Sh4_YTCpgpvDPcP6WfBHvGC9oB-4QifRz-gUYg4C65EXHrA7TjMAzj4CHo9a6zIQN4VjeWyW_p5L1ueEaw9Jn8xa9gurW85DezyosVZrZYUNthLQcPeuePvCpK2Or4CP1tuaLXSiLtAzynJYcvyELR4tPWNyvN_qqEpYQC-ld9a0P8zTZAXtTtwbR3/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel
This email message may contain legally privileged and/or confidential information. If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited. If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you.
[[alternative HTML version deleted]]
More information about the Bioc-devel
mailing list