[Bioc-devel] Package reference manuals in html
Laurent Gatto
lg390 at cam.ac.uk
Fri Dec 23 19:14:13 CET 2016
Dear Sean and Robert,
On 23 December 2016 17:24, Robert M. Flight wrote:
> Yes, this seems like a time where using a non-standard site directory
> on Github is useful, or as Sean said, using a separate branch to serve
> the html content.
It's not clear to me how a separate branch is the solution. In git-svn,
devel, release_3_4, ... are just that, different branches. How would you
set a gh-pages branch that contains a single dir so that it doesn't
conflict with master?
If I could set branch-specific directives in my .gitignore file, then I
would set this up for the Bioc branches. This seems to suggest it is
possible, but I haven't tried yet
http://stackoverflow.com/questions/1836742/using-git-how-do-i-ignore-a-file-in-one-branch-but-have-it-committed-in-another
Would it be possible to set something on the svn side to ignore/delete
the docs dir automatically, independently of what git does?
Laurent
> -Robert
>
> On Fri, Dec 23, 2016 at 12:19 PM Sean Davis <seandavi at gmail.com> wrote:
>
> Github allows you to set the branch for the docs directory if I
> recall. Perhaps a separate branch with a docs directory (not
> master) is a viable way to go?
>
> Sean
>
> > On Dec 23, 2016, at 12:16 PM, Laurent Gatto <lg390 at cam.ac.uk>
> wrote:
> >
> >
> > There's actually another side-effect for Bioconductor. The
> package
> > website is (by default) generated in the package's ./docs
> > directory. This is very handy, as it can be set directly on
> Github as a
> > Github page and becomes available as https://username.github.io/
> pkgname.
> >
> > This also means that the docs directory (which is about 5.5M for
> > MSnbase) ends up on hedgehog. It is easy for it not to be part of
> the
> > package build artefact using .Rbuildignore, but I am not sure how
> to
> > easily push it to github but not hedgehog when using git-svn.
> >
> > Laurent
> >
> > On 23 December 2016 16:36, Laurent Gatto wrote:
> >
> >> Dear all,
> >>
> >> I'm following up re my online references suggestion with my
> recent
> >> experience with Hadley's pkgdown package
> >>
> >> https://github.com/hadley/pkgdown
> >>
> >> It doesn't address the cross-package issue (which is a difficult
> one
> >> anyway), but does pretty much everything else (with some caveats
> though,
> >> see below).
> >>
> >> Here are 2 examples
> >>
> >> - MSnbase: http://lgatto.github.io/MSnbase/
> >> - hpar: http://lgatto.github.io/hpar/
> >>
> >> It uses the REAMDE file as index page, creates html documents
> for all Rd
> >> files in man, an article tab for vignettes (but see below) and a
> News
> >> tab (but see below)
> >>
> >> The biggest caveats is that only Rmd vignettes are taken into
> account;
> >> Rnw are completely ignored (they don't show up at all in the
> Articles
> >> tab). This is not going to be tackled by the developer [1].
> >>
> >> [1] https://github.com/hadley/pkgdown/issues/220
> >>
> >> I had a quick look at the code and patching pkgdown (and
> probably
> >> rmarkdown) to build Rnw/pdf vignettes would take too much time I
> could
> >> devote at the moment. I would be satisfied if the Rnw were not
> build but
> >> at least there were links in the Articles tab pointing to the
> vignettes
> >> Bioconductor landing pages. On the other hand, I am migrating to
> >> BiocStyle's html_document2 with the nice floating table of
> contents...
> >>
> >> Regarding the News tab, only NEWS.md (markdown format) are
> considered;
> >> NEWS in Rd are ignored too.
> >>
> >> Hope this helps.
> >>
> >> Best wishes,
> >>
> >> Laurent
> >>
> >> On 16 March 2016 23:33, Andrzej Oleś wrote:
> >>
> >>> Hi all,
> >>>
> >>> I had a discussion earlier today with Martin and Dan on
> providing
> >>> online man pages for Bioconductor packages. As we dived into
> >>> implementation details, it turned out that this idea is a
> little bit
> >>> more complex and resource-intensive than originally
> anticipated.
> >>>
> >>> The main problem in generating man pages in a repository-wide
> fashion
> >>> seems to be the cross-linking of packages. Briefly, in order to
> >>> generate the links, apparently one needs to generate the html
> pages in
> >>> an R installation which is aware of the other packages. For
> example,
> >>> the Rd macro \linkS4class{ClassName} takes as argument only the
> class
> >>> name, and the corresponding package containing the class
> definition is
> >>> "automagically" resolved by R. I'm not sure how this could be
> done
> >>> manually, on a per-package basis. So by the end of the day, in
> order to
> >>> generate static man pages, we would need to maintain a complete
> BioC
> >>> repo installation, possibly on a system with the
> --enable-prebuilt-html
> >>> configure option. Unfortunately, it seems unfeasible to exploit
> the
> >>> build servers for this, as this would significantly increase
> the
> >>> computational burden. This is because currently only around 2/5
> of all
> >>> software and data packages are actually being installed by the
> build
> >>> system. The rest which does not have any reverse dependencies
> is
> >>> skipped. Installing the remaining 3/5 of packages on a regular
> basis,
> >>> not to mention the heavy annotation packages, is a little bit
> of an
> >>> overkill. So piggy-backing on the existing infrastructure
> doesn't seem
> >>> realistic.
> >>>
> >>> On top of this, even if we would have access to a machine with
> a
> >>> complete, up-to-date BioC installation (maybe by just updating
> the
> >>> packages after the repo gets rebuild rather than re-installing
> them
> >>> each time from scratch), it remains an open question how
> "external"
> >>> links to, let's say, CRAN packages, or even base R packages,
> should be
> >>> handled.
> >>>
> >>> A lightweight and easy to implement alternative for those
> willing to
> >>> share self-hosted documentation of their packages, could be to
> provide
> >>> in the package DESCRIPTION file a "Documentation" field
> containing a
> >>> link to external resource, which would then appear on the
> package
> >>> landing page next to the vignettes and pdf manual. The obvious
> >>> downsides of this solution are: 1. no package cross-links, and
> 2. the
> >>> burden of keeping the documentation in sync with the package
> version on
> >>> BioC would be in maintainer's hands...
> >>>
> >>> I will try to contact the authors of rdocumentation.org - maybe
> they
> >>> have some useful comments or even code which they would be
> willing to
> >>> share. In any case, it would be good to know what their
> experience is
> >>> and why did they stop maintaining their service. Maybe the BioC
> >>> community could jump in and help them to resolve the
> bottlenecks and
> >>> keep the website up to date.
> >>>
> >>> Cheers,
> >>> Andrzej
> >>>
> >>>
> >>> On Tue, Mar 8, 2016 at 4:36 PM, Andrzej Oleś <
> andrzej.oles at gmail.com>
> >>> wrote:
> >>>
> >>> Hi Martin,
> >>>
> >>> thank you for your suggestions - I would be happy to
> contribute to
> >>> this! I could help with developing the scripts for
> generating man
> >>> pages, and integrating them with the website layout.
> >>>
> >>> As for rendering the man pages, I suggest that we try a
> similar
> >>> approach to the one used by knitr::knit_rd() rather than
> plain
> >>> tools::Rd2HTML(). It has the advantage that the examples are
> >>> actually run, and the results, e.g. plots, are included in
> the
> >>> output documents. I hope you can appreciate the added value
> by
> >>> comparing the following man page rendered using
> tools::Rd2HTML()
> >>> and knitr::knit_rd(), respectively.
> >>> http://www.huber.embl.de/users/aoles/man/Image.html
> >>> http://www.huber.embl.de/users/aoles/man/Image-knitr.html
> >>> Regarding the additional dependencies: we kind of already
> rely on
> >>> knitr when compiling vignettes, so this this shouldn't add
> much to
> >>> the maintenance burden.
> >>>
> >>> Cheers,
> >>> Andrzej
> >>>
> >>> On Fri, Mar 4, 2016 at 2:20 PM, Morgan, Martin <
> >>> Martin.Morgan at roswellpark.org> wrote:
> >>>
> >>> One thing about accessing the html versions locally
> (e.g., via
> >>> ? with options(help_type="html")] or help.start() or
> Rstudio)
> >>> is that you get the version relevant to your R /
> Bioconductor,
> >>> rather than whatever is at the top of google; I guess
> the same
> >>> applies to the pdf versions, and the reason that there
> isn't
> >>> more current confusion is because the online pdf
> versions are
> >>> not as useful as the off-line help system.
> >>>
> >>> I think Laurent was interested in an integration of help
> pages
> >>> across packages (which is the appeal of
> rdocumentation.org?),
> >>> not just rendering the help pages in html rather than
> pdf? An
> >>> integration of help pages would definitely be a big job
> with
> >>> substantial development and maintenance; we will not be
> >>> undertaking this ourselves.
> >>>
> >>> For the more limited case of adding a (directory of)
> html files
> >>> for the the manual, it's not impossible that we could
> find the
> >>> resources to do this in the next 6 months.
> >>>
> >>> One intermediate and helpful step for those willing to
> help
> >>> would be to develop the code to process help pages into
> a style
> >>> consistent with the bioconductor web site. One place
> where this
> >>> could be implemented would be the BiocStyle package
> (https://
> >>> github.com/Bioconductor-mirror/BiocStyle but hmm, seems
> like
> >>> there's a slightly out of sync version at https://
> github.com/
> >>> Bioconductor/BiocStyle that would be more
> convenient...).
> >>> Perhaps this really means only developing a css style
> sheet and
> >>> R's tools::Rd2HTML() (I'm very reluctant to introduce
> >>> dependencies into the build system, and am very
> conservative
> >>> about inclusion of fancy features in the html -- these
> become
> >>> significant maintenance burdens moving forward).
> >>>
> >>> The web site is generated by https://github.com/
> Bioconductor/
> >>> bioconductor.org, with the style sheet at https://
> github.com/
> >>> Bioconductor/bioconductor.org/blob/master/assets/style/
> >>> bioconductor.css. The package landing pages are
> templated using
> >>> layouts/_bioc_views_package_detail.html. The idea would
> be to
> >>> end up with layouts/_bioc_man_index.html and
> >>> _bioc_man_body.html that wrapped output from BiocStyle
> in the
> >>> overall bioc page.
> >>>
> >>> The implementation suggestions above are just a sketch
> and
> >>> could be quite misguided. If there's interest then
> probably we
> >>> should set up a hangout to discuss in a little more
> detail.
> >>>
> >>> Martin
> >>>
> >>> ________________________________________
> >>> From: Bioc-devel <bioc-devel-bounces at r-project.org> on
> behalf
> >>> of Hartley, Stephen (NIH/NHGRI) [F] <
> stephen.hartley at nih.gov>
> >>> Sent: Wednesday, March 2, 2016 11:46 AM
> >>> To: Laurent Gatto; bioc-devel
> >>> Subject: Re: [Bioc-devel] Package reference manuals in
> html
> >>>
> >>> I'd like to second this. Currently Bioconductor hosts
> the pdf
> >>> reference manuals, but those are often sub-ideal. The
> page
> >>> breaks make it harder to read, the fixed width basically
> makes
> >>> it either too small or too big depending on your
> display, you
> >>> can't navigate cross-package links, and in general using
> >>> paper-formatted software documentation is just poor
> form.
> >>>
> >>> Yihui, the creator of knitr, has a blog post where he
> shows how
> >>> to do this. There are a lot of ways to do this, and it's
> >>> generally pretty straightforward.
> >>> http://yihui.name/en/2012/10/build-static-html-help/
> >>>
> >>> You can also use a function in knitr, knit_rd(), which
> builds
> >>> the examples as well and inserts the output right onto
> the
> >>> page. That's what I used to make the docs for QoRTs
> (http://
> >>> hartleys.github.io/QoRTs/Rhtml/index.html) and
> JunctionSeq (
> >>> http://hartleys.github.io/JunctionSeq/Rhtml/index.html).
> >>>
> >>> Or you can use the staticdocs package, which does the
> same
> >>> basic thing but prettier (see ggplot2's docs: http://
> >>> docs.ggplot2.org/current/)
> >>>
> >>> The nuclear option, of course, is to do what CRAN does
> and
> >>> rebuild R on (one of) the servers using the
> >>> --enable-prebuilt-html configure option. That might
> affect
> >>> other things, though, and might not be ideal.
> >>>
> >>> Does any of this seem like a viable option for
> Bioconductor? I
> >>> think it could be an incredibly valuable resource for
> the
> >>> community. Are there any technical issues that haven't
> been
> >>> considered in the above?
> >>>
> >>> Regards,
> >>> Steve Hartley
> >>>
> >>> -----Original Message-----
> >>> From: Laurent Gatto [mailto:lg390 at cam.ac.uk]
> >>> Sent: Tuesday, March 01, 2016 6:42 AM
> >>> To: bioc-devel
> >>> Subject: [Bioc-devel] Package reference manuals in html
> >>>
> >>>
> >>> Dear all,
> >>>
> >>> I find the http://www.rdocumentation.org/ site very
> useful to
> >>> refer to nicely formatted online man pages individually.
> >>> Unfortunately, this resource is terribly outdated and
> not
> >>> maintained anymore.
> >>>
> >>> I was wondering if Bioconductor had any interest in
> serving an
> >>> html version of individual reference manuals in addition
> to the
> >>> pdf that are already available on the package landing
> pages.
> >>>
> >>> Is there anything I or any other members of the
> community could
> >>> help with to get this up and running?
> >>>
> >>> Best wishes,
> >>>
> >>> Laurent
> >>>
> >>> _______________________________________________
> >>> Bioc-devel at r-project.org mailing list
> >>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
> >>>
> >>> _______________________________________________
> >>> Bioc-devel at r-project.org mailing list
> >>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
> >>>
> >>>
> >>> This email message may contain legally privileged and/or
> >>> confidential information. If you are not the intended
> >>> recipient(s), or the employee or agent responsible for
> the
> >>> delivery of this message to the intended recipient(s),
> you are
> >>> hereby notified that any disclosure, copying,
> distribution, or
> >>> use of this email message is prohibited. If you have
> received
> >>> this message in error, please notify the sender
> immediately by
> >>> e-mail and delete this email message from your computer.
> Thank
> >>> you.
> >>> _______________________________________________
> >>> Bioc-devel at r-project.org mailing list
> >>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
> >
> >
> > --
> > Laurent Gatto | @lgatt0
> > http://cpu.sysbiol.cam.ac.uk/
> > http://lgatto.github.io/
> >
> > _______________________________________________
> > Bioc-devel at r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/bioc-devel
>
> _______________________________________________
> Bioc-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
--
Laurent Gatto | @lgatt0
http://cpu.sysbiol.cam.ac.uk/
http://lgatto.github.io/
More information about the Bioc-devel
mailing list