[Bioc-devel] Package reference manuals in html

Laurent Gatto lg390 at cam.ac.uk
Fri Dec 23 19:25:25 CET 2016


On 23 December 2016 17:52, Kasper Daniel Hansen wrote:

> Seems to be a good start.
>
> But I don't understand why NEWS.md is ok and NEWS.Rd is ignored, given
> than NEWS.md is not (I think) compatible and NEWS.Rd is.

What is irritating is that NEWS.md is not (yet?) supported by news()!

I say not yet because there's a currenlty unused format argument to
news, so there's hope.

Laurent

> Best,
> Kasper
>
> On Fri, Dec 23, 2016 at 12:24 PM, Robert M. Flight <rflight79 at gmail.com
>> wrote:
>
>     Yes, this seems like a time where using a non-standard site
>     directory on
>     Github is useful, or as Sean said, using a separate branch to serve
>     the
>     html content.
>
>     -Robert
>
>     On Fri, Dec 23, 2016 at 12:19 PM Sean Davis <seandavi at gmail.com>
>     wrote:
>
>     > Github allows you to set the branch for the docs directory if I
>     recall.
>     > Perhaps a separate branch with a docs directory (not master) is a
>     viable
>     > way to go?
>     >
>     > Sean
>     >
>     > > On Dec 23, 2016, at 12:16 PM, Laurent Gatto <lg390 at cam.ac.uk>
>     wrote:
>     > >
>     > >
>     > > There's actually another side-effect for Bioconductor. The
>     package
>     > > website is (by default) generated in the package's ./docs
>     > > directory. This is very handy, as it can be set directly on
>     Github as a
>     > > Github page and becomes available as https://username.github.io
>     /pkgname.
>     > >
>     > > This also means that the docs directory (which is about 5.5M
>     for
>     > > MSnbase) ends up on hedgehog. It is easy for it not to be part
>     of the
>     > > package build artefact using .Rbuildignore, but I am not sure
>     how to
>     > > easily push it to github but not hedgehog when using git-svn.
>     > >
>     > > Laurent
>     > >
>     > > On 23 December 2016 16:36, Laurent Gatto wrote:
>     > >
>     > >> Dear all,
>     > >>
>     > >> I'm following up re my online references suggestion with my
>     recent
>     > >> experience with Hadley's pkgdown package
>     > >>
>     > >> https://github.com/hadley/pkgdown
>     > >>
>     > >> It doesn't address the cross-package issue (which is a
>     difficult one
>     > >> anyway), but does pretty much everything else (with some
>     caveats though,
>     > >> see below).
>     > >>
>     > >> Here are 2 examples
>     > >>
>     > >> - MSnbase: http://lgatto.github.io/MSnbase/
>     > >> - hpar: http://lgatto.github.io/hpar/
>     > >>
>     > >> It uses the REAMDE file as index page, creates html documents
>     for all Rd
>     > >> files in man, an article tab for vignettes (but see below) and
>     a News
>     > >> tab (but see below)
>     > >>
>     > >> The biggest caveats is that only Rmd vignettes are taken into
>     account;
>     > >> Rnw are completely ignored (they don't show up at all in the
>     Articles
>     > >> tab). This is not going to be tackled by the developer [1].
>     > >>
>     > >> [1] https://github.com/hadley/pkgdown/issues/220
>     > >>
>     > >> I had a quick look at the code and patching pkgdown (and
>     probably
>     > >> rmarkdown) to build Rnw/pdf vignettes would take too much time
>     I could
>     > >> devote at the moment. I would be satisfied if the Rnw were not
>     build but
>     > >> at least there were links in the Articles tab pointing to the
>     vignettes
>     > >> Bioconductor landing pages. On the other hand, I am migrating
>     to
>     > >> BiocStyle's html_document2 with the nice floating table of
>     contents...
>     > >>
>     > >> Regarding the News tab, only NEWS.md (markdown format) are
>     considered;
>     > >> NEWS in Rd are ignored too.
>     > >>
>     > >> Hope this helps.
>     > >>
>     > >> Best wishes,
>     > >>
>     > >> Laurent
>     > >>
>     > >> On 16 March 2016 23:33, Andrzej Oleś wrote:
>     > >>
>     > >>> Hi all,
>     > >>>
>     > >>> I had a discussion earlier today with Martin and Dan on
>     providing
>     > >>> online man pages for Bioconductor packages. As we dived into
>     > >>> implementation details, it turned out that this idea is a
>     little bit
>     > >>> more complex and resource-intensive than originally
>     anticipated.
>     > >>>
>     > >>> The main problem in generating man pages in a repository-wide
>     fashion
>     > >>> seems to be the cross-linking of packages. Briefly, in order
>     to
>     > >>> generate the links, apparently one needs to generate the html
>     pages in
>     > >>> an R installation which is aware of the other packages. For
>     example,
>     > >>> the Rd macro \linkS4class{ClassName} takes as argument only
>     the class
>     > >>> name, and the corresponding package containing the class
>     definition is
>     > >>> "automagically" resolved by R. I'm not sure how this could be
>     done
>     > >>> manually, on a per-package basis. So by the end of the day,
>     in order to
>     > >>> generate static man pages, we would need to maintain a
>     complete BioC
>     > >>> repo installation, possibly on a system with the
>     --enable-prebuilt-html
>     > >>> configure option. Unfortunately, it seems unfeasible to
>     exploit the
>     > >>> build servers for this, as this would significantly increase
>     the
>     > >>> computational burden. This is because currently only around 2
>     /5 of all
>     > >>> software and data packages are actually being installed by
>     the build
>     > >>> system. The rest which does not have any reverse dependencies
>     is
>     > >>> skipped. Installing the remaining 3/5 of packages on a
>     regular basis,
>     > >>> not to mention the heavy annotation packages, is a little bit
>     of an
>     > >>> overkill. So piggy-backing on the existing infrastructure
>     doesn't seem
>     > >>> realistic.
>     > >>>
>     > >>> On top of this, even if we would have access to a machine
>     with a
>     > >>> complete, up-to-date BioC installation (maybe by just
>     updating the
>     > >>> packages after the repo gets rebuild rather than
>     re-installing them
>     > >>> each time from scratch), it remains an open question how
>     "external"
>     > >>> links to, let's say, CRAN packages, or even base R packages,
>     should be
>     > >>> handled.
>     > >>>
>     > >>> A lightweight and easy to implement alternative for those
>     willing to
>     > >>> share self-hosted documentation of their packages, could be
>     to provide
>     > >>> in the package DESCRIPTION file a "Documentation" field
>     containing a
>     > >>> link to external resource, which would then appear on the
>     package
>     > >>> landing page next to the vignettes and pdf manual. The
>     obvious
>     > >>> downsides of this solution are: 1. no package cross-links,
>     and 2. the
>     > >>> burden of keeping the documentation in sync with the package
>     version on
>     > >>> BioC would be in maintainer's hands...
>     > >>>
>     > >>> I will try to contact the authors of rdocumentation.org -
>     maybe they
>     > >>> have some useful comments or even code which they would be
>     willing to
>     > >>> share. In any case, it would be good to know what their
>     experience is
>     > >>> and why did they stop maintaining their service. Maybe the
>     BioC
>     > >>> community could jump in and help them to resolve the
>     bottlenecks and
>     > >>> keep the website up to date.
>     > >>>
>     > >>> Cheers,
>     > >>> Andrzej
>     > >>>
>     > >>>
>     > >>> On Tue, Mar 8, 2016 at 4:36 PM, Andrzej Oleś <
>     andrzej.oles at gmail.com>
>     > >>> wrote:
>     > >>>
>     > >>>  Hi Martin,
>     > >>>
>     > >>>  thank you for your suggestions - I would be happy to
>     contribute to
>     > >>>  this! I could help with developing the scripts for
>     generating man
>     > >>>  pages, and integrating them with the website layout.
>     > >>>
>     > >>>  As for rendering the man pages, I suggest that we try a
>     similar
>     > >>>  approach to the one used by knitr::knit_rd() rather than
>     plain
>     > >>>  tools::Rd2HTML(). It has the advantage that the examples
>     are
>     > >>>  actually run, and the results, e.g. plots, are included in
>     the
>     > >>>  output documents. I hope you can appreciate the added
>     value by
>     > >>>  comparing the following man page rendered using
>     tools::Rd2HTML()
>     > >>>  and knitr::knit_rd(), respectively.
>     > >>>  http://www.huber.embl.de/users/aoles/man/Image.html
>     > >>>  http://www.huber.embl.de/users/aoles/man/Image-knitr.html
>     > >>>  Regarding the additional dependencies: we kind of already
>     rely on
>     > >>>  knitr when compiling vignettes, so this this shouldn't add
>     much to
>     > >>>  the maintenance burden.
>     > >>>
>     > >>>  Cheers,
>     > >>>  Andrzej
>     > >>>
>     > >>>  On Fri, Mar 4, 2016 at 2:20 PM, Morgan, Martin <
>     > >>>  Martin.Morgan at roswellpark.org> wrote:
>     > >>>
>     > >>>    One thing about accessing the html versions locally
>     (e.g., via
>     > >>>    ? with options(help_type="html")] or help.start() or
>     Rstudio)
>     > >>>    is that you get the version relevant to your R /
>     Bioconductor,
>     > >>>    rather than whatever is at the top of google; I guess
>     the same
>     > >>>    applies to the pdf versions, and the reason that there
>     isn't
>     > >>>    more current confusion is because the online pdf
>     versions are
>     > >>>    not as useful as the off-line help system.
>     > >>>
>     > >>>    I think Laurent was interested in an integration of
>     help pages
>     > >>>    across packages (which is the appeal of
>     rdocumentation.org?),
>     > >>>    not just rendering the help pages in html rather than
>     pdf? An
>     > >>>    integration of help pages would definitely be a big
>     job with
>     > >>>    substantial development and maintenance; we will not
>     be
>     > >>>    undertaking this ourselves.
>     > >>>
>     > >>>    For the more limited case of adding a (directory of)
>     html files
>     > >>>    for the the manual, it's not impossible that we could
>     find the
>     > >>>    resources to do this in the next 6 months.
>     > >>>
>     > >>>    One intermediate and helpful step for those willing to
>     help
>     > >>>    would be to develop the code to process help pages
>     into a style
>     > >>>    consistent with the bioconductor web site. One place
>     where this
>     > >>>    could be implemented would be the BiocStyle package
>     (https://
>     > >>>    github.com/Bioconductor-mirror/BiocStyle but hmm,
>     seems like
>     > >>>    there's a slightly out of sync version at https://
>     github.com/
>     > >>>    Bioconductor/BiocStyle that would be more
>     convenient...).
>     > >>>    Perhaps this really means only developing a css style
>     sheet and
>     > >>>    R's tools::Rd2HTML() (I'm very reluctant to introduce
>     > >>>    dependencies into the build system, and am very
>     conservative
>     > >>>    about inclusion of fancy features in the html -- these
>     become
>     > >>>    significant maintenance burdens moving forward).
>     > >>>
>     > >>>    The web site is generated by https://github.com/
>     Bioconductor/
>     > >>>    bioconductor.org, with the style sheet at https://
>     github.com/
>     > >>>    Bioconductor/bioconductor.org/blob/master/assets/style
>     /
>     > >>>    bioconductor.css. The package landing pages are
>     templated using
>     > >>>    layouts/_bioc_views_package_detail.html. The idea
>     would be to
>     > >>>    end up with layouts/_bioc_man_index.html and
>     > >>>    _bioc_man_body.html that wrapped output from BiocStyle
>     in the
>     > >>>    overall bioc page.
>     > >>>
>     > >>>    The implementation suggestions above are just a sketch
>     and
>     > >>>    could be quite misguided. If there's interest then
>     probably we
>     > >>>    should set up a hangout to discuss in a little more
>     detail.
>     > >>>
>     > >>>    Martin
>     > >>>
>     > >>>    ________________________________________
>     > >>>    From: Bioc-devel <bioc-devel-bounces at r-project.org> on
>     behalf
>     > >>>    of Hartley, Stephen (NIH/NHGRI) [F] <
>     stephen.hartley at nih.gov>
>     > >>>    Sent: Wednesday, March 2, 2016 11:46 AM
>     > >>>    To: Laurent Gatto; bioc-devel
>     > >>>    Subject: Re: [Bioc-devel] Package reference manuals in
>     html
>     > >>>
>     > >>>    I'd like to second this. Currently Bioconductor hosts
>     the pdf
>     > >>>    reference manuals, but those are often sub-ideal. The
>     page
>     > >>>    breaks make it harder to read, the fixed width
>     basically makes
>     > >>>    it either too small or too big depending on your
>     display, you
>     > >>>    can't navigate cross-package links, and in general
>     using
>     > >>>    paper-formatted software documentation is just poor
>     form.
>     > >>>
>     > >>>    Yihui, the creator of knitr, has a blog post where he
>     shows how
>     > >>>    to do this. There are a lot of ways to do this, and
>     it's
>     > >>>    generally pretty straightforward.
>     > >>>    http://yihui.name/en/2012/10/build-static-html-help/
>     > >>>
>     > >>>    You can also use a function in knitr, knit_rd(), which
>     builds
>     > >>>    the examples as well and inserts the output right onto
>     the
>     > >>>    page. That's what I used to make the docs for QoRTs
>     (http://
>     > >>>    hartleys.github.io/QoRTs/Rhtml/index.html) and
>     JunctionSeq (
>     > >>>    http://hartleys.github.io/JunctionSeq/Rhtml/index.html
>     ).
>     > >>>
>     > >>>    Or you can use the staticdocs package, which does the
>     same
>     > >>>    basic thing but prettier (see ggplot2's docs: http://
>     > >>>    docs.ggplot2.org/current/)
>     > >>>
>     > >>>    The nuclear option, of course, is to do what CRAN does
>     and
>     > >>>    rebuild R on (one of) the servers using the
>     > >>>    --enable-prebuilt-html configure option. That might
>     affect
>     > >>>    other things, though, and might not be ideal.
>     > >>>
>     > >>>    Does any of this seem like a viable option for
>     Bioconductor? I
>     > >>>    think it could be an incredibly valuable resource for
>     the
>     > >>>    community. Are there any technical issues that haven't
>     been
>     > >>>    considered in the above?
>     > >>>
>     > >>>    Regards,
>     > >>>    Steve Hartley
>     > >>>
>     > >>>    -----Original Message-----
>     > >>>    From: Laurent Gatto [mailto:lg390 at cam.ac.uk]
>     > >>>    Sent: Tuesday, March 01, 2016 6:42 AM
>     > >>>    To: bioc-devel
>     > >>>    Subject: [Bioc-devel] Package reference manuals in
>     html
>     > >>>
>     > >>>
>     > >>>    Dear all,
>     > >>>
>     > >>>    I find the http://www.rdocumentation.org/ site very
>     useful to
>     > >>>    refer to nicely formatted online man pages
>     individually.
>     > >>>    Unfortunately, this resource is terribly outdated and
>     not
>     > >>>    maintained anymore.
>     > >>>
>     > >>>    I was wondering if Bioconductor had any interest in
>     serving an
>     > >>>    html version of individual reference manuals in
>     addition to the
>     > >>>    pdf that are already available on the package landing
>     pages.
>     > >>>
>     > >>>    Is there anything I or any other members of the
>     community could
>     > >>>    help with to get this up and running?
>     > >>>
>     > >>>    Best wishes,
>     > >>>
>     > >>>    Laurent
>     > >>>
>     > >>>    _______________________________________________
>     > >>>    Bioc-devel at r-project.org mailing list
>     > >>>    https://stat.ethz.ch/mailman/listinfo/bioc-devel
>     > >>>
>     > >>>    _______________________________________________
>     > >>>    Bioc-devel at r-project.org mailing list
>     > >>>    https://stat.ethz.ch/mailman/listinfo/bioc-devel
>     > >>>
>     > >>>
>     > >>>    This email message may contain legally privileged and/
>     or
>     > >>>    confidential information. If you are not the intended
>     > >>>    recipient(s), or the employee or agent responsible for
>     the
>     > >>>    delivery of this message to the intended recipient(s),
>     you are
>     > >>>    hereby notified that any disclosure, copying,
>     distribution, or
>     > >>>    use of this email message is prohibited. If you have
>     received
>     > >>>    this message in error, please notify the sender
>     immediately by
>     > >>>    e-mail and delete this email message from your
>     computer. Thank
>     > >>>    you.
>     > >>>    _______________________________________________
>     > >>>    Bioc-devel at r-project.org mailing list
>     > >>>    https://stat.ethz.ch/mailman/listinfo/bioc-devel
>     > >
>     > >
>     > > --
>     > > Laurent Gatto | @lgatt0
>     > > http://cpu.sysbiol.cam.ac.uk/
>     > > http://lgatto.github.io/
>     > >
>     > > _______________________________________________
>     > > Bioc-devel at r-project.org mailing list
>     > > https://stat.ethz.ch/mailman/listinfo/bioc-devel
>     >
>     > _______________________________________________
>     > Bioc-devel at r-project.org mailing list
>     > https://stat.ethz.ch/mailman/listinfo/bioc-devel
>
>         [[alternative HTML version deleted]]
>
>     _______________________________________________
>     Bioc-devel at r-project.org mailing list
>     https://stat.ethz.ch/mailman/listinfo/bioc-devel


-- 
Laurent Gatto | @lgatt0
http://cpu.sysbiol.cam.ac.uk/
http://lgatto.github.io/



More information about the Bioc-devel mailing list