[Bioc-devel] Package reference manuals in html
Laurent Gatto
lg390 at cam.ac.uk
Fri Dec 23 23:38:10 CET 2016
Dear all,
I ended up implementing the branch specific .gitingore hack described in
on Stackoverflow
http://stackoverflow.com/questions/29579546/git-excludesfile-for-a-branch/29583813#29583813
My notes and implementation are available here
http://lgatto.github.io/branch-specific-gitignore/
Best wishes,
Laurent
On 23 December 2016 18:14, Laurent Gatto wrote:
> Dear Sean and Robert,
>
> On 23 December 2016 17:24, Robert M. Flight wrote:
>
>> Yes, this seems like a time where using a non-standard site directory
>> on Github is useful, or as Sean said, using a separate branch to serve
>> the html content.
>
> It's not clear to me how a separate branch is the solution. In git-svn,
> devel, release_3_4, ... are just that, different branches. How would you
> set a gh-pages branch that contains a single dir so that it doesn't
> conflict with master?
>
> If I could set branch-specific directives in my .gitignore file, then I
> would set this up for the Bioc branches. This seems to suggest it is
> possible, but I haven't tried yet
>
> http://stackoverflow.com/questions/1836742/using-git-how-do-i-ignore-a-file-in-one-branch-but-have-it-committed-in-another
>
> Would it be possible to set something on the svn side to ignore/delete
> the docs dir automatically, independently of what git does?
>
> Laurent
>
>> -Robert
>>
>> On Fri, Dec 23, 2016 at 12:19 PM Sean Davis <seandavi at gmail.com> wrote:
>>
>> Github allows you to set the branch for the docs directory if I
>> recall. Perhaps a separate branch with a docs directory (not
>> master) is a viable way to go?
>>
>> Sean
>>
>> > On Dec 23, 2016, at 12:16 PM, Laurent Gatto <lg390 at cam.ac.uk>
>> wrote:
>> >
>> >
>> > There's actually another side-effect for Bioconductor. The
>> package
>> > website is (by default) generated in the package's ./docs
>> > directory. This is very handy, as it can be set directly on
>> Github as a
>> > Github page and becomes available as https://username.github.io/
>> pkgname.
>> >
>> > This also means that the docs directory (which is about 5.5M for
>> > MSnbase) ends up on hedgehog. It is easy for it not to be part of
>> the
>> > package build artefact using .Rbuildignore, but I am not sure how
>> to
>> > easily push it to github but not hedgehog when using git-svn.
>> >
>> > Laurent
>> >
>> > On 23 December 2016 16:36, Laurent Gatto wrote:
>> >
>> >> Dear all,
>> >>
>> >> I'm following up re my online references suggestion with my
>> recent
>> >> experience with Hadley's pkgdown package
>> >>
>> >> https://github.com/hadley/pkgdown
>> >>
>> >> It doesn't address the cross-package issue (which is a difficult
>> one
>> >> anyway), but does pretty much everything else (with some caveats
>> though,
>> >> see below).
>> >>
>> >> Here are 2 examples
>> >>
>> >> - MSnbase: http://lgatto.github.io/MSnbase/
>> >> - hpar: http://lgatto.github.io/hpar/
>> >>
>> >> It uses the REAMDE file as index page, creates html documents
>> for all Rd
>> >> files in man, an article tab for vignettes (but see below) and a
>> News
>> >> tab (but see below)
>> >>
>> >> The biggest caveats is that only Rmd vignettes are taken into
>> account;
>> >> Rnw are completely ignored (they don't show up at all in the
>> Articles
>> >> tab). This is not going to be tackled by the developer [1].
>> >>
>> >> [1] https://github.com/hadley/pkgdown/issues/220
>> >>
>> >> I had a quick look at the code and patching pkgdown (and
>> probably
>> >> rmarkdown) to build Rnw/pdf vignettes would take too much time I
>> could
>> >> devote at the moment. I would be satisfied if the Rnw were not
>> build but
>> >> at least there were links in the Articles tab pointing to the
>> vignettes
>> >> Bioconductor landing pages. On the other hand, I am migrating to
>> >> BiocStyle's html_document2 with the nice floating table of
>> contents...
>> >>
>> >> Regarding the News tab, only NEWS.md (markdown format) are
>> considered;
>> >> NEWS in Rd are ignored too.
>> >>
>> >> Hope this helps.
>> >>
>> >> Best wishes,
>> >>
>> >> Laurent
>> >>
>> >> On 16 March 2016 23:33, Andrzej Oleś wrote:
>> >>
>> >>> Hi all,
>> >>>
>> >>> I had a discussion earlier today with Martin and Dan on
>> providing
>> >>> online man pages for Bioconductor packages. As we dived into
>> >>> implementation details, it turned out that this idea is a
>> little bit
>> >>> more complex and resource-intensive than originally
>> anticipated.
>> >>>
>> >>> The main problem in generating man pages in a repository-wide
>> fashion
>> >>> seems to be the cross-linking of packages. Briefly, in order to
>> >>> generate the links, apparently one needs to generate the html
>> pages in
>> >>> an R installation which is aware of the other packages. For
>> example,
>> >>> the Rd macro \linkS4class{ClassName} takes as argument only the
>> class
>> >>> name, and the corresponding package containing the class
>> definition is
>> >>> "automagically" resolved by R. I'm not sure how this could be
>> done
>> >>> manually, on a per-package basis. So by the end of the day, in
>> order to
>> >>> generate static man pages, we would need to maintain a complete
>> BioC
>> >>> repo installation, possibly on a system with the
>> --enable-prebuilt-html
>> >>> configure option. Unfortunately, it seems unfeasible to exploit
>> the
>> >>> build servers for this, as this would significantly increase
>> the
>> >>> computational burden. This is because currently only around 2/5
>> of all
>> >>> software and data packages are actually being installed by the
>> build
>> >>> system. The rest which does not have any reverse dependencies
>> is
>> >>> skipped. Installing the remaining 3/5 of packages on a regular
>> basis,
>> >>> not to mention the heavy annotation packages, is a little bit
>> of an
>> >>> overkill. So piggy-backing on the existing infrastructure
>> doesn't seem
>> >>> realistic.
>> >>>
>> >>> On top of this, even if we would have access to a machine with
>> a
>> >>> complete, up-to-date BioC installation (maybe by just updating
>> the
>> >>> packages after the repo gets rebuild rather than re-installing
>> them
>> >>> each time from scratch), it remains an open question how
>> "external"
>> >>> links to, let's say, CRAN packages, or even base R packages,
>> should be
>> >>> handled.
>> >>>
>> >>> A lightweight and easy to implement alternative for those
>> willing to
>> >>> share self-hosted documentation of their packages, could be to
>> provide
>> >>> in the package DESCRIPTION file a "Documentation" field
>> containing a
>> >>> link to external resource, which would then appear on the
>> package
>> >>> landing page next to the vignettes and pdf manual. The obvious
>> >>> downsides of this solution are: 1. no package cross-links, and
>> 2. the
>> >>> burden of keeping the documentation in sync with the package
>> version on
>> >>> BioC would be in maintainer's hands...
>> >>>
>> >>> I will try to contact the authors of rdocumentation.org - maybe
>> they
>> >>> have some useful comments or even code which they would be
>> willing to
>> >>> share. In any case, it would be good to know what their
>> experience is
>> >>> and why did they stop maintaining their service. Maybe the BioC
>> >>> community could jump in and help them to resolve the
>> bottlenecks and
>> >>> keep the website up to date.
>> >>>
>> >>> Cheers,
>> >>> Andrzej
>> >>>
>> >>>
>> >>> On Tue, Mar 8, 2016 at 4:36 PM, Andrzej Oleś <
>> andrzej.oles at gmail.com>
>> >>> wrote:
>> >>>
>> >>> Hi Martin,
>> >>>
>> >>> thank you for your suggestions - I would be happy to
>> contribute to
>> >>> this! I could help with developing the scripts for
>> generating man
>> >>> pages, and integrating them with the website layout.
>> >>>
>> >>> As for rendering the man pages, I suggest that we try a
>> similar
>> >>> approach to the one used by knitr::knit_rd() rather than
>> plain
>> >>> tools::Rd2HTML(). It has the advantage that the examples are
>> >>> actually run, and the results, e.g. plots, are included in
>> the
>> >>> output documents. I hope you can appreciate the added value
>> by
>> >>> comparing the following man page rendered using
>> tools::Rd2HTML()
>> >>> and knitr::knit_rd(), respectively.
>> >>> http://www.huber.embl.de/users/aoles/man/Image.html
>> >>> http://www.huber.embl.de/users/aoles/man/Image-knitr.html
>> >>> Regarding the additional dependencies: we kind of already
>> rely on
>> >>> knitr when compiling vignettes, so this this shouldn't add
>> much to
>> >>> the maintenance burden.
>> >>>
>> >>> Cheers,
>> >>> Andrzej
>> >>>
>> >>> On Fri, Mar 4, 2016 at 2:20 PM, Morgan, Martin <
>> >>> Martin.Morgan at roswellpark.org> wrote:
>> >>>
>> >>> One thing about accessing the html versions locally
>> (e.g., via
>> >>> ? with options(help_type="html")] or help.start() or
>> Rstudio)
>> >>> is that you get the version relevant to your R /
>> Bioconductor,
>> >>> rather than whatever is at the top of google; I guess
>> the same
>> >>> applies to the pdf versions, and the reason that there
>> isn't
>> >>> more current confusion is because the online pdf
>> versions are
>> >>> not as useful as the off-line help system.
>> >>>
>> >>> I think Laurent was interested in an integration of help
>> pages
>> >>> across packages (which is the appeal of
>> rdocumentation.org?),
>> >>> not just rendering the help pages in html rather than
>> pdf? An
>> >>> integration of help pages would definitely be a big job
>> with
>> >>> substantial development and maintenance; we will not be
>> >>> undertaking this ourselves.
>> >>>
>> >>> For the more limited case of adding a (directory of)
>> html files
>> >>> for the the manual, it's not impossible that we could
>> find the
>> >>> resources to do this in the next 6 months.
>> >>>
>> >>> One intermediate and helpful step for those willing to
>> help
>> >>> would be to develop the code to process help pages into
>> a style
>> >>> consistent with the bioconductor web site. One place
>> where this
>> >>> could be implemented would be the BiocStyle package
>> (https://
>> >>> github.com/Bioconductor-mirror/BiocStyle but hmm, seems
>> like
>> >>> there's a slightly out of sync version at https://
>> github.com/
>> >>> Bioconductor/BiocStyle that would be more
>> convenient...).
>> >>> Perhaps this really means only developing a css style
>> sheet and
>> >>> R's tools::Rd2HTML() (I'm very reluctant to introduce
>> >>> dependencies into the build system, and am very
>> conservative
>> >>> about inclusion of fancy features in the html -- these
>> become
>> >>> significant maintenance burdens moving forward).
>> >>>
>> >>> The web site is generated by https://github.com/
>> Bioconductor/
>> >>> bioconductor.org, with the style sheet at https://
>> github.com/
>> >>> Bioconductor/bioconductor.org/blob/master/assets/style/
>> >>> bioconductor.css. The package landing pages are
>> templated using
>> >>> layouts/_bioc_views_package_detail.html. The idea would
>> be to
>> >>> end up with layouts/_bioc_man_index.html and
>> >>> _bioc_man_body.html that wrapped output from BiocStyle
>> in the
>> >>> overall bioc page.
>> >>>
>> >>> The implementation suggestions above are just a sketch
>> and
>> >>> could be quite misguided. If there's interest then
>> probably we
>> >>> should set up a hangout to discuss in a little more
>> detail.
>> >>>
>> >>> Martin
>> >>>
>> >>> ________________________________________
>> >>> From: Bioc-devel <bioc-devel-bounces at r-project.org> on
>> behalf
>> >>> of Hartley, Stephen (NIH/NHGRI) [F] <
>> stephen.hartley at nih.gov>
>> >>> Sent: Wednesday, March 2, 2016 11:46 AM
>> >>> To: Laurent Gatto; bioc-devel
>> >>> Subject: Re: [Bioc-devel] Package reference manuals in
>> html
>> >>>
>> >>> I'd like to second this. Currently Bioconductor hosts
>> the pdf
>> >>> reference manuals, but those are often sub-ideal. The
>> page
>> >>> breaks make it harder to read, the fixed width basically
>> makes
>> >>> it either too small or too big depending on your
>> display, you
>> >>> can't navigate cross-package links, and in general using
>> >>> paper-formatted software documentation is just poor
>> form.
>> >>>
>> >>> Yihui, the creator of knitr, has a blog post where he
>> shows how
>> >>> to do this. There are a lot of ways to do this, and it's
>> >>> generally pretty straightforward.
>> >>> http://yihui.name/en/2012/10/build-static-html-help/
>> >>>
>> >>> You can also use a function in knitr, knit_rd(), which
>> builds
>> >>> the examples as well and inserts the output right onto
>> the
>> >>> page. That's what I used to make the docs for QoRTs
>> (http://
>> >>> hartleys.github.io/QoRTs/Rhtml/index.html) and
>> JunctionSeq (
>> >>> http://hartleys.github.io/JunctionSeq/Rhtml/index.html).
>> >>>
>> >>> Or you can use the staticdocs package, which does the
>> same
>> >>> basic thing but prettier (see ggplot2's docs: http://
>> >>> docs.ggplot2.org/current/)
>> >>>
>> >>> The nuclear option, of course, is to do what CRAN does
>> and
>> >>> rebuild R on (one of) the servers using the
>> >>> --enable-prebuilt-html configure option. That might
>> affect
>> >>> other things, though, and might not be ideal.
>> >>>
>> >>> Does any of this seem like a viable option for
>> Bioconductor? I
>> >>> think it could be an incredibly valuable resource for
>> the
>> >>> community. Are there any technical issues that haven't
>> been
>> >>> considered in the above?
>> >>>
>> >>> Regards,
>> >>> Steve Hartley
>> >>>
>> >>> -----Original Message-----
>> >>> From: Laurent Gatto [mailto:lg390 at cam.ac.uk]
>> >>> Sent: Tuesday, March 01, 2016 6:42 AM
>> >>> To: bioc-devel
>> >>> Subject: [Bioc-devel] Package reference manuals in html
>> >>>
>> >>>
>> >>> Dear all,
>> >>>
>> >>> I find the http://www.rdocumentation.org/ site very
>> useful to
>> >>> refer to nicely formatted online man pages individually.
>> >>> Unfortunately, this resource is terribly outdated and
>> not
>> >>> maintained anymore.
>> >>>
>> >>> I was wondering if Bioconductor had any interest in
>> serving an
>> >>> html version of individual reference manuals in addition
>> to the
>> >>> pdf that are already available on the package landing
>> pages.
>> >>>
>> >>> Is there anything I or any other members of the
>> community could
>> >>> help with to get this up and running?
>> >>>
>> >>> Best wishes,
>> >>>
>> >>> Laurent
>> >>>
>> >>> _______________________________________________
>> >>> Bioc-devel at r-project.org mailing list
>> >>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>> >>>
>> >>> _______________________________________________
>> >>> Bioc-devel at r-project.org mailing list
>> >>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>> >>>
>> >>>
>> >>> This email message may contain legally privileged and/or
>> >>> confidential information. If you are not the intended
>> >>> recipient(s), or the employee or agent responsible for
>> the
>> >>> delivery of this message to the intended recipient(s),
>> you are
>> >>> hereby notified that any disclosure, copying,
>> distribution, or
>> >>> use of this email message is prohibited. If you have
>> received
>> >>> this message in error, please notify the sender
>> immediately by
>> >>> e-mail and delete this email message from your computer.
>> Thank
>> >>> you.
>> >>> _______________________________________________
>> >>> Bioc-devel at r-project.org mailing list
>> >>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>> >
>> >
>> > --
>> > Laurent Gatto | @lgatt0
>> > http://cpu.sysbiol.cam.ac.uk/
>> > http://lgatto.github.io/
>> >
>> > _______________________________________________
>> > Bioc-devel at r-project.org mailing list
>> > https://stat.ethz.ch/mailman/listinfo/bioc-devel
>>
>> _______________________________________________
>> Bioc-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
--
Laurent Gatto | @lgatt0
http://cpu.sysbiol.cam.ac.uk/
http://lgatto.github.io/
More information about the Bioc-devel
mailing list