[Rd] Recent and upcoming changes to R-devel

Tue Jul 5 16:17:25 CEST 2011

Dear Duncan,

On 07/05/2011 03:25 PM, Duncan Murdoch wrote:
> On 05/07/2011 6:52 AM, Tobias Verbeke wrote:
>> L.S.
>>
>> On 07/05/2011 02:16 AM, Mark.Bravington at csiro.au wrote:
>> > I may have misunderstood, but:
>> >
>> > Please could we have an optional installation that does not*not*
>> byte-compile base and recommended?
>> >
>> > Reason: it's not possible to debug byte-compiled code-- at least not
>> with the 'debug' package, which is quite widely used. I quite often
>> end up using 'mtrace' on functions in base/recommended packages to
>> figure out what they are doing. And sometimes I (and others)
>> experiment with changing functions in base/recommended to improve
>> functionality. That seems to be harder with BC versions, and might
>> even be impossible, as best I can tell from hints in the documentation
>> of 'compile').
>> >
>> > Personally, if I had to choose only one, I'd rather live with the
>> speed penalty from not byte-compiling. But of course, if both are
>> available, I could install both.
>>
>> I completely second this request. All speed improvements and the byte
>> compiler in particular are leaps forward and I am very grateful and
>> admiring towards the people that make this happen.
>>
>> That being said, 'moving away' from the sources (with the lazy loading
>> files and byte-compilation) may be a step back for R package developers
>> that (during development and maybe on separate development installations
>> [as opposed to production installations of R]) require
>> the sources of all packages to be efficient in their work.
>>
>> As many of you know there is an open source Eclipse/StatET visual
>> debugger ready and for that application as well (similar to Mark's
>> request) presence of non-compiled code is highly desirable.
>>
>> For the particular purpose of debugging R packages, I would even plead
>> to go beyond the current options and support the addition of an
>> R package install option that allows to include the sources (e.g. in
>> a standard folder Rsrc/) in installed packages.
>>
>> I am fully aware that one can always fetch the source tarballs from
>> CRAN for that purpose, but it would be much more easy if a simple
>> installation option could put the R sources of a package in a separate
>> folder [or archive inside an existing folder] such that R development
>> tools (such as the Eclipse/StatET IDE) can offer inspection of sources
>> or display them (e.g. during debugging) out of the box.
>>
>> If one has the srcref, one can always load the absolutely correct source
>> code this way, even if one doesn't know the parent function with
>> the source attribute.
>>
>> Any comments?
>
> I think these requests have already been met. If you modify the body of
> a closure (as trace() does), then the byte compiled version is
> discarded, and you go back to the regular interpreted code. If you
> install packages with the R_KEEP_PKG_SOURCE=yes environment variable
> set, the you keep all source for all functions. (It's attached to the
> function itself, not as a file that may be out of date.) It's possible
> that byte compiling turns off R_KEEP_PKG_SOURCE, but that is something
> that is either easily fixed, or avoided by re-installing without byte
> compiling.

Many thanks for your reaction. Is the R_KEEP_PKG_SOURCE=yes environment
variable also supported during R installation ?

I hope I'm not overlooking anything, but when compiling

ftp://ftp.stat.math.ethz.ch/Software/R/R-devel.tar.gz

a few minutes ago I encountered the following issue:

[...]

building package 'tools'
mkdir -p -- ../../../library/tools
make[4]: Entering directory `/home/tobias/rAdmin/R-devel/src/library/tools'
mkdir -p -- ../../../library/tools/R
mkdir -p -- ../../../library/tools/po
make[4]: Leaving directory `/home/tobias/rAdmin/R-devel/src/library/tools'
make[4]: Entering directory `/home/tobias/rAdmin/R-devel/src/library/tools'
make[5]: Entering directory 
`/home/tobias/rAdmin/R-devel/src/library/tools/src'
making text.d from text.c
making init.d from init.c
making Rmd5.d from Rmd5.c
making md5.d from md5.c
gcc -std=gnu99 -I../../../../include  -I/usr/local/include 
-fvisibility=hidden -fpic  -g -O2 -c text.c -o text.o
gcc -std=gnu99 -I../../../../include  -I/usr/local/include 
-fvisibility=hidden -fpic  -g -O2 -c init.c -o init.o
gcc -std=gnu99 -I../../../../include  -I/usr/local/include 
-fvisibility=hidden -fpic  -g -O2 -c Rmd5.c -o Rmd5.o
gcc -std=gnu99 -I../../../../include  -I/usr/local/include 
-fvisibility=hidden -fpic  -g -O2 -c md5.c -o md5.o
gcc -std=gnu99 -shared -L/usr/local/lib64 -o tools.so text.o init.o 
Rmd5.o md5.o -L../../../../lib -lR
make[6]: Entering directory 
`/home/tobias/rAdmin/R-devel/src/library/tools/src'
make[6]: `Makedeps' is up to date.
make[6]: Leaving directory 
`/home/tobias/rAdmin/R-devel/src/library/tools/src'
make[6]: Entering directory 
`/home/tobias/rAdmin/R-devel/src/library/tools/src'
mkdir -p -- ../../../../library/tools/libs
make[6]: Leaving directory 
`/home/tobias/rAdmin/R-devel/src/library/tools/src'
make[5]: Leaving directory 
`/home/tobias/rAdmin/R-devel/src/library/tools/src'
make[4]: Leaving directory `/home/tobias/rAdmin/R-devel/src/library/tools'
Error in parse(n = -1, file = file) :
   function is too long to keep source (at line 2967)
Error: unable to load R code in package ‘tools’
Execution halted

[...]

tobias at openanalytics:~/rAdmin$ echo $R_KEEP_PKG_SOURCE
yes

I do not have this issue when R_KEEP_PKG_SOURCE is set
to 'false' during compilation.

Best,
Tobias

>> P.S. One could even consider a post-install option e.g. to add 'real'
>> R sources (and source references) to Windows packages (which are by
>> definition already 'installed' and for which such information is not
>> by default included in the CRAN binaries of these packages).
>>
>> >> > Prof Brian Ripley wrote:
>> >> > There was an R-core meeting the week before last, and various
>> planned
>> >> > changes will appear in R-devel over the next few weeks.
>> >> >
>> >> > These are changes planned for R 2.14.0 scheduled for Oct 31. As we
>> >> > are sick of people referring to R-devel as '2.14' or '2.14.0', that
>> >> > version number will not be used until we reach 2.14.0 alpha. You
>> >> > will be able to have a package depend on an svn version number when
>> >> > referring to R-devel rather than using R (>= 2.14.0).
>> >> >
>> >> > All packages are installed with lazy-loading (there were 72 CRAN
>> >> > packages and 8 BioC packages which opted out). This means that the
>> >> > code is always parsed at install time which inter alia simplifies
>> the
>> >> > descriptions. R 2.13.1 RC warns on installation about packages which
>> >> > ask not to be lazy-loaded, and R-devel ignores such requests (with a
>> >> > warning).
>> >> >
>> >> > In the near future all packages will have a name space. If the
>> >> > sources do not contain one, a default NAMESPACE file will be added.
>> >> > This again will simplify the descriptions and also a lot of internal
>> >> > code. Maintainers of packages without name spaces (currently 42% of
>> >> > CRAN) are encouraged to add one themselves.
>> >> >
>> >> > R-devel is installed with the base and recommended packages
>> >> > byte-compiled (the equivalent of 'make bytecode' in R 2.13.x, but
>> >> > done less inefficiently). There is a new option R CMD INSTALL
>> >> > --byte-compile to byte-compile contributed packages, but that
>> remains
>> >> > optional.
>> >> > Byte-compilation is quite expensive (so you definitely want to do it
>> >> > at install time, which requires lazy-loading), and relatively few
>> >> > packages benefit appreciably from byte-compilation. A larger number
>> >> > of packages benefit from byte-compilation of R itself: for example
>> >> > AER runs its checks 10% faster. The byte-compiler technology is
>> >> > thanks to Luke Tierney.
>> >> >
>> >> > There is support for figures in Rd files: currently with a
>> first-pass
>> >> > implementation (thanks to Duncan Murdoch).
>> > ______________________________________________
>> > R-devel at r-project.org mailing list
>> > https://stat.ethz.ch/mailman/listinfo/r-devel
>> >
>>
>> ______________________________________________
>> R-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>