[ESS-bugs] ESS 13.09-1 with old R versions fails pretty badly

[Martin Maechler]

> Hi Martin, 

> Nice job. Thanks. 

> This is quite a change, so we will have to postpone the release for yet
> another week.

Yes indeed, or more:
Currently 13.09-1 fails with typically all R version < 3.0.0 for me,
and 13.09 does not... See below.

> I have a couple of concerns so far. 


>      @@ -87,6 +89,9 @@ distclean: clean
>       ess-custom.el: ../VERSION
>       	perl -pi -e 's/".*"/"$(ESSVERSION)"/ if /ess-version/' $@
      
>      +ess-r-d.el: ../etc/ESSR-VERSION
>      +	sed -i -e '/let.*ESSR-version/s/"[-.0-9]*"/"$(ESSR_VER)"/' $@
>      +
      
>       (defun ess--R-load-ESSR ()
>      -  "LOAD/INSTALL/UPDATE ESSR"
>      -  (let* ((ESSR-version "1.0.1") ; <- FIXME: smart way to automate this?
>      +  "Load/INSTALL/Update ESSR"
>      +  (let* ((ESSR-version "1.0.2") ; <- This is auto-updated via make
>                (up-to-date (ess-boolean-command


> This will not work for those who don't build. 

That's no problem.
All of us (ESS core) can build. 
People working from svn or git *must* build ... that's been true
in the past too, as we build quite a few doc things as well,
that are not in svn (but in the release) for good reason.

For release, the release master
*must* build anyway, so the release tarball will be fine always.

> It will also not work with
> MELPA. I will fix that by parsing the ESSR/DESCRIPTION directly.

Do we really care for MELPA for *UN*released versions of ESS ??
I definitely don't want to.
Remember we have releases (and may have prereleases if desired),
and that's what 99.95% of the users should start from.
The others (even if they are more than 0.05% must build, yes,
that's true for all free software I've known.
     
>      all: ESSR-VERSION $(ESSR_tarball) library/ESSR
     
>      ## happens "above" as it is need also in ../lisp/ :
>      ESSR-VERSION: $(ESSR_FILES)
>      	(cd .. ; make etc/ESSR-VERSION)
     
>      $(ESSR_tarball): $(ESSR_FILES)
>      	R CMD build ESSR 
>      library/ESSR: $(ESSR_tarball)
>      	R CMD INSTALL -l library ESSR
     

> So we are switching to ESS specific library? Good. 

No... well not typically.  The above library/ESSR is another way
to provide a workaround when

   install.packages("ESSR...")
   library(ESSR)

is failing.  But I now think this will all get moot, see below.

> But, it should work also without make, and that means that we
> need to ship library/ESSR with SVN.

No!  SVN/Git is for people who can build -- in any case.
*We* and only we do release and the release contains everything.

People working from svn must be able to run 'make', really.
We can pretty easily also do explicit "pre-release" tarballs if
there are really people who cannot run make.

> This pre-installed version should be built with a reasonably old
> version of R like 2.15 or even 2.14. Otherwise, as you pointed out last
> time, users will see warnings.

Well, yesterday on the way home I've started pondering further
and later last night I've spent a sleepless hour (or more),
thinking about all the different ways that install.packages()
and library() differ from R versions and installed packages also
often fail when not used with the correct R version,
and in future R may still change in the workings of
install.packages() ..

and sorry, but I've come to the conclusion that I very strongly
would like us to revert from ESSR package to the way ESS 13.09
worked. Source the R code into an environment that we add to the
search() path so it looks pretty similar to an attached package
from the user point of view.

In practice there are so many small ways that the package
installation can fail, partly fail, almost work, etc, and the
same with the subsequent library() call, and we want to do this
basically without the user having to do anything.

Also, for 95%-99% of users packages are from a repository they
can install / update from, ask for help about on CRAN or
bioconductor, and all that is different for ESSR anyway, as you
very correctly diagnosed, Vitalie.

The advantage of having such non-repository package over what
we've had in 13.09 seems really small to me,
 - speed in loading, once the package is installed for the correct version of R
 - (one help page which still has to be complemented by ESS documentation)
compared to the added brittleness and non-robustness in the
whole setup.

I know, I had advocated the use of a package ESSR quite a bit
for a long time, but now feel that it almost only adds
complexity, and dependencies of ESS on the exact setup of R,
that I think we should revert that part.
Note also that we see popping up of other non-free and
alternative versions of "almost R", and using those from ESS
will be easy {and one of our selling points against Rstudio} if
we don't depend on many "R internals".

Yes, I feel bad about this; I've spent my several hours
yesterday that are mostly lost, but of course Vitalie, you've
spent more than me on this.
But I strongly believe we save ourselves and even more our users
much  hair tearing in the future if we keep this as simple as we
did in ESS 13.09.

The small speed loss (we are talking fractions of a second anyway),
could be eliminated in the future by providing optional save()
and load()ing of the ESSR code.


Martin





>  >>> Martin Maechler on Tue, 29 Oct 2013 19:49:15 +0100 wrote:

>  > I've worked on this,  but got "hung up" on the FIXME in
>  >      ess-r-d.el
>  > to autogenerate the ESSR package version number used there,
>  > and committed my current changes... should be working but
>  > are unfinished, notably for the first simple workaround source()
>  > workaround.

>  > More is planned in two days only (as I have to prepare other
>  > stuff urgently for Thursday morning and am already busy with
>  > something else a big part of tomorrow).

>  > Martin

>  > _______________________________________________
>  > ESS-bugs ESS-bugs at r-project.org
>  > https://stat.ethz.ch/mailman/listinfo/ess-bugs

>  > _______________________________________________
>  > ESS-core list: https://stat.ethz.ch/mailman/listinfo/ess-core