[R] static vs. lexical scope

[Richard O'Keefe]

I didn't say R's rules were "a mystery" or "overly complicated", but
that they are "weird".
When I started trying to compile R, with() did not exist, if I
remember correctly.
Half the point of compiling is to do variable lookups at compile time.
In an example like
  function (...) {
     use x
     x <- ...
     use x
}
it's easy enough to tell one x from the other.  Now change it slightly:
  function (...) {
     use x
     if (...) x <- ...
     use x
}
Now we cannot tell whether the second use refers to an inner x or the outer one.
Consider next
> f <- function () {
+    g <- function () { x }
+    x <- 2
+    g()'
+ }
> x <- 1
> f()
Here's the snag: the x in g does not refer to the x that is visible at
the time when
g is *defined* (as in every other language with lexical scope I've
used) but to the
x that is visible when g is *called*.  In other languages, you can 'trim' the
lexical environment of a closure to just the variables that the
closure mentions.
In R you cannot.

It gets nastier.  A function can *remove* a variable from another
function's frame.
(See ?rm and note the 'envir' argument and then read ?sys.frame.)

The best I was able to come up with was a scheme where each statically visible
variable had a slot for its value and a reserved object.
Mention of x =>
   t := static slot for x
   if t is the reserved object: t := full lookup("x")
Assignment to x =>
  static slot for  := t

That doesn't work with "with", and then of course there are "active
bindings" nowadays,
see ?bindenv.

On Fri, 27 Sep 2019 at 02:59, Duncan Murdoch <murdoch.duncan using gmail.com> wrote:
>
> On 26/09/2019 9:44 a.m., Richard O'Keefe wrote:
> > Actually, R's scope rules are seriously weird.
> > I set out to write an R compiler, wow, >20 years ago.
> > Figured out how to handle optional and keyword parameters efficiently,
> > figured out a lot of other things, but choked on the scope rules.
> > Consider
> >
> >> x <- 1
> >> f <- function () {
> > +   a <- x
> > +   x <- 2
> > +   b <- x
> > +   c(a=a, b=b)
> > + }
> >> f()
> > a b
> > 1 2
> >> x
> > [1] 1
> >
> > It's really not clear what is going on here.
>
> This is all pretty clear:  in the first assignment, x is found in the
> global environment, because it does not exist in the evaluation frame.
> In the second assignment, a new variable is created in the evaluation
> frame.  In the third assignment, that new variable is used to set the
> value of b.
>
> > However, ?assign can introduce new variables into an environment,
> > and from something like
> >    with(df, x*2-y)
> > it is impossible for a compiler to tell which, if either, of x and y is to
> > be obtained from df and which from outside.  And of course ?with
> > is just a function:
> >
> >> df <- data.frame(y=24)
> >> w <- with
> >> w(df, x*2-y)
> > [1] -22
> >
> > So you cannot in general tell *which* function can twist the environment
> > in which its arguments will be evaluated.
>
> It's definitely hard to compile R because of the scoping rules, but that
> doesn't make the scoping rules unclear.
>
> > I got very tired of trying to explore a twisty maze of documentation and
> > trying to infer a specification from examples.  I would come up with an
> > ingenious mechanism for making the common case tolerable and the
> > rare cases possible, and then I'd discover a bear trap I hadn't seen.
> > I love R, but I try really hard not to be clever with it.
>
> I think the specification is really pretty simple.  I'm not sure it is
> well documented anywhere, but I think I understand it pretty well, and
> it doesn't seem overly complicated to me.
>
> > So while R's scoping is *like* lexical scoping, it is *dynamic* lexical
> > scoping, to coin a phrase.
>
> I'd say it is regular lexical scoping but with dynamic variable
> creation. Call that dynamic lexical scoping if you want, but it's not
> really a mystery.
>
> Duncan Murdoch
>
> >
> > On Thu, 26 Sep 2019 at 23:56, Martin Møller Skarbiniks Pedersen
> > <traxplayer using gmail.com> wrote:
> >>
> >> On Wed, 25 Sep 2019 at 11:03, Francesco Ariis <fa-ml using ariis.it> wrote:
> >>>
> >>> Dear R users/developers,
> >>> while ploughing through "An Introduction to R" [1], I found the
> >>> expression "static scope" (in contraposition to "lexical scope").
> >>>
> >>> I was a bit puzzled by the difference (since e.g. Wikipedia conflates the
> >>> two) until I found this document [2].
> >>
> >>
> >> I sometimes teach a little R, and they might ask about static/lexical scope.
> >> My short answer is normally that S uses static scoping and R uses
> >> lexical scoping.
> >> And most all modern languages uses lexical scoping.
> >> So if they know Java, C, C# etc. then the scoping rules for R are the same.
> >>
> >> I finally says that it is not a full answer but enough for most.
> >>
> >> Regards
> >> Martin
> >>
> >> ______________________________________________
> >> R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> >> https://stat.ethz.ch/mailman/listinfo/r-help
> >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> >> and provide commented, minimal, self-contained, reproducible code.
> >
> > ______________________________________________
> > R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> >
>