[Rd] attributes of environments

[Gabor Grothendieck]

On 7/5/06, Duncan Murdoch <murdoch at stats.uwo.ca> wrote:
> On 7/5/2006 8:06 PM, Gabor Grothendieck wrote:
> > On 7/5/06, Duncan Murdoch <murdoch at stats.uwo.ca> wrote:
> >> On 7/5/2006 4:33 PM, Gabor Grothendieck wrote:
> >>> On 7/5/06, Duncan Murdoch <murdoch at stats.uwo.ca> wrote:
> >>>> On 7/5/2006 3:47 PM, Gabor Grothendieck wrote:
> >>>>> On 7/5/06, Duncan Murdoch <murdoch at stats.uwo.ca> wrote:
> >>>>>> On 7/5/2006 2:23 PM, Gabor Grothendieck wrote:
> >>>>>>  I think by this time I have shown that subclassing of
> >>>>>>> environments does not work yet it could if it were designed differently
> >>>>>>> and furthermore there are significant problems with the workarounds.
> >>>>>> I don't think you've shown that subclassing of environments doesn't
> >>>>>> work.  You have an example that shows that shows that R implements
> >>>>>> Henrik's "Case 2" rather than his "Case 1", but as Thomas and I said,
> >>>>>> that really has nothing to do with subclassing.
> >>>>>>
> >>>>>> Subclassing is about defining a new class, not about copying objects.
> >>>>>> You can (and did!) define a new class which inherits from the
> >>>>>> environment class.
> >>>>> But by subclassing in the way allowed one comes up with something that
> >>>>> is not useful.
> >>>> You haven't shown that.  Show an example where you define a new class
> >>>> that should inherit from environment but doesn't.
> >>>>
> >>>> All you've shown so far is that when you try to change the class of an
> >>>> object to a new class, it appears that the class of another object also
> >>>> changes.  (The explanation being that they are really just different
> >>>> names for the same object.)
> >>> But that is not how oo works.  When one defines a child its a
> >>> delta to the parent.  It does not change the parent.
> >> There are dozens of different definitions of "object oriented", but
> >> generally in the ones I know about, subclassing is something you do to a
> >> class, not to an object.  (In some languages classes are objects, but
> >> not all objects are classes.)
> >>
> >> It is possible to have an object with class c('myenv', 'environment').
> >> As far as I know, methods applied to that object will dispatch to the
> >> myenv method if one is defined, or to the environment method or default
> >> method if not.  That's exactly how things should work, and that's how
> >> they worked in the example you showed.
> >>
> >> Because environments have unusual semantics, it wouldn't surprise me
> >> very much if there were some errors in the implementation of UseMethod
> >> or NextMethod.  If there are, then you'd have a valid complaint.  But so
> >> far you've just made an unsupported claim.
> >>
> >>> Your parenthesized statement discussed why it works that way
> >>> under the current design but that is not inevitable.  The current
> >>> design is not the only possibility.
> >>>
> >>>>> That is why tcltk and Henrik's package wrap environments in lists and define
> >>>>> a completely different class but by doing that they are not able to take
> >>>>> advantage of inheritance.
> >>>> I think they did that because they wanted explicit references to
> >>>> objects, rather than the built-in implicit ones.  I've wanted explicit
> >>>> references to things on a number of occasions too, but that's really
> >>>> unrelated to inheritance as far as I can see.
> >>> They are defining environments with special features yet they can't make
> >>> use of inheritance as they injected the environment object into their object
> >>> rather than subclassing it -- understandable given the current limitations.
> >> I think their worry was that attaching the special features to the
> >> environment would leave those features at risk of being thrown away by
> >> some other code that attached its own features to that environment.  But
> >> this has nothing to do with subclassing, it has everything to do with
> >> the semantics of references.
> >>
> >> If you want to complain about the semantics of references in R, do that,
> >> but don't bring up the red herring of subclassing (unless you really
> >> have code that demonstrates that CallMethod or NextMethod don't work as
> >> expected.)
> >
> > Perhaps the example I gave previously does not adequately
> > convey the problem.  Lets try another example.
> >
> > #1 uses R as it currently exists.
> >
> > 1. Define an environment e1 with component A.  Now
> > suppose we wish to define a "subobject" for which
> > $ is overridden so that it will upper case the
> > right arg.  We can do this:
> >
> > e1 <- new.env()
> > e1$A <- 1
> >
> > # now define a "subobject" and override $
> > e2 <- structure(list(env = e1), class = c("myenvironment", "environment"))
>
> I think this shows one of the problems of the S3 object system.  You've
> declared e2 to be an environment, but it's not.  There's nothing in S3
> to stop you from doing that, but the code won't work, as you observed.
>
> > "$.myenvironment" <- function(obj, x) obj[["env"]][[toupper(x)]]
> >
> > e2$a # 1
> > e2[["A"]]  # NULL since e2 cannot usefully inherit [[
> >
> > To really get this to work we would have to
> > redefine all of environment's methods.  I won't do that
> > here to keep things small.
> >
> >
> > 2. However, if it were possible to have the class
> > attached to e2 rather than being attached to the environment itself then
> > e2 could still inherit all the other methods of "environment" yet
> > be distinct from e1 even though the two would share the same
> > environment:
>
> That's something different from what I'd expect from an object.  I don't
> expect objects to inherit from other objects, I expect the class of
> objects to inherit from the class of other objects.  This means the
> objects share properties and behaviour, but not necessarily data.


With environments new possibilities become available and this sharing
of data is one of them.   From the viewpoint of S3 the inheritance is
still the usual S3 inheritance but it would allow the same environment
to be used in different objects sharing the same data.  It does not really
extend the inheritance concept with respect to data as the data in e1
cannot be overridden by e2 without changing e1.  It only allows the
methods to be overridden.  Yet it would provide a useful step without
being too drastic. The fact that existing code motivated this example
suggests that it would be useful.

>
> In some languages they can share data too.  In S3 inheritance is almost
> meaningless, so there's really nothing shared, except a claim that any
> method you don't override will still somehow work:  but there's no
> possible way to verify that claim.  That's why John Chambers wrote the
> S4 system.  In S4 objects a descendant class will inherit properties,
> and it will inherite behaviour to some extent, but even S4 doesn't
> really encapsulate behaviour as much as some other object systems (e.g.
> Java) do.
> >
> > # this assumes a attributes are associated with variables
> > # does not work as intended in R currently since the class(e2)<-
> > statement also changes e1:
> >
> > e1 <- new.env()
> > e1$A <- 1
> > e2 <- e1
> > class(e2) <- c("myenvironment", "environment")
> > "$.myenvironment" <- function(obj, x) obj[[toupper(x)]]
> > e2$a # 1
> > e2[["A"]]  # 1
> >
> > Now all of environment's methods can be used and e1
> > can still be used in the usual way.
> >
> > If we typed the above into R now then e1 would be changed
> > too.  Thus e2 is no longer a "subobject" of e1.  It is e1.
> > e1$a # 1  -- oops!
> >
> > Although this example may seem simple I think it represents the essence
> > of  the situation that exists in a number of packages.  Any package
> > that currently uses the list(env=...an.environment...) idiom could
> > likely usefully
> > make use of this.  Note that in #1 we had to redefine all the methods of
> > environment to get access to them but if the functionality assumed in
> > #2 existed then it would inherit them and no further work need be done.
>
> But you can still do some of what you want as follows.
>
> Instead of saying e2 <- e1, say e2 <- copy(e1), where copy is a function
> you write that copies the content and attributes of e1.  Changes to e2
> won't affect e1.  Then change the class of e2, and e1 is unaffected.
>
> If you'd like some e2 changes (i.e. content changes) to also affect e1
> but not others (i.e. attribute changes), then you're right, you can't do
> that in the current S3 system.  I'd say that's because R doesn't do
> references as well as it should.  I'd like to have a way to say a
> variable "r" is a reference to something else, so that if you do
>

If you are going to copy them you might as well use lists.

> s <- r
>
> then s becomes a reference to the same thing.  I had a discussion last
> year with Luke Tierney about this, and he convinced me that it's a
> really good idea to distinguish references from the things they refer
> to, and it's also a good idea not to allow references to named objects.
> (If you did an assignment from the thing r refers to directly to another
> variable, you'd get a copy.  Changes to the copy would have no effect on
> the thing r refers to.)
>
> He put together an implementation of this idea using current R
> functions; the only problem was that since the parser didn't know
> anything about it, the syntax was a little clunky.
>
> You couldn't use this to do exactly what you want unless all of R was
> rewritten so that environments were implemented as this new kind of
> reference.  Since environments play such a central role in low level
> processing in R, that's never going to happen.  But you could probably
> accomplish more of what you want to do than now.
>
> Duncan Murdoch
>

I think the idea here of separating the attributes from the .environment
is less drastic than changing or proposing a new oo system or adding a
new reference facility.  Its more incremental in nature and simply makes
the existing sharing of environments more useful by leveraging the
existing S3 inheritance system in a way which cannot (usefully) be done now.