[Rd] Pb with lapply()

[Martin Maechler]

>>>>> "HP" == Herve Pages <hpages at fhcrc.org>
>>>>>     on Thu, 31 Jan 2008 10:26:31 -0800 writes:

    HP> Hi, If needed, lapply() tries to convert its first
    HP> argument into a list before it starts doing something
    HP> with it:

    >> lapply
    HP> function (X, FUN, ...) 
    HP> {
    HP>   FUN <- match.fun(FUN)
    HP>   if (!is.vector(X) || is.object(X)) 
    HP>       X <- as.list(X)
    HP>   .Internal(lapply(X, FUN))
    HP> }

    HP> But in practice, things don't always seem to "work" as suggested by
    HP> this code (at least to the eyes of a naive user).

Yes.  That is the infamous problem of what I'd call a mental conflict
between namespaces and function-centered OOP (as in S3 or S4),
or put differently, the problem that not all R functions are
S4 generics right from the start :
 
Both lapply() and as.list() are in the base namespace.
Consequently, lapply() will always call base::as.list() and
unfortunately  base::as.list() is not becoming an S4 generic by your

    >> setClass("A", representation(data="list"))
    HP> [1] "A"
    >> setMethod("as.list", "A", function(x, ...) x at data)
    HP> Creating a new generic function for "as.list" in ".GlobalEnv"
    HP> [1] "as.list"

See: it's an S4 generic only in .GlobalEnv, but to really work
it should be an S4 generic in base.


    HP> Seems like using force() inside lapply() would solve the problem:

Well, it's not force() that makes it work,
it's the fact that you define a version of lapply outside "base"
and that of course does see your as.list() generic in .GlobalEnv ..

    HP> lapply2 <- function(X, FUN, ...)
    HP> {
    HP> FUN <- match.fun(FUN)
    HP> if (!is.vector(X) || is.object(X))
    HP> X <- force(as.list(X))
    HP> .Internal(lapply(X, FUN))
    HP> }

    HP> It works now:

    [...........]


Now one "solution" to the problem is to redefine  base::as.list()
to be your S4 generic.  Most smart useRs I know would call this
a terrible hack though...
and yes, I'm guilty of committing that hack -- inside the Matrix package, 
not for as.list() but for as.matrix():

The consequence of that is that e.g.  eigen() *does* work for
all our matrices, because eigen starts with as.matrix() and that
needs to work as a proper (i.e. S4) generic in order to work as
it should. 

A much better solution to the underlying deeper problem would be
to find a way where ___ conceptually ___ part (or all of)
'methods' would be inside of 'base', and  as.list(), as.matrix()
etc all S4 (and S3 simultaneously) generics. 
[and we have pondered of using  'base4' for that,
 which would contain part of current base and part of current methods;
 but there have been different ideas].

One workaround/solution used recently (notably for the group generics)
was to make more of these functions into
*primitive* functions with C-internal S3 and S4 method dispatch.
That maybe a desideratum for "now".

Martin Maechler, ETH Zurich (and R core team)