[Rd] Pb with lapply()

[Herve Pages]

Thanks Martin for your patient and detailed explanations!

So with the base4 solution, base would import base4 so lapply() could
call base4::as.list()? Sounds good.
Making as.list() a *primitive* function with C-internal S3 and S4
method dispatch: like for length(), etc...? Sounds good too.

I'm wondering why can't as.list() just be made an S4 generic in base.
Or in methods, which seems to be a more appropriate place for S4 generics.
But then I guess lapply() would have to be moved to methods too and that
would break code importing base but not methods...

Sounds like there is no simple solution indeed.

Cheers,
H.


Martin Maechler wrote:
>>>>>> "HP" == Herve Pages <hpages at fhcrc.org>
>>>>>>     on Thu, 31 Jan 2008 10:26:31 -0800 writes:
> 
>     HP> Hi, If needed, lapply() tries to convert its first
>     HP> argument into a list before it starts doing something
>     HP> with it:
> 
>     >> lapply
>     HP> function (X, FUN, ...) 
>     HP> {
>     HP>   FUN <- match.fun(FUN)
>     HP>   if (!is.vector(X) || is.object(X)) 
>     HP>       X <- as.list(X)
>     HP>   .Internal(lapply(X, FUN))
>     HP> }
> 
>     HP> But in practice, things don't always seem to "work" as suggested by
>     HP> this code (at least to the eyes of a naive user).
> 
> Yes.  That is the infamous problem of what I'd call a mental conflict
> between namespaces and function-centered OOP (as in S3 or S4),
> or put differently, the problem that not all R functions are
> S4 generics right from the start :
>  
> Both lapply() and as.list() are in the base namespace.
> Consequently, lapply() will always call base::as.list() and
> unfortunately  base::as.list() is not becoming an S4 generic by your
> 
>     >> setClass("A", representation(data="list"))
>     HP> [1] "A"
>     >> setMethod("as.list", "A", function(x, ...) x at data)
>     HP> Creating a new generic function for "as.list" in ".GlobalEnv"
>     HP> [1] "as.list"
> 
> See: it's an S4 generic only in .GlobalEnv, but to really work
> it should be an S4 generic in base.
> 
> 
>     HP> Seems like using force() inside lapply() would solve the problem:
> 
> Well, it's not force() that makes it work,
> it's the fact that you define a version of lapply outside "base"
> and that of course does see your as.list() generic in .GlobalEnv ..
> 
>     HP> lapply2 <- function(X, FUN, ...)
>     HP> {
>     HP> FUN <- match.fun(FUN)
>     HP> if (!is.vector(X) || is.object(X))
>     HP> X <- force(as.list(X))
>     HP> .Internal(lapply(X, FUN))
>     HP> }
> 
>     HP> It works now:
> 
>     [...........]
> 
> 
> Now one "solution" to the problem is to redefine  base::as.list()
> to be your S4 generic.  Most smart useRs I know would call this
> a terrible hack though...
> and yes, I'm guilty of committing that hack -- inside the Matrix package, 
> not for as.list() but for as.matrix():
> 
> The consequence of that is that e.g.  eigen() *does* work for
> all our matrices, because eigen starts with as.matrix() and that
> needs to work as a proper (i.e. S4) generic in order to work as
> it should. 
> 
> A much better solution to the underlying deeper problem would be
> to find a way where ___ conceptually ___ part (or all of)
> 'methods' would be inside of 'base', and  as.list(), as.matrix()
> etc all S4 (and S3 simultaneously) generics. 
> [and we have pondered of using  'base4' for that,
>  which would contain part of current base and part of current methods;
>  but there have been different ideas].
> 
> One workaround/solution used recently (notably for the group generics)
> was to make more of these functions into
> *primitive* functions with C-internal S3 and S4 method dispatch.
> That maybe a desideratum for "now".
> 
> Martin Maechler, ETH Zurich (and R core team)
>