[Rd] S4 NAMESPACE method imports and exports do not include (promoted?) generics

[John Chambers]

As I mentioned earlier in the thread, the main hangup remains the 
treatment of implicit generics--the kludge that turns on method dispatch 
for a function in another package, unique() in your original example.

For 2.15.0, my current plan is to make the export/import of implicit 
generics as automatic as is feasible:

1.  exportMethods() would export the corresponding implicit generic, if 
that's what the generic function in PkgB is.  It would not automatically 
export non-implicit generics, those that the package defines on its own. 
  These should be exported and documented explicitly, since the package 
author is responsible for their design.  (In practice, non-implicit 
means generics with "PkgB" as their package slot.)

2. importMethodsFrom() would correspondingly automatically import 
implicit generics, if they were not already in the namespace.

3. the documentation requirements for check would not require 
documenting implicit generics if there were corresponding exported 
methods (of course, the package author is still responsible for 
documenting the exported methods themselves).

John

On 12/22/11 10:01 AM, Martin Morgan wrote:
> On 12/18/2011 11:04 AM, John Chambers wrote:
>>
>>
>> On 12/17/11 6:02 AM, Michael Lawrence wrote:
>>> I guess what it boils down to is whether it makes sense for PkgB to have
>>> exportMethods(unique) when PkgB does not export(unique) or have PkgA in
>>> Depends. And whether it makes sense for PkgC to have
>>> importMethodsFrom(PkgB, unique) without importFrom(PkgA, unique). If it
>>> is not feasible/sensible to support implicit passing of generics up the
>>> dependency stack, then R should probably emit some sort of warning/error
>>> when methods are exported or imported without the corresponding generic.
>>>
>>> The fact that a generic is being created for an implicit generic defined
>>> in 'base' is not really the issue here.
>>
>> On the contrary, it's why we need to do some thinking about what we want.
>>
>> Try replacing "unique" with "sum" throughout Martin's example, adjusting
>> the method definition appropriately. No problems arise, because sum() is
>> a primitive. Exporting and importing methods works as implied by the
>> extensions manual, but via the implicit generic for sum().
>>
>> The essential point is that the methods being imported are "for" the
>> generic implied by the function in package "base". PkgA is the
>> irrelevant aspect if the methods come from PkgB and the implied generic
>> comes from "base". The problem is that there is no "flag" for
>> non-primitive functions that says "Methods have been defined for this
>> function (base::unique in this case), so some calls should carry out
>> method dispatch."
>>
>> The extensions manual stated that exportMethods() would "export the
>> generic". We could make that true or make it unnecessary, by having
>> importMethods() look for a generic in the referenced package and infer
>> the implicit generic (maybe with a message). Otherwise, we're adding
>> inconsistent requirements for primitives vs true functions in the base
>> package.
>
> One advantage of the exportMethods solution is that a user of PkgB gets
> access to the unique generic without having all of PkgA on the search
> path; it would be unfortunate if the exported generic also carried with
> it the burden of documenting the generic (again).
>
> Even with the insights from this thread, I find myself spending a very
> long time working out the appropriate NAMESPACE declarations for my
> current projects.
>
> Martin
>
>>
>> John
>>>
>>> Thanks,
>>> Michael
>>>
>>> On Fri, Dec 16, 2011 at 4:52 PM, John Chambers <jmc at r-project.org
>>> <mailto:jmc at r-project.org>> wrote:
>>>
>>> The key point here is that setGeneric("unique") is done that way,
>>> without other argument, whoever does it. That creates the generic
>>> from the implicit generic corresponding to base::unique. If package
>>> A had done anything else, the resulting methods tables would NOT
>>> refer to package "base" but to package "PkgA" and it's that version
>>> of the generic that would need to be imported.
>>>
>>> So it's not particularly relevant that we're dealing with
>>> PkgA::unique() if the generic function was created from base::unique
>>> by the standard call. That's what would make an automatic
>>> imputation of the generic from importMethods() possible.
>>>
>>>
>>> On 12/16/11 2:57 PM, Martin Morgan wrote:
>>>
>>> On 12/16/2011 12:19 PM, John Chambers wrote:
>>>
>>> The subject heading is correct if referring to
>>> exportMethods() and
>>> importMethodsFrom(). They refer to the methods tables, not
>>> the generic
>>> functions, whatever the extensions manual says.
>>>
>>> Looking into the code of namespaceImportMethods() will
>>> illustrate this.
>>> It just deals with lists of method tables obtained from
>>> .getGenerics()
>>> which in spite of its name also only looks for the method
>>> table metadata
>>> objects.
>>>
>>> As I vaguely recall, there was some concern at one time
>>> about having
>>> extra copies of the generic version of the function.
>>>
>>> The fundamental problem is that creating methods for
>>> unique(), say, does
>>> not change the way calls to base::unique() work. Therefore,
>>> all packages
>>> that want to use methods have to ensure that a generic
>>> version of unique
>>> gets in between. Primitive functions are an exception
>>> because the method
>>> dispatch is in the C code and has a rule for checking when
>>> given an S4
>>> object. There is no corresponding provision for evaluating a
>>> call to a
>>> regular function.
>>>
>>> If the importing package has a setGeneric() for the relevant
>>> function
>>> then its own namespace has the generic version of the
>>> function. (That is
>>> a workaround, but I inferred that was what you were trying
>>> to avoid.)
>>>
>>> Fixes seem possible, but some care is needed. If exportMethods
>>> automatically exported the generic function, it really is no
>>> different
>>> from export() for that function.
>>>
>>>
>>> export() somehow implies ownership of the generic, e.g.,
>>> responsibility
>>> for documentation. I can see in the scenario below that PkgB
>>> might be
>>> expected to Depends: PkgA if it intends for the user to access
>>> PkgB's
>>> methods on PkgA::unique.
>>>
>>> namespaceImportMethods() could try to supply the generic
>>> function if it
>>> is not already present. If it does not find the generic in
>>> the namespace
>>> being imported, it would essentially have to call
>>> setGeneric(), assuming
>>> the non-generic function exists in the specified package
>>> (e.g., in base
>>> for unique()).
>>>
>>>
>>> In the example below for PkgC the 'unique' generic is in PkgB's
>>> namespace imports
>>>
>>> > getNamespaceImports("PkgB")
>>> $base
>>> [1] TRUE
>>>
>>> $PkgA
>>> unique
>>> "unique"
>>>
>>> I guess PkgB could have Depends: PkgA, not importFrom(PkgA,
>>> unique), and
>>> then defined and exported a method on PkgA::unique found on the
>>> search
>>> path, so that the generic wasn't available to PkgC. But I'd be
>>> happy if
>>> the generic found in either PkgB's namespace or namespace
>>> imports were
>>> imported along with the method. Not sure that I like the idea of
>>> calling
>>> setGeneric() -- PkgA could have done something non-standard --
>>> and would
>>> rather an error.
>>>
>>> Thans for your attention.
>>>
>>> Martin
>>>
>>>
>>> Comments?
>>> John
>>>
>>>
>>>
>>> On 12/16/11 6:16 AM, Martin Morgan wrote:
>>>
>>> tar of Pkgs A, B, C attached. Martin
>>>
>>> On 12/15/2011 03:34 PM, Martin Morgan wrote:
>>>
>>> In
>>>
>>> > R.version.string
>>> [1] "R Under development (unstable) (2011-12-15 r57901)"
>>>
>>> section 1.6.6 of 'Writing R Extensions' says
>>>
>>> Note that exporting methods on a generic in the
>>> namespace will
>>> also export the generic, and exporting a generic in the
>>> namespace will also export its methods.
>>>
>>> and
>>>
>>> Note that importMethodsFrom will also import any
>>> generics defined in
>>> the namespace on those methods
>>>
>>> However, if PkgA promotes 'unique' to a generic and
>>> exports that
>>>
>>> DESCRIPTION:
>>> Imports: methods
>>>
>>> R/f.R:
>>> setGeneric("unique")
>>>
>>> NAMESPACE:
>>> export(unique)
>>>
>>> and PkgB creates and exports a method on unique
>>>
>>> DESCRIPTION
>>> Imports: methods, PkgA
>>>
>>> R/f.R:
>>> setClass("B", representation(b="numeric"))
>>> setMethod(unique, "B",
>>> function(x, incomparables=FALSE, ...) unique(x at b))
>>>
>>> NAMESPACE:
>>> importFrom(PkgA, unique)
>>> exportClasses(B)
>>> exportMethods(unique)
>>>
>>> and PkgC wants to import PkgB's classes and methods
>>>
>>> DESCRIPTION
>>> Imports: methods, PkgB
>>>
>>> R/f.R
>>> cunique <- function(x) unique(x)
>>>
>>> NAMESPACE
>>> importMethodsFrom(PkgB, unique)
>>> export(cunique)
>>>
>>> then
>>>
>>> (a) the 'unique' generic is not available to the
>>> user of PkgB
>>>
>>> > library(PkgB)
>>> > unique(new("B", b=1:5))
>>> Error in unique.default(new("B", b = 1:5)) :
>>> unique() applies only to vectors
>>>
>>> and (b) the generic has not been imported to PkgC's
>>> namespace
>>>
>>> > cunique(new("B", b=1:5))
>>> Error in unique.default(b) : unique() applies only
>>> to vectors
>>>
>>> A workaround is for PkgB to also export(unique), and
>>> for PkgC to also
>>> importFrom(PkgA, unique), but is this the intention?
>>>
>>> This is arising from Bioconductor efforts to place
>>> commonly promoted
>>> functions and S3 classes into a single package, to
>>> avoid conflicts when
>>> the same function is promoted independently by
>>> several packages.
>>>
>>> Martin
>>>
>>>
>>>
>>>
>>>
>>>
>>> ________________________________________________
>>> R-devel at r-project.org <mailto:R-devel at r-project.org> mailing list
>>> https://stat.ethz.ch/mailman/__listinfo/r-devel
>>> <https://stat.ethz.ch/mailman/listinfo/r-devel>
>>>
>>>
>
>