[Rd] xrealloc namespace conflict

[Patrick Perry]

Thanks for the response. It looks like there is a dlopen flag that will 
fix the issue on Linux (RTLD_DEEPBIND); on Mac OS, symbols get resolved 
by searching in the library first, so the flag isn't needed. I don't 
know what the default behavior is on other platforms. See, e.g. 
https://software.intel.com/en-us/articles/ensuring-shared-library-uses-intel-math-functions 
.

The use of RTLD_DEEPBIND is not portable, and "should only be used as a 
last resort" according to Ulrich Drepper: 
https://software.intel.com/sites/default/files/m/a/1/e/dsohowto.pdf .

I'll probably end up prefixing all of my functions with "corpus_" to 
avoid future namespace clashers.


Patrick


Dirk Eddelbuettel wrote:
>
> On 11 May 2017 at 12:16, Patrick Perry wrote:
> | I've done a bit more investigation into this issue. Here is my current
> | understanding of the situation:
> |
> | 1. I have a package on CRAN (corpus-0.3.1) that passes tests on all
> | platforms except for Linux.
> | 2. My package defines a C function, "xrealloc", for internal use.
> | 3. The libreadline library that R links to defines a different version
> | of "xrealloc".
> | 4. On Linux, when I load my package, references to "xrealloc" get linked
> | to the libreadline version of that function.
> | 5. This causes one of my tests to fail, calling exit(2), because the
> | libreadline version of xrealloc does not allow calls of the form
> | "xrealloc(ptr, 0)".
> |
> | I can work around this issue pretty easily, by either renaming my
> | version of xrealloc to something else, or by avoiding calls of the form
> | "xrealloc(ptr, 0)". So, this is not a major issue, but it's a little
> | unsettling to see this behavior when my package does not explicitly link
> | to or use anything from libreadline.
> |
> | Is there a reason this behavior is only manifesting on Linux? Is there
> | something wrong with the way I'm compiling my package on that platform?
> | Is this just some quirk about the way R is loading dynamic libraries on
> | Linux? I'd appreciate any insight into the issue.
>
> It may just be the flat namespace and linking order. AFAIK there is 
> nothing
> in C preventing so maybe you 'just got lucky' on the other platforms. 
> See eg
> http://stackoverflow.com/questions/7998770/
>
> But then I don't use pure C that after anymore ... and in C++ you 
> could just
> wrap a namespace around your code, avoiding the issue.
>
>
> Dirk


> Dirk Eddelbuettel <mailto:edd at debian.org>
> May 11, 2017 at 1:29 PM
> On 11 May 2017 at 12:16, Patrick Perry wrote:
> | I've done a bit more investigation into this issue. Here is my current
> | understanding of the situation:
> |
> | 1. I have a package on CRAN (corpus-0.3.1) that passes tests on all
> | platforms except for Linux.
> | 2. My package defines a C function, "xrealloc", for internal use.
> | 3. The libreadline library that R links to defines a different version
> | of "xrealloc".
> | 4. On Linux, when I load my package, references to "xrealloc" get 
> linked
> | to the libreadline version of that function.
> | 5. This causes one of my tests to fail, calling exit(2), because the
> | libreadline version of xrealloc does not allow calls of the form
> | "xrealloc(ptr, 0)".
> |
> | I can work around this issue pretty easily, by either renaming my
> | version of xrealloc to something else, or by avoiding calls of the form
> | "xrealloc(ptr, 0)". So, this is not a major issue, but it's a little
> | unsettling to see this behavior when my package does not explicitly 
> link
> | to or use anything from libreadline.
> |
> | Is there a reason this behavior is only manifesting on Linux? Is there
> | something wrong with the way I'm compiling my package on that platform?
> | Is this just some quirk about the way R is loading dynamic libraries on
> | Linux? I'd appreciate any insight into the issue.
>
> It may just be the flat namespace and linking order. AFAIK there is 
> nothing
> in C preventing so maybe you 'just got lucky' on the other platforms. 
> See eg
> http://stackoverflow.com/questions/7998770/
>
> But then I don't use pure C that after anymore ... and in C++ you 
> could just
> wrap a namespace around your code, avoiding the issue.
>
>
> Dirk
>
> Patrick Perry <mailto:pperry at stern.nyu.edu>
> May 11, 2017 at 12:16 PM
> I've done a bit more investigation into this issue. Here is my current 
> understanding of the situation:
>
> 1. I have a package on CRAN (corpus-0.3.1) that passes tests on all 
> platforms except for Linux.
> 2. My package defines a C function, "xrealloc", for internal use.
> 3. The libreadline library that R links to defines a different version 
> of "xrealloc".
> 4. On Linux, when I load my package, references to "xrealloc" get 
> linked to the libreadline version of that function.
> 5. This causes one of my tests to fail, calling exit(2), because the 
> libreadline version of xrealloc does not allow calls of the form 
> "xrealloc(ptr, 0)".
>
> I can work around this issue pretty easily, by either renaming my 
> version of xrealloc to something else, or by avoiding calls of the 
> form "xrealloc(ptr, 0)". So, this is not a major issue, but it's a 
> little unsettling to see this behavior when my package does not 
> explicitly link to or use anything from libreadline.
>
> Is there a reason this behavior is only manifesting on Linux? Is there 
> something wrong with the way I'm compiling my package on that 
> platform? Is this just some quirk about the way R is loading dynamic 
> libraries on Linux? I'd appreciate any insight into the issue.
>
>
> Patrick
>
> p.s. Here are some references:
>
> My package Makevars are at 
> https://github.com/patperry/r-corpus/blob/master/src/Makevars ; my 
> version of "xrealloc" is in corpus/src/xalloc.c
>
> You can see the source for the libreadline xrealloc at 
> https://github.com/JuliaLang/readline/blob/master/xmalloc.c#L67
>
>
> Patrick Perry wrote:
>>
>> I have a package on CRAN now (corpus-0.3.1) that is currently failing
>> tests on Linux, but passing on all other architectures:
>>
>> https://cran.r-project.org/web/checks/check_results_corpus.html
>>
>> I believe that the issue arrises from a namespace class between
>> "xrealloc", which my package provides for internal use, but which R
>> also seems to provide (possibly as part of TRE in
>> src/extra/tre/xmalloc.c). It looks like my package is not picking up
>> my custom xrealloc, but using an xrealloc provided by R.
>>
>> Besides the fact that I am linking to the wrong xrealloc, I think my
>> tests are failing for the same reason that the following code
>> segfaults on Linux (Debian, with R 3.4.0):
>>
>> test <- inline::cfunction(language='C',
>>     otherdefs='void *xmalloc(size_t); void *xrealloc(void *, size_t);',
>>     body = 'void *ptr = xmalloc(256); xrealloc(ptr, 0); return
>> R_NilValue;')
>> test()
>> ## xrealloc: out of virtual memory
>>
>> It seems that the R xrealloc doesn't like being given a size of 0,
>> even though this behavior is well-defined for realloc (it should free
>> the memory). Based on my failing CRAN tests, it looks like this is a
>> Linux-specific issue.
>>
>> Is there a way to modify my Makevars to force the linker to choose my
>> version of xrealloc for my package-specific code? My current Makevars
>> are at https://github.com/patperry/r-corpus/blob/master/src/Makevars
>>
>> Thanks in advance for any help.
>>
>>
>> Patrick
>
>
>> Patrick Perry <mailto:pperry at stern.nyu.edu>
>> May 6, 2017 at 5:36 PM
>> I have a package on CRAN now (corpus-0.3.1) that is currently failing 
>> tests on Linux, but passing on all other architectures:
>>
>> https://cran.r-project.org/web/checks/check_results_corpus.html
>>
>> I believe that the issue arrises from a namespace class between 
>> "xrealloc", which my package provides for internal use, but which R 
>> also seems to provide (possibly as part of TRE in 
>> src/extra/tre/xmalloc.c). It looks like my package is not picking up 
>> my custom xrealloc, but using an xrealloc provided by R.
>>
>> Besides the fact that I am linking to the wrong xrealloc, I think my 
>> tests are failing for the same reason that the following code 
>> segfaults on Linux (Debian, with R 3.4.0):
>>
>> test <- inline::cfunction(language='C',
>>     otherdefs='void *xmalloc(size_t); void *xrealloc(void *, size_t);',
>>     body = 'void *ptr = xmalloc(256); xrealloc(ptr, 0); return 
>> R_NilValue;')
>> test()
>> ## xrealloc: out of virtual memory
>>
>> It seems that the R xrealloc doesn't like being given a size of 0, 
>> even though this behavior is well-defined for realloc (it should free 
>> the memory). Based on my failing CRAN tests, it looks like this is a 
>> Linux-specific issue.
>>
>> Is there a way to modify my Makevars to force the linker to choose my 
>> version of xrealloc for my package-specific code? My current Makevars 
>> are at https://github.com/patperry/r-corpus/blob/master/src/Makevars
>>
>> Thanks in advance for any help.
>>
>>
>> Patrick
>>
>
> Patrick Perry <mailto:pperry at stern.nyu.edu>
> May 6, 2017 at 5:36 PM
> I have a package on CRAN now (corpus-0.3.1) that is currently failing 
> tests on Linux, but passing on all other architectures:
>
> https://cran.r-project.org/web/checks/check_results_corpus.html
>
> I believe that the issue arrises from a namespace class between 
> "xrealloc", which my package provides for internal use, but which R 
> also seems to provide (possibly as part of TRE in 
> src/extra/tre/xmalloc.c). It looks like my package is not picking up 
> my custom xrealloc, but using an xrealloc provided by R.
>
> Besides the fact that I am linking to the wrong xrealloc, I think my 
> tests are failing for the same reason that the following code 
> segfaults on Linux (Debian, with R 3.4.0):
>
> test <- inline::cfunction(language='C',
>     otherdefs='void *xmalloc(size_t); void *xrealloc(void *, size_t);',
>     body = 'void *ptr = xmalloc(256); xrealloc(ptr, 0); return 
> R_NilValue;')
> test()
> ## xrealloc: out of virtual memory
>
> It seems that the R xrealloc doesn't like being given a size of 0, 
> even though this behavior is well-defined for realloc (it should free 
> the memory). Based on my failing CRAN tests, it looks like this is a 
> Linux-specific issue.
>
> Is there a way to modify my Makevars to force the linker to choose my 
> version of xrealloc for my package-specific code? My current Makevars 
> are at https://github.com/patperry/r-corpus/blob/master/src/Makevars
>
> Thanks in advance for any help.
>
>
> Patrick
>


	[[alternative HTML version deleted]]