[R-pkg-devel] Wrong mailing list: Could the 100 byte path length limit be lifted?
McGrath, Justin M
jmcgr@th @end|ng |rom ||||no|@@edu
Wed Dec 13 14:56:09 CET 2023
> Thanks. Pursuing this a bit further, from ?tar "Known problems":
> The handling of file paths of more than 100 bytes. These
> were unsupported in early versions of ‘tar’, and supported in
> one way by POSIX ‘tar’ and in another by GNU ‘tar’ and yet
> another by the POSIX ‘pax’ command which recent ‘tar’
> programs often support. The internal implementation warns on
> paths of more than 100 bytes, uses the ‘ustar’ way from the
> 1998 POSIX standard which supports up to 256 bytes (depending
> on the path: in particular the final component is limited to
> 100 bytes) if possible, otherwise the GNU way (which is
> widely supported, including by ‘untar’).
R CMD check --as-cran gives a WARNING if a path is not ustar compatible, and gives the same message as a NOTE if it is not v7 compatible.
The path in the example below is ustar compatible, but not v7 compatible.
***********
* checking for portable file names ... NOTE
Found the following non-portable file path:
BioCro/inc/boost/numeric/odeint/stepper/generation/generation_controlled_adams_bashforth_moulton_really_long_file_name_one_two_three.hpp
Tarballs are only required to store paths of up to 100 bytes and cannot
store those of more than 256 bytes, with restrictions including to 100
bytes for the final component.
***********
The code is in library/tools/R/check.R. An equivalent test is performed by R CMD build in library/tools/utils/tar.R.
AFAIK, CRAN rejects a package regardless whether the message is given as a NOTE or a WARNING. ustar is 35 years old and is older than R itself, and thus anyone using R in the last several decades will have ustar support. Any path stored by ustar will be suitable for any common file system in use for the last several decades. Overall, if a path is supported by ustar, there is no reason to generate any message, even if that path is not v7 compatible. If the NOTE about v7 incompatibility is still desired, then could CRAN not reject packages for which the check reports a NOTE?
Best wishes,
Justin
________________________________________
From: Martin Maechler <maechler using stat.math.ethz.ch>
Sent: Wednesday, December 13, 2023 2:22 AM
To: Ben Bolker; McGrath, Justin M
Cc: Simon Urbanek; Duncan Murdoch
Subject: Re: [R-pkg-devel] Wrong mailing list: Could the 100 byte path length limit be lifted?
>>>>> Ben Bolker
>>>>> on Tue, 12 Dec 2023 15:48:11 -0500 writes:
> Thanks. Pursuing this a bit further, from ?tar "Known problems":
> The handling of file paths of more than 100 bytes. These
> were unsupported in early versions of ‘tar’, and supported in
> one way by POSIX ‘tar’ and in another by GNU ‘tar’ and yet
> another by the POSIX ‘pax’ command which recent ‘tar’
> programs often support. The internal implementation warns on
> paths of more than 100 bytes, uses the ‘ustar’ way from the
> 1998 POSIX standard which supports up to 256 bytes (depending
> on the path: in particular the final component is limited to
> 100 bytes) if possible, otherwise the GNU way (which is
> widely supported, including by ‘untar’).
> This issue is reminiscent of the "invalid uid value replaced ..."
> warning, which has happened to me a lot but which CRAN has never
> actually flagged when I've submitted a package:
> https://urldefense.com/v3/__https://stackoverflow.com/questions/30599326/warning-message-during-building-an-r-package-invalid-uid-value-replaced-by-that__;!!DZ3fjg!7eG1psjusIaxJny-wr4z56esZliTkCKpBunUO0IPxzllvu3oSRehH6ZnQh8vJN6ZkaAAmNXHaAHfrqwULOipdsJh2VOFww$
> (By the way, this thread now seems firmly on-topic for r-pkg-devel,
> even if the ultimate answers must come from the CRAN maintainers ...)
Well, yes, in parts at least.
Originally, even when it did not seem clear, it mostly seemed about the
part of the help(tar) page you mention above and the
implementation of tar() .. i.e. R-devel.
... but it's not really important anymore now, and hence this goes only to you.
Best regards,
Martin
> cheers
> Ben
> On 2023-12-12 3:41 p.m., Duncan Murdoch wrote:
>> I don't know what the warning looks like, but the ?tar help page
>> discusses the issues.
>>
>> Duncan Murdoch
>>
>> On 12/12/2023 3:10 p.m., Ben Bolker wrote:
>>> FWIW the R-windows FAQ says:
>>>
>>> Yet another complication is a 260 character limit on the length of the
>>> entire path name imposed by Windows. The limit applies only to some
>>> system functions, and hence it is possible to create a long path using
>>> one application yet inaccessible to another. It is sometimes possible to
>>> reduce the path length by creating a drive mapping using subst and
>>> accessing files via that drive. As of Windows 10 version 1607 and R 4.3,
>>> one can remove this limit via Windows registry by setting
>>> Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem\LongPathsEnabled
>>> to 1. Long paths still may not always work reliably: some applications
>>> or packages may not be able to work with them and Windows cannot execute
>>> an application with long path as the current directory.
>>>
>>> I'm having trouble finding the specific check for path lengths > 100
>>> in the R source tree. It would be helpful to have the exact wording of
>>> the NOTE/WARNING (?) that is thrown ... (I know I could make my own
>>> mini-package with a long path length in it somewhere but ...)
>>>
>>> cheers
>>> Ben Bolker
>>>
>>> On 2023-12-12 2:57 p.m., Simon Urbanek wrote:
>>>> Justin,
>>>>
>>>> now that you clarified what you are actually talking about, this is a
>>>> question about the CRAN policies, so you should really direct it to
>>>> the CRAN team as it is their decision (R-devel would be appropriate
>>>> if this was a limitation in R itself, and R-package-devel would be
>>>> appropriate if you wanted help with refactoring to adhere to the
>>>> policy). There are still path limits on various platforms (even if
>>>> they are becoming more rare), so I'd personally question the source
>>>> rather than the policy, but then your email was remarkably devoid of
>>>> any details.
>>>>
>>>> Cheers,
>>>> Simon
>>>>
>>>>
>>>>> On Dec 13, 2023, at 6:03 AM, McGrath, Justin M
>>>>> <jmcgrath using illinois.edu> wrote:
>>>>>
>>>>> When submitting a package to CRAN, it is required that path names be
>>>>> shorter than 100 bytes, with the reason that paths longer than that
>>>>> cannot be made into portable tar files. This error is reported by `R
>>>>> CMD check --as-cran`. Since this pertains only to developing
>>>>> packages, this seemed like the appropriate list, but if you don't
>>>>> think so, I can instead ask on R-devel.
>>>>>
>>>>> Best wishes,
>>>>> Justin
>>>>>
>>>>> ________________________________________
>>>>> From: Martin Maechler <maechler using stat.math.ethz.ch>
>>>>> Sent: Tuesday, December 12, 2023 10:13 AM
>>>>> To: McGrath, Justin M
>>>>> Cc: r-package-devel using r-project.org
>>>>> Subject: Wrong mailing list: [R-pkg-devel] Could the 100 byte path
>>>>> length limit be lifted?
>>>>>
>>>>>>>>>> McGrath, Justin M
>>>>>>>>>> on Tue, 12 Dec 2023 15:03:28 +0000 writes:
>>>>>
>>>>> We include other software in our source code. It has some long
>>>>> paths so a few of the files end up with paths longer than 100
>>>>> bytes, and we need to manually rename them whenever we pull in
>>>>> updates.
>>>>> The 100 byte path limit is from tar v7, and since
>>>>> POSIX1.1988, there has not been a path length limit. That
>>>>> standard is 35 years old now, so given that there is
>>>>> probably no one using an old version of tar that also
>>>>> wants to use the latest version of R, could the 100 byte
>>>>> limit be lifted? Incidentally, I am a big proponent of
>>>>> wide, long-term support, but it's hard to see that this
>>>>> change would negatively impact anyone.
>>>>>
>>>>> Best wishes,
>>>>> Justin
>>>>>
>>>>> Wrong mailing list:
>>>>>
>>>>> This is a topic for R-devel, not at all R-package-devel,
>>>>> but be more accurate in what you are talking about, only between
>>>>> the line I could read that it is about some variants of using
>>>>> 'tar'.
>>>>>
>>>>> Best regards,
>>>>> Martin
>>>>> ---
>>>>>
>>>>> Martin Maechler
>>>>> ETH Zurich and R Core team
> ______________________________________________
> R-package-devel using r-project.org mailing list
> https://urldefense.com/v3/__https://stat.ethz.ch/mailman/listinfo/r-package-devel__;!!DZ3fjg!7eG1psjusIaxJny-wr4z56esZliTkCKpBunUO0IPxzllvu3oSRehH6ZnQh8vJN6ZkaAAmNXHaAHfrqwULOipdsJI-ZBa6A$
More information about the R-package-devel
mailing list