[R-pkg-devel] Feedback on "Using Rust in CRAN packages"

[Hiroaki Yutani]

Thank you for the correction. I see.

Best,
Yutani

2023年7月13日(木) 16:08 Tomas Kalibera <tomas.kalibera using gmail.com>:

>
> On 7/13/23 05:08, Hiroaki Yutani wrote:
> > I actually use cargo vendor.
> >
> >
> https://github.com/yutannihilation/string2path/blob/main/src/rust/vendor.sh
> >
> > One thing to note is that, prior to R 4.3.0, the vendored directories hit
> > the Windows' path limit so I had to put them into a TAR file. I haven't
> > tested on R 4.3.0, but probably this problem is solved by this
> improvement.
> > So, if you target only R >= 4.3, you can just cargo vendor.
> >
> >
> https://blog.r-project.org/2023/03/07/path-length-limit-on-windows/index.html
>
> I wouldn't rely on that long paths on Windows are supported even in R >=
> 4.3, because it requires at least Windows 10 1607, and it needs to be
> enabled system-wide in Windows - so, users/admins have to do that, and
> it impacts also other applications. The blog post has more details and
> recommendations.
>
> Best
> Tomas
>
> >
> > Best,
> > Yutani
> >
> > 2023年7月13日(木) 11:50 Kevin Ushey <kevinushey using gmail.com>:
> >
> >> Package authors could use 'cargo vendor' to include Rust crate sources
> >> directly in their source R packages. Would that be acceptable?
> >>
> >> Presumedly, the vendored sources would be built using the versions
> >> specified in an accompanying Cargo.lock as well.
> >>
> >> https://doc.rust-lang.org/cargo/commands/cargo-vendor.html
> >>
> >>
> >> On Wed, Jul 12, 2023, 7:35 PM Simon Urbanek <
> simon.urbanek using r-project.org>
> >> wrote:
> >>
> >>> Yutani,
> >>>
> >>> I'm not quite sure your reading fully matches the intent of the policy.
> >>> Cargo.lock is not sufficient, it is expected that the package will
> provide
> >>> *all* the sources, it is not expected to use cargo to resolve them from
> >>> random (possibly inaccessible) places. So the package author is
> expected to
> >>> either include the sources in the package *or* (if prohibitive due to
> >>> extreme size) have a release tar ball available at a fixed, secure,
> >>> reliable location (I was recommending Zenodo.org for that reason -
> GitHub
> >>> is neither fixed nor reliable by definition).
> >>>
> >>> Based on that, I'm not sure I fully understand the scope of your
> proposal
> >>> for improvement. Carlo.lock is certainly the first step that the
> package
> >>> author should take in creating the distribution tar ball so you can
> fix the
> >>> versions, but it is not sufficient as the next step involves
> collecting the
> >>> related sources. We don't want R users to be involved in that can of
> worms
> >>> (especially since the lock file itself provides no guarantees of
> >>> accessibility of the components and we don't want to have to manually
> >>> inspect it), the package should be ready to be used which is why it
> has to
> >>> do that step first. Does that explain the intent better? (In general,
> the
> >>> downloading at install time is actually a problem, because it's not
> >>> uncommon to use R in environments that have no Internet access, but the
> >>> download is a concession for extreme cases where the tar balls may be
> too
> >>> big to make it part of the package, but it's yet another can of
> worms...).
> >>>
> >>> Cheers,
> >>> Simon
> >>>
> >>>
> >>>
> >>>> On 13/07/2023, at 12:37 PM, Hiroaki Yutani <yutani.ini using gmail.com>
> >>> wrote:
> >>>> Hi,
> >>>>
> >>>> I'm glad to see CRAN now has its official policy about Rust [1]!
> >>>> It seems it probably needs some feedback from those who are familiar
> >>> with
> >>>> the Rust workflow. I'm not an expert, but let me leave some quick
> >>> feedback.
> >>>> This email is sent to the R-package-devel mailing list as well as to
> >>> cran@~
> >>>> so that we can publicly discuss.
> >>>>
> >>>> It seems most of the concern is about how to make the build
> >>> deterministic.
> >>>> In this regard, the policy should encourage including "Cargo.lock"
> file
> >>>> [2]. Cargo.lock is created on the first compile, and the resolved
> >>> versions
> >>>> of dependencies are recorded. As long as this file exists, the
> >>> dependency
> >>>> versions are locked to the ones in this file, except when the package
> >>>> author explicitly updates the versions.
> >>>>
> >>>> Cargo.lock also records the SHA256 checksums of the crates if they are
> >>> from
> >>>> crates.io, Rust's official crate registry. If the checksums don't
> >>> match,
> >>>> the build will fail with the following message:
> >>>>
> >>>>     error: checksum for `foo v0.1.2` changed between lock files
> >>>>
> >>>>     this could be indicative of a few possible errors:
> >>>>
> >>>>         * the lock file is corrupt
> >>>>         * a replacement source in use (e.g., a mirror) returned a
> >>> different
> >>>> checksum
> >>>>         * the source itself may be corrupt in one way or another
> >>>>
> >>>>     unable to verify that `foo v0.1.2` is the same as when the
> lockfile
> >>> was
> >>>> generated
> >>>>
> >>>> For dependencies from Git repositories, Cargo.lock records the commit
> >>>> hashes. So, the version of the source code (not the version of the
> >>> crate)
> >>>> is uniquely determined. That said, unlike cargo.io, it's possible
> that
> >>> the
> >>>> commit or the Git repository itself has disappeared at the time of
> >>>> building, which makes the build fail. So, it might be reasonable the
> >>> CRAN
> >>>> policy prohibits the use of Git dependency unless the source code is
> >>>> bundled. I have no strong opinion here.
> >>>>
> >>>> Accordingly, I believe this sentence
> >>>>
> >>>>> In practice maintainers have found it nigh-impossible to meet these
> >>>> conditions whilst downloading as they have too little control.
> >>>>
> >>>> is not quite true. More specifically, these things
> >>>>
> >>>>> The standard way to download a Rust ‘crate’ is by its version number,
> >>> and
> >>>> these have been changed without changing their number.
> >>>>> Downloading a ‘crate’ normally entails downloading its dependencies,
> >>> and
> >>>> that is done without fixing their version numbers
> >>>>
> >>>> won't happen if the R package does include Cargo.lock because
> >>>>
> >>>> - if the crate is from crates.io, "the version can never be
> >>> overwritten,
> >>>> and the code cannot be deleted" there [3]
> >>>> - if the crate is from a Git repository, the commit hash is unique in
> >>> its
> >>>> nature. The version of the crate might be the same between commits,
> but
> >>> a
> >>>> git dependency is specified by the commit hash, not the version of the
> >>>> crate.
> >>>>
> >>>> I'm keen to know what problems the CRAN maintainers have experienced
> >>> that
> >>>> Cargo.lock cannot solve. I hope we can help somehow to improve the
> >>> policy.
> >>>> Best,
> >>>> Yutani
> >>>>
> >>>> [1]: https://cran.r-project.org/web/packages/using_rust.html
> >>>> [2]:
> >>> https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html
> >>>> [3]: https://doc.rust-lang.org/cargo/reference/publishing.html
> >>>>
> >>>>        [[alternative HTML version deleted]]
> >>>>
> >>>> ______________________________________________
> >>>> R-package-devel using r-project.org mailing list
> >>>> https://stat.ethz.ch/mailman/listinfo/r-package-devel
> >>>>
> >>> ______________________________________________
> >>> R-package-devel using r-project.org mailing list
> >>> https://stat.ethz.ch/mailman/listinfo/r-package-devel
> >>>
> >       [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > R-package-devel using r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-package-devel
>

	[[alternative HTML version deleted]]