[R-pkg-devel] Feedback on "Using Rust in CRAN packages"

[Kevin Ushey]

Package authors could use 'cargo vendor' to include Rust crate sources
directly in their source R packages. Would that be acceptable?

Presumedly, the vendored sources would be built using the versions
specified in an accompanying Cargo.lock as well.

https://doc.rust-lang.org/cargo/commands/cargo-vendor.html


On Wed, Jul 12, 2023, 7:35 PM Simon Urbanek <simon.urbanek using r-project.org>
wrote:

> Yutani,
>
> I'm not quite sure your reading fully matches the intent of the policy.
> Cargo.lock is not sufficient, it is expected that the package will provide
> *all* the sources, it is not expected to use cargo to resolve them from
> random (possibly inaccessible) places. So the package author is expected to
> either include the sources in the package *or* (if prohibitive due to
> extreme size) have a release tar ball available at a fixed, secure,
> reliable location (I was recommending Zenodo.org for that reason - GitHub
> is neither fixed nor reliable by definition).
>
> Based on that, I'm not sure I fully understand the scope of your proposal
> for improvement. Carlo.lock is certainly the first step that the package
> author should take in creating the distribution tar ball so you can fix the
> versions, but it is not sufficient as the next step involves collecting the
> related sources. We don't want R users to be involved in that can of worms
> (especially since the lock file itself provides no guarantees of
> accessibility of the components and we don't want to have to manually
> inspect it), the package should be ready to be used which is why it has to
> do that step first. Does that explain the intent better? (In general, the
> downloading at install time is actually a problem, because it's not
> uncommon to use R in environments that have no Internet access, but the
> download is a concession for extreme cases where the tar balls may be too
> big to make it part of the package, but it's yet another can of worms...).
>
> Cheers,
> Simon
>
>
>
> > On 13/07/2023, at 12:37 PM, Hiroaki Yutani <yutani.ini using gmail.com> wrote:
> >
> > Hi,
> >
> > I'm glad to see CRAN now has its official policy about Rust [1]!
> > It seems it probably needs some feedback from those who are familiar with
> > the Rust workflow. I'm not an expert, but let me leave some quick
> feedback.
> > This email is sent to the R-package-devel mailing list as well as to
> cran@~
> > so that we can publicly discuss.
> >
> > It seems most of the concern is about how to make the build
> deterministic.
> > In this regard, the policy should encourage including "Cargo.lock" file
> > [2]. Cargo.lock is created on the first compile, and the resolved
> versions
> > of dependencies are recorded. As long as this file exists, the dependency
> > versions are locked to the ones in this file, except when the package
> > author explicitly updates the versions.
> >
> > Cargo.lock also records the SHA256 checksums of the crates if they are
> from
> > crates.io, Rust's official crate registry. If the checksums don't match,
> > the build will fail with the following message:
> >
> >    error: checksum for `foo v0.1.2` changed between lock files
> >
> >    this could be indicative of a few possible errors:
> >
> >        * the lock file is corrupt
> >        * a replacement source in use (e.g., a mirror) returned a
> different
> > checksum
> >        * the source itself may be corrupt in one way or another
> >
> >    unable to verify that `foo v0.1.2` is the same as when the lockfile
> was
> > generated
> >
> > For dependencies from Git repositories, Cargo.lock records the commit
> > hashes. So, the version of the source code (not the version of the crate)
> > is uniquely determined. That said, unlike cargo.io, it's possible that
> the
> > commit or the Git repository itself has disappeared at the time of
> > building, which makes the build fail. So, it might be reasonable the CRAN
> > policy prohibits the use of Git dependency unless the source code is
> > bundled. I have no strong opinion here.
> >
> > Accordingly, I believe this sentence
> >
> >> In practice maintainers have found it nigh-impossible to meet these
> > conditions whilst downloading as they have too little control.
> >
> > is not quite true. More specifically, these things
> >
> >> The standard way to download a Rust ‘crate’ is by its version number,
> and
> > these have been changed without changing their number.
> >> Downloading a ‘crate’ normally entails downloading its dependencies, and
> > that is done without fixing their version numbers
> >
> > won't happen if the R package does include Cargo.lock because
> >
> > - if the crate is from crates.io, "the version can never be overwritten,
> > and the code cannot be deleted" there [3]
> > - if the crate is from a Git repository, the commit hash is unique in its
> > nature. The version of the crate might be the same between commits, but a
> > git dependency is specified by the commit hash, not the version of the
> > crate.
> >
> > I'm keen to know what problems the CRAN maintainers have experienced that
> > Cargo.lock cannot solve. I hope we can help somehow to improve the
> policy.
> >
> > Best,
> > Yutani
> >
> > [1]: https://cran.r-project.org/web/packages/using_rust.html
> > [2]: https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html
> > [3]: https://doc.rust-lang.org/cargo/reference/publishing.html
> >
> >       [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > R-package-devel using r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-package-devel
> >
>
> ______________________________________________
> R-package-devel using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>

	[[alternative HTML version deleted]]