[R-pkg-devel] Links from manual return 403 and 404 responses when running devtools::check_rhub()

[Ivan Krylov]

On Wed, 14 Sep 2022 00:32:45 +0000
"Gao, Song via R-package-devel" <r-package-devel using r-project.org> wrote:

> The issue happened when running devtools::check_rhub(), there are a
> lot of 403 responses when testing the links in the manual. The
> details log is attached The links in the manual are public ones
> without authentication, and anyone can access them from anywhere.

A browser deemed by Cloudflare to be operated by a human can access the
links, that's true. When R checks the links, it uses cURL to send a
HEAD request to the URL. Cloudflare deems these requests as coming from
a robot (which is more or less correct, I guess) and blocks them,
returning a 403 error.

R is not going to pretend to be a human operating a browser, because
there's no winning this cat-and-mouse game: it's already costly, and
bot-detecting systems will just adapt and classify the new behaviour as
bot-like.

Use the CRAN comments page to explain that the links returning 403 are
for humans only and would work if actually clicked. Alternatively, you
could ask Cloudflare not to block HEAD requests to your website coming
from cURL, but that might be against their business model.

-- 
Best regards,
Ivan