[Rd] ongoing work: tracing support for R; Dtrace and eBPF

[elijah wright]

Hi folks - I've put together some recent work to add support for Dtrace and eBPF profiling of the core R interpreter - it's pretty experimental but it DOES work. Side project, but hopefully becoming more useful in the long run.

There's a patch here - not terribly big, actually - showing the minimal changes required to add the pragmas into the core to make it traceable.

patch-diff.githubusercontent.com/raw/wch/r-source/pull/25.patch

My thinking is that this sort of works in concert / cooperation with the profiling work that Brian Ripley did to the core, 20-some years ago now. ;-)

If you're not terribly familiar with the two technologies involved here - Dtrace is a technology that came from the Solaris world - essentially you're measuring the impact and occurrence rate of syscalls as the eval() loop in the center of the R core runs. It's also supported natively on macOS. eBPF, similarly, is a technology on the Linux side of the world - a derivative of the good old Berkeley Packet Filter, it allows for adding custom fragments to a running kernel and measuring performance (and other...) as a system runs.

Included is a harness to allow you to use lima to test the Linux-y eBPF bits on a relatively current macOS install, inside of a VM - it's as small as I could reasonably make it,

The demo here - is to actually run the standard demo() loop, on top of an instrumented R engine. The included harness script does that, and outputs PDFs showing you the (minimal) impact of trace probes, as well as how much time various parts of the system are spending on their tasks.

There is future work to be done, some of which is already in progress - for example, I've been working on a patch to the Matrix package to allow some of the same techniques to be applied there, sort of bubbling the same technology and techniques to places where they can be maximally useful. (Matrix, as a part of r-recommended, has a TON of other packages that are dependent upon it - enough to make it a prime target for improvements of any sort.)

I hope this lands nicely for folks doing related work - I was quite surprised by how straightforward the patch appears, given how much work it's setting up.

best, and thanks for all the previous contributions you've made to R and the larger performance tooling ecosystem...

--e
	[[alternative HTML version deleted]]