[Rd] Detecting physical CPUs in detectCores() on Linux platforms

Dirk Eddelbuettel edd @end|ng |rom deb|@n@org
Tue Aug 8 04:16:59 CEST 2023


On 8 August 2023 at 13:17, Simon Urbanek wrote:
| To be honest I think the motivation of this thread is dubious at best: it is a bad idea to use detectCore() blindly to specify parallelization and we explicitly say it's a bad idea - any sensible person will set it according to the demands, the hardware and the task. The number of cores is only partially relevant - e.g. if any I/O is involved you want to oversubscribe the CPU. If you have other users you want to only use a fraction etc. That doesn't mean that the we couldn't do a better job, but if you have to use detectCores() then you are already in trouble to start with.

As I often say, "life's a bitch and then you die".  Detecting hardware
capabilties at run-time is no small task.

When 1 1/2 decades ago I filled in maintaining slurm for Debian (and
interfacing open-mpi) that latter project added an entire library for that
scope (libhwloc).  For all I know it is still best practices.  So in that
sense no point grepping / reading /proc/cpuinfo as that has clear limits.

Dirk

-- 
dirk.eddelbuettel.com | @eddelbuettel | edd using debian.org



More information about the R-devel mailing list