[Rd] Detecting physical CPUs in detectCores() on Linux platforms
Dirk Eddelbuettel
edd @end|ng |rom deb|@n@org
Tue Aug 8 04:16:59 CEST 2023
On 8 August 2023 at 13:17, Simon Urbanek wrote:
| To be honest I think the motivation of this thread is dubious at best: it is a bad idea to use detectCore() blindly to specify parallelization and we explicitly say it's a bad idea - any sensible person will set it according to the demands, the hardware and the task. The number of cores is only partially relevant - e.g. if any I/O is involved you want to oversubscribe the CPU. If you have other users you want to only use a fraction etc. That doesn't mean that the we couldn't do a better job, but if you have to use detectCores() then you are already in trouble to start with.
As I often say, "life's a bitch and then you die". Detecting hardware
capabilties at run-time is no small task.
When 1 1/2 decades ago I filled in maintaining slurm for Debian (and
interfacing open-mpi) that latter project added an entire library for that
scope (libhwloc). For all I know it is still best practices. So in that
sense no point grepping / reading /proc/cpuinfo as that has clear limits.
Dirk
--
dirk.eddelbuettel.com | @eddelbuettel | edd using debian.org
More information about the R-devel
mailing list