[Rd] Get Logical processor count correctly whether NUMA is enabled or disabled

[Srinivasan, Arunkumar]

Dear R-devel list,

R's detectCores() function internally calls "ncpus" function to get the total number of logical processors. However, this doesnot seem to take NUMA into account on Windows machines.

On a machine having 48 processors (24 cores) in total and windows server 2012 installed, if NUMA is enabled and has 2 nodes (node 0 and node 1 each having 24 CPUs), then R's detectCores() only detects 24 instead of the total 48. If NUMA is disabled, detectCores() returns 48.

Similarly, on a machine with 88 cores (176 processors) and windows server 2012, detectCores() with NUMA disabled only returns the maximum value of 64. If NUMA is enabled with 4 nodes (44 processors each), then detectCores() will only return 44. This is particularly limiting since we cannot get to use all processors by enabling/disabling NUMA in this case.

We think this is because R's ncpus.c file uses "PSYSTEM_LOGICAL_PROCESSOR_INFORMATION" (https://msdn.microsoft.com/en-us/library/windows/desktop/ms683194(v=vs.85).aspx) instead of "PSYSTEM_LOGICAL_PROCESSOR_INFORMATION_EX" (https://msdn.microsoft.com/en-us/library/windows/desktop/dd405488(v=vs.85).aspx). Specifically, quoting from the first link:

"On systems with more than 64 logical processors, the GetLogicalProcessorInformation function retrieves logical processor information about processors in the https://msdn.microsoft.com/en-us/library/windows/desktop/dd405503(v=vs.85).aspx to which the calling thread is currently assigned. Use the https://msdn.microsoft.com/en-us/library/windows/desktop/dd405488(v=vs.85).aspx function to retrieve information about processors in all processor groups on the system."

Therefore, it might be possible to get the right count of total processors even with NUMA enabled by using "GetLogicalProcessorInformationEX".  It'd be nice to know what you think.

Thank you very much,
Arun.

--
Arun Srinivasan
Analyst, Millennium Management LLC 
50 Berkeley Street | London, W1J 8HD


######################################################################

The information contained in this communication is confi...{{dropped:30}}