[R] Help needed to understand an error message produced from furrr and future packages

Hiroto Miyoshi h|roto-m|yo@h| @end|ng |rom e-m@||@jp
Thu Nov 25 13:19:09 CET 2021


Dear R-users

I need help to understand the error message from furrr function.
I am trying to build a parallel compute system which combines two
desktop computers, one of which is a host computer, and runs ubuntu
over wsl2, and the other is a slave, which runs ubuntu. as its OS.
They are mutually connected on LAN.

The host computer has 8 physical cores (16 logical cores), and the
slave has 4 physical cores(8 logical cores).

I wrote a code chunk, which is;

 > nodes<-c(rep("localhost",7),rep("192.168.1.11",4))
 > plan(list(tweak(cluster, workers = nodes),tweak(multicore,workers=2)))
 > system.time(VCtransfrm("typeIII"))

in which VCtransfrm() is the target function, in which future_pmap and 
future_map
are being called tporogically.   The variable "typeIII" shows the file 
which is sent to
the VCtransfrm function. the typeIII file is the largest and has 165MB 
of data while
a typeII file is smaller and has only 7 MB of data.

The chunk runs just fine when the typeII data is fed.  However, when the 
typeIII data
was fed, it gave the following error messages  and returned to the R 
prompt.  Oddly,
multiple R sessions were still running under the host computers when I 
obsered its
behaviour by the top command of ubuntu.  The error messages are:

Error in unserialize(node$con) :
   ClusterFuture (<none>) failed to receive results from cluster 
RichSOCKnode #10 (PID 47955 on localhost ‘localhost’). The reason 
reported was ‘error reading from connection’. Post-mortem diagnostic: No 
process exists with this PID, i.e. the localhost worker is no longer 
alive. Detected a non-exportable reference (‘externalptr’) in one of the 
globals (‘...furrr_fn’ of class ‘function’) used in the future 
expression. The total size of the 8 globals exported is 3.77 MiB. The 
three largest globals are ‘...furrr_chunk_args’ (3.30 MiB of class 
‘list’), ‘...furrr_fn’ (456.55 KiB of class ‘function’) and 
‘...furrr_map_fn’ (11.91 KiB of class ‘function’)
Timing stopped at: 2.285 4.291 37.53

I hastily add that the part of "multicore" in the chunk is changed to 
"multisession",
the chunk runs without a problem even when the typeIII file is fed.

I need to understand what this messages mean and how to fix this 
problem.Since
the chunk runs just fine for the smaller data, I reasoned that the 
problem could not
be a logical matter of the code.

Please direct me to the solution of the problem.
Any suggestion will be greatly appreciated.

Sincerely,

Hiroto



More information about the R-help mailing list