[R] Help needed to understand an error message produced from furrr and future packages

Hiroto Miyoshi h|roto-m|yo@h| @end|ng |rom e-m@||@jp
Thu Nov 25 13:19:09 CET 2021

Dear R-users

I need help to understand the error message from furrr function.
I am trying to build a parallel compute system which combines two
desktop computers, one of which is a host computer, and runs ubuntu
over wsl2, and the other is a slave, which runs ubuntu. as its OS.
They are mutually connected on LAN.

The host computer has 8 physical cores (16 logical cores), and the
slave has 4 physical cores(8 logical cores).

I wrote a code chunk, which is;

 > nodes<-c(rep("localhost",7),rep("",4))
 > plan(list(tweak(cluster, workers = nodes),tweak(multicore,workers=2)))
 > system.time(VCtransfrm("typeIII"))

in which VCtransfrm() is the target function, in which future_pmap and 
are being called tporogically.   The variable "typeIII" shows the file 
which is sent to
the VCtransfrm function. the typeIII file is the largest and has 165MB 
of data while
a typeII file is smaller and has only 7 MB of data.

The chunk runs just fine when the typeII data is fed.  However, when the 
typeIII data
was fed, it gave the following error messages  and returned to the R 
prompt.  Oddly,
multiple R sessions were still running under the host computers when I 
obsered its
behaviour by the top command of ubuntu.  The error messages are:

Error in unserialize(node$con) :
   ClusterFuture (<none>) failed to receive results from cluster 
RichSOCKnode #10 (PID 47955 on localhost ‘localhost’). The reason 
reported was ‘error reading from connection’. Post-mortem diagnostic: No 
process exists with this PID, i.e. the localhost worker is no longer 
alive. Detected a non-exportable reference (‘externalptr’) in one of the 
globals (‘...furrr_fn’ of class ‘function’) used in the future 
expression. The total size of the 8 globals exported is 3.77 MiB. The 
three largest globals are ‘...furrr_chunk_args’ (3.30 MiB of class 
‘list’), ‘...furrr_fn’ (456.55 KiB of class ‘function’) and 
‘...furrr_map_fn’ (11.91 KiB of class ‘function’)
Timing stopped at: 2.285 4.291 37.53

I hastily add that the part of "multicore" in the chunk is changed to 
the chunk runs without a problem even when the typeIII file is fed.

I need to understand what this messages mean and how to fix this 
the chunk runs just fine for the smaller data, I reasoned that the 
problem could not
be a logical matter of the code.

Please direct me to the solution of the problem.
Any suggestion will be greatly appreciated.



More information about the R-help mailing list