[Rd] User interrupts parallel excution. Why it works or why not?

[Jiefei Wang]

Hi all,

I just notice this interesting problem a few days before, but I cannot
find an answer for it. Say if you have a long-running job in a cluster
made by the parallel package and you decide to stop the execution by
pressing ctr + c in the terminal or the stop button in Rstudio for
some reason. After the interrupt, is the cluster still valid or not?
Below is a simple example code

library(parallel)
cl <- makeCluster(1)
## run and interrupt it
parLapply(cl, 1, function(i){Sys.sleep(10);Sys.getpid()})
## run another apply function to check the cluster status
parLapply(cl, 1, function(i)i)

>From my test result, the answer is yes. The worker is interrupted
immediately and the cluster is ready for the next command, but when I
create the worker manually, things seem different.

library(parallel)
cl <- makeCluster(1, manual = TRUE)
## run and interrupt it
parLapply(cl, 1, function(i){Sys.sleep(10);Sys.getpid()})
## run another apply function to check the cluster status
parLapply(cl, 1, function(i)i)

It seems like the worker does not know the manager has been
interrupted and still runs the current task. I have to wait for 10
seconds before I can get the result from the last line of the code and
the return value is the PID from the first apply function.

Both cases are reasonable, but it is surprising to see them at the
same time. I start to wonder how the user interrupt is handled, so I
looked at the code in the parallel package. However, it looks like
there is no related code, there is no try-catch statement in the
manager's code to handle the user interrupt, but the worker just
magically knows it should stop the current execution.

I can see this behavior in both Win and Ubuntu. It is kind of beyond
my knowledge, so I wonder if anyone can help me with it. Does the
cluster support the user interrupt? Why the above code works or not
works? Many thanks!

Best,
Jiefei