[R] mclapply enters into an infinite loop....

[Ivan Krylov]

В Sat, 20 May 2023 10:59:18 +0000
akshay kulkarni <akshay_e4 using hotmail.com> пишет:

> By "holding a lock", you mean a bug in the process right

Well... one person's bug ("your threaded program breaks if I fork() the
process") is another person's documented behaviour ("of course it
does, it says 'please do not fork()' right in the manual"). Depending
on how you're running R (which operating system? are you using a
front-end? a multi-threaded BLAS?), this may be applicable, hence the
question about sessionInfo().

A GUI program may need multiple threads in order to stay responsive
(without looking hung and forgetting to repaint its window or
something) while doing something significant in the background. A
multi-threaded linear algebra library may create a thread per CPU core
in order to take your matrix products faster. Inside a process, some
resources have to be shared between the threads, like the bathroom in a
flat rented by multiple people together. "Holding a lock" is like
taking the key to the bathroom with the intent to occupy it for a
while: a completely normal action that prevents embarrassing accidents.

When you fork() a process, though, it ends up creating an exact copy of
the flat (process) just for you (the current thread), with none of the
flat-mates (other threads). If the bathroom was locked by someone else,
it stays locked, and there's no key anywhere.

There do exist mechanisms like pthread_atfork() that could be used to
prevent problems like this, but for that to work, *every* piece of code
that may be creating threads and taking locks inside your process
should be using them right. (The flat-mates need to know to give you
any keys they might be holding before the flat is fork()ed. It's enough
for one of them to forget one key to cause a problem.) The pragmatic
solution may be to avoid fork() and threads being used together.

-- 
Best regards,
Ivan