[R] mclapply enters into an infinite loop....

[akshay kulkarni]

Dear Ivan,
                 I was on RHEL 8 with aws Ec2 z1d instance.

I infer by your words that this "bug" happens only sometimes..But I disagree...I run the same code multiple times within a span of 5 minutes, and the problem persisted....Anyway I have removed the bug in the function and I hope that mclapply runs properly now( if it doesn't you are my only saviour!)

Many thanks for your geekish explanation.

THanking you,
Yours sincerely,
AKSHAY M KULKARNI
________________________________
From: Ivan Krylov <krylov.r00t using gmail.com>
Sent: Saturday, May 20, 2023 6:00 PM
To: akshay kulkarni <akshay_e4 using hotmail.com>
Cc: r-help using r-project.org <r-help using r-project.org>
Subject: Re: [R] mclapply enters into an infinite loop....

� Sat, 20 May 2023 10:59:18 +0000
akshay kulkarni <akshay_e4 using hotmail.com> �����:

> By "holding a lock", you mean a bug in the process right

Well... one person's bug ("your threaded program breaks if I fork() the
process") is another person's documented behaviour ("of course it
does, it says 'please do not fork()' right in the manual"). Depending
on how you're running R (which operating system? are you using a
front-end? a multi-threaded BLAS?), this may be applicable, hence the
question about sessionInfo().

A GUI program may need multiple threads in order to stay responsive
(without looking hung and forgetting to repaint its window or
something) while doing something significant in the background. A
multi-threaded linear algebra library may create a thread per CPU core
in order to take your matrix products faster. Inside a process, some
resources have to be shared between the threads, like the bathroom in a
flat rented by multiple people together. "Holding a lock" is like
taking the key to the bathroom with the intent to occupy it for a
while: a completely normal action that prevents embarrassing accidents.

When you fork() a process, though, it ends up creating an exact copy of
the flat (process) just for you (the current thread), with none of the
flat-mates (other threads). If the bathroom was locked by someone else,
it stays locked, and there's no key anywhere.

There do exist mechanisms like pthread_atfork() that could be used to
prevent problems like this, but for that to work, *every* piece of code
that may be creating threads and taking locks inside your process
should be using them right. (The flat-mates need to know to give you
any keys they might be holding before the flat is fork()ed. It's enough
for one of them to forget one key to cause a problem.) The pragmatic
solution may be to avoid fork() and threads being used together.

--
Best regards,
Ivan

	[[alternative HTML version deleted]]