[Bioc-devel] [Untrusted Server] Rprintf in a multi-threaded environment

[Yang Liao]

Thanks, Shian!

Yes, I think the only way to make it work is to send the messages to the main thread.

Cheers,
Yang

-----Original Message-----
From: Shian Su <su.s using wehi.edu.au>
Sent: Tuesday, 29 January 2019 3:22 PM
To: Yang Liao <liao using wehi.edu.au>
Cc: bioc-devel using r-project.org
Subject: Re: [Untrusted Server][Bioc-devel] Rprintf in a multi-threaded environment

Hi Yang,

That was my SO thread you found, and as you can see, the issue was never clearly explained to me besides the warning in the docs. I don’t believe anything’s changed to allow the R API to be called from different threads, so extra effort is still required.

What I ended up doing in my code was the hack shown in the second code-snippet, handling output only in the main thread. For your situation depending on what kind of work you are doing, you may consider a thread safe queue to push your progress messages to and a condition variable to ask the main thread to flush the queue to output. I imagine this would be simpler if your do all your work in spawned threads and leave the main thread free to only print output periodically.

Kind regards,
Shian Su

> On 29 Jan 2019, at 2:24 pm, Yang Liao <liao using wehi.edu.au> wrote:
>
> Hi,
>
> I'm not sure if some C developers have gone through this problem: it seems that Rprintf cannot work safely in a multi-threaded environment. In particular, if I call Rprintf() from a then-created thread while the stack size checking is enabled (ie the "R_CStackLimit" pointer isn't set to -1), it is very likely to end up with some fatal error messages like:
>
> Error: C stack usage  847645293284 is too close to the limit
>> Error: C stack usage  847336061668 is too close to the limit
>> Error: C stack usage  847666277092 is too close to the limit
>> Error: C stack usage  847346551524 is too close to the limit
>> Error: C stack usage  847367531236 is too close to the limit
>> Error: C stack usage  847357041380 is too close to the limit
>> Error: C stack usage  847378021092 is too close to the limit
>> Error: C stack usage  847655787236 is too close to the limit
>
> , and the R session terminates in a segfault.
> After I used all means to confirm that there was no memory leakage and the real stack use was minimum, I thought it can only be the Rprintf issue. I then disabled all screen outputs from the then-created threads and the error was gone. It was also reported on stackoverflow:
> https://stackoverflow.com/questions/50092949/why-does-rcout-and-rprintf-cause-stack-limit-error-when-multithreading
> I tried using a semaphore to protect all Rprintf calls but it didn't prevent the error.
>
> Since my program needs to report some messages from the worker threads (created by the main thread), I wonder if there is a solution to safely do so, or I have to pipe the messages to the main thread, which in turn calls Rprintf? I hope not to change "R_CStackLimit" to disable the stack size checks because it generates a "NOTE" in R check.
>
> Cheers,
> Yang
>
> _______________________________________________
>
> The information in this email is confidential and intend...{{dropped:15}}
>
> _______________________________________________
> Bioc-devel using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel

_______________________________________________

The information in this email is confidential and intended solely for the addressee.
You must not disclose, forward, print or use it without the permission of the sender.

The Walter and Eliza Hall Institute acknowledges the Wurundjeri people of the Kulin
Nation as the traditional owners of the land where our campuses are located and
the continuing connection to country and community.
_______________________________________________