[Rd] parallel: Race-condition concern regarding graphics devices in a multi-thread environment

[Henrik Bengtsson]

On Fri, Apr 5, 2013 at 2:44 PM, Simon Urbanek
<simon.urbanek at r-project.org> wrote:
> Henrik,
>
> there are no threads.

Oh my... all this time I've thought mc* were working with threads.
...despite the 'parallel' vignette writing it in black and white: "In
principle the workers could be implemented by threads\footnote{only
`in principle' since the R interpreter is not thread-safe.} or
lightweight processes, but in the current implementation they are full
processes."  Note to self: lapply(rep(c(9,14), times=100),
fortunes::fortune)


> Hence you're on the wrong track there, it has nothing to do with device numbers/devices.The device number will be the same in all processes and that's ok.

Got it; at the moment the main process is forked, each child *process*
gets a copy of the existing list of devices (devices which they then
may interact with but that's bad style - or is ignored?!?), whereas
any newly created graphics device is unique to each child process.
Device indices are local to each child processes.   In case someone
else follows this or finds this thread later, the following example
illustrates this:

library("parallel");
jpeg("foo0.jpg");
idx <- dev.cur();
plot(1:10, main="main");

dummy <- function(i) {
  before <- dev.list();
  filename <- sprintf("foo%d.png", i);
  png(filename);
  plot(1:10, main=i);
  # No dev.off(); tests what happens if you forget
  # to close device (see below).
  list(before=before, after=dev.list());
}

res <- mclapply(1:2, FUN=dummy, mc.cores=2);
str(res);
## List of 2
##  $ :List of 2
##   ..$ before: Named int 2
##   .. ..- attr(*, "names")= chr "jpeg"
##   ..$ after : Named int [1:2] 2 3
##   .. ..- attr(*, "names")= chr [1:2] "jpeg" "png"
##  $ :List of 2
##   ..$ before: Named int 2
##   .. ..- attr(*, "names")= chr "jpeg"
##   ..$ after : Named int [1:2] 2 3
##   .. ..- attr(*, "names")= chr [1:2] "jpeg" "png"

# When child processes finishes, any open devices are closed.
str(dev.list());
##  Named int 2
## - attr(*, "names")= chr "jpeg"

# Close the opened jpeg device.
dev.off(idx);

# All devices closed
str(dev.list())
##  NULL


> The issue here really is which png() pack-end are you using? (You didn't say).

I was considering the general case (where I don't know), e.g.
dev.new(<user specified>).


> Some back-ends (like X11) cannot be run in forked environment, so you can't use them.  I don't use png() myself, but I do know that CairoPNG() from the Cairo package works when forked.

Test/example:

dummy <- function(i) {
  x11();
  plot(1:10, main=i);
}

res <- mclapply(1:2, FUN=dummy, mc.cores=2);
## Warning message:
## In mclapply(1:2, FUN = dummy, mc.cores = 2) :
##   all scheduled cores encountered errors in user code
print(res);
## [[1]]
## [1] "Error in x11() : a forked child should not open a graphics device\n"
## [[2]]
## [1] "Error in x11() : a forked child should not open a graphics device\n"


Thanks for your help Simon,

Henrik

>
> Cheers,
> Simon
>
>
> On Apr 5, 2013, at 12:23 PM, Henrik Bengtsson <hb at biostat.ucsf.edu> wrote:
>
>> Hi,
>>
>> I'm trying to figure out how to safely make sure that I close the same
>> graphics device that I opened earlier in a thread (and not one opened
>> by a parallel thread).  In a *single-thread* environment, one can do
>> the following to open and close a device:
>>
>> makePlot <- function(i) {
>>  filename <- sprintf("foo%d.png", i);
>>  png(filename);
>>  idx <- dev.cur();
>>  on.exit(dev.off(idx));
>>  plot(1:10, col=i);
>>  title(main=i);
>>  filename;
>> } # makePlot()
>>
>> However, in a *multi-thread* environment, e.g.
>>
>> res <- mclapply(1:4, FUN=makePlot, mc.cores=4);
>>
>> the following race-condition problem may occur, because of the
>> non-atomicity(?) of (i) png() and (ii) dev.cur():
>>
>> 1. Thread #1: png("foo1.png")
>> 2. Thread #2: png("foo2.png")
>> 3. Thread #1: idx <- dev.cur();  # idx == 3
>> 4. Thread #2: idx <- dev.cur();  # idx == 3 (same device!)
>> ...
>> 5. Thread #1: dev.off(idx); # Closes device #3
>> 6. Thread #2: plot(1:10, col=2); # Trying to plot, which opens a new
>> on-screen device (or to another active device) since device #3 is
>> closed.
>> 7. Thread #2: dev.off(idx) # Trying to close device #3, which is already closed.
>>
>> On this topic, there are some clues/notes on concern in the vignette
>> of the 'parallel' package, but not enough to answer my concerns/solve
>> my problems.  Any other references/discussion on this (other that
>> source code)?
>>
>>
>> Q1. Is there already a built-in protection against the above race condition?
>>
>> Q2. If not on Q1, is there a way/strategy to get hold of the device
>> (index) for the graphics devices that was opened by png() without
>> using dev.cur()?
>>
>> If not on Q2, what about transitioning to have graphics device
>> functions return the opened device (index) (cf. connections), e.g. idx
>> <- png(...).  A quick look at ?png and ?x11 show that those functions
>> does not return anything (although code inspections show that they may
>> return something, but always NULL when I try).
>>
>>
>> Comments?
>>
>> Henrik
>>
>> ______________________________________________
>> R-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>>
>