[R] pdf() under multicore (was Comments disappearing from local functions (R 2.14.0))

[Patrick Connolly]

On Thu, 24-Nov-2011 at 09:17PM +0000, Prof Brian Ripley wrote:

|> On Fri, 25 Nov 2011, Patrick Connolly wrote:
|> 
|> [... about a completely different topic]
|> 
|> >With the one using multicore, I get a strange error that didn't occur
|> >with R-2.13.1.  A function which runs on 7 cores produces a swag of
|> >PDF files and then returns a list of information.  New to the 2.14.0
|> >is an error like this:
|> >
|> >"Error in dev.off() : internal error in PDF_endpage\n"
|> >
|> >>From what I can work out, that is a C message and tricky to trace.
|> 
|> Easy to trace; you look for it in the code! It indicates that you

I did find it at about line 7050 in
RHOME/src/library/grDevices/src/devPS.c

But being a bear with such a small brain (and even smaller when it
comes to C), it was rather tricky for me to trace that it had to do
with the 'if (pd->useCompression) {' line about 10 before that one.

|> used PDF compression (new to 2.14.0) and that the page-stream file
|> got corrupted.  So simply use compress=FALSE to get the previous
|> behaviour.

Wow! simple once you know.  That would certainly explain why it's not
a problem with the postscript device.


|> 
|> However, mixing graphics devices with multicore is probably asking
|> for trouble.  If your fork with an open graphics device you
|> certainly are since the parent and child will share file handles.
|> But even if each child opens a separate pdf() device and closes it
|> before it returns, there still are many pitfalls as the parent and
|> child share a session temporary directory and all assume they have
|> exclusive access to it.  My guess is that two pdf() devices in
|> different processes are using the same temporary file, or the same
|> FILE stream associated with a temporary file.  And only
|> onefile=FALSE or compress=TRUE use such temporary files.

That sounds like a pretty good guess.


|> 
|> >However, if I use a postscript device instead of PDF, it works
|> >fine.  It's a simple matter to convert a postscript file to pdf,
|> >so that's easy to deal with.  It's impossible to make a toy
|> >example of the problem which doesn't appear when run on a single
|> >core.  Maybe if I knew exactly where the problem arises I could
|> >make a toy example but it's rather a lot of code I'm dealing with.
|> >The fact that the two problems I'm encountering did not occur with
|> >earlier versions makes me less suspicious of my own code.
|> 
|> You could at least have outlined what you are doing.

In my defence, I did indicate that I was making a bunch of PDF files
in parallel and getting a list together at the end.  The inclusion of
gbm in the attached packages would give a hint at what was happening
in the parallel processes.  Would have knowing more about what was
happening with gbm have made it any clearer?  


As usual, Brian's insights were spot on and very much appreciated.



|> 
|> -- 
|> Brian D. Ripley,                  ripley at stats.ox.ac.uk
|> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
|> University of Oxford,             Tel:  +44 1865 272861 (self)
|> 1 South Parks Road,                     +44 1865 272866 (PA)
|> Oxford OX1 3TG, UK                Fax:  +44 1865 272595

-- 
~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.   
   ___    Patrick Connolly   
 {~._.~}                   Great minds discuss ideas    
 _( Y )_  	         Average minds discuss events 
(:_~*~_:)                  Small minds discuss people  
 (_)-(_)  	                      ..... Eleanor Roosevelt
	  
~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.