[R] Does R ever stop responding without a message?

Mon Jul 15 18:34:25 CEST 2013

It is a (very) large marked point process object that is being split (see
original post), and that is some other kind of structure. So I think all
bets are off, in terms of what kind of duration to expect.

The folks on r-sig-geo might be able to help with suggestions for faster
approaches.

In terms of monitoring the process, I have a couple of suggestions.

The first is to turn on profiling (see ?Rprof). Unless things changed in R
3.x.x, Rprof writes to a file as it profiles, and one could watch the file
grow to determine that processing is still taking place. Admittedly, it's
a cheap trick, and runs the risk of slowing the process, especially if the
profile file gets big.

The other is to do the split manually with a loop. This way, one could
insert cat() statements to track the progress. This is probably what I
would do first. Indeed, one could also write timestamps and track the rate
at which splitting is taking place (i.e., is it getting slower and slower).

-Don

-- 
Don MacQueen

Lawrence Livermore National Laboratory
7000 East Ave., L-627
Livermore, CA 94550
925-423-1062

On 7/15/13 2:50 AM, "Jim Holtman" <jholtman at gmail.com> wrote:

>a 'split' function should not take hours to complete.  I will not even
>wait a couple of minutes for one until I look at other approaches.  Can
>you at least show what the command looks like and what the structure of
>the data is.  Are you trying to split a dataframe on multiple criteria?
>I tend to use 'data.table' for some of these operation, or split the
>indices of a dataframe instead of the data.  My own criteria is that if a
>single base function call is taking more than a minute, I see if there
>are options.
>
>This is also a case of "tell me what you want to do, not how you want to
>do it".
>
>Sent from my iPad
>
>On Jul 14, 2013, at 23:24, L S <losedaghat at gmail.com> wrote:
>
>> Thanks Jim and David for your helpful feedback.  I still have not
>>terminated RStudio (and it still has not gone to completion).  A few
>>observations I forgot to mention is that the red "stop" icon is showing
>>in RStudio so I am unable to enter any new commands.  Also there is a
>>blinking cursor under the "> command" I ran with no > before the cursor.
>> 
>> David--Looking at the activity monitor, RStudio is using anywhere from
>>0.4-1% of CPU while another process called rsession is using anywhere
>>from 17-23% CPU.  Does this mean it has given up processing the R
>>command?  If so, is there anything I can do to ensure this won't happen
>>next time I try to run the command with my data?
>> 
>> (Also, how can you tell if the process is using virtual memory?)
>> 
>> Jim--What I'm running now is just a single instruction using the split
>>function, so I would not be able to put in the progress messages.  But,
>>that's a great idea! I will keep that in mind if I run into problems
>>with scripts in the future.
>> 
>> Thank you.
>> 
>> 
>> On Sun, Jul 14, 2013 at 6:46 PM, jim holtman <jholtman at gmail.com> wrote:
>> If you are writing a script that you know will take a long time to
>>process, "pepper" it with "progress" reports so you know what part of
>>the script it is in and when it is going around loop.  On some of my
>>long scripts, I will print out a message every n'th time through the
>>loop so that I know if it making progress.  I position them so that I
>>get out a message every minute or so.  Also make sure the GUI is not
>>buffered, or better, follow each status with a 'flush.console()' to put
>>the message out to the screen.  This will give you a feeling of what
>>progress you are making.
>> 
>> 
>> On Sun, Jul 14, 2013 at 5:40 PM, L S <losedaghat at gmail.com> wrote:
>> Hi,
>> 
>> Have any of you ever encountered a situation where R stops processing an
>> instruction but does not give a "not responding" message?
>> 
>> The reason I ask is I am working in RStudio (Mac OS/X 10.7, 1.8 Ghz i7,
>>4
>> GB DDR3) and the instruction I entered in the command line pane is still
>> being processed since last night around 7 PM.  I expected it to take on
>>the
>> order of 8-12 hours to complete, but I'm nearing 24 hours with no
>>progress,
>> no messages, etc.
>> 
>> The command is essentially splitting the a ppp (marked point pattern)
>>into
>> a tessellation/grid.  There are about a couple million points in the
>> pattern and each has a vector of four marks.  That said, I processed a
>> similar command with 700,000 points in the pattern (each point with a
>> vector of two marks) yesterday and it took only around 3 hours.
>> 
>> Is there anyway I can be certain the command is being processed?  I
>>don't
>> want to abort prematurely if I know it will go to completion.  How long
>> would you give it before you knew for certain it would not complete?
>> 
>> Any guidance you could offer would be much appreciated.
>> 
>> Thanks,
>> Lily
>> 
>>         [[alternative HTML version deleted]]
>> 
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>>http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>> 
>> 
>> 
>> -- 
>> Jim Holtman
>> Data Munger Guru
>>  
>> What is the problem that you are trying to solve?
>> Tell me what you want to do, not how you want to do it.
>> 
>
>	[[alternative HTML version deleted]]
>
>______________________________________________
>R-help at r-project.org mailing list
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.