[Rd] how to determine if a function's result is invisible

[Duncan Murdoch]

On 10/29/2006 8:00 AM, Philippe Grosjean wrote:
> [I cut in this message that becomes very long]
> 
> Duncan Murdoch wrote:
> [...]
>> And also thanks to Gabor bringing it up:  and that's really the solution 
>> to this second problem.  If you want to do something unusual and don't 
>> see a way to do it, ask on R-devel.  If the solution you get requires 
>> undocumented functions calls or other kludges, suggest a clean solution 
>> to it.
> 
> But that's exactly what I am doing!

So far I don't think we have a "clean solution".  See comments below.

> OK, now the problem formulated in simple terms: I need a R function that 
> parses and executes one or several lines of R code and returns EXACTLY 
> the same output (including presentation of errors, warnings, etc.) as if 
> I issued the same code at the command prompt. Something like:

If you need that, you should write it yourself from components that R 
provides.  "The same output" should not be a well defined concept:  it's 
up to the GUI to decide what the output is, i.e. how values and errors 
are presented, etc.

What R needs is a way for you to write that function.  In particular, 
you should have a way to parse source code (you do), you should have a 
way to execute parsed code (you do), and you should have a way to work 
out what R would print and what error messages or warnings it might want 
to display. It's mainly the 3rd part that is incomplete now, and 
withVisible is intended to help with that.  As you've pointed out, the 
parsing also has some problems with communicating errors to you (e.g. 
letting you know when the input is incomplete).

> execute("some R code\nanother line of R code\netc")
> 
> The output of execute() would be character strings corresponding to what 
> is printed at the R console. May be you don't need such function 
> personally, but I can cite at least two examples where we need it: R 
> Commander and Tinn-R... and I am sure it could benefit for other R GUIs too.

And I disagree with that claim.  Those GUIs don't need to replicate R's 
console, they need to present results and error messages in an 
appropriate way.  The way this is done in existing R console 
implementations is a very old and robust model (the teletype), but it's 
not necessarily an appropriate model for all GUIs.

> Now, the best suggestion I can do for this is the awful code below. 
> Several points to finalize, and as you say, several parts of the code 
> are subject to problems in future versions of R, because workarounds are 
> rather fragile against possible future changes in R.
> 
> [...]
>> The code below looks fragile to me:  it depends on the format of the 
>> error message report.  I'm hoping to make syntax errors more informative 
>> in the next release, and that will probably change the format of the 
>> reports.
> 
> Yes, I know! That is why this code is NOT released to CRAN, and also why 
> I do NOT propose it as to John Fox or Jose-Claudio Faria as a patch for 
> R Commander or Tinn-R, respectively. We are precisely discussing the 
> problem to find better solutions (the R GUI API?). For instance, could 
> you suggest a better isLineComplete() function? I am pretty sure one 
> could get something better using C code, which is something I have not 
> attempted to do.

I don't know a better isLineComplete function currently, but if parse() 
returned more structured error information, it would be easy to write 
one.  (Of course, we might not need one in that case...)

I think that's a better approach, and it's definitely something that I 
am interested in working on.  It seems to me to be a waste of my time to 
make internal changes that will not only allow, but *force* all GUI 
writers to replicate the R console.  It should be easy to do that (and 
it's not now), but it should also be easy to do something quite different.
> 
>> More information about an error is available at the C level, but I'm not 
>> sure how much of that is published in the API.
>>
>> Now would be a good time to suggest the ideal thing for parse() to 
>> return, from your point of view.  It may not be doable, but if it is, 
>> and it would work in other situations, there's a chance it will make it 
>> into 2.5.0.
> 
> OK, you have it formulated hereabove.

Not yet...

>>> isLineComplete <- function(x) {
>>>      # x is a character string vector with R code
>>>      # The function determines if the parser is satisfied with it
>>>      # or it needs more input (code is continued at the next line)
>>>
>>>      # First parse code
>>>      con <- textConnection(x)
>>>      expr <- try(parse(con), silent = TRUE)
>>>      close(con)
>>>
>>>      # Determine if this code is correctly parsed
>>>      if (inherits(expr, "try-error")) {
>>>          results <- expr
>>>          # Determine if it is an incomplete line
>>>          if (length(grep("\n2:", results)) == 1) return(FALSE)
>>>      }
>>>      # Note: here, we return TRUE also if the code is wrong
>>>      # but one could enhance the function to return something
>>>      # else in case of wrong R code
>>>      return(TRUE)
>>> }
>>>
>>>  > isLineComplete("ls()")
>>> [1] TRUE
>>>  > isLineComplete("ls(")
>>> [1] FALSE
>>>  > isLineComplete("ls())")
>>> [1] TRUE
>>>
>>> - For functions like: xxx.yyy(), it is impossible to know if they are 
>>> xxx methods for yyy S3 objects, or simply a function called xxx.yyy() 
>>> without parsing the code, or interrogating R another way.
>> I think you shouldn't care.  You should leave the evaluation of 
>> expressions up to R.
> 
> Ah, ha! Do you mean that the only valid way to make syntax highlighting 
> of R code is by parsing that code with R?

I think the only reliable way would involve R.  Whether other methods 
are "valid" or not depends on their goals.

 > This is too bad that many
> excellent syntax highlighting engines around cannot be used then. For 
> instance, the best I could do, using GeSHi, is materialized in the R 
> Wiki. You could certainly spot several places where syntax highlighting 
> of R code is wrong in the Wiki, but it was the best I could do without 
> parsing the code in R, indeed. This was about three months of work to 
> get that result.

I think a syntax highlighting engine that can't work with the R parser 
couldn't do an excellent job.  Take a look at the MacOSX GUI:  it can 
provide syntax hints on functions you've just written.  It needs to be 
working closely with R to do that, it can't use a big pre-compiled table 
of existing functions.

>> R should provide a way for a GUI to evaluate 
>> something, and should tell the GUI as much as the GUI needs to know to 
>> operate, but the GUI shouldn't try to be an R interpreter.
> 
> Really? An embedded R interpreter is absolutely not needed in a R GUI? I 
> am afraid I do not agree with this, especially because I thing that a 
> GUI should only help users to write code: it should display correct R 
> code and show the user how it works at the command prompt... to 
> encourage him to switch to the prompt as soon as possible. And that 
> would certainly benefit from an embedded R interpreter in the GUI. 
> Again, the same actual examples: R Commander and Tinn-R... plus many 
> others.

I think you misunderstood me.  I think the GUI should do all those 
things, but you as a GUI writer shouldn't have to write an R interpreter 
to do them.  You should be able to ask R to do that kind of work for you.
> 
> [...]
>> I think it's unlikely that I will be doing anything with the 
>> error/warning mechanism, but there are other people who might.  So I'd 
>> suggest you propose a change that would meet your needs, and see if 
>> anyone responds.
> 
> ...??? But that's what I am doing in this thread? Do I need to formulate 
> it in a different way, perhaps?

Yes, I think so.  I think it would help to break down your questions to 
small, self-contained tasks, and work on them one at a time.  What's the 
most important thing you'd like R to do for you that you can't figure 
out?  Perhaps it's already there, and you just need someone to tell you 
what to do, or maybe it's not there, and R needs some changes.

In the latter case, you'll have to convince someone in R Core that your 
problem is worth solving.  That means making it appear that the value 
gained (from their point of view, not yours) exceeds the amount of work 
required so much that they should put aside other work to solve your 
problem.  (There are very few idle cycles in the R Core to work on low 
priority jobs.)

You can decrease the amount of their work by actually designing and 
submitting a nice clean patch to implement the change, but convincing 
them of the value of the change is harder.  So make a clear, short 
argument about why the change would be valuable.

So far I've been convinced that exposing the R_Visible flag is a good 
idea, and that's done.  I'm also convinced that it would be valuable to 
make more error information available from the parser, but I don't yet 
know the best way to do that.

I'm not convinced that the error and warning mechanism needs changes, 
and even if I were, I would not work on that problem:  it would just 
take too much of my time to learn the internals of the error handling 
well enough to make changes.  So you'll have to convince someone else of 
the need for changes there.

Duncan Murdoch