[Rd] on ptr_R_WriteErrConsole

[Byron Ellis]

Yup, I like that idea too though it seems like it should be done by  
routing through the connection interface. It would be interesting to  
see how much the current stdout relies on information that wouldn't  
necessarily be available in an event stream. Of course, if you're  
going to go this route you may as well go for the gusto and push the  
eventloop up to the connection level a la CLIM.

On Oct 18, 2005, at 5:36 AM, A.J. Rossini wrote:

> I fully support Duncan M (though it's painless for me).
>
> What I wanted to do, and explored, when I was visiting Peter D in
> Copenhagen nearly 2 years ago, was to XML-ize the output stream, so
> that I could mark it up.  From that, it would be reasonably simple to
> parse output (provided you marked it up on the way out, which was the
> difficult part, I fully admit, which was where I failed to get a
> distributable implementation).
>
> (the goal was to provide multimedia output streams, i.e. text, audio,
> static graphics, or dynamic graphics, as required).
>
> The person asking about a log4j a while back (or what I'm looking at
> right now, log4cl) probably wanted the same thing.
>
>
> On 10/18/05, Duncan Murdoch <murdoch at stats.uwo.ca> wrote:
>
>> I'll volunteer to do your testing on Windows, provided nobody  
>> suggests a
>> way to do what you want without your patches.  That is, if there's  
>> a way
>> for a custom front-end to separate the streams of text coming to
>> ptr_R_WriteConsole through REvprintf from that coming through  
>> Rvprintf,
>> then your changes are not needed.  If there is currently no way to do
>> that, then I think there should be.
>>
>> The justification for this change is to facilitate experimentation  
>> with
>> different front ends, as you describe below.  That's something I  
>> think
>> we should support.
>>
>> Forcing both streams through ptr_R_WriteConsole without giving
>> information about their origin is a decision about a particular  
>> kind of
>> user interface.  Decisions like that should be made by the writer  
>> of the
>> front-end.
>>
>> I agree with Luke and Brian that your second suggestion (which would
>> allow separation of text into regular output, warnings, and errors)
>> would tie us down too much in the internal details.
>>
>> Duncan Murdoch
>>
>>
>>
>>
>> Thomas Friedrichsmeier wrote:
>>
>>>> I second the point about not having made a case, and also that  
>>>> (from the
>>>> part deleted below) about needing to understand the internals  
>>>> before
>>>> proposing changes.
>>>>
>>>
>>>
>>> I'll try to make the case below point d). I fully accept  
>>> rejection of the more
>>> daring part of my proposal, but the part about  
>>> ptr_R_WriteErrConsole is
>>> really straightforward, and I have really made sure to understand  
>>> the
>>> internal going ons, here.
>>> My reply will be lengthy, because this really is important to me.
>>>
>>>
>>>
>>>> a) I took a quick look at the patch, and it seems to be only  
>>>> half the
>>>> story.  R_WriteConsole is also supported under Windows, in a  
>>>> different
>>>> (and more flexible) way.  I think the part not covered is a lot  
>>>> more
>>>> problematic, as I don't see how to introduce R_WriteErrConsole in a
>>>> backwards compatible way.
>>>>
>>>
>>>
>>> I admit I've been neglectant to the Windows code. I don't have a  
>>> windows
>>> machine to compile and test any changes I make in my local  
>>> sources, and that
>>> lead me to be careless. so probably the patch is not correct.
>>> Do I believe there is still a backwards compatible way to introduce
>>> ptr_R_WriteErrConsole / R_WriteErrConsole?
>>>
>>> void R_WriteErrConsole(char *buf, int len) {ptr_R_WriteErrConsole 
>>> (buf, len);}
>>> ...
>>> /* standard initialization: */
>>> ptr_R_WriteErrConsole = R_WriteConsole;
>>>
>>> *R_WriteConsole*. This ensures that if you do not touch the new
>>> ptr_R_WriteErrConsole, you get *exactly* the old behavior. I  
>>> can't be much
>>> easier than that.
>>>
>>>
>>>
>>>> b) There are several inaccuracies in the account.  First,  
>>>> `channels' are
>>>> not what we are talking about here.  sink() allow output and  
>>>> messages to
>>>> be sent to different connections, but the console is connected  
>>>> to fixed
>>>> terminal connections stdout() and stderr().  There is a long- 
>>>> standing
>>>> design for R consoles (sketched in src/unix/system.txt) that has  
>>>> all the
>>>> output going to the console.  This is even true of the standard  
>>>> Unix
>>>> terminal front end.  The only circumstances in which stdout()  
>>>> and stderr()
>>>> are separated are when R is used for scripting.  Part of the  
>>>> motivation
>>>> for this is to ensure that stdout() and stderr() appear in an  
>>>> integrated
>>>> way (in particular, interleaved in the correct order) on a console.
>>>>
>>>
>>>
>>> Ok, so I have been simplifying, and using my own terminology.  
>>> What I'm saying
>>> is, output generally goes two routes *throughout* R sources:
>>>
>>> "message":
>>> [warnings]->...->REprintf->REvprintf
>>> [errors]->...->REprintf->REvprintf
>>> REprintf->REvprintf
>>>
>>> "output":
>>> [print]->...->Rprintf->Rvprintf
>>> Rprintf->Rvprintf
>>>
>>> So it is not like I'm making up any new distinction, here. I'm  
>>> referring to an
>>> existing one, avaible via already public API. See also this  
>>> comment from
>>> printutils.c:
>>>
>>> /* =========
>>>  * Printing:
>>>  * =========
>>>  *
>>>  * All printing in R is done via the functions Rprintf and REprintf
>>>  * or their (v) versions Rvprintf and REvprintf.
>>>  * These routines work exactly like (v)printf(3).  Rprintf writes to
>>>  * ``standard output''.        It is redirected by the sink()  
>>> function,
>>>  * and is suitable for ordinary output.        REprintf writes to
>>>  * ``standard error'' and is useful for error messages and warnings.
>>>  * It is not redirected by sink().
>>>
>>> All I'm dealing with is, what should happen in REvprintf, the  
>>> last station in
>>> what I call the "message route":
>>>
>>> Currently it is basically (and yes, I'm simplifying again):
>>> void REvprintf(const char *format, va_list arg)
>>> {
>>>       /* A: if there is a sink(type="message"), print to that and  
>>> return */
>>>       /* B: if RConsoleFile is not NULL, print to that and return */
>>>       /* C: if none of the above, call R_WriteConsole */
>>> }
>>>
>>> All I'm talking about is case C, and effectively, my proposal  
>>> would change
>>> this to
>>>       /* C: ptr_R_WriteErrConsole has been modified by stubborn  
>>> developer, print to
>>> that and return */
>>>       /* D: if none of the above, call R_WriteConsole */
>>>
>>>
>>>
>>>> No other console designer has seen a need for this, and there is  
>>>> at least
>>>> the potential for presenting very misleading information to end  
>>>> users.
>>>>
>>>
>>>
>>> Let me assure you this: The interface pointers give me enough  
>>> rope already to
>>> shoot myself in the foot. I have enough opportunity to present  
>>> extremely
>>> misleading information to end users. Re-read my answer to a) to  
>>> see the
>>> change really, really, really would not affect any code that does  
>>> not
>>> explicitely ask for this extra inch of rope.
>>> Also let me assure you this: Other GUI designers are much  
>>> interested in an
>>> easy way to find out which portions of the output have come the  
>>> "message"
>>> route, as well. We do crazy stuff like using separate file-sinks,  
>>> grepping
>>> for "warning", or "error", and things like that. I've seen hacks  
>>> and hacks,
>>> and I've used hack after hack. But I think there is a good  
>>> opportunity for a
>>> solid solution.
>>>
>>>
>>>
>>>> So
>>>> one ramification of allowing consoles to present stdout() and  
>>>> stderr()
>>>> separately would have to be a review of how they are used.  This  
>>>> is not
>>>> hypothetical: I have struggled many times over the years with an  
>>>> R-like
>>>> system which when scripted wrote error messages in inappropriate  
>>>> places on
>>>> a file, at one point sending prompts to stderr yet echoed input  
>>>> to stdout.
>>>> We've worked hard in R to avoid that kind of thing, mainly by  
>>>> having a
>>>> single route to the console.  (There are still residual problems  
>>>> if C++ or
>>>> Fortran I/O is used, BTW, and note that R_ReadConsole also  
>>>> *writes* to the
>>>> console.)
>>>>
>>>
>>>
>>> And the single route to the console will remain intact using  
>>> R_WriteConsole.
>>> I'm only asking for the *opportunity*, not the *obligation* to  
>>> intercept the
>>> one call to R_WriteConsole in REvprintf.
>>>
>>>
>>>
>>>> c) Anything in R involving more than one of the three main  
>>>> families of
>>>> platforms is NOT a `small addition': it involves testing and  
>>>> subsequent
>>>> maintenance by two or three people.
>>>>
>>>
>>>
>>> So, yes, somebody would have to add corresponding code to windows.
>>> Unfortunately, I don't think I qualify for this, as I just can't  
>>> test on
>>> windows. I'm fairly confident, the change I did in gnuwin32 will  
>>> ensure
>>> nothing is broken, but you would want a parallel to  
>>> ptr_R_WriteErrConsole in
>>> windows for consistency's sake.
>>> But please: Don't conjure up a maintenance nightmare for this  
>>> simple change.
>>>
>>>
>>>
>>>> d) There is an existing mechanism that could be used.  If you want
>>>> file-like stderr and stdout, you could drive R via a file-like  
>>>> interface
>>>> (e.g. ptys).  That is not easy on non-Unix-alike platforms (and was
>>>> probably impossible on classic MacOS R), but I understood Thomas  
>>>> was using
>>>> KDE.  (There are live examples of this route, even on Windows.)
>>>>
>>>
>>>
>>> And it's not like I haven't ventured along that route. Do you  
>>> know how much
>>> fun it is to use two separate ptys, then try to make sure the  
>>> output arrives
>>> in a sensible order, i.e. you don't get all warnings before all  
>>> output or
>>> vice versa, or some strange mixture? I'm not the infailable  
>>> programmer, and
>>> I'm not an expert in R internals. But before I go into lengthy  
>>> discussions, I
>>> have checked my options.
>>>
>>> So again: Why do I want something like this?
>>> A GUI may want to do some things, which are not needed on the  
>>> console. One
>>> such thing is to identify "message" output. There are several  
>>> uses for this:
>>> 1) Highlight "message" output to bring it to the users attention
>>> 2) Offer context help on "message"s. Of course this is easier  
>>> said that done,
>>> but the first step in this, would be to find out which portions  
>>> of the output
>>> are messages.
>>> 3) Show "message"s that come up in operations that would usually  
>>> redirect the
>>> output elsewhere, and not show it to the user directly. Much like  
>>> the
>>> scripting situation you depicted in b)
>>>
>>> And again: Why can't I just use current facilities?
>>> 1) Condition Handling: Probably the way to go for my advanced  
>>> needs. But
>>> totally useless, if I want to catch messages generated using  
>>> direct calls to
>>> REprintf/REvprintf. Those are abundant. They have to be dealt  
>>> with. The only
>>> way to do this is to use a mechanism available in/below/after  
>>> REvprintf.
>>> 2) Ptys, sinks: See above
>>> 3) Grepping: come on now. Could not even solve most needs,  
>>> impossible due to
>>> internationalization...
>>> 4) Using mind reading? When I first posted about these matters to  
>>> r-devel (and
>>> yes, before that I tried my luck on r-help), it was a plain  
>>> support question:
>>> How can I do this? See also here:
>>> https://stat.ethz.ch/pipermail/r-devel/2005-October/034975.html
>>> I did not receive any reply on this. What should I conclude?
>>>
>>> I'll gladly accept alternative solutions. They are not the above  
>>> though. And
>>> I've written why they are not before.
>>>
>>>
>>>
>>>> I have spent far longer (and written more) than I intended on  
>>>> this.  The
>>>> length of correspondence so far (and much in a prolix style) is  
>>>> all part
>>>> of the support costs.  One thing the R project can not afford is to
>>>> explain to individual users how internals work -- we have not  
>>>> even been
>>>> able to find the time to write down for the core team how a lot  
>>>> of the
>>>> internals work, and some developments are being held up by  
>>>> that.  So this
>>>> has to weighed against considering proposals which would appear  
>>>> to help
>>>> just one user.
>>>>
>>>
>>>
>>> Sorry about writing more and more lengthy mails. I don't really  
>>> want to. I
>>> have better things to do as well. But this is important, and -  
>>> sorry to say -
>>> IMO you've simply overlooked a number of points. All I can do is  
>>> restate
>>> them, trying to make extra sure to get my point across.
>>>
>>>
>>>
>>>> I suspect that we will only want to go forward if a concise and  
>>>> strong
>>>> case can be made from a group of developers who have tested it  
>>>> out on all
>>>> three main families of platforms.
>>>>
>>>
>>>
>>> And where would you think, I could conjure such a group of  
>>> developers, if not
>>> on this list?
>>>
>>> Regards
>>> Thomas Friedrichsmeier
>>>
>>> ______________________________________________
>>> R-devel at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>
>>
>> ______________________________________________
>> R-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>>
>
>
> --
> best,
> -tony
>
> blindglobe at gmail.com
> Muttenz, Switzerland.
> "Commit early,commit often, and commit in a repository from which  
> we can easily
> roll-back your mistakes" (AJR, 4Jan05).
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

---
Byron Ellis (ellis at stat.harvard.edu)
"Oook" -- The Librarian