[Rd] on ptr_R_WriteErrConsole

[A.J. Rossini]

I fully support Duncan M (though it's painless for me).

What I wanted to do, and explored, when I was visiting Peter D in
Copenhagen nearly 2 years ago, was to XML-ize the output stream, so
that I could mark it up.  From that, it would be reasonably simple to
parse output (provided you marked it up on the way out, which was the
difficult part, I fully admit, which was where I failed to get a
distributable implementation).

(the goal was to provide multimedia output streams, i.e. text, audio,
static graphics, or dynamic graphics, as required).

The person asking about a log4j a while back (or what I'm looking at
right now, log4cl) probably wanted the same thing.


On 10/18/05, Duncan Murdoch <murdoch at stats.uwo.ca> wrote:
> I'll volunteer to do your testing on Windows, provided nobody suggests a
> way to do what you want without your patches.  That is, if there's a way
> for a custom front-end to separate the streams of text coming to
> ptr_R_WriteConsole through REvprintf from that coming through Rvprintf,
> then your changes are not needed.  If there is currently no way to do
> that, then I think there should be.
>
> The justification for this change is to facilitate experimentation with
> different front ends, as you describe below.  That's something I think
> we should support.
>
> Forcing both streams through ptr_R_WriteConsole without giving
> information about their origin is a decision about a particular kind of
> user interface.  Decisions like that should be made by the writer of the
> front-end.
>
> I agree with Luke and Brian that your second suggestion (which would
> allow separation of text into regular output, warnings, and errors)
> would tie us down too much in the internal details.
>
> Duncan Murdoch
>
>
>
>
> Thomas Friedrichsmeier wrote:
> >>I second the point about not having made a case, and also that (from the
> >>part deleted below) about needing to understand the internals before
> >>proposing changes.
> >
> >
> > I'll try to make the case below point d). I fully accept rejection of the more
> > daring part of my proposal, but the part about ptr_R_WriteErrConsole is
> > really straightforward, and I have really made sure to understand the
> > internal going ons, here.
> > My reply will be lengthy, because this really is important to me.
> >
> >
> >>a) I took a quick look at the patch, and it seems to be only half the
> >>story.  R_WriteConsole is also supported under Windows, in a different
> >>(and more flexible) way.  I think the part not covered is a lot more
> >>problematic, as I don't see how to introduce R_WriteErrConsole in a
> >>backwards compatible way.
> >
> >
> > I admit I've been neglectant to the Windows code. I don't have a windows
> > machine to compile and test any changes I make in my local sources, and that
> > lead me to be careless. so probably the patch is not correct.
> > Do I believe there is still a backwards compatible way to introduce
> > ptr_R_WriteErrConsole / R_WriteErrConsole?
> >
> > void R_WriteErrConsole(char *buf, int len) {ptr_R_WriteErrConsole(buf, len);}
> > ...
> > /* standard initialization: */
> > ptr_R_WriteErrConsole = R_WriteConsole;
> >
> > *R_WriteConsole*. This ensures that if you do not touch the new
> > ptr_R_WriteErrConsole, you get *exactly* the old behavior. I can't be much
> > easier than that.
> >
> >
> >>b) There are several inaccuracies in the account.  First, `channels' are
> >>not what we are talking about here.  sink() allow output and messages to
> >>be sent to different connections, but the console is connected to fixed
> >>terminal connections stdout() and stderr().  There is a long-standing
> >>design for R consoles (sketched in src/unix/system.txt) that has all the
> >>output going to the console.  This is even true of the standard Unix
> >>terminal front end.  The only circumstances in which stdout() and stderr()
> >>are separated are when R is used for scripting.  Part of the motivation
> >>for this is to ensure that stdout() and stderr() appear in an integrated
> >>way (in particular, interleaved in the correct order) on a console.
> >
> >
> > Ok, so I have been simplifying, and using my own terminology. What I'm saying
> > is, output generally goes two routes *throughout* R sources:
> >
> > "message":
> > [warnings]->...->REprintf->REvprintf
> > [errors]->...->REprintf->REvprintf
> > REprintf->REvprintf
> >
> > "output":
> > [print]->...->Rprintf->Rvprintf
> > Rprintf->Rvprintf
> >
> > So it is not like I'm making up any new distinction, here. I'm referring to an
> > existing one, avaible via already public API. See also this comment from
> > printutils.c:
> >
> > /* =========
> >  * Printing:
> >  * =========
> >  *
> >  * All printing in R is done via the functions Rprintf and REprintf
> >  * or their (v) versions Rvprintf and REvprintf.
> >  * These routines work exactly like (v)printf(3).  Rprintf writes to
> >  * ``standard output''.        It is redirected by the sink() function,
> >  * and is suitable for ordinary output.        REprintf writes to
> >  * ``standard error'' and is useful for error messages and warnings.
> >  * It is not redirected by sink().
> >
> > All I'm dealing with is, what should happen in REvprintf, the last station in
> > what I call the "message route":
> >
> > Currently it is basically (and yes, I'm simplifying again):
> > void REvprintf(const char *format, va_list arg)
> > {
> >       /* A: if there is a sink(type="message"), print to that and return */
> >       /* B: if RConsoleFile is not NULL, print to that and return */
> >       /* C: if none of the above, call R_WriteConsole */
> > }
> >
> > All I'm talking about is case C, and effectively, my proposal would change
> > this to
> >       /* C: ptr_R_WriteErrConsole has been modified by stubborn developer, print to
> > that and return */
> >       /* D: if none of the above, call R_WriteConsole */
> >
> >
> >>No other console designer has seen a need for this, and there is at least
> >>the potential for presenting very misleading information to end users.
> >
> >
> > Let me assure you this: The interface pointers give me enough rope already to
> > shoot myself in the foot. I have enough opportunity to present extremely
> > misleading information to end users. Re-read my answer to a) to see the
> > change really, really, really would not affect any code that does not
> > explicitely ask for this extra inch of rope.
> > Also let me assure you this: Other GUI designers are much interested in an
> > easy way to find out which portions of the output have come the "message"
> > route, as well. We do crazy stuff like using separate file-sinks, grepping
> > for "warning", or "error", and things like that. I've seen hacks and hacks,
> > and I've used hack after hack. But I think there is a good opportunity for a
> > solid solution.
> >
> >
> >>So
> >>one ramification of allowing consoles to present stdout() and stderr()
> >>separately would have to be a review of how they are used.  This is not
> >>hypothetical: I have struggled many times over the years with an R-like
> >>system which when scripted wrote error messages in inappropriate places on
> >>a file, at one point sending prompts to stderr yet echoed input to stdout.
> >>We've worked hard in R to avoid that kind of thing, mainly by having a
> >>single route to the console.  (There are still residual problems if C++ or
> >>Fortran I/O is used, BTW, and note that R_ReadConsole also *writes* to the
> >>console.)
> >
> >
> > And the single route to the console will remain intact using R_WriteConsole.
> > I'm only asking for the *opportunity*, not the *obligation* to intercept the
> > one call to R_WriteConsole in REvprintf.
> >
> >
> >>c) Anything in R involving more than one of the three main families of
> >>platforms is NOT a `small addition': it involves testing and subsequent
> >>maintenance by two or three people.
> >
> >
> > So, yes, somebody would have to add corresponding code to windows.
> > Unfortunately, I don't think I qualify for this, as I just can't test on
> > windows. I'm fairly confident, the change I did in gnuwin32 will ensure
> > nothing is broken, but you would want a parallel to ptr_R_WriteErrConsole in
> > windows for consistency's sake.
> > But please: Don't conjure up a maintenance nightmare for this simple change.
> >
> >
> >>d) There is an existing mechanism that could be used.  If you want
> >>file-like stderr and stdout, you could drive R via a file-like interface
> >>(e.g. ptys).  That is not easy on non-Unix-alike platforms (and was
> >>probably impossible on classic MacOS R), but I understood Thomas was using
> >>KDE.  (There are live examples of this route, even on Windows.)
> >
> >
> > And it's not like I haven't ventured along that route. Do you know how much
> > fun it is to use two separate ptys, then try to make sure the output arrives
> > in a sensible order, i.e. you don't get all warnings before all output or
> > vice versa, or some strange mixture? I'm not the infailable programmer, and
> > I'm not an expert in R internals. But before I go into lengthy discussions, I
> > have checked my options.
> >
> > So again: Why do I want something like this?
> > A GUI may want to do some things, which are not needed on the console. One
> > such thing is to identify "message" output. There are several uses for this:
> > 1) Highlight "message" output to bring it to the users attention
> > 2) Offer context help on "message"s. Of course this is easier said that done,
> > but the first step in this, would be to find out which portions of the output
> > are messages.
> > 3) Show "message"s that come up in operations that would usually redirect the
> > output elsewhere, and not show it to the user directly. Much like the
> > scripting situation you depicted in b)
> >
> > And again: Why can't I just use current facilities?
> > 1) Condition Handling: Probably the way to go for my advanced needs. But
> > totally useless, if I want to catch messages generated using direct calls to
> > REprintf/REvprintf. Those are abundant. They have to be dealt with. The only
> > way to do this is to use a mechanism available in/below/after REvprintf.
> > 2) Ptys, sinks: See above
> > 3) Grepping: come on now. Could not even solve most needs, impossible due to
> > internationalization...
> > 4) Using mind reading? When I first posted about these matters to r-devel (and
> > yes, before that I tried my luck on r-help), it was a plain support question:
> > How can I do this? See also here:
> > https://stat.ethz.ch/pipermail/r-devel/2005-October/034975.html
> > I did not receive any reply on this. What should I conclude?
> >
> > I'll gladly accept alternative solutions. They are not the above though. And
> > I've written why they are not before.
> >
> >
> >>I have spent far longer (and written more) than I intended on this.  The
> >>length of correspondence so far (and much in a prolix style) is all part
> >>of the support costs.  One thing the R project can not afford is to
> >>explain to individual users how internals work -- we have not even been
> >>able to find the time to write down for the core team how a lot of the
> >>internals work, and some developments are being held up by that.  So this
> >>has to weighed against considering proposals which would appear to help
> >>just one user.
> >
> >
> > Sorry about writing more and more lengthy mails. I don't really want to. I
> > have better things to do as well. But this is important, and - sorry to say -
> > IMO you've simply overlooked a number of points. All I can do is restate
> > them, trying to make extra sure to get my point across.
> >
> >
> >>I suspect that we will only want to go forward if a concise and strong
> >>case can be made from a group of developers who have tested it out on all
> >>three main families of platforms.
> >
> >
> > And where would you think, I could conjure such a group of developers, if not
> > on this list?
> >
> > Regards
> > Thomas Friedrichsmeier
> >
> > ______________________________________________
> > R-devel at r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>


--
best,
-tony

blindglobe at gmail.com
Muttenz, Switzerland.
"Commit early,commit often, and commit in a repository from which we can easily
roll-back your mistakes" (AJR, 4Jan05).