[Bioc-devel] BrowserViz and sub-protocols

Michael Lawrence lawrence.michael at gene.com
Tue Apr 7 01:48:41 CEST 2015


On Mon, Apr 6, 2015 at 10:05 AM, Tim Triche, Jr. <tim.triche at gmail.com>
wrote:

> Kitware seems to have had good experiences with WAMP, which is a good
> sign, given the complexity of their typical visualization projects
> (stemming from old HPC projects in fluid dynamics, astrophysics, realtime
> anatomical reconstruction, etc). Thus WAMP seems like a great way to
> leverage the experiences of an open source focused, medium sized shop known
> for doing highly technical work with some similar aims to Paul's pkg.
>
> Avro's IDL becomes onerous for large projects rather fast.  The lack of
> inheritance can be excruciating when it is pushed beyond its typical scope.
> Best not to poke that sleeping dog ;-)
>
>
Hmm, thanks for pointing that out. Is there no good solution for
serializing objects, outside of simple lists and maps?


> --t
>
> > On Apr 6, 2015, at 9:35 AM, Michael Lawrence <lawrence.michael at gene.com>
> wrote:
> >
> > Sounds like a good plan. It does seem like there is a trend away from
> type
> > formalism in application development. Everything is essentially JSON.
> Have
> > you checked out WAMP? It seems very similar to your solution; it might
> pay
> > off to stick with standards, even if it takes more up-front investment.
> > It's JSON-based, so it doesn't solve the high-level type problem. Very
> few
> > serialization formats do. Avro has a nice IDL, but that's out of the
> hadoop
> > world and would be foreign to the web. There is someting to be said for
> > simplicity, but the incurred technical debt can become burdensome.
> >
> >
> >> On Mon, Apr 6, 2015 at 8:39 AM, Paul Shannon <pshannon at fredhutch.org>
> wrote:
> >>
> >> Hi Michael,
> >>
> >> Thanks for the link to the subprotocol negotiaion discussion in "High
> >> Performance Browser Networking".  There's a number of things in that
> >> discussion of which I was not aware.  Very helpful.
> >>
> >> There may be real merit -- that is, a good trade-off between some added
> >> complexity and extra clarity -- to adding a mandatory content-type
> subfield
> >> to the payload or to the message.  Or adding full sub-protocol
> negotiation
> >> as the reference you supplied discusses.
> >>
> >> For the first release of BrowserViz (and its demo subclass,
> >> BrowserVizDemo, along with RCyjs) I would like to keep things as simple
> as
> >> possible.  By which I mean:  provide (as we do now) the mechanism by
> which
> >> content-type can be specified, without promulgating the policy that
> every
> >> message must have that payload subfield.
> >>
> >> In practice I have found it quick and easy in both Javascript and R to
> >> examine the incoming 4-field message (cmd, status, callback, payload):
> >>
> >> 1) Is the cmd field something I can respond to?  (Do I have a handler
> >> registered for that cmd?)
> >> 2) Does the status indicate trouble? A request?
> >> 3) If 1 & 2 suggest proceeding, then  names(msg$payload) can help me
> >> figure out if the incoming data is structured as I expect.
> >>
> >> One could reasonably object that this is all "programming by (ad hoc)
> >> agreement", rather than by transparent, self-describing, clearly
> >> negotiatied data and exchange standards.
> >>
> >> In order to arrive at such standards, to figure out how formal we should
> >> go, let's experiment for a release cycle.  This could be accomplished
> >> without any changes to BrowserViz by using the status field
> >> (status="summarizedExperimentRequest"), or by always using two fields in
> >> every message payload (payload$contentType, payload$content).  If a few
> >> people collaborated on (let's say) a SummarizedExperiment browser viz
> >> webapp, those people could agree on how to do this, try it out, perhaps
> >> discover a few sub-varieties of content-type are actually needed.  Then,
> >> when things have settled down, we could explore adding some formality to
> >> the process.
> >>
> >> How does that strike you?
> >>
> >> Thanks for the discussion!
> >>
> >> - Paul
> >>
> >>
> >>
> >> On Apr 3, 2015, at 3:11 PM, Michael Lawrence <lawrence.michael at gene.com
> >
> >> wrote:
> >>
> >>> The high-level type issue is sort of discussed here:
> >>>
> >>
> http://chimera.labs.oreilly.com/books/1230000000545/ch17.html#_subprotocol_negotiation
> >>>
> >>> What about extending your protocol so that the payload consists of two
> >> fields:
> >>> content-type and content, where the content-type adheres to the media
> >> type specification? This is analogous to how every S4 object has a class
> >> attribute.
> >>>
> >>> Thanks for the discussion!
> >>>
> >>> On Fri, Apr 3, 2015 at 2:57 PM, Paul Shannon <
> >> paul.thurmond.shannon at gmail.com> wrote:
> >>> Hi Michael,
> >>>
> >>> Thanks for the clarification.   You make a good point about caching:
> >> nobody wants to have to reinvent and re-engineer that!   If too many
> >> features like that become important, then the simplicity of websockets
> will
> >> have an attendant cost.
> >>>
> >>> With regard to more formalism around the payload:  I can imagine that
> >> there will be circumstances in which that is essential.  A
> >> SummarizedExperiment browser would be very useful, and would clearly
> >> benefit from a standard payload structure.
> >>>
> >>> I figure, however, that that's above my pay-grade ;).   I leave that up
> >> to people like you.  And I stand ready to add any features such
> formalized
> >> structures might require.  I'd like to think (I may be naive) that the
> >> architects of SummarizedExperiment and heavy users of it could devise
> and
> >> negotiate some standard JSON representation of the class, translators to
> >> and from, which all could then be used by BrowserViz without BrowserViz
> >> needing to know it's there.
> >>>
> >>> The "binary JSON" format may be useful in some circumstances:
> >> http://bjson.org
> >>>
> >>> - Paul
> >>>
> >>> On Apr 3, 2015, at 2:43 PM, Michael Lawrence <
> lawrence.michael at gene.com>
> >> wrote:
> >>>
> >>>>
> >>>>
> >>>> On Fri, Apr 3, 2015 at 2:00 PM, Paul Shannon <
> >> paul.thurmond.shannon at gmail.com> wrote:
> >>>> Hi Michael,
> >>>>
> >>>> Great to get your response, comments and questions.  Answers attempted
> >> below.
> >>>>
> >>>> I think our overriding difference lies in our contrasting experience
> of
> >> complexity.  I have come to see websockets as minimal, simple, fast and
> >> flexible, whereas you see them quite differently.  I would like to
> >> understand your views on this; I could be overlooking somce important
> and
> >> maybe costly complicating features, perhaps because of my fondness for
> >> other simplifying features.
> >>>>
> >>>>
> >>>> I was just referring to potential complexity. If one attempted to
> >> reimplement the useful features of HTTP (like caching), complexity
> would be
> >> introduced into application code, while you really want that complexity
> in
> >> the protocol implementation. HTTP seems most appropriate for when the
> >> server is acting as the data model. Outside of that, I see the benefits
> of
> >> web sockets.
> >>>>
> >>>> I think you missed my question about RPC. Also, any thoughts as to
> more
> >> formalism around the payload? We obviously have complex data structures,
> >> and it would be nice to communicate semantics somehow to the web
> browser.
> >> For example, could there be some convention for representing a
> >> SummarizedExperiment? Could a payload contain the equivalent of a media
> >> type that an R/JS library could understand to marshal objects? Could
> there
> >> be some way to query for the types of payload a command supports?
> >>>>
> >>>> I've seen stuff like WAMP, but they seem to lack the ability to
> declare
> >> high-level types. Maybe that's just out of style?
> >>>>
> >>>>
> >>>> More below...
> >>>>
> >>>> I look forward to hearing back from you.
> >>>>
> >>>> - Paul
> >>>>
> >>>> On Apr 3, 2015, at 1:45 PM, Michael Lawrence <
> lawrence.michael at gene.com>
> >> wrote:
> >>>>
> >>>>> Thanks to Val's excellent newsletter, I've had my first glance at
> >>>>> BrowserViz. I'm glad to see something that is more flexible and
> >> low-level
> >>>>> than e.g. shiny.
> >>>>>
> >>>>> I'm curious about the motivation behind web sockets. I guess any
> >>>>> application with an R-driven web UI actually has two UIs: the R
> >> console and
> >>>>> the browser. But what if the R session is headless, or if there is no
> >> need
> >>>>> for commands in R to affect the browser? Then the web socket layer
> >> brings
> >>>>> mostly unneeded complexity.
> >>>>
> >>>> I see websockets (like TCP sockets) as musch simpler than HTTP.  No
> >> headers,
> >>>> no explicit server and explicit client (once the connection is open).
> >>>> Could you explain the complexity you see?
> >>>>
> >>>>> An interesting comparison to BrowserViz is not
> >>>>> shiny but OpenCPU. It's purely HTTP-based and still manages to
> maintain
> >>>>> state (not sure how efficiently). I guess one advantage of web
> sockets
> >> is
> >>>>> that one can program imperatively instead of declaratively on the
> >> server,
> >>>>> i.e., the server can send a command to show a popup in response to
> some
> >>>>> event, instead of returning a "declaration" that the popup should be
> >> shown.
> >>>>
> >>>> Exactly!
> >>>>
> >>>>>
> >>>>> So essentially web sockets are more natural for implementing
> >> server-side
> >>>>> controllers (think MVC), instead of just the data model, but man,
> it's
> >> a
> >>>>> shame to lose the features of HTTP.
> >>>>
> >>>> I'll confess:  I set out to -shed- the features of HTTP.  Isn't it a
> >> protocol designed for serving up web pages on demand?  Not for fast,
> >> lighweight peer-to-peer communications?
> >>>>
> >>>>
> >>>>>
> >>>>> Ultimately, I think we want web apps that are easy to develop and
> >> maintain,
> >>>>> and run equally well from either a useR's session or a remote client
> >>>>> communicating to a dedicated, headless server. Is the generality of
> >>>>> websockets worth the complexity?
> >>>>
> >>>>>
> >>>>> As an aside, it would seem relatively straight-forward to implement a
> >>>>> simple bi-directional RPC mechanism between R and JS using the
> standard
> >>>>> protocol (i.e., hide details like the callback). Does that sound
> >> reasonable?
> >>>>>
> >>>>> I was also a bit surprised about the need to copy/paste the JS
> >> boilerplate.
> >>>>> Certainly there must be javascript frameworks with a more elegant
> >> solution
> >>>>> to extensibility.
> >>>>
> >>>> You are correct on this, and I only need to correct the vignette,
> >> written many weeks ago.
> >>>> I now provide "BrowserViz.jx", a simple Javascript module.  It is used
> >> by both
> >>>> BrowserVizDemo and RCyjs.  Sorry to have sent out an obsolete
> vignette.
> >>>>
> >>>>>
> >>>>>     [[alternative HTML version deleted]]
> >>>>>
> >>>>> _______________________________________________
> >>>>> Bioc-devel at r-project.org mailing list
> >>>>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
> >>>>
> >>>>
> >>>
> >>>
> >>
> >>
> >
> >    [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > Bioc-devel at r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/bioc-devel
>

	[[alternative HTML version deleted]]



More information about the Bioc-devel mailing list