[Bioc-devel] BrowserViz and sub-protocols

Dan Tenenbaum dtenenba at fredhutch.org
Tue Apr 7 02:08:58 CEST 2015 


----- Original Message -----
> From: "Michael Lawrence" <lawrence.michael at gene.com>
> To: "Tim Triche, Jr." <tim.triche at gmail.com>
> Cc: "Michael Lawrence" <lawrence.michael at gene.com>, bioc-devel at r-project.org, "Paul Shannon" <pshannon at fredhutch.org>
> Sent: Monday, April 6, 2015 4:48:41 PM
> Subject: Re: [Bioc-devel] BrowserViz and sub-protocols
> 
> On Mon, Apr 6, 2015 at 10:05 AM, Tim Triche, Jr.
> <tim.triche at gmail.com>
> wrote:
> 
> > Kitware seems to have had good experiences with WAMP, which is a
> > good
> > sign, given the complexity of their typical visualization projects
> > (stemming from old HPC projects in fluid dynamics, astrophysics,
> > realtime
> > anatomical reconstruction, etc). Thus WAMP seems like a great way
> > to
> > leverage the experiences of an open source focused, medium sized
> > shop known
> > for doing highly technical work with some similar aims to Paul's
> > pkg.
> >
> > Avro's IDL becomes onerous for large projects rather fast.  The
> > lack of
> > inheritance can be excruciating when it is pushed beyond its
> > typical scope.
> > Best not to poke that sleeping dog ;-)
> >
> >
> Hmm, thanks for pointing that out. Is there no good solution for
> serializing objects, outside of simple lists and maps?
> 

Protocol buffers?

https://github.com/dcodeIO/ProtoBuf.js/tree/master/examples/websocket

Dan


> 
> > --t
> >
> > > On Apr 6, 2015, at 9:35 AM, Michael Lawrence
> > > <lawrence.michael at gene.com>
> > wrote:
> > >
> > > Sounds like a good plan. It does seem like there is a trend away
> > > from
> > type
> > > formalism in application development. Everything is essentially
> > > JSON.
> > Have
> > > you checked out WAMP? It seems very similar to your solution; it
> > > might
> > pay
> > > off to stick with standards, even if it takes more up-front
> > > investment.
> > > It's JSON-based, so it doesn't solve the high-level type problem.
> > > Very
> > few
> > > serialization formats do. Avro has a nice IDL, but that's out of
> > > the
> > hadoop
> > > world and would be foreign to the web. There is someting to be
> > > said for
> > > simplicity, but the incurred technical debt can become
> > > burdensome.
> > >
> > >
> > >> On Mon, Apr 6, 2015 at 8:39 AM, Paul Shannon
> > >> <pshannon at fredhutch.org>
> > wrote:
> > >>
> > >> Hi Michael,
> > >>
> > >> Thanks for the link to the subprotocol negotiaion discussion in
> > >> "High
> > >> Performance Browser Networking".  There's a number of things in
> > >> that
> > >> discussion of which I was not aware.  Very helpful.
> > >>
> > >> There may be real merit -- that is, a good trade-off between
> > >> some added
> > >> complexity and extra clarity -- to adding a mandatory
> > >> content-type
> > subfield
> > >> to the payload or to the message.  Or adding full sub-protocol
> > negotiation
> > >> as the reference you supplied discusses.
> > >>
> > >> For the first release of BrowserViz (and its demo subclass,
> > >> BrowserVizDemo, along with RCyjs) I would like to keep things as
> > >> simple
> > as
> > >> possible.  By which I mean:  provide (as we do now) the
> > >> mechanism by
> > which
> > >> content-type can be specified, without promulgating the policy
> > >> that
> > every
> > >> message must have that payload subfield.
> > >>
> > >> In practice I have found it quick and easy in both Javascript
> > >> and R to
> > >> examine the incoming 4-field message (cmd, status, callback,
> > >> payload):
> > >>
> > >> 1) Is the cmd field something I can respond to?  (Do I have a
> > >> handler
> > >> registered for that cmd?)
> > >> 2) Does the status indicate trouble? A request?
> > >> 3) If 1 & 2 suggest proceeding, then  names(msg$payload) can
> > >> help me
> > >> figure out if the incoming data is structured as I expect.
> > >>
> > >> One could reasonably object that this is all "programming by (ad
> > >> hoc)
> > >> agreement", rather than by transparent, self-describing, clearly
> > >> negotiatied data and exchange standards.
> > >>
> > >> In order to arrive at such standards, to figure out how formal
> > >> we should
> > >> go, let's experiment for a release cycle.  This could be
> > >> accomplished
> > >> without any changes to BrowserViz by using the status field
> > >> (status="summarizedExperimentRequest"), or by always using two
> > >> fields in
> > >> every message payload (payload$contentType, payload$content).
> > >>  If a few
> > >> people collaborated on (let's say) a SummarizedExperiment
> > >> browser viz
> > >> webapp, those people could agree on how to do this, try it out,
> > >> perhaps
> > >> discover a few sub-varieties of content-type are actually
> > >> needed.  Then,
> > >> when things have settled down, we could explore adding some
> > >> formality to
> > >> the process.
> > >>
> > >> How does that strike you?
> > >>
> > >> Thanks for the discussion!
> > >>
> > >> - Paul
> > >>
> > >>
> > >>
> > >> On Apr 3, 2015, at 3:11 PM, Michael Lawrence
> > >> <lawrence.michael at gene.com
> > >
> > >> wrote:
> > >>
> > >>> The high-level type issue is sort of discussed here:
> > >>>
> > >>
> > http://chimera.labs.oreilly.com/books/1230000000545/ch17.html#_subprotocol_negotiation
> > >>>
> > >>> What about extending your protocol so that the payload consists
> > >>> of two
> > >> fields:
> > >>> content-type and content, where the content-type adheres to the
> > >>> media
> > >> type specification? This is analogous to how every S4 object has
> > >> a class
> > >> attribute.
> > >>>
> > >>> Thanks for the discussion!
> > >>>
> > >>> On Fri, Apr 3, 2015 at 2:57 PM, Paul Shannon <
> > >> paul.thurmond.shannon at gmail.com> wrote:
> > >>> Hi Michael,
> > >>>
> > >>> Thanks for the clarification.   You make a good point about
> > >>> caching:
> > >> nobody wants to have to reinvent and re-engineer that!   If too
> > >> many
> > >> features like that become important, then the simplicity of
> > >> websockets
> > will
> > >> have an attendant cost.
> > >>>
> > >>> With regard to more formalism around the payload:  I can
> > >>> imagine that
> > >> there will be circumstances in which that is essential.  A
> > >> SummarizedExperiment browser would be very useful, and would
> > >> clearly
> > >> benefit from a standard payload structure.
> > >>>
> > >>> I figure, however, that that's above my pay-grade ;).   I leave
> > >>> that up
> > >> to people like you.  And I stand ready to add any features such
> > formalized
> > >> structures might require.  I'd like to think (I may be naive)
> > >> that the
> > >> architects of SummarizedExperiment and heavy users of it could
> > >> devise
> > and
> > >> negotiate some standard JSON representation of the class,
> > >> translators to
> > >> and from, which all could then be used by BrowserViz without
> > >> BrowserViz
> > >> needing to know it's there.
> > >>>
> > >>> The "binary JSON" format may be useful in some circumstances:
> > >> http://bjson.org
> > >>>
> > >>> - Paul
> > >>>
> > >>> On Apr 3, 2015, at 2:43 PM, Michael Lawrence <
> > lawrence.michael at gene.com>
> > >> wrote:
> > >>>
> > >>>>
> > >>>>
> > >>>> On Fri, Apr 3, 2015 at 2:00 PM, Paul Shannon <
> > >> paul.thurmond.shannon at gmail.com> wrote:
> > >>>> Hi Michael,
> > >>>>
> > >>>> Great to get your response, comments and questions.  Answers
> > >>>> attempted
> > >> below.
> > >>>>
> > >>>> I think our overriding difference lies in our contrasting
> > >>>> experience
> > of
> > >> complexity.  I have come to see websockets as minimal, simple,
> > >> fast and
> > >> flexible, whereas you see them quite differently.  I would like
> > >> to
> > >> understand your views on this; I could be overlooking somce
> > >> important
> > and
> > >> maybe costly complicating features, perhaps because of my
> > >> fondness for
> > >> other simplifying features.
> > >>>>
> > >>>>
> > >>>> I was just referring to potential complexity. If one attempted
> > >>>> to
> > >> reimplement the useful features of HTTP (like caching),
> > >> complexity
> > would be
> > >> introduced into application code, while you really want that
> > >> complexity
> > in
> > >> the protocol implementation. HTTP seems most appropriate for
> > >> when the
> > >> server is acting as the data model. Outside of that, I see the
> > >> benefits
> > of
> > >> web sockets.
> > >>>>
> > >>>> I think you missed my question about RPC. Also, any thoughts
> > >>>> as to
> > more
> > >> formalism around the payload? We obviously have complex data
> > >> structures,
> > >> and it would be nice to communicate semantics somehow to the web
> > browser.
> > >> For example, could there be some convention for representing a
> > >> SummarizedExperiment? Could a payload contain the equivalent of
> > >> a media
> > >> type that an R/JS library could understand to marshal objects?
> > >> Could
> > there
> > >> be some way to query for the types of payload a command
> > >> supports?
> > >>>>
> > >>>> I've seen stuff like WAMP, but they seem to lack the ability
> > >>>> to
> > declare
> > >> high-level types. Maybe that's just out of style?
> > >>>>
> > >>>>
> > >>>> More below...
> > >>>>
> > >>>> I look forward to hearing back from you.
> > >>>>
> > >>>> - Paul
> > >>>>
> > >>>> On Apr 3, 2015, at 1:45 PM, Michael Lawrence <
> > lawrence.michael at gene.com>
> > >> wrote:
> > >>>>
> > >>>>> Thanks to Val's excellent newsletter, I've had my first
> > >>>>> glance at
> > >>>>> BrowserViz. I'm glad to see something that is more flexible
> > >>>>> and
> > >> low-level
> > >>>>> than e.g. shiny.
> > >>>>>
> > >>>>> I'm curious about the motivation behind web sockets. I guess
> > >>>>> any
> > >>>>> application with an R-driven web UI actually has two UIs: the
> > >>>>> R
> > >> console and
> > >>>>> the browser. But what if the R session is headless, or if
> > >>>>> there is no
> > >> need
> > >>>>> for commands in R to affect the browser? Then the web socket
> > >>>>> layer
> > >> brings
> > >>>>> mostly unneeded complexity.
> > >>>>
> > >>>> I see websockets (like TCP sockets) as musch simpler than
> > >>>> HTTP.  No
> > >> headers,
> > >>>> no explicit server and explicit client (once the connection is
> > >>>> open).
> > >>>> Could you explain the complexity you see?
> > >>>>
> > >>>>> An interesting comparison to BrowserViz is not
> > >>>>> shiny but OpenCPU. It's purely HTTP-based and still manages
> > >>>>> to
> > maintain
> > >>>>> state (not sure how efficiently). I guess one advantage of
> > >>>>> web
> > sockets
> > >> is
> > >>>>> that one can program imperatively instead of declaratively on
> > >>>>> the
> > >> server,
> > >>>>> i.e., the server can send a command to show a popup in
> > >>>>> response to
> > some
> > >>>>> event, instead of returning a "declaration" that the popup
> > >>>>> should be
> > >> shown.
> > >>>>
> > >>>> Exactly!
> > >>>>
> > >>>>>
> > >>>>> So essentially web sockets are more natural for implementing
> > >> server-side
> > >>>>> controllers (think MVC), instead of just the data model, but
> > >>>>> man,
> > it's
> > >> a
> > >>>>> shame to lose the features of HTTP.
> > >>>>
> > >>>> I'll confess:  I set out to -shed- the features of HTTP.
> > >>>>  Isn't it a
> > >> protocol designed for serving up web pages on demand?  Not for
> > >> fast,
> > >> lighweight peer-to-peer communications?
> > >>>>
> > >>>>
> > >>>>>
> > >>>>> Ultimately, I think we want web apps that are easy to develop
> > >>>>> and
> > >> maintain,
> > >>>>> and run equally well from either a useR's session or a remote
> > >>>>> client
> > >>>>> communicating to a dedicated, headless server. Is the
> > >>>>> generality of
> > >>>>> websockets worth the complexity?
> > >>>>
> > >>>>>
> > >>>>> As an aside, it would seem relatively straight-forward to
> > >>>>> implement a
> > >>>>> simple bi-directional RPC mechanism between R and JS using
> > >>>>> the
> > standard
> > >>>>> protocol (i.e., hide details like the callback). Does that
> > >>>>> sound
> > >> reasonable?
> > >>>>>
> > >>>>> I was also a bit surprised about the need to copy/paste the
> > >>>>> JS
> > >> boilerplate.
> > >>>>> Certainly there must be javascript frameworks with a more
> > >>>>> elegant
> > >> solution
> > >>>>> to extensibility.
> > >>>>
> > >>>> You are correct on this, and I only need to correct the
> > >>>> vignette,
> > >> written many weeks ago.
> > >>>> I now provide "BrowserViz.jx", a simple Javascript module.  It
> > >>>> is used
> > >> by both
> > >>>> BrowserVizDemo and RCyjs.  Sorry to have sent out an obsolete
> > vignette.
> > >>>>
> > >>>>>
> > >>>>>     [[alternative HTML version deleted]]
> > >>>>>
> > >>>>> _______________________________________________
> > >>>>> Bioc-devel at r-project.org mailing list
> > >>>>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
> > >>>>
> > >>>>
> > >>>
> > >>>
> > >>
> > >>
> > >
> > >    [[alternative HTML version deleted]]
> > >
> > > _______________________________________________
> > > Bioc-devel at r-project.org mailing list
> > > https://stat.ethz.ch/mailman/listinfo/bioc-devel
> >
> 
> 	[[alternative HTML version deleted]]
> 
> _______________________________________________
> Bioc-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>

More information about the Bioc-devel mailing list