[R] Announcing TIBCO Enterprise Runtime for R

Wed Jul 17 02:06:34 CEST 2013

Hi Michael

Thanks for your questions. Posting them here is fine, or you can also post them on the TERR Community site: https://www.tibcommunity.com/community/products/analytics/terr.

Initial answers below--feel free to ask follow ups:

> 1. Do you have concrete benchmarks of what sorts of operations are faster? In particular, are you, like Revolution, licensing MKL?

Yes we have benchmarks, and yes we license MKL. It's challenging to come up with comprehensive, meaningful benchmarks, and so we are looking to the community for benchmarks they'd like to see. Feel free to post anything you are interested in on the community site, or email me directly. 
- Generally we will be much faster for pure R language operations (e.g., for loops), since we have written a new, R-compatible interpreter from the ground up. The performance ratio will tend to improve with larger data sets, since we have implemented highly efficient memory management.
- Since we license MKL (and distribute it in the free Developer's Edition), we will have all the related matrix-operation speed advantages.
- For more detailed info and some benchmarks, see the presentation that Michael Sannella (the chief architect for TERR) gave at useR 2013. It's now available on slideshare: http://www.slideshare.net/loubajukyorgan/sannella-use-r2013terrmemorymanagement 

> 2. Your white-paper boasts of superior memory management; can you give more details? Are we now reference counting, or abandoning copy-on-write to play better with fork(), or running concurrently? In turn, does this mean the C-API is not kept as is and any package with compiled code isn't usable?

See Michael's presentation "Memory Management in TERR", referenced above for full details. In particular, yes, full reference counting has been implemented. 

Our goal is to have packages with compiled code work as-is, without modification. To do this, since the internals of our engine are different, we need to emulate the APIs for these packages to work. We have done this for many APIs and packages so far. See the Community site for a list of packages known to run successfully. 

> 3. You speak of being more robust: is this at a language level (i.e.,something simpler to use than try()/tryCatch()) or an implementation level? If the latter, what non-robustnesses are being addressed?

Primarily, the greater robustness comes from the improvements to memory management. Beyond that, we do extensive automated testing (using the deep array of tests we have built up over the years for S+ development) to ensure reliability, and we have done some work at the language level (around improvements to signal handling, throwing and catching errors, etc.). If you'd like more info the last point, email me directly and I can get you in touch with Michael.

> 4. What are y'all's plans for supporting TERR as 'regular R' evolves? Will it track or will things diverge over time?

Our intent is to keep up with current R as closely as possible. In talking to our customers, we have heard loud and clear that they love the idea of an enterprise R engine, but they don't want to be locked into a proprietary flavor of R. 

> 5. Known (and not intended to change) differences?

Very minor items. Full list on the Community Site. 

> 6. Is the whole R-level API exposed or only a selected subset? I'm thinking in particular of (1) things like .colMeans() which seem rather tied to the implementation; (2) graphics devices; (3) the promise mechanism and copy-on-write semantics?

As mentioned in #2 above, we are implementing the R APIs over time. For a full list of what we have and haven't yet implemented, check out the community site, and feel free to leave comments about what you'd like us to prioritize next. 

Regards, and thanks again for your interest.
Lou

-----Original Message-----
From: R. Michael Weylandt [mailto:michael.weylandt at gmail.com] 
Sent: Thursday, July 11, 2013 1:59 PM
To: Louis Bajuk-Yorgan
Cc: r-help
Subject: Re: [R] Announcing TIBCO Enterprise Runtime for R

Hi Louis,

I apologize in advance if this isn't the right forum; feel free to direct me elsewhere.

Can you say a bit more about what exactly constitute the advantages of TERR over R as most readers of this list know it? Some random points of interest to me:

1. Do you have concrete benchmarks of what sorts of operations are faster? In particular, are you, like Revolution, licensing MKL?
2. Your white-paper boasts of superior memory management; can you give more details? Are we now reference counting, or abandoning copy-on-write to play better with fork(), or running concurrently? In turn, does this mean the C-API is not kept as is and any package with compiled code isn't usable?
3. You speak of being more robust: is this at a language level (i.e.,something simpler to use than try()/tryCatch()) or an implementation level? If the latter, what non-robustnesses are being addressed?
4. What are y'all's plans for supporting TERR as 'regular R' evolves?
Will it track or will things diverge over time?
5. Known (and not intended to change) differences?
6. Is the whole R-level API exposed or only a selected subset? I'm thinking in particular of (1) things like .colMeans() which seem rather tied to the implementation; (2) graphics devices; (3) the promise mechanism and copy-on-write semantics?

Cheers,
Michael

On Wed, Jul 10, 2013 at 4:42 AM, Louis Bajuk-Yorgan <lbajuk at tibco.com> wrote:
> In honor of the kickoff of useR 2013 today, I'm proud to announce the availability of TIBCO Enterprise Runtime for R (or TERR for short), our new enterprise-grade, high-performance statistical engine, fully compatible with the R language.
>
> For more information on TERR, and a link to download the free Developer's Edition via the TERR Community site, check out http://spotfire.tibco.com/terr--or come to my talk at useR on Thursday morning.
>
> As part of our development of TERR, we have also contributed new packages to CRAN: sjdbc (a JDBC driver interface, previously developed for S-PLUS) and tibbrConnector (an R interface to tibbr, TIBCO's Social Network for the Enterprise).
>
> ----------------------------------
> Lou Bajuk-Yorgan
> @loubajuk
> TIBCO Spotfire
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide 
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.