[BioC] plans for binary version of *.CEL files

[Rafael A. Irizarry]

thanks to David Kulp for his clear explanation. here are some further 
points that Ben Bolstad, Robert Gentleman and I see as relevant to 
Bioconductor users:
 
 Affymetrix is switching (slowly) to a proprietary format. They will, as we
 understand it, have CEL files (in their current format) for one year.
 
 When they switch we are unlikely to be able to make use of the new
 format. Writing code to read proprietary formats is illegal in the USA.
 Bioconductor will therefore *not* accept contributions that do this.
 We simply do not have the legal resources to determine what is or is
 not legal. This means that we must rely on the public APIs
 described by David Kulp below.

 As we see it now, it is likely that there will be two steps
 involved in the reading of probe level data into Bioconductor packages: 
 1) converting the proprietary format to a format comaptible with  
 bioconductor 2) reading in the files using the affy package. Under these 
 conditions we expect a decrease in performance. Users unhappy with  this 
 should make their concerns known to their Affymetrix representative or 
 bring them up in the Affymetrix Developer Network (as opposed 
 to the Bioconductor mailing list).
 
On Wed, 25 Jun 2003, Kulp, David wrote:

> The details are fairly simple.
> 
> Binary formats are being introduced to reduce file size and speed data
> access.  Developers have basically two options:
> 
> 1. Affymetrix sells a developer's license to a linkable library for reading
> GeneChip file formats.  (There are also APIs/libraries available for
> accessing backend servers.)  Currently Affymetrix provides Windows and java
> implementations.  There are no end-user costs or distribution restrictions
> on software that is compiled using this library.  Thus, Bioconductor
> developers can choose to write to the API or provide a compile-time switch
> to do so for those with the Affymetrix library.  Anyone compiling from
> source would need a license.  Bioconductor can provide free precompiled
> binaries.
> 
> 2. Affymetrix provides a MAGE-ML exporter (available for download from the
> website) which outputs data into an ASCII format that is open and easily
> parseable.  It contains all of the "CEL" data you expect.  There is no cost
> to use this format.  Because the individual cell data is stored in an
> accompanying ASCII file, there is no significant "XML bloat".  
> 
> The first option may be the preferred option in terms of speed and long-term
> compatibility.  It also is the easiest to use for the end-user because no
> conversion is required.  The second option is simple, free, and comparable
> to performance of the current ASCII CEL files.
> 
> I recommend joining the Affymetrix Developer Network for more information.
> 
> David Kulp
> 
> -----Original Message-----
> From: Warnes, Gregory R [mailto:gregory_r_warnes at groton.pfizer.com] 
> Sent: Wednesday, June 25, 2003 7:37 AM
> To: 'James MacDonald'; sgh at breastcenter.tmc.edu; ririzarr at jhsph.edu;
> rafa at jhu.edu
> Cc: bioconductor at stat.math.ethz.ch
> Subject: RE: [BioC] plans for binary version of *.CEL files
> 
> 
> I second that.
> 
> -G
> 
> > -----Original Message-----
> > From: James MacDonald [mailto:jmacdon at med.umich.edu]
> > Sent: Wednesday, June 25, 2003 10:17 AM
> > To: sgh at breastcenter.tmc.edu; ririzarr at jhsph.edu; rafa at jhu.edu
> > Cc: bioconductor at stat.math.ethz.ch
> > Subject: Re: [BioC] plans for binary version of *.CEL files
> > 
> > 
> > All I ever do these days is analyze Affy data, so I am very interested
> > in knowing the details. I promise I won't be bored ;-D
> > 
> > Jim
> > 
> > 
> > 
> > James W. MacDonald
> > UMCCC Microarray Core Facility
> > 1500 E. Medical Center Drive
> > 7410 CCGC
> > Ann Arbor MI 48109
> > 734-647-5623
> > 
> > >>> ririzarr at jhsph.edu 06/25/03 09:41AM >>>
> > there are several 
> > alternatives we are considering. i wont bore you with details (unless
> > you 
> > ask me to). according 
> > to affymetrix, ascii cel files will be supproted for one more 
> > year. but
> > 
> > there is a possibility that things will become more difficult for 
> > bioconductor  users in the future.  bioconductor is talking to
> > affymetrix 
> > about this issue. 
> > 
> > you may want to contact your affymetrix sales rep and explain your 
> > situation.
> > 
> > 
> > On Wed, 25 Jun 2003, Susan G. Hilsenbeck wrote:
> > 
> > > How does, or how will the affy packages in bioconductor handle the
> > newer 
> > > binary versions of the *.cel files?
> > > 
> > >
> > 
> > _______________________________________________
> > Bioconductor mailing list
> > Bioconductor at stat.math.ethz.ch 
> > https://www.stat.math.ethz.ch/mailman/listinfo/bioconductor
> > 
> > _______________________________________________
> > Bioconductor mailing list
> > Bioconductor at stat.math.ethz.ch
> > https://www.stat.math.ethz.ch/mailman/listinfo/bioconductor
> > 
> 
> 
> LEGAL NOTICE\ Unless expressly stated otherwise, this messag...{{dropped}}
> 
> _______________________________________________
> Bioconductor mailing list
> Bioconductor at stat.math.ethz.ch
> https://www.stat.math.ethz.ch/mailman/listinfo/bioconductor
> 
>