Thanks Laura,

I deal with huge data sets and have to do alot of fancy juggling of data to get the job done in R.

I have recently been granted access to a cluster at a university which means 64 bit machines with 8gb of memory, which could prove to be a saviour.... hopefully.

Simon.


  ----- Original Message ----- 
  From: Laura Arsanto 
  To: simon.pickett@bto.org 
  Sent: Wednesday, May 06, 2009 3:09 PM
  Subject: RE: [R] Do you use R for data manipulation?



  dear simon,

  my job now is doing a benchmark between SAS and open source tools (like R, weka, etc) for data and text mining, so I'm using both of them.

  personally I would prefer using the open source tools and I really would do it if possible, but at the moment...they simply do not work! and 99% of cases because of the dimensions of data!


  > From: simon.pickett@bto.org
  > To: ghina84@hotmail.it; jrkrideau@yahoo.ca; r-help@stat.math.ethz.ch; fjbuch@gmail.com
  > CC: ross.lazarus@gmail.com; gregory_warnes@urmc.rochester.edu; greg@warnes.net
  > Subject: Re: [R] Do you use R for data manipulation?
  > Date: Wed, 6 May 2009 15:01:33 +0100
  > 
  > My institute uses SAS religiously, I am the only R "heathen".
  > 
  > I have resisted learning to use SAS because I dont see the point after years 
  > of using R and I like being able to do everything using one program. 
  > However, my colleagues maintain that SAS is "better" for programming without 
  > really ever giving me a good reason why other than memory issues.
  > 
  > dont want to hi-jack the thread but would be interested in hearing some 
  > other views, especially since my organisation spends (wastes?) alot of money 
  > every year on SAS licences...
  > 
  > Simon.
  > 
  > ----- Original Message ----- 
  > From: "Laura Arsanto" <ghina84@hotmail.it>
  > To: <jrkrideau@yahoo.ca>; <r-help@stat.math.ethz.ch>; <fjbuch@gmail.com>
  > Cc: <ross.lazarus@gmail.com>; <gregory_warnes@urmc.rochester.edu>; 
  > <greg@warnes.net>
  > Sent: Wednesday, May 06, 2009 2:53 PM
  > Subject: Re: [R] Do you use R for data manipulation?
  > 
  > 
  > 
  > 
  > I used R for my master thesis (with big effort, anyway) and now I find 
  > difficult to use R in my daily work, becasue it has really serious problems 
  > with datasets of big dimension, both in the data manipulation step and in 
  > the analysis step.
  > 
  > But I really would love to use it, as I like its transparence, compared to 
  > other software.
  > 
  > Laura
  > 
  > ***********
  > 
  > > Date: Wed, 6 May 2009 06:42:45 -0700
  > > From: jrkrideau@yahoo.ca
  > > To: r-help@stat.math.ethz.ch; fjbuch@gmail.com
  > > CC: ross.lazarus@gmail.com; gregory_warnes@urmc.rochester.edu; 
  > > greg@warnes.net
  > > Subject: Re: [R] Do you use R for data manipulation?
  > >
  > >
  > >
  > >
  > > --- On Wed, 5/6/09, Farrel Buchinsky <fjbuch@gmail.com> wrote:
  > >
  > > > Is R an appropriate tool for data
  > > > manipulation and data reshaping and data
  > > > organizing? I think so but someone who recently joined our
  > > > group thinks not.
  > >
  > > I only do small scale projects and am by no means a programmer. Isn't Perl 
  > > something for earings?
  > >
  > > That said, I find R to be extremely useful at data manipulation and have 
  > > used it exclusively in my last three projects. The different data 
  > > structures alone are worth their weight in gold, if for nothing else than 
  > > making it harder to make stupid mistakes in coding.
  > >
  > > > The new recruit believes that python or another language is
  > > > a far better tool for developing data manipulation scripts that can be
  > > > then used by> several members of our research group. Her assessment is
  > > > that R is useful> only when it comes to data analysis and working with
  > > > statistical models.
  > >
  > > Any reason that she thinks this? How well does she know R? It is not 
  > > exactly a language that one picks up in a week, especially if one is 
  > > coming from using a stats package like SAS or SPSS. As an ex-SAS and 
  > > SYSTAT user it took me weeks to just get comfortable with the power of 
  > > subscripting and the ability to do all kinds of calculations "in-line".
  > >
  > > > So what do you think:
  > > > 1)R is a phenomenally powerful and flexible tool and since you are going 
  > > > > to do analyses in R you might as well use it to read data in and merge
  > > > it and reshape it to whatever you need.
  > >
  > > Definately. I am not a computer scientist or a statistician. I usually am 
  > > working as a single contractor and normally with small datasets as part of 
  > > a larger project. R does what I want, usually very elegantly (albeit 
  > > perhaps after a lot of headbanging and calls for help to the R-list) and 
  > > it would be stupid for me to use more than one language when it is not 
  > > needed.
  > >
  > > Another plus is that I can easily leave my data analysis work and a 
  > > working copy of R with the client. He/she may have a problem seeing what 
  > > I did but it is clearly readable & replicable by either the client or 
  > > another consultant.
  > >
  > > > OR
  > > > 2) Are you crazy? Nobody in their right mind uses R to pipe
  > > > the data around their lab and assemble it for analysis.
  > >
  > > Well I don't work in a lab but why complicate things? If everyone is using 
  > > the same tools then you have a good situation. Others who do work in labs 
  > > can address this point more cogently
  > >
  > > >From a personnel point of view do you expect everyone in the lab to be 
  > > >proficient with R and, for example, Perl? What happens when/if you lose 
  > > >your Perl expert(s)? I've had occasions where I waited a week for data 
  > > >simply because the division's MS Access "expert" was on holiday and the 
  > > >only other "Access" person there only knew how to enter data and run the 
  > > >monthly reports. Anything more complicated required the "expert".
  > >
  > >
  > >
  > >
  > >
  > > __________________________________________________________________
  > > Make your browsing faster, safer, and easier with the new Internet 
  > > Explorer® 8. Optimized for Yahoo! Get it Now for Free! at 
  > > http://downloads.yahoo.com/ca/internetexplorer/
  > >
  > > ______________________________________________
  > > R-help@r-project.org mailing list
  > > https://stat.ethz.ch/mailman/listinfo/r-help
  > > PLEASE do read the posting guide 
  > > http://www.R-project.org/posting-guide.html
  > > and provide commented, minimal, self-contained, reproducible code.
  > 
  > _________________________________________________________________
  > [[elided Hotmail spam]]
  > 
  > [[alternative HTML version deleted]]
  > 
  > 
  > 
  > 
  > --------------------------------------------------------------------------------
  > 
  > 
  > > ______________________________________________
  > > R-help@r-project.org mailing list
  > > https://stat.ethz.ch/mailman/listinfo/r-help
  > > PLEASE do read the posting guide 
  > > http://www.R-project.org/posting-guide.html
  > > and provide commented, minimal, self-contained, reproducible code.
  > > 
  > 


------------------------------------------------------------------------------
  Tante nuove funzioni su Messenger! Scarica la nuova versione! 
	[[alternative HTML version deleted]]


