[R-sig-teaching] importing and processing large datasets in R (fwd)

[Bob]

I am one of the people who lobbied for the creation of this list long
ago.  I am not sure R is a great choice for a first course in
statistics, but I thought that if someone chose to use it, then they
and their students might need all the help they could get to make it
easier for the class.  But right from the beginning, the bulk of the
posts to the list were like this latest one quoted below -- questions
about how to do something with R that has no obvious connection to
pedagogy or to using R in a first course.  This means that those of us
interested in the actual topic of this list get lots of off-topic
messages, while those who post the messages reach only a small
audience that may not be interested in their question.  Some off topic
posts are answered, some ingnored, and some posters get redirected
(even scolded) toward a more appropriate list.  I see only losers in
this process.

So my question is whether this list really serves any useful purpose,
or does it just siphon off queries that should have gone elsewhere?
Those who post those queries would be likely to get an answer, and get
it sooner, if they posted to an appropriate list in the first place.
My own answer is that this list is not useful at the present time.
Possibly in the future more people will be interested in R for an
introductory course and then they might be glad if this list were
still alive, but so far...

So I am wondering what others on the list think.

Here's the official description of this list.

Special Interest Group (SIG) on teaching statistics with R. The
primary purpose of the group is to provide a forum where instructors
using R in their statistics courses can share ideas, teaching
materials, and experiences. One particular focus of the SIG is to
provide helpful support to instructors new to R who are teaching
introductory statistics courses populated with students with little
experience in statistics, statistical software, and command line
interfaces. 

Here is where most posts to this list really should have gone.

R-help

    The ?main? R mailing list, for discussion about problems and
    solutions using R, announcements (not covered by ?R-announce? or
    ?R-packages?, see above), about the availability of new
    functionality for R and documentation of R, comparison and
    compatibility with S-plus, and for the posting of nice examples
    and benchmarks.

Forwarded message:
> 
> Hi Everyone,
> 
> I am a little new to R and the first problem I am facing is the dilemma
> whether R is suitable for files of size 2 GB's and slightly more then 2
> Million rows. When I try importing the data using read.table, it seems to
> take forever and I have to cancel the command. Are there any special
> techniques or methods which i can use or some tricks of the game that I
> should keep in mind in order to be able to do data analysis on such large
> files using R?


------->  First-time AP Stats. teacher?  Help is on the way! See
http://courses.ncssm.edu/math/Stat_Inst/Stats2007/Bob%20Hayden/Relief.html
      _
     | |          Robert W. Hayden
     | |          142 Main Street
    /  |          Apartment 104
   |   |          Jaffrey, New Hampshire 03452  USA
   |   |          email: bob@ the site below
  /    |          website: http://statland.org
 | x   /          phone: (603) 532-7224 (home)
 ''''''