[R-sig-DB] request for examples

[M. Edward Borasky]

Well ... I don't know if the company I work for will let me send out any
real data, but I can tell you what I do with databases and R. I collect
large quantities of Linux and Windows performance data. For example, I will
have perhaps 50 - 250 columns of high-frequency samples taken, say, every 15
seconds over a 12-hour period. A typical benchmarking project will take two
weeks (Saturdays and Sundays included!), giving about a dozen test cases to
be processed. Excel can deal with the columns all right, but after 65536
rows it rolls over and plays dead. And even if I just left the files in CSV
format, R's "read.csv" function runs out of memory on my 128 MB workstation
fairly quickly - somewhere in the vicinity of a 15 or 20 MB CSV file.

So, I load all the raw data into tables in a Microsoft Access database. I
then write queries to format the data, add case tags (factors) to the rows,
do some date/time calculations, etc. Then I assign a Data Set Name to the
".mdb" file and read from those queries using RODBC. I'm told all this magic
can be made to work on Linux with PostGres, but we're mostly a Windows shop
so I have Access and SQL Server available. Given the quantity of data I
have, Access and SQL help me organize things as well, plus I can do much of
the inevitable data cleaning much more easily with Access queries than I can
in R. As far as I'm concerned it's a match made in heaven.

-----Original Message-----
From: r-sig-db-admin using stat.math.ethz.ch
[mailto:r-sig-db-admin using stat.math.ethz.ch]On Behalf Of Paul Murrell
Sent: Sunday, May 12, 2002 7:13 PM
To: r-sig-db using stat.math.ethz.ch
Subject: [R-sig-DB] request for examples

Hi

I hope you don't mind this "cold call", but this seems like a really
good place to contact people with interest/experience/expertise in stats
and databases ...

I am busy producing a course on statistical computing for stage II
students (to be delivered in the second half of this year).

I will be teaching them about some databases issues:  advantages of
databases as a way to store information, how to design databases
properly, how to retrieve information using SQL.

What I am seriously lacking are some killer examples.

Would anyone be able to help me with any of the following ...

(i)  killer examples where a database is clearly a superior method of
storing information than, say, plain text files or spreadsheets or
statistical-package-specific formats

(ii)  an actual real-life statistical database that could be copied to a
local server for the students to practise accessing

(iii)  killer examples where an important data source is stored in a
database therefore requiring something like SQL knowledge to get access
to the information.

I would also obviously be interested in any general comments regarding
which database issues people think are the most crucial for statistics
students to learn.

Again, apologies if this approach is an imposition.
Any help would be greatly appreciated.

Paul

Dr Paul Murrell
Department of Statistics
The University of Auckland
Auckland
New Zealand
_______________________________________________
R-sig-DB mailing list -- R Special Interest Group
R-sig-DB using stat.math.ethz.ch
http://www.stat.math.ethz.ch/mailman/listinfo/r-sig-db