[R] Thoughts for faster indexing

[Rainer M Krug]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



On 11/21/13, 12:34 , Jim Holtman wrote:
> you need to show the statement in context with the rest of the
> script.  you need to tell us what you want to do, not how you want
> to do it.

Agreed - a few details will result in guesses (see my guess below)

> 
> Sent from my iPad
> 
> On Nov 20, 2013, at 15:16, Noah Silverman
> <noahsilverman at g.ucla.edu> wrote:
> 
>> Hello,
>> 
>> I have a fairly large data.frame.  (About 150,000 rows of 100 
>> variables.) There are case IDs, and multiple entries for each ID,
>> with a date stamp.  (i.e. records of peoples activity.)
>> 
>> 
>> I need to iterate over each person (record ID) in the data set,
>> and then process their data for each date.  The processing part
>> is fast, the date part is fast.  Locating the records is slow.
>> I've even tried using data.table, with ID set as the index, and
>> it is still slow.
>> 
>> The line with the slow process (According to Rprof) is:
>> 
>> 
>> j <- which( d$id == person )

Possibly use

d_by_id <- split(d, d$id)

which splits the data.frame d into a listt, where each list represents
the data.frame of one id.

But: Just a guess.

Cheers,

Rainer

>> 
>> (I then process all the records indexed by j, which seems fast
>> enough.)
>> 
>> where d is my data.frame or data.table
>> 
>> I thought that using the data.table indexing would speed things
>> up, but not in this case.
>> 
>> Any ideas on how to speed this up?
>> 
>> 
>> Thanks!
>> 
>> -- Noah Silverman, M.S., C.Phil UCLA Department of Statistics 
>> 8117 Math Sciences Building Los Angeles, CA 90095
>> 
>> ______________________________________________ 
>> R-help at r-project.org mailing list 
>> https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the
>> posting guide http://www.R-project.org/posting-guide.html and
>> provide commented, minimal, self-contained, reproducible code.
> 
> ______________________________________________ R-help at r-project.org
> mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do
> read the posting guide http://www.R-project.org/posting-guide.html 
> and provide commented, minimal, self-contained, reproducible code.
> 

- -- 
Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation
Biology, UCT), Dipl. Phys. (Germany)

Centre of Excellence for Invasion Biology
Stellenbosch University
South Africa

Tel :       +33 - (0)9 53 10 27 44
Cell:       +33 - (0)6 85 62 59 98
Fax :       +33 - (0)9 58 10 27 44

Fax (D):    +49 - (0)3 21 21 25 22 44

email:      Rainer at krugs.de

Skype:      RMkrug
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJSjg/+AAoJENvXNx4PUvmC/kcH/3eaMvOTCAvA6ewwH/XJHH6X
B4BgstscvvJ3yArFeWqLIV0CEgk3da4c28+4Jk50vnltRVwUieFxKA1UK6Ef3gPl
pZUg9TaUNAeHPfkrxQSrYIa+hLWMZ1Ybe6GM1OlXnkc9ZBT9KS+rX3HFfr9rdyFI
Rv7SgrylUnpZIyiMeAzQS/FBzozV3G6mGu8FJ8YW5mHCqajI2alK3B3BBREzuLsL
ZMSuFDPTzxrE63O+uU6yFDibhz/4chKVz6CEF52WUgpgP+X4rW/DcLDrDfXxEvwM
ZDHcOZ8FJsuDl1lb1bdzSyS61KfzWls37i9VtOozqQwSFbaHbcdV16jHCPDzRPA=
=u5ol
-----END PGP SIGNATURE-----