Thanks for your help!  Your original solution would have worked just fine
too if I didn't have a number of other large data frames loaded.  This is
my first time working with such large data sets, so I'd never previously
run out of available memory.  I finished up the project, though I'm sure
all be back to this mailing list in the future!


On Sun, Jan 27, 2013 at 11:54 AM, arun kirshna [via R] <
ml-node+s789695n4656784h48@n4.nabble.com> wrote:

> Hi,
> I tried with bigger dataset.
>
> set.seed(25)
> names <- sample(c("bob", "joe", "[hidden email]<http://user/SendEmail.jtp?type=node&node=4656784&i=0>",
> "emily", "[hidden email]<http://user/SendEmail.jtp?type=node&node=4656784&i=1>"),5e6,replace=TRUE)
>
> set.seed(1651)
> emails
>  <- sample(c("[hidden email]<http://user/SendEmail.jtp?type=node&node=4656784&i=2>",
> "[hidden email] <http://user/SendEmail.jtp?type=node&node=4656784&i=3>", "[hidden
> email] <http://user/SendEmail.jtp?type=node&node=4656784&i=4>",
>  "[hidden email] <http://user/SendEmail.jtp?type=node&node=4656784&i=5>",
> "[hidden email] <http://user/SendEmail.jtp?type=node&node=4656784&i=6>"),5e6,replace=TRUE)
>
>
>  df <- data.frame(names, emails)
>  dim(df)
> #[1] 5000000       2
>  df[]<-lapply(df,as.character)
>  system.time(df[,1][grep("@",df$names)]<- "" )
> #   user  system elapsed
> #  1.732   0.108   1.844
>  system.time(dfNew1<-df[grep("\\w+",df$names),])
> #   user  system elapsed
> #  0.896   0.024   0.923
>  system.time(dfNew2<- df[df$names!="",])
> #   user  system elapsed
>  # 0.460   0.028   0.490
> A.K.
>
>
>
>
>
>
>
> ________________________________
> From: Yasha Podeswa <[hidden email]<http://user/SendEmail.jtp?type=node&node=4656784&i=7>>
>
> To: arun <[hidden email]<http://user/SendEmail.jtp?type=node&node=4656784&i=8>>
>
> Cc: R help <[hidden email]<http://user/SendEmail.jtp?type=node&node=4656784&i=9>>;
> Uwe Ligges <[hidden email]<http://user/SendEmail.jtp?type=node&node=4656784&i=10>>
>
> Sent: Sunday, January 27, 2013 2:05 PM
> Subject: Re: [R] Removing values containing a specific character
>
>
> You two were 100% right, it was just a memory issue.  This was part of a
> bigger project where I had a number of data frames loaded, all with 1-5
> million rows. Cleaned up my code to have less data frames loaded at once,
> and everything is working great.  Thanks for the help!
> On Jan 27, 2013 9:46 AM, "arun" <[hidden email]<http://user/SendEmail.jtp?type=node&node=4656784&i=11>>
> wrote:
>
> Hi Yasha,
>
> >
> > I guess you got Uwe's response.
> >
> > I created `df2` with the intention of getting the two results from the
> original dataset.
> >For example, after you get the first result
> >df[,1][grep("@",df$names)]<- ""
> >#you can get the second result by:
> >df[df$names!="",]
> > # names             emails
> >#1   bob       [hidden email]<http://user/SendEmail.jtp?type=node&node=4656784&i=12>
> >#2   joe [hidden email]<http://user/SendEmail.jtp?type=node&node=4656784&i=13>
> >#4 emily   [hidden email]<http://user/SendEmail.jtp?type=node&node=4656784&i=14>
> >
> >#or
> >df[grep("\\w+",df$names),]
> >#  names             emails
> >#1   bob       [hidden email]<http://user/SendEmail.jtp?type=node&node=4656784&i=15>
> >#2   joe [hidden email]<http://user/SendEmail.jtp?type=node&node=4656784&i=16>
> >#4 emily   [hidden email]<http://user/SendEmail.jtp?type=node&node=4656784&i=17>
> >
> >But, I am  not sure how this will work over a 5.5 million rows.
> >A.K.
> >
> >
> >
> >
> >----- Original Message -----
> >From: ypodeswa <[hidden email]<http://user/SendEmail.jtp?type=node&node=4656784&i=18>>
>
> >To: [hidden email]<http://user/SendEmail.jtp?type=node&node=4656784&i=19>
> >Cc:
> >Sent: Sunday, January 27, 2013 1:11 AM
> >Subject: Re: [R] Removing values containing a specific character
> >
> >Actually, it worked perfectly for my sample data, but my actual data has
> >5.5 million rows, and grep doesn't seem to work with over a million rows.
> >Any idea on a workaround?
> >
> >
> >On Sat, Jan 26, 2013 at 9:37 PM, Yasha Podeswa <[hidden email]<http://user/SendEmail.jtp?type=node&node=4656784&i=20>>
> wrote:
> >
> >> Awesome, thanks Arun, that's exactly what I was looking for!
> >>
> >>
> >> On Sat, Jan 26, 2013 at 9:21 PM, arun kirshna [via R] <
> >> [hidden email] <http://user/SendEmail.jtp?type=node&node=4656784&i=21>>
> wrote:
> >>
> >>> Hi,
> >>> Try this:
> >>> df[]<-lapply(df,as.character)
> >>> df2<-df
> >>> df[,1][grep("@",df$names)]<- ""
> >>> df
> >>>   #names             emails
> >>> #1   bob      [hidden email]<http://user/SendEmail.jtp?type=node&node=4656784&i=22>
> >>> #2   joe [hidden email]<http://user/SendEmail.jtp?type=node&node=4656784&i=23>
> >>> #3          [hidden email]<http://user/SendEmail.jtp?type=node&node=4656784&i=24>
> >>> #4 emily  [hidden email]<http://user/SendEmail.jtp?type=node&node=4656784&i=25>
> >>> #5          [hidden email]<http://user/SendEmail.jtp?type=node&node=4656784&i=26>
> >>>
> >>> #2nd part:
> >>>
> >>>  df2[-grep("@",df2$names),]
> >>>   names             emails
> >>> #1   bob      [hidden email]<http://user/SendEmail.jtp?type=node&node=4656784&i=27>
> >>> #2   joe [hidden email]<http://user/SendEmail.jtp?type=node&node=4656784&i=28>
> >>> #4 emily  [hidden email]<http://user/SendEmail.jtp?type=node&node=4656784&i=29>
> >>> A.K.
> >>>
> >>> ------------------------------
> >>>  If you reply to this email, your message will be added to the
> >>> discussion below:
> >>>
> >>>
> http://r.789695.n4.nabble.com/Removing-values-containing-a-specific-character-tp4656744p4656749.html
> >>>  To unsubscribe from Removing values containing a specific character,
> click
> >>> here<
>
> >>> .
> >>> NAML<
> http://r.789695.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
> >>>
> >>
> >>
> >
> >
> >
> >
> >--
> >View this message in context:
> http://r.789695.n4.nabble.com/Removing-values-containing-a-specific-character-tp4656744p4656751.html
> >Sent from the R help mailing list archive at Nabble.com.
> >    [[alternative HTML version deleted]]
> >
> >______________________________________________
> >[hidden email] <http://user/SendEmail.jtp?type=node&node=4656784&i=30>mailing list
> >https://stat.ethz.ch/mailman/listinfo/r-help
> >PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> >and provide commented, minimal, self-contained, reproducible code.
> >
> >
>
> ______________________________________________
> [hidden email] <http://user/SendEmail.jtp?type=node&node=4656784&i=31>mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>
> ------------------------------
>  If you reply to this email, your message will be added to the discussion
> below:
>
> http://r.789695.n4.nabble.com/Removing-values-containing-a-specific-character-tp4656744p4656784.html
>  To unsubscribe from Removing values containing a specific character, click
> here<http://r.789695.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4656744&code=eXBvZGVzd2FAZ21haWwuY29tfDQ2NTY3NDR8LTEyMTY0MzM4NDk=>
> .
> NAML<http://r.789695.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>




--
View this message in context: http://r.789695.n4.nabble.com/Removing-values-containing-a-specific-character-tp4656744p4656790.html
Sent from the R help mailing list archive at Nabble.com.
	[[alternative HTML version deleted]]

