[R] How can I avoid nested 'for' loops or quicken the process?
Bert Gunter
gunter.berton at gene.com
Tue Dec 23 19:49:18 CET 2008
FWIW:
Good advice below! -- after all, the first rule of optimizing code is:
Don't!
For the record (yet again), the apply() family of functions (and their
packaged derivatives, of course) are "merely" vary carefully written for()
loops: their main advantage is in code readability, not in efficiency gains,
which may well be small or nonexistent. True efficiency gains require
"vectorization", which essentially moves the for() loops from interpreted
code to (underlying) C code (on the underlying data structures): e.g.
compare rowMeans() [vectorized] with ave() or apply(..,1,mean).
Cheers,
Bert Gunter
Genentech Nonclinical Statistics
-----Original Message-----
From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On
Behalf Of Daniel Nordlund
Sent: Tuesday, December 23, 2008 10:01 AM
To: r-help at r-project.org
Subject: Re: [R] How can I avoid nested 'for' loops or quicken the process?
Avoiding multiple nested for loops (as requested in the subject) is usually
a good idea, especially if you can take advantage of vectorized functions.
You were able redesign your code to use a single for loop. I presume there
was a substantial improvement in program speed. How much additional time is
saved by using apply to eliminate the final for loop? Is it worth the
additional programming time? Enquiring minds want to know. :-)
Dan
Daniel Nordlund
Bothell, WA USA
> -----Original Message-----
> From: r-help-bounces at r-project.org
> [mailto:r-help-bounces at r-project.org] On Behalf Of Brigid Mooney
> Sent: Tuesday, December 23, 2008 8:36 AM
> To: David Winsemius
> Cc: r-help at r-project.org
> Subject: Re: [R] How can I avoid nested 'for' loops or
> quicken the process?
>
> --------------------------------------------------------------
> ----------------------------------------------------------------
> Problem Description: (reproducible code below)
> --------------------------------------------------------------
> ----------------------------------------------------------------
> I cannot seem to get as.data.frame() to work as I would expect.
>
> Results2 seems to contain repeated column titles for each
> row, as well as a
> row name 'investment' (which is not intended), like:
> Results2
> [[1]]
> OutTotInvestment OutNumInvestments OutDolProf OutPerProf
> OutNetGains OutLong OutShort OutInvestment OutStoploss
> OutComission OutPenny
> OutVolume OutNumU OutAccDefn
> investment 30000 3 -450 -0.015
> -0.0154 0.75 -0.5 10000 -0.015 2e-04
> 3 0.02 2 0
> [[2]]
> OutTotInvestment OutNumInvestments OutDolProf OutPerProf
> OutNetGains OutLong OutShort OutInvestment OutStoploss
> OutComission OutPenny
> OutVolume OutNumU OutAccDefn
> investment 30000 3 -450 -0.015
> -0.0154 1.5 -0.5 10000 -0.015 2e-04
> 3 0.02 2 0
> ...
>
> When I try to apply 'as.data.frame', it concatenates
> incremental numbers to
> the repeated row headers and gives:
> as.data.frame(Results2)
> OutTotInvestment OutNumInvestments OutDolProf OutPerProf
> OutNetGains OutLong OutShort OutInvestment OutStoploss
> OutComission OutPenny
> OutVolume OutNumU OutAccDefn
> investment 30000 3 -450 -0.015
> -0.0154 0.75 -0.5 10000 -0.015 2e-04
> 3 0.02 2 0
> OutTotInvestment.1 OutNumInvestments.1
> OutDolProf.1 OutPerProf.1
> OutNetGains.1 OutLong.1 OutShort.1 OutInvestment.1 OutStoploss.1
> OutComission.1 OutPenny.1
> investment 30000 3 -450
> -0.015 -0.0154 1.5 -0.5 10000
> -0.015 2e-04 3
> OutVolume.1 OutNumU.1 OutAccDefn.1 OutTotInvestment.2
> OutNumInvestments.2 OutDolProf.2 OutPerProf.2 OutNetGains.2 OutLong.2
> OutShort.2 OutInvestment.2
> investment 0.02 2 0
> 30000 3 -450 -0.015 -0.0154
> 0.75 -1 10000
> ...
>
> which is a data frame of dimension 1 224, when I am looking
> for a data frame
> like Results of dimension 16 14.
>
>
> --------------------------------------------------------------
> ----------------------------------------------------------------
> Reproducible code:
> --------------------------------------------------------------
> ----------------------------------------------------------------
> # --------------------------------------------------------------
> # FUNCTION calcProfit
> # --------------------------------------------------------------
> calcProfit <- function(IterParam, marketData, dailyForecast) #, long,
> short, investment, stoploss, comission, penny, volume, numU, accDefn)
> {
> if (class(IterParam) == "numeric")
> {
> long <- IterParam["long"]
> short <- IterParam["short"]
> investment <- IterParam["investment"]
> stoploss <- IterParam["stoploss"]
> comission <- IterParam["comission"]
> penny <- IterParam["penny"]
> volume <- IterParam["volume"]
> numU <- IterParam["numU"]
> accDefn <- IterParam["accDefn"]
> } else {
> long <- IterParam$long
> short <- IterParam$short
> investment <- IterParam$investment
> stoploss <- IterParam$stoploss
> comission <- IterParam$comission
> penny <- IterParam$penny
> volume <- IterParam$volume
> numU <- IterParam$numU
> accDefn <- IterParam$accDefn
> }
>
> compareMarket <- merge(dailyForecast, marketData, by.x="SymbolID",
> by.y="SymbolID")
>
> weight <- ifelse(rep(accDefn,
> times=length(compareMarket$weight))==1,
> compareMarket$weight, compareMarket$CPweight)
>
> position <- ifelse((weight<=short &
> compareMarket$OpeningPrice > penny &
> compareMarket$noU>=numU), "S",
> ifelse((weight>=long & compareMarket$OpeningPrice > penny &
> compareMarket$noU>=numU), "L", NA))
> positionTF <- ifelse(position=="L" | position=="S", TRUE, FALSE)
>
> estMaxInv <-
> volume*compareMarket$MinTrVol*compareMarket$YesterdayClose
>
> investbySymbol <- ifelse(positionTF==TRUE, ifelse(estMaxInv >=
> investment, investment, 0))
>
> opClProfit <- ifelse(position=="L",
> compareMarket$ClosingPrice/compareMarket$OpeningPrice-1,
> ifelse(position=="S",
> 1-compareMarket$ClosingPrice/compareMarket$OpeningPrice, 0.0))
>
> Gains <- investbySymbol*ifelse(opClProfit <= stoploss, stoploss,
> opClProfit)
>
> ProfitTable <- data.frame(SymbolID=compareMarket$SymbolID,
> investbySymbol, Gains, percentGains=Gains/investbySymbol,
> LessComm=rep(comission, times=length(Gains)),
> NetGains=Gains/investbySymbol-2*comission)
>
> AggregatesTable <- data.frame( OutTotInvestment =
> sum(ProfitTable$investbySymbol, na.rm=TRUE),
> OutNumInvestments = sum(ProfitTable$investbySymbol,
> na.rm=TRUE)/investment, OutDolProf = sum(ProfitTable$Gains,
> na.rm=TRUE),
> OutPerProf = sum(ProfitTable$Gains,
> na.rm=TRUE)/sum(ProfitTable$investbySymbol, na.rm=TRUE),
> OutNetGains = sum(ProfitTable$Gains,
> na.rm=TRUE)/sum(ProfitTable$investbySymbol,
> na.rm=TRUE)-2*comission, OutLong
> = long,
> OutShort = short, OutInvestment = investment, OutStoploss =
> stoploss, OutComission = comission, OutPenny = penny,
> OutVolume = volume,
> OutNumU = numU, OutAccDefn = accDefn )
>
> return(AggregatesTable)
> }
>
>
> # Sample iteration parameters (these can be vectors of
> arbitrary length)
> # Need to iterate through all possible combinations of these
> parameters
> Param <- list(long=c(.75, 1.5),
> short=c(-.5, -1),
> investment=10000,
> stoploss=c(-.015),
> comission=.0002,
> penny=3,
> volume=c(.02, .01),
> numU=2,
> accDefn=0:1 )
> CombParam <- expand.grid(Param)
>
> # Create sample X and Y data frames for function call
> Y <- data.frame(SymbolID=10:14, OpeningPrice =
> c(1,3,10,20,60), ClosingPrice
> = c(2,2.5,11,18,61.5), YesterdayClose= c(1,3,10,20,60), MinTrVol =
> rep(10000000, times=5))
> X <- data.frame(SymbolID=10:14, weight = c(1, .5, -3, -.75, 2),
> CPweight=c(1.5, .25, -1.75, 2, -1), noU = c(2,3,4,2,10))
>
>
> for (i in 1:length(CombParam$long))
> {
> if(i==1)
> { Results <- calcProfit(CombParam[i,], X, Y)
> } else {
> Results <- rbind(Results, calcProfit(CombParam[i,], X, Y))
> }
> }
>
>
> Results2 <- apply(CombParam, 1, calcProfit, X, Y)
>
> --------------------------------------------------------------
> ----------------------------------------------------------------
>
> On Tue, Dec 23, 2008 at 11:15 AM, David Winsemius
> <dwinsemius at comcast.net>wrote:
>
> >
> > On Dec 23, 2008, at 10:56 AM, Brigid Mooney wrote:
> >
> > Thank you again for your help.
> >>
> >> snip
> >
> >>
> >>
> > -------------
> >> With the 'apply' call, Results2 is of class list.
> >>
> >> Results2 <- apply(CombParam, 1, calcProfit, X, Y)
> >>
> >>
> --------------------------------------------------------------
> -----------------------------------------
> >>
> >> How can I get convert Results2 from a list to a data frame
> like Results?
> >>
> >
> > Have you tried as.data.frame() on Results2? Each of its
> elements should
> > have the proper structure.
> >
> > You no longer have a reproducible example, but see this
> session clip:
> > > lairq <- apply(airquality,1, function(x) x )
> > > str(lairq)
> > num [1:6, 1:153] 41 190 7.4 67 5 1 36 118 8 72 ...
> > - attr(*, "dimnames")=List of 2
> > ..$ : chr [1:6] "Ozone" "Solar.R" "Wind" "Temp" ...
> > ..$ : NULL
> > > is.data.frame(lairq)
> > [1] FALSE
> > > is.data.frame(rbind(lairq))
> > [1] FALSE
> > > is.data.frame( as.data.frame(lairq) )
> > --
> > David Winsemius
> >
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
More information about the R-help
mailing list