[R-sig-hpc] Parallelizing function with subfunctions using snowfall
Joakim Ringblom
joakim.ringblom at ki.se
Thu Aug 25 10:39:40 CEST 2011
Hi
I have a function (jr.parsimest) which is rather time consuming and since I want to do use it several times with different input I tried to parallelize the process using snowfall. The code works fine when running in serial with sfInit(parallel=FALSE) but I have a problem getting it to work with sfInit(parallel=TRUE, cpus=8).
The problem is that the function jr.parsimest contains several subfunctions that in turn have several more subfunctions and so on and it seems that I have to export all of them with sfExport or sfExportall. This will be rather tricky and tedious at best, since I don´t really know some of the subfunctions with subfunctions since they come from a non-Cran package not written by me. I also get a feeling that this is not the way to do things but I am not able confirm it anywhere.
Is my approach fundamentally wrong? Is there a better way of doing things?
Parallelizing a time-consuming subfunction is of course an idea but I rather parallelize the "topfunction" if possible.
I don´t think my code is important for the questions but I post it here anyhow just in case. I run the program on windows 7 computer at the moment but I would like to run it on unix later on if that matters.
Any help would be greatly appreciated.
/
Thanks in advance
Joakim Ringblom
PhD-student
Karolinska Institutet
Stockholm, Sweden
joakim.ringblom at ki.se
paramvect<-c(1,0.52, 1.3,4)
paramsd<-c(1,0.52, 1.3,4)
paramsd<-c(0,0,0,0)
ressd<-c(0.01)
doser<-c(0,0.381,0.475,0.564)
gruppstorlek<-c(10,10,10,10)
n<-1
nrdiff<-4
cef<-0.05
opiter<-2
searchnr<-1
bestnr<-gruppstorlek
outtable<-matrix(0,opiter,27+2*length(doser))
colnames(outtable)<-c("Medel-BMD","Medel-BMDL","Medel-BMDU","Medel-BMDL/BMD","Medel-BMDtrue", "Medel-SquaredError", "Medel-APE", "BMD-Varians","BMDL-Varians","BMDU-Varians","BMDL/BMD-Varians","BMDtrue-Varians", "SquaredError-Varians", "APE-Varians", "Antal BMD-NAs", "Parameter","Parameter","Parameter","Parameter", "SD","SD","SD","SD", "Ressd", 1:length(doser), 1:length(doser),"n", "CES", "searchnr")
for (k in 1:opiter){
iternr<-1
grupplist<-list()
grupplist[[iternr]]<-gruppstorlek
for (i in 1:length(doser)){
arbnr<-gruppstorlek
if (arbnr[i]>=nrdiff){
arbnr[i]<-arbnr[i]-nrdiff
for (j in 1:length(doser)){
tempnr<-arbnr
if (!(i==j)){
searchnr<-searchnr+1
iternr<-iternr+1
tempnr[j]=tempnr[j]+nrdiff
grupplist[[iternr]]<-tempnr
}
}
}
}
sfInit(parallel=TRUE, cpus=8)
svar<-sfLapply(grupplist, jr.parsimest, paramvect=paramvect, paramsd=paramsd, ressd=ressd, doser=doser, n=n, CES=cef, outtable=outtable,searchnr=searchnr)
sfStop()
besteffect<-svar[[1]]
for (i in 2:length(svar)){
if (svar[[i]][7]<besteffect[7]){
besteffect<-svar[[i]]
}
}
bestnr<-c(besteffect[29],besteffect[30],besteffect[31],besteffect[32])
if(all(gruppstorlek==bestnr & nrdiff>1)){nrdiff<-nrdiff/2}
gruppstorlek<-bestnr
outtable[k,]<-besteffect
write.table(outtable, file="utskrift.txt")
}
return(bestnr)
More information about the R-sig-hpc
mailing list