[Rd] lazy loading in SparkR

[Tóth Zoltán]

Hi,

I'm using SparkR and R won't read the promises from the SparkR package,
only if I run lazyLoad manually.

.libPaths(c(file.path(Sys.getenv("SPARK_HOME"), "R", "lib"), .libPaths()))
print(.libPaths())
# [1] "/private/tmp/spark-1.5/spark-1.5.0-SNAPSHOT-bin-hadoop2.6/R/lib"
# [2] "/usr/local/lib/R/3.2/site-library"
# [3] "/usr/local/Cellar/r/3.2.2/R.framework/Versions/3.2/Resources/library"

library(SparkR)
# Works

if(!exists("parallelize",mode="function")){
    # Here is the trick. For some reason, SparkR doesn't load it's
functions/promises automatically here.
    # We can fix it with an internal function:
    lazyLoad(filebase = file.path(SPARK_HOME, "R/lib/SparkR/R/SparkR"),
envir = parent.frame(), filter = function(x) TRUE)
}

# parallelize() works now

Please help me understand what goes wrong.

Here is the whole repro:
https://gist.github.com/zoltanctoth/971ae374dace93de12f0#file-sparkr-rstudio-parallelize-r-L21-L31

And the relevant files from the Spark installation:
https://www.dropbox.com/sh/0xp2wgkbrl7ov64/AAAHRWdMecGMkVdo0GUs6fmYa?dl=0

Thanks,
Best,
Zoltan

--
https://www.linkedin.com/in/zoltanctoth

	[[alternative HTML version deleted]]