[Rd] R in a sandbox/jail

[Roger D. Peng]

I've not tried an automated system like you describe but I have tried to make 
test scripts available that compare the output of their programs to correct 
output.  That way the students can check their progress as they work.  In 
general, I find this approach doesn't work well for grading because in order to 
specify a homework to the point where the students' output matches your output, 
the homework ends up being 10 pages long (not unlike a legal document).

For example, in a number of instances I'd say "the function should return a 
number..." and then I'd compare the returned result to the actual answer.  But a 
student would 'print' the number instead, not knowing the difference between 
'return+autoprint' and 'print'.  I found it difficult to write a test script 
that gives partial credit for getting the answer right but the concept wrong. 
Perhaps that's a problem with the way I grade things, but I found it difficult 
to write an acceptable test script.

-roger

Barry Rowlingson wrote:
> Someone recently suggested building a system for automatically testing
> student's R programs. They would upload them to our Virtual Learning
> Environment, which would then run the code on some inputs and see if
> it got the right output. If it does, the student scores points for
> that course.
> 
> My first thought was "you want to run unchecked, student-submitted
> code on a server that has access to students' grades?".
> 
> Can this be done securely? The idea might be to run R in a
> chroot-jail, freshly generated for each run. The jail would not be
> able to access anything outside of it, and once the R session has
> finished the calling process can pick up the output from within the
> jail.
> 
> Maybe that's overkill. Perhaps if you run the user's code as an
> ordinary user and store the answers/results in a directory only root
> can read that would work (given no local root exploits). Other
> precautions could include limiting the runtime or cputime for the R
> session. It might be necessary to limit network access too.
> 
>  Anyone done anything like this? Personally I think there are too many
> other problems with automated systems like this, particularly that
> just because a program produces the correct output that makes it a
> good one. Sure, at the production stage that's a requirement, but I'd
> rather students learnt to program well than to program correctly -
> since correctness follows goodness but goodness does not follow
> correctness. But that's an argument for another day!
> 
> Barry
> 
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
> .
> 

-- 
Roger D. Peng  |  http://www.biostat.jhsph.edu/~rpeng/