[R-sig-hpc] Error: serialization is too large to store in a raw vector

[Stephen Weston]

Hi Fong,

A vector of length 40503 is definitely not too large.  The code that
you included looks fine.  Is it possible that in the full version of the
script the variable "ps" is defined outside of the foreach loop?
It would be automatically exported in that case, and if it was large
enough, it could cause the serialization error.  If that is the case,
you could fix the problem by using the foreach argument
".noexport='ps'".

If that isn't the case, could you post a complete, self-contained
example that demonstrates the problem?

- Steve


On Tue, Aug 10, 2010 at 4:07 PM, Fong Chun Chan
<fongchun at interchange.ubc.ca> wrote:
> Hi all,
>
> I am getting this serialization error when I am trying to run a simple
> parallel process script.  I just want to test if it works.
>
> foreach (i = 1:length(U133_ps), .combine="rbind") %dopar% {
> + ps <- U133_ps[i]
> + ps
> + }
> Error: serialization is too large to store in a raw vector
>
> U133_ps is a single column vector of length 40503.  Is this too large of a
> dataset to iterate over?  If not, how can I get around this problem?
>
> Thanks,
>
> Fong
>
>        [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-hpc mailing list
> R-sig-hpc at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>