[R-sig-Geo] Memory...

[Giuseppe Amatulli]

Ciao Maurizio
- Try to build up your table outside R and enter in R just with the
columns/rows that you need.

- If you are fitting the model use y=resp  , x=prdictors rather others
formulas  (e.g.:  y ~  )
- Every time that you delate a file rm() use gc() to reallocate memory

- Bash and awk can handling large data set, and can be useful for
preselecting columns and row and also for making some basic
statistical analyse ( see
http://www.spatial-ecology.net/dokuwiki/doku.php?id=wiki:geo_tools :
Dealing with simple statistic in large txt file )

- You can also sampling your data adding a random column with awk and
make an if condition
awk '{ if ( rand() > 0.5)    print $1   }' input.txt   > output.txt
check for more examples and issues how use rand() & srand()

- Sorting/ranking action are very demanding in R so try to do in bash.

- Should be possible to use the swap from R but it will became extremely slow.

Ciao

--
Giuseppe Amatulli
Web: www.spatial-ecology.net


On 7 March 2013 05:29, Maurizio Marchi <mauriziomarchi85 at gmail.com> wrote:
> Hi everybody,
> I have to calculate some new columns in a data frame with 1971000 lines adding
> 57 new columns but R crashes every time...
> I think it's a problem of memory allocation, so I would like to know how I
> can change the memory settings...
> I'm on Ubuntu 12.04 64-bit
>
> Thanks,
> --
> Maurizio Marchi
>  ID skype: maurizioxyz
> *Ubuntu 12.04 (Precise Pangolin)*
> *"Il bello dell'open-source è che le domande possono essere poste alla
> fonte"*
> *utente linux 552.742*
>
>         [[alternative HTML version deleted]]
>
>
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>



--
Giuseppe Amatulli
Web: www.spatial-ecology.net