[R-sig-Geo] Errore in sprintf(fmt, ...) ?
Roger Bivand
Roger.Bivand at nhh.no
Tue Dec 5 08:59:45 CET 2006
On Tue, 5 Dec 2006, massimodisasha wrote:
Please respect the time of people who reply to your questions. You send
messages which indicate that you are just fumbling in ignorance. You are
trying to run a very large problem on difficult hardware (yes, recently we
even found a case of console R on Intel Mac working when R.App failed),
using somebody else's script.
0) Why such large gaps in time between your replies - are you really
interested in a resolution? My suggestion of using reasRAST6() wasn't
answering your problem anyway, but was sent Sat, 2 Dec 2006 16:03:53,
after a previous longish gap.
1) Why such a large area?
2) Why this script and methodology - why clara() and this choice of
variables - you are classifying 5M observations on 7 variables? Wouldn't
it be possible to set up criteria for classification using a sample or
sub-region, and use the same rules in for example r.mapcalc?
3) Why an Intel Mac using R.app(?)?
4) Why not try harder to resolve the problems yourself, since you are
trying to do things that probably nobody else on the list can reproduce on
a similar platform? You say Dylan's script works for a smaller region,
doesn't that tell you something? Please try to do this on a Linux
platform, then at least some of us could try to reproduce it.
4a) Or make the difficult object available on a website as:
save(x, file="very_large_geomorp.RData", compress=TRUE)
if this doesn't cause memory problems too, that is.
5) Instead of copying to morph, use x at data directly - it is already a data
frame.
str(x at data)
summary(x at data)
In any case in GRASS doing r.stats or whatever would show that there are
NAs in your region, wouldn't it?
6) Instead of R.app(?), use R in a regular console (but maybe you are
doing that anyway); recent experience indicates that Intel Mac is not
necessarily well supported for larger tasks, especially R.app.
Roger
> hi,
> i've tryied to use
> readRAST6 instead of readFLOAT6sp
> this is the log:
>
> > library(cluster)
> > library(cluster)
> > x <- readRAST6(c
> ("er","crosc","longc","slope","profc","minic","maxic"))
>
> .....
> .......
> ........
> Percent complete: 100%
> r.out.bin complete.
> Creating BIL support files ...
> Header File = /grassdata/remote/onearth/.tmp/powerbook-g4-12-di-
> epifanio.local/maxic.hdr
> World File = /grassdata/remote/onearth/.tmp/powerbook-g4-12-di-
> epifanio.local/maxic.wld
> Exporting Raster as double values(bytes=8)
> Using the Current Region settings ...
> north=4516560.000000
> south=4483080.000000
> east=527440.000000
> west=485740.000000
> r=1674
> c=2085
>
> Percent complete: 100%
> r.out.bin complete.
> > gc()
> used (Mb) gc trigger (Mb) max used (Mb)
> Ncells 416932 11.2 741108 19.8 692659 18.5
> Vcells 23007923 175.6 194536624 1484.2 242896455 1853.2
> > morph <- as(x, "data.frame")
> R(22751) malloc: *** vm_allocate(size=8421376) failed (error code=3)
> R(22751) malloc: *** error: can't allocate region
> R(22751) malloc: *** set a breakpoint in szone_error to debug
> Errore: memoria 'vector' esaurita (raggiunto il limite?)
> R(22751) malloc: *** vm_allocate(size=8421376) failed (error code=3)
> R(22751) malloc: *** error: can't allocate region
> R(22751) malloc: *** set a breakpoint in szone_error to debug
> >
> ...
> R not respond,
>
> i force to quit the shell.
>
> i cant continue to check morph:
>
> str(morph) # look inside morph
> summary(morph)
> sapply(morph, function(x) any(!is.finite(x)))
> # check morph for NAs etc.
>
>
> Massimo
>
>
>
> Il giorno 03/dic/06, alle ore 12:00, r-sig-geo-
> request at stat.math.ethz.ch ha scritto:
>
> > On Sat, 2 Dec 2006, Massimo Di Stefano wrote:
> >
> >> Roger:
> >>
> >> This is the log,
> >>
> >>> library(spgrass6)
> >> Carico il pacchetto richiesto: sp
> >> Carico il pacchetto richiesto: maptools
> >> Carico il pacchetto richiesto: foreign
> >> Carico il pacchetto richiesto: rgdal
> >> Geospatial Data Abstraction Library extensions to R
> >> successfully loaded
> >>> library(cluster)
> >>> sessionInfo()
> >> R version 2.4.0 (2006-10-03)
> >> powerpc-apple-darwin8.7.0
> >>
> >> locale:
> >> it_IT/it_IT/it_IT/C/it_IT/it_IT
> >>
> >> attached base packages:
> >> [1] "methods" "stats" "graphics" "grDevices"
> >> "utils" "datasets"
> >> [7] "base"
> >>
> >> other attached packages:
> >> cluster spgrass6 rgdal maptools foreign sp
> >> "1.11.2" "0.3-3" "0.5-1" "0.6-3" "0.8-18" "0.9-4"
> >>> gc()
> >> used (Mb) gc trigger (Mb) max used (Mb)
> >> Ncells 408935 11.0 597831 16 531268 14.2
> >> Vcells 320324 2.5 786432 6 472905 3.7
> >>> gmeta6 <- gmeta6()
> >>> x <-
> >> readFLOAT6sp(c("er","crosc","longc","slope","profc","minic","maxic"))
> >> Warning messages:
> >> 1: perl = TRUE ? implementato solo nei locale UTF-8
> >> 2: perl = TRUE ? implementato solo nei locale UTF-8
> >> 3: perl = TRUE ? implementato solo nei locale UTF-8
> >> 4: perl = TRUE ? implementato solo nei locale UTF-8
> >> 5: perl = TRUE ? implementato solo nei locale UTF-8
> >> 6: perl = TRUE ? implementato solo nei locale UTF-8
> >> 7: perl = TRUE ? implementato solo nei locale UTF-8
> >
> > Nowadays I would suggest readRAST6() instead of readFLOAT6sp(),
> > because it
> > uses r.out.bin rather than r.out.arc from GRASS. The perl=TRUE
> > warnings
> > seems to be coming from string comparison somewhere inside a function
> > called by readFLOAT6sp(), I don't think they are the problem.
> >
> >
> >
> >>> gc()
> >> used (Mb) gc trigger (Mb) max used
> >> (Mb)
> >> Ncells 418204 11.2 741108 19.8 699974
> >> 18.7
> >> Vcells 22643234 172.8 191414408 1460.4 238993684
> >> 1823.4
> >>> morph <- data.frame(cbind(x$er, x$crosc, x$longc,
> >> x$slope, x$profc, x$minic, x$maxic))
> >
> > morph <- as(x, "data.frame")
> >
> > str(morph) # look inside morph
> > summary(morph)
> >
> > sapply(morph, function(x) any(!is.finite(x)))
> > # check morph for NAs etc.
> >
> >
> >>> gc()
> >> used (Mb) gc trigger (Mb) max used
> >> (Mb)
> >> Ncells 418451 11.2 741108 19.8 699974
> >> 18.7
> >> Vcells 46682176 356.2 153131526 1168.4 238993684
> >> 1823.4
> >> <er, x$crosc, x$longc, x$slope/max(x$slope), x$profc,
> >> x$minic, x$maxic))
> >>> gc()
> >> used (Mb) gc trigger (Mb) max used
> >> (Mb)
> >> Ncells 418453 11.2 741108 19.8 699974
> >> 18.7
> >> Vcells 46682176 356.2 137841862 1051.7 238993684
> >> 1823.4
> >>> names(morph) <-
> >> c("er","crosc","longc","slope_n","profc","minic","maxic")
> >>> gc()
> >> used (Mb) gc trigger (Mb) max used
> >> (Mb)
> >> Ncells 418456 11.2 741108 19.8 699974
> >> 18.7
> >> Vcells 46682180 356.2 137841862 1051.7 238993684
> >> 1823.4
> >>> morph.clara <- clara(morph, k=5, stand=F)
> >> Errore in clara(morph, k = 5, stand = F) :
> >> Observations
> >> 1929241,1931311,1933381,1935451,1937521,1939591,1941661,1943731,19458
> >> 01,1947871,1949941,1952011,1954081,1956151,1958221,1960291,1962361,19
> >> 64431,1966501,1968571,1970641,1972711,1974781,1976851,1978921,1980991
> >> ,
> >> 1983061,1985131,1987201,1989271,1991341,1993411,1995481,1997551,19996
> >> 21,2001691,2003761,2005831,2007901,2009971,2012041,2014111,2016181,20
> >> 18251,2020321,2022391,2024461,2026531,2028601,2030671,2032741,2034811
> >> ,
> >> 2036881,2038951,2041021,2043091,2045161,2047231,2049301,2051371,20534
> >> 41,2055511,2057581,205965
> >> 1,2061721,2063791,2065861,2067931,2070001,2072071,2074141,2076211,207
> >> 8281,2080351,2082421,2084491,2086561,2088631,2090701,2092771,2094841,
> >> 2096911,2098981,2101051,2103121,2105191,2107261,2109331,2111401,21134
> >> 71,2115541,2117611,2119681,2121751,2123821,21258
> >> 91,2127961,2130031,2132101,2134171,2136241,2138311,2140381,2142451,21
> >> 44521,2146591,2148661,2150731,2152801,2154871,2156941,2159011,2161081
> >> ,2163151,2165221,2167291,2169361,2171431
> >>> traceback()
> >> 2: stop(ngettext(length(i), sprintf("Observation %d
> >> has", i[1]),
> >> sprintf("Observations %s have", paste(i,
> >> collapse = ","))),
> >> " *only* NAs --> omit for clustering")
> >> 1: clara(morph, k = 5, stand = F)
> >>
> >> is this log usefool ?
> >
> > Yes, I think the problem is firstly that there are NAs in morph,
> > and then
> > that the string listing the row numbers of the NAs exceeds 8K
> > characters
> > (your earlier message had: Errore in sprintf(fmt, ...) : La lunghezza
> > della stringa eccede la dimensione del buffer di 8192)
> >
> >
> > So try clara on morph once you have omitted the NAs. If the NAs are
> > for
> > all variables in a row, coercing from SpatialGridDataFrame to
> > SpatialPixelsDataFrame will work, if only some, you need to set the
> > whole
> > row to NA, coerce, cluster, add the clara results to the
> > SpatialPixelsDataFrame, coerce back to SpatialGridDataFrame, then
> > back to
> > GRASS.
> >
> > Once this is resolved, please post back to the list as well as to
> > me, NAs
> > do happen, expecially in geomorphometrics, for example around the
> > edges
> > where the filter is not fully within the window and/or mask.
> >
> > Best wishes,
> >
> > Roger
>
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at stat.math.ethz.ch
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no
More information about the R-sig-Geo
mailing list