[R] missing value where TRUE/FALSE needed
jim holtman
jholtman at gmail.com
Fri Dec 23 18:09:44 CET 2011
Given that the maximum floating point value is:
$double.xmax
[1] 1.797693e+308
and the number you are trying to calculate is 5E323 you are exceeding
the size of numbers you can process.
Have a happy holiday and glad I could help.
On Fri, Dec 23, 2011 at 11:24 AM, Michael Pearmain
<michael.pearmain at gmail.com> wrote:
> Thanks for al your help with Jim,
> I realise this is really a case of debugging (and so falls to the code
> writer) using your help i have traced the NaN to be from the c1 calculation
> in the FindGradient function.
> there is a place where we use the calculation of c1...
>
> which at the start was calculated as
> c1 <- (2*pi) ** (n.cols/2)
>
> when i use
> c1 <- (2*pi) ** (811/2)
>> [1] Inf
>
> So i think i have found a ceiling on the number of cols i can use to make a
> calcuation.
>
> Many thanks again for your help, especially on 23rd Dec!
>
> Mike
>
>
>
> On 23 December 2011 16:01, jim holtman <jholtman at gmail.com> wrote:
>>
>> The next step is to now evaluate the objects used in the calculation;
>> e.g., look at the objects in the statement:
>>
>> change <- sqrt(sum((next.x - x) * (next.x - x)))
>>
>> What do each of them show? Most likely you have something in the data
>> that causing this statement to evaluate to NaN (not a number) and this
>> is caused by the values of one of these as not being what you think it
>> is. From the value of 'i', it looks like the first iteration (or at
>> least when it has the value of 1). Notice that "X" is defined as a
>> matrix with NAs as the initial values and you are picking up the first
>> row (X[i,]) which on the initial pass is an NA. So start by examining
>> the values of all the objects used up to that point and what the
>> values of each expression is.
>>
>> On Fri, Dec 23, 2011 at 10:38 AM, Michael Pearmain
>> <michael.pearmain at gmail.com> wrote:
>> > Thanks for guidance;
>> >
>> > the return from the dump is below:
>> >
>> > So i can see that the NaN is causing NA, as its evaluating to NA in the
>> > ifelse.
>> > So i guess my next question is why this happens for my large data set
>> > but
>> > not he iris dataset?
>> >
>> > Thank you again for your help on this, i really appreciate you taking
>> > the
>> > time to help
>> >
>> > 1: DensitiesAndAttractors(data = cluster.data, smoothing = 0.34, tol =
>> > 1e-05,
>> > 2: denclue_par.R#136: sfSapply(rep(1:n.rows), function(i) {
>> > 3: sapply(x, fun, ..., simplify = simplify, USE.NAMES = USE.NAMES)
>> > 4: lapply(X, FUN, ...)
>> > 5: FUN(1:9985[[1]], ...)
>> >
>> > Selection: 5
>> > Called from: function ()
>> > {
>> > if (.isMethodsDispatchOn()) {
>> > tState <- tracingState(FALSE)
>> > on.exit(tracingState(tState))
>> > }
>> > calls <- sys.calls()
>> > from <- 0L
>> > n <- length(calls)
>> > if (identical(sys.function(n), recover))
>> > n <- n - 1L
>> > for (i in rev(seq_len(n))) {
>> > calli <- calls[[i]]
>> > fname <- calli[[1L]]
>> > if (!is.na(match(deparse(fname)[1L], c("methods::.doTrace",
>> > ".doTrace")))) {
>> > from <- i - 1L
>> > break
>> > }
>> > }
>> > if (from == 0L)
>> > for (i in rev(seq_len(n))) {
>> > calli <- calls[[i]]
>> > fname <- calli[[1L]]
>> > if (!is.name(fname) || is.na(match(as.character(fname),
>> > c("recover", "stop", "Stop")))) {
>> > from <- i
>> > break
>> > }
>> > }
>> > if (from > 0L) {
>> > if (!interactive()) {
>> > try(dump.frames())
>> > cat(gettext("recover called non-interactively; frames
>> > dumped,
>> > use debugger() to view\n"))
>> > return(NULL)
>> > }
>> > else if (identical(getOption("show.error.messages"),
>> > FALSE))
>> > return(NULL)
>> > calls <- limitedLabels(calls[1L:from])
>> > repeat {
>> > which <- menu(calls, title = "\nEnter a frame number, or 0
>> > to
>> > exit ")
>> > if (which)
>> > eval(substitute(browser(skipCalls = skip), list(skip = 7
>> > -
>> > which)), envir = sys.frame(which))
>> > else break
>> > }
>> > }
>> > else cat(gettext("No suitable frames for recover()\n"))
>> > }()
>> > Browse[1]> ls()
>> > [1] "change" "find.gradient" "i" "iters"
>> > [5] "next.x" "notconverged" "x"
>> > Browse[1]> change
>> > [1] NaN
>> > Browse[1]> i
>> > [1] 1
>> > Browse[1]>
>> >
>> >
>> > On 23 December 2011 15:22, jim holtman <jholtman at gmail.com> wrote:
>> >>
>> >> According to your dump, if you look in section 5 (this is the function
>> >> being call by 'sapply') you should do an 'ls()' and see the object
>> >> 'notconverged' and if you type its name, I am guessing that is that
>> >> the value NA. Also look at value of 'change' and 'tol' to determine
>> >> how the 'ifelse' was evaluated.
>> >>
>> >> Let me know how it goes.
>> >>
>> >> On Fri, Dec 23, 2011 at 9:35 AM, Michael Pearmain
>> >> <michael.pearmain at gmail.com> wrote:
>> >> > Hi Jim,
>> >> >
>> >> > Sorry to email directly, i wasn't sure what i was looking for, can
>> >> > you
>> >> > give
>> >> > me a little guidance then i can post back to the list? Many thanks
>> >> >
>> >> >> density.data <- DensitiesAndAttractors(data = cluster.data,
>> >> > smoothing = 0.34,
>> >> > tol = 0.00001,
>> >> > parallel = FALSE)
>> >> > density.data <- DensitiesAndAttractors(data = cluster.data,
>> >> > + smoothing = 0.34,
>> >> > + tol = 0.00001,
>> >> > + parallel = FALSE)
>> >> > Explicit sfStop() is missing: stop now.
>> >> > snowfall 1.84 initialized: sequential execution, one CPU.
>> >> >
>> >> > Error in while (notconverged) { : missing value where TRUE/FALSE
>> >> > needed
>> >> >
>> >> > Enter a frame number, or 0 to exit
>> >> >
>> >> > 1: DensitiesAndAttractors(data = cluster.data, smoothing = 0.34, tol
>> >> > =
>> >> > 1e-05,
>> >> > 2: denclue_par.R#136: sfSapply(rep(1:n.rows), function(i) {
>> >> > 3: sapply(x, fun, ..., simplify = simplify, USE.NAMES = USE.NAMES)
>> >> > 4: lapply(X, FUN, ...)
>> >> > 5: FUN(1:9985[[1]], ...)
>> >> >
>> >> > Selection: 1
>> >> > Called from: sfSapply(rep(1:n.rows), function(i) {
>> >> > notconverged <- TRUE
>> >> > x <- (X[i, ])
>> >> > iters <- as.integer(1)
>> >> > while (notconverged) {
>> >> > find.gradient <- FindGradient(x)
>> >> > next.x <- x + stepsize * find.gradient$gradient
>> >> > change <- sqrt(sum((next.x - x) * (next.x - x)))
>> >> > notconverged <- ifelse(change > tol, TRUE, FALSE)
>> >> > x <- next.x
>> >> > iters <- iters + 1
>> >> > }
>> >> > return(c(densities[i, ] <- find.gradient$density, attractors[i,
>> >> > ] <- x))
>> >> > })
>> >> > Browse[1]> ls()
>> >> > [1] "attractors" "c1" "c2" "c3"
>> >> > "cores"
>> >> >
>> >> > [6] "data" "densities" "FindGradient" "Kexp"
>> >> > "n.cols"
>> >> >
>> >> > [11] "n.rows" "parallel" "smoothing" "stepsize"
>> >> > "tol"
>> >> >
>> >> > [16] "X"
>> >> > Browse[1]>
>> >> >
>> >> > Enter a frame number, or 0 to exit
>> >> >
>> >> > 1: DensitiesAndAttractors(data = cluster.data, smoothing = 0.34, tol
>> >> > =
>> >> > 1e-05,
>> >> > 2: denclue_par.R#136: sfSapply(rep(1:n.rows), function(i) {
>> >> > 3: sapply(x, fun, ..., simplify = simplify, USE.NAMES = USE.NAMES)
>> >> > 4: lapply(X, FUN, ...)
>> >> > 5: FUN(1:9985[[1]], ...)
>> >> >
>> >> > Selection: 2
>> >> > Called from: sapply(x, fun, ..., simplify = simplify, USE.NAMES =
>> >> > USE.NAMES)
>> >> > Browse[1]> ls()
>> >> > [1] "fun" "simplify" "USE.NAMES" "x"
>> >> > Browse[1]>
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > On 23 December 2011 13:26, jim holtman <jholtman at gmail.com> wrote:
>> >> >>
>> >> >> Does this look similar to the error you are getting:
>> >> >>
>> >> >> > while(NA == TRUE) 1
>> >> >> Error in while (NA == TRUE) 1 : missing value where TRUE/FALSE
>> >> >> needed
>> >> >>
>> >> >> SO 'notconverged' is probably equal to NA. BTW, what is the value
>> >> >> of
>> >> >> 'tol'; I do not see it defined. So when computing 'notconverged'
>> >> >> you
>> >> >> have generated an NA. You can test it to see if this is true.
>> >> >>
>> >> >> You can use the following command:
>> >> >>
>> >> >> options(error=utils::recover)
>> >> >>
>> >> >> and then learn how to use the 'browser' to examine variables when
>> >> >> the
>> >> >> error occurs.
>> >> >>
>> >> >> On Fri, Dec 23, 2011 at 5:44 AM, Michael Pearmain
>> >> >> <michael.pearmain at gmail.com> wrote:
>> >> >> > Merry Xmas to all,
>> >> >> >
>> >> >> > I am writing a function and curiously this runs sometimes on one
>> >> >> > data
>> >> >> > set
>> >> >> > and fails on another and i cannot figure out why.
>> >> >> > Any help much appreciated.
>> >> >> >
>> >> >> > If i run the code below with
>> >> >> > data <- iris[ ,1:4]
>> >> >> > The code runs fine, but if i run on a large dataset i get the
>> >> >> > following
>> >> >> > error (showing data structures as matrix is large)
>> >> >> >
>> >> >> >> str(cluster.data)
>> >> >> > num [1:9985, 1:811] 0 0 0 0 0 0 0 0 0 0 ...
>> >> >> > - attr(*, "dimnames")=List of 2
>> >> >> > ..$ : NULL
>> >> >> > ..$ : chr [1:811] "1073949105" "1073930585" "1073843224"
>> >> >> > "1073792624"
>> >> >> > ...
>> >> >> > #(This is intended to be chr)
>> >> >> >> str(iris)
>> >> >> > 'data.frame': 150 obs. of 5 variables:
>> >> >> > $ Sepal.Length: num 5.1 4.9 4.7 4.6 5 5.4 4.6 5 4.4 4.9 ...
>> >> >> > $ Sepal.Width : num 3.5 3 3.2 3.1 3.6 3.9 3.4 3.4 2.9 3.1 ...
>> >> >> > $ Petal.Length: num 1.4 1.4 1.3 1.5 1.4 1.7 1.4 1.5 1.4 1.5 ...
>> >> >> > $ Petal.Width : num 0.2 0.2 0.2 0.2 0.2 0.4 0.3 0.2 0.2 0.1 ...
>> >> >> > $ Species : Factor w/ 3 levels "setosa","versicolor",..: 1 1
>> >> >> > 1 1
>> >> >> > 1
>> >> >> > 1 1
>> >> >> > 1 1 1 ...
>> >> >> >> str(as.matrix(iris[,1:4]))
>> >> >> > num [1:150, 1:4] 5.1 4.9 4.7 4.6 5 5.4 4.6 5 4.4 4.9 ...
>> >> >> > - attr(*, "dimnames")=List of 2
>> >> >> > ..$ : NULL
>> >> >> > ..$ : chr [1:4] "Sepal.Length" "Sepal.Width" "Petal.Length"
>> >> >> > "Petal.Width"
>> >> >> >
>> >> >> > n.cols <- ncol(data)
>> >> >> > n.rows <- nrow(data)
>> >> >> > X <- as.matrix(data)
>> >> >> > stepsize <- 0.05
>> >> >> > c1 <- (2 * pi) ** (n.cols / 2)
>> >> >> > c2 <- n.rows * (smoothing ** (n.cols + 2))
>> >> >> > c3 <- n.rows * (smoothing ** n.cols)
>> >> >> >
>> >> >> > Kexp <- function(sqs){
>> >> >> > return (exp((-1 * sqs) / (2 * smoothing ** 2)))
>> >> >> > }
>> >> >> >
>> >> >> > FindGradient <- function(x){
>> >> >> > XmY <- t(x - t(X))
>> >> >> > sqsum <- rowSums(XmY * XmY)
>> >> >> > K <- sapply(sqsum, Kexp)
>> >> >> > dens <- ((c1 * c3) ** -1) * sum(K)
>> >> >> > grad <- -1 * ((c1 * c2) ** -1) * colSums(K * XmY)
>> >> >> > return (list(gradient = grad,
>> >> >> > density = dens))
>> >> >> > }
>> >> >> >
>> >> >> > attractors <- matrix(0, n.rows, n.cols)
>> >> >> > densities <- matrix(0, n.rows)
>> >> >> >
>> >> >> >
>> >> >> >> density.attractors <-
>> >> >> > sapply(rep(1:n.rows), function(i) {
>> >> >> > notconverged <- TRUE
>> >> >> > # For each row loop through and find the attractor and
>> >> >> > density
>> >> >> > value.
>> >> >> > x <- (X[i, ])
>> >> >> > iters <- as.integer(1)
>> >> >> > # Run gradient ascent for each point to obtain x*
>> >> >> > while(notconverged == TRUE) {
>> >> >> > find.gradient <- FindGradient(x)
>> >> >> > next.x <- x + stepsize * find.gradient$gradient
>> >> >> > change <- sqrt(sum((next.x - x) * (next.x - x)))
>> >> >> > notconverged <- ifelse(change > tol, TRUE, FALSE)
>> >> >> > x <- next.x
>> >> >> > iters <- iters + 1
>> >> >> > }
>> >> >> >
>> >> >> > # store the attractor and density value
>> >> >> > return(c(densities[i, ] <- find.gradient$density,
>> >> >> > attractors[i, ] <- x))
>> >> >> > })
>> >> >> >
>> >> >> > Error in while (notconverged == TRUE) { :
>> >> >> > missing value where TRUE/FALSE needed
>> >> >> >>
>> >> >> >
>> >> >> > Any help would be great
>> >> >> >
>> >> >> > Mike
>> >> >> >
>> >> >> > [[alternative HTML version deleted]]
>> >> >> >
>> >> >> > ______________________________________________
>> >> >> > R-help at r-project.org mailing list
>> >> >> > https://stat.ethz.ch/mailman/listinfo/r-help
>> >> >> > PLEASE do read the posting guide
>> >> >> > http://www.R-project.org/posting-guide.html
>> >> >> > and provide commented, minimal, self-contained, reproducible code.
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Jim Holtman
>> >> >> Data Munger Guru
>> >> >>
>> >> >> What is the problem that you are trying to solve?
>> >> >> Tell me what you want to do, not how you want to do it.
>> >> >
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Jim Holtman
>> >> Data Munger Guru
>> >>
>> >> What is the problem that you are trying to solve?
>> >> Tell me what you want to do, not how you want to do it.
>> >
>> >
>>
>>
>>
>> --
>> Jim Holtman
>> Data Munger Guru
>>
>> What is the problem that you are trying to solve?
>> Tell me what you want to do, not how you want to do it.
>
>
--
Jim Holtman
Data Munger Guru
What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.
More information about the R-help
mailing list