[R-pkg-devel] Strange behaviour of function called from within a function

[John Fox]

Dear John,

Please see below interline:

On 2022-08-13 1:15 p.m., J C Nash wrote:
> This posting is to answer "what I'm trying to do". Otherwise can be 
> skipped.
> 
> My nlsr package has a completely different structure (philosophy?) from 
> nls(). My aim is to
> try VERY hard to get solutions and not give up with "singular gradient" 
> or other issues as
> nls() does. But the package is quite lean and doesn't return a lot of 
> useful bits and pieces,
> such as nls returned functions and model stuff. So I've got a wrapnlsr() 
> function that runs my
> solver, then passes the best parameters to nls(). This won't always 
> work, for example, if
> the Jacobian at the best parameters is still singular, but very often it 
> will, and the
> wrapper then returns all the goodies and a user can use the output as 
> per nls().
> 
> Because my activities don't often get into those spin-offs of nls() -- 
> for instance the
> profile plots -- I really need some collaborators if nlsr is going to 
> get more capable in
> that regard. However, I feel the wrapper is at least a start.

If I understand correctly, there are two contexts here: (1) the 
interface for your wrapnlsr() function, and (2) how nlsr() calls other 
functions, including nls().

The first should probably work like a standard statistical modelling 
function, including how the weights argument is evaluated. The second is 
hidden from the user, and is where the scoping issue that you reported 
arose. My suggestion to follow a standard statistical modelling function 
here is probably off-base, and the previous two solutions, one from me 
and one from Noah, both work.

With respect to my solution: if you actually adopt it, I'd prefer not to 
use a name like data$weights, for fear of overwriting an existing 
variable in data called weights, but rather some like .weights or even 
..weights.. .

> 
> For anyone interested, I have been tightening up the nlsr package and 
> also trying to build a
> quite comprehensive collection of examples. Some areas of improvement 
> are in applying bounds
> and masks (fixed parameters) and using selfStart models. There is still 
> plenty to do, but the
> base package nlsr should be upgraded on CRAN in a few weeks. If anyone 
> is anxious to try it,
> let me know. The changes are mostly internal. The examples collection is 
> going to take a bit
> longer to get tidy, but I'll be happy to share those too. Material is on 
> Gitlab or Github,
> though definitely a work in progress and updated daily, often with loose 
> ends.

I'm probably the wrong person for this -- I'm familiar with nonlinear 
regression, but your knowledge of the computational details vastly 
exceeds mine.

> 
> Best, JN
> 
> PS. Why would we "evaluate" the weights? The nls() man-page says they 
> are fixed numbers, but
> the nls.R code I pointed to does evaluate them, and that seems to be the 
> trouble. Is that a bug?

What I meant by "evaluate" was associate a value with the symbol 
weights. The scoping issue was that nls() ended up looking in the global 
environment, where it found the weights() function rather than the local 
variable (i.e., argument) weights in your function.

Best,
  John

> 
> On 2022-08-13 12:58, John Fox wrote:
>> Dear John,
>>
>> After further thought, it's probably a better idea to evaluate the 
>> weights argument in the same environment as the formula rather than to 
>> bypass nonstandard evaluation. You could use lm() as a guide. I don't 
>> entirely understand what you're trying to do so maybe this suggestion 
>> is off-base.
>>
>> Best,
>>   John
>>
>> On 2022-08-13 9:41 a.m., J C Nash wrote:
>>>
>>> Thanks to John Fox and Noah Greifer. Both their approaches resolved 
>>> my immediate
>>> problem.
>>>
>>> That is, to provide a summary of the fix of my example code,
>>>
>>> tw <- function(formula, data, start, control, trace, weights) {
>>>    firstcoef <- c(b1=199, b2=50, b3=0.3)
>>>    cat("firstcoef:\n")
>>>    print(firstcoef)
>>>    cat("weights:"); print(weights)
>>> # Following fails -- closure error
>>> #  secondw<-nls(formula, data, firstcoef, control, algorithm=NULL, 
>>> TRUE, weights=weights)
>>> # from noah.greifer using gmail.com # this works OK
>>>    secondw <- do.call("nls", list(formula, data, firstcoef, control, 
>>> algorithm=NULL, TRUE, weights = weights))
>>> #  As does putting weights in the data dataframe (here not active)
>>> #  data$weights <- weights # from John Fox
>>> #  secondw<-nls(formula, data, firstcoef, control, algorithm=NULL, 
>>> TRUE, weights=weights)
>>>    secondw
>>> }
>>>
>>> Afraid I avoid the wonders of non-standard evaluation, and this time 
>>> it jumped up and bit me.
>>> But then I remember what machine instruction 260000800009 did on an 
>>> IBM 1620.
>>>
>>> The swiftness of reply from John and Noah was much appreciated.
>>>
>>> Best, JN
-- 
John Fox, Professor Emeritus
McMaster University
Hamilton, Ontario, Canada
web: https://socialsciences.mcmaster.ca/jfox/