[R-SIG-Mac] [R] lme4 + R 2.11.0 + mac unavailable

John Maindonald john.maindonald at anu.edu.au
Sun Aug 1 05:00:08 CEST 2010 

I have wondered about what OSX may do to take advantage of
multiple processors somewhat automatically.  I'd expect a few
new traps, beyond what happens for threading on single processor
systems.  I had a brief look at the document:
http://developer.apple.com/mac/library/documentation/General/Conceptual/ConcurrencyProgrammingGuide/Introduction/Introduction.html

As I understand this, recent versions of MacOS do have some 
relatively automatic concurrency abilities, but the code writer 
(always?) must invoke them.  I'd not expect the threads to be the 
same from one run to another, or are attempts made to ensure this?  
Great care is in any case required, for any use of concurrency that
is handled by the OS, to ensure that the calculations in the different 
threads are totally independent, even to ensuring that rounding 
errors do not change.

Brian's results with the R reference BLAS  (which are not threaded?)
perhaps suggest that threading/concurrency in this part of the code 
are not issues?

Does lme4 on the Mac call code, apart from the BLAS, where there 
may be some automatic invocation of concurrency?

I do remember listening to a talk some decades ago about unfortunate
consequences of some compiler optimizations, primarily for numerical 
error. Of course, the whole art has become much more sophisticated
(and challenging) since then.  

The Wikepedia article on Program Optimization is interesting, but it
says nothing about the scope for introducing inconsistencies
between runs, or (worse) inflating numerical error.
http://en.wikipedia.org/wiki/Program_optimization

This seems an omission -- is it assumed that such things do not
nowadays ever happen?

Of course, the problem may lie elsewhere.

John Maindonald             email: john.maindonald at anu.edu.au
phone : +61 2 (6125)3473    fax  : +61 2(6125)5549
Centre for Mathematics & Its Applications, Room 1194,
John Dedman Mathematical Sciences Building (Building 27)
Australian National University, Canberra ACT 0200.
http://www.maths.anu.edu.au/~johnm

On 31/07/2010, at 10:07 PM, Martin Maechler wrote:

> On Sat, Jul 31, 2010 at 04:25, John Maindonald
> <john.maindonald at anu.edu.au> wrote:
>> Is optimisation of code a possibility, i.e., the same algebraic calculations
>> done  in a slightly different way depending on the state of various registers?
> 
> I'm really not a Mac specialist and neither one in those low level
> system / CPU computations,  but indeed, exactly something like that
> has been my current guess about what's happening...
> 
>> E.g., (ab)c vs a(bc), but it would almost certainly be more subtle than that.
> yes, more subtle ( I hope).  We know that computer arithmetic does
> *not*  fulfill most of these basic arithmetic laws, but remember that
> here, we cannot imagine how the numeric matrices that are used in the
> computations could *start*  differently, and then I'd wonder how
> differences came about...
> and recall that the differences we've seen where not just in the last
> few significant bits
> 
>> That seems to me the sort of thing that is likely to be Mac-specific.
> 
> interesting...  so you have seen other evidence of such behavior?
> 
>> 
>> John Maindonald             email: john.maindonald at anu.edu.au
>> phone : +61 2 (6125)3473    fax  : +61 2(6125)5549
>> Centre for Mathematics & Its Applications, Room 1194,
>> John Dedman Mathematical Sciences Building (Building 27)
>> Australian National University, Canberra ACT 0200.
>> http://www.maths.anu.edu.au/~johnm
>> 
>> On 31/07/2010, at 5:09 AM, Martin Maechler wrote:
>> 
>>> On Fri, Jul 30, 2010 at 17:29, Simon Urbanek
>>> <simon.urbanek at r-project.org> wrote:
>>>> 
>>>> On Jul 29, 2010, at 7:51 PM, Martin Maechler wrote:
>>>> 
>>>>>>>>>> "KB" == Ken Beath <ken at kjbeath.com.au>
>>>>>>>>>>   on Wed, 28 Jul 2010 01:19:34 -0000 (GMT) writes:
>>>>> 
>>>>>   KB> On Tue, July 27, 2010 11:07 pm, John Maindonald wrote:
>>>>>>> Binaries for lme4 are not for the time being available either on
>>>>>>> CRAN or on r-forge?
>>>>>>> 
>>>>> 
>>>>>   KB> There was a discussion about this on R-Sig-ME. It fails one of the checks
>>>>>   KB> on the machine used to build the mac packages. It may pass on machines
>>>>>   KB> with a later version of MacOS, but there didn't seem to be a consensus of
>>>>>   KB> whether failing the checks was a problem.
>>>>> 
>>>>> The underlyign issue seems to remain unresolved.
>>>>> My best guess is still to low-level bugs somewhere in the libraries
>>>>> used.
>>>>> 
>>>>> However, to help users,
>>>>> I'm willing to *not* run those tests when *running* on the Mac.
>>>>> 
>>>>> What does
>>>>>    Sys.info()[["sysname"]]
>>>>> 
>>>>> return on the Mac, i.e., different versions of R running on the
>>>>> Mac?
>>>>> From grepping around I'd expect it give  "Darwin"  in all cases.
>>>>> Is that true?
>>>>> 
>>>> 
>>>> ... why don't you just compare the values with tolerance? That seems to make more sense that to disable the test altogether ...
>>>> 
>>> 
>>> Doug and I have explained in several occasions (but maybe not on R-SIG-Mac)
>>> that this is *NOT* the point here.
>>> It's not about "basically the same computations" it's about
>>> identical computations, i.e. from identical numerical matrices and so
>>> the use of identical() has been very much on purpose, and is different
>>> from the well justified typical use of
>>> all.equal() which we use in many other places.
>>> 
>>> And what people where reporting *is* a bit disturbing: The same
>>> computations giving different answers for *exactly* the same
>>> computation, e.g., depending if they had used
>>> library(lme4)  or  require(lme4)  to load the package ...
>>> and the difference was not just the last significant bit, but changes
>>> at about the 7-th significant digit.
>>> 
>>> Martin
>>> 
>>> 
>>>> Cheers,
>>>> Simon
>>>> 
>>> 
>>> _______________________________________________
>>> R-SIG-Mac mailing list
>>> R-SIG-Mac at stat.math.ethz.ch
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>> 
>> 
> 
> _______________________________________________
> R-SIG-Mac mailing list
> R-SIG-Mac at stat.math.ethz.ch
> https://stat.ethz.ch/mailman/listinfo/r-sig-mac

More information about the R-SIG-Mac mailing list