[R-SIG-Mac] How to make Mac 64-bit version feature complete?

[Simon Urbanek]

Adam,

On Dec 14, 2011, at 11:24 AM, Adam Strzelecki wrote:

> Simon wrote:
>>> I have recently started using R for my PhD work. Aside of that I am Mac and iOS developer since few years. As far I can see R package for Mac comes in two flavors, R (32-bit) and R64 (64-bit).
>> 
>> Not really - it's one R with three architectures (i386, x86_64 and ppc) - you can choose them using R --arch <arch>.
> 
> I've used work "flavor" intentionally, I know 32-bit version is FAT binary for both i386 and PPC (which are both 32-bit FYI) :)
> 
> Anyway I refer only to R.app and R64.app asking for the reason of coming in "two flavors". It is very uncommon and against OSX standards to provide two separate apps for 64-bits and 32-bits, where one can choose that via Finder (but usually it is not necessary at all), see: http://kb2.adobe.com/cps/509/cpsid_50983/images/Safari_32-bit_mode1.png
> 
>>> Correctly if I am wrong, but the only reason of such separation (rather doing single universal version for all PPC, i386 and x86_64) is Carbon dependency for various R packages?
>> 
>> Yes, you are wrong, that is not the rationale. The main reason is the same as why Apple uses separate files for i386, x86_64 and ppc: portability (see gcc, for example).
> 
> Frankly I do not see any place where Apple does use separate files for each supported architecture,

Try  ls /usr/lib/gcc/*/4.2.1 and notice separate directories for ppc vs intel (first level) and 32-bit vs 64-bit (second level; top dir is 32-bit 64-has its own subdir).

ginaz:bin$ ls /usr/lib/gcc/*/4.2.1 
/usr/lib/gcc/i686-apple-darwin10/4.2.1:
crt3.o              install-tools       libgcc_eh.a         libgfortran.a       libgomp.a           x86_64
finclude            libcc_kext.a        libgcc_static.a     libgfortranbegin.a  libgomp.spec
include             libgcc.a            libgcov.a           libgfortranbegin.la libstdc++.dylib

/usr/lib/gcc/powerpc-apple-darwin10/4.2.1:
crt2.o              include             libgcc.a            libgcov.a           libgfortranbegin.la libstdc++.dylib
crt3.o              install-tools       libgcc_eh.a         libgfortran.a       libgomp.a           ppc64
finclude            libcc_kext.a        libgcc_static.a     libgfortranbegin.a  libgomp.spec


> here's output from `file` on couple of Apple apps and libs on my Lion 10.7.2, all of them are FAT (universal binaries):
> 
> $ file /usr/lib/libgcc_s.1.dylib 
> /usr/lib/libgcc_s.1.dylib: Mach-O universal binary with 2 architectures
> /usr/lib/libgcc_s.1.dylib (for architecture x86_64):	Mach-O 64-bit dynamically linked shared library x86_64
> /usr/lib/libgcc_s.1.dylib (for architecture i386):	Mach-O dynamically linked shared library i386
> 
> $ file /usr/bin/gcc
> /usr/bin/gcc: Mach-O universal binary with 2 architectures
> /usr/bin/gcc (for architecture i386):	Mach-O executable i386
> /usr/bin/gcc (for architecture x86_64):	Mach-O 64-bit executable x86_64
> 

> $ file /Applications/Safari.app/Contents/MacOS/Safari 
> /Applications/Safari.app/Contents/MacOS/Safari: Mach-O universal binary with 2 architectures
> /Applications/Safari.app/Contents/MacOS/Safari (for architecture i386):	Mach-O executable i386
> /Applications/Safari.app/Contents/MacOS/Safari (for architecture x86_64):	Mach-O 64-bit executable x86_64
> 
>> In addition, there is no benefit in creating universal binaries, since they are very Darwin-specific and bring no benefit in this context.
> 
> There's a huge benefit of doing that. I do develop Mac apps, and FAT binaries and libs makes the other apps referring to them to refer to single file path regardless of architecture.

That's why R framework does that so the discussion is really moot.


> So this is benefit for developers. Another benefit is for Mac users, that they run single app regardless if they run on PPC, i386 only Intel (first Intel Macs ?!) or latest 64-bit Macs.
> 

If they want to do that. However, with R users typically don't want to do that - you are missing important points - see below. There is nothing preventing you from lipo'ing R from R64.app into R.app if that's what you want. It doesn't change anything, though - it's still exactly the same binary.


>> Think about the consequences - R code can be conditioned on the architecture, so when building packages, you have to run them for every architecture separately which is unnecessary difficult with universal binaries (you can't simply use -arch foo -arch bar like with gcc). It's much easier if you use the classic multi-lib approach where you can run separate architectures at will. (Technically the only difference is that we use thin binaries instead of fat binaries, so it's only the format on disk, not anything else)
> 
> Using multiple -arch arguments is not the only way of producing universal binary, you may have classic separate build process for each architecture, then during post-build you may use "lipo" command to produce single FAT binary. Anyway using separate build process for each architecture on Mac is only justified if each arch has different library dependencies and #ifdef/#ifs are not enough.
> 

Please think for a moment and re-read what I said. This is not about compilation, but about R. You are completely missing the fact that there is interpreted code involved that can be (and in practice is) arch-conditional. But, again, I don't think you wanted to go there - this is about the package build process which is a whole different story and one that's certainly not Mac-specific.


> I have nothing against of having separate libraries if the user does not have to care about it. So having two R.app and R64.app IMHO only leads to confusion.
> 
>> As a side note - the R framework and all its libraries are universal as you can see, so "native" Mac apps that link the framework can be universal (and e.g. the GUI is).
> 
> Yes, I know /Library/Frameworks/R.framework/R is single universal library, but I do not understand why /Applications/R.app isn't :)
> 

a) historical reason (64-bit R was optional) b) easier way to choose architecture c) capability to run them in parallel


>> Also your subject doesn't make sense - all architectures in R have exactly the same functionality. None of them is "limited" in any way. A side-effect of the multi-lib approach is that users can choose to compile only specific architectures (more for convenience than anything), but CRAN only builds complete packages (i.e., all architectures).
> 
> If they both have same functionality then I don't see any benefit of using 32-bit version on 64-bit OS and CPU, since 64-bit code is faster, can address more than 4GB RAM so load bigger datasets.


That is plain wrong. 64-bit is not faster, it can be both slower or faster depending on the task. And it always uses more memory. There are plenty of machines with 4GB of RAM or less where running 64-bit R can cause more trouble than you want. 


> 2GB limit refers to Windows, so was wrong with that in former mail.
> 
> However in case of R for OSX, 32-bit rgl package uses native OpenGL rendering via Carbon+AGL API, while 64-bit version uses X11 - Carbon is deprecated 32-bit only API. So this makes difference.
> 
> That's why I was asking whether there are any other packages or components of R that relay on some deprecated 32-bit libs such as Carbon and offered my help to port these parts to 64-bit Cocoa API bringing native interface.
> 

Fell free, this has nothing to do with R, though. You may want to ask the maintainers of the packages in question whether they would welcome a patch.


> Charlie wrote:
>> Another important thing to note is that the multi-lib approach works well on other platforms like Windows and Linux where "universal binaries" do not exist. Using the same approach on OS X makes for one less special case to worry about.
> 
> If everybody was thinking like that we would all use Windows and drive only Ford cars, because it is simpler when everybody uses the same stuff :) For me FAT binaries is an advantage not a problem. Windows is just a mess, so I wouldn't treat Windows as a reference here. Linux makes you do not care about architecture once you install it (ie. Ubuntu or other distro) and use built in package manager. You just select package "R" and it just copies appropriate architecture binaries, and you get single R application.
> 

That's wrong as well, you can run multi-lib on Linux (typically it's done on x64 systems to run legacy x86 code, but it is very common for other architectures like MIPS). You choose the architecture the same way as on OS X using R --arch <arch>.


> Once again, the reason of raising this discussion was a purpose of having two separate apps - R64.app and R.app as it is very uncommon for Mac users and IMHO leads to confusion. I cannot name single other application that makes such distinction. I believe there should be only one R.app using FAT binary for all archs, running best architecture (like 64-bit on recent Macs) where possible. Also all packages referring to Carbon API should be rewritten for Cocoa.
> 
> Altogether please forgive me if I may sound like "mr wise guy" here. My only intention is to help making R more Mac user friendly.
> 

Well, IMHO you don't make much sense - the only thing you seem to object is the fact that we have R.app/R64.app instead of fat R.app. However, that makes no difference whatsoever - it is just a convention that comes from the way R is run (the GUI is just an add-on, it's not "the R") and either approach works just as well. It doesn't limit anything in any way so I fail to see any connection with it and you offering help. We could certainly use an experienced Mac programmer - but the good ones don't start pointless discussions but typically provide patches and we go from there (quite successfully).

Cheers,
Simon