[R-SIG-Mac] OpenJPEG in GDAL [Was: Problems with R and Big Sur]

Gilberto Camara g||berto@c@m@r@@|npe @end|ng |rom gm@||@com
Tue Dec 8 12:23:22 CET 2020 

Dear Simon 

Apologies for the late response. Thank you so much for including the OpenJPEG driver in rgdal. For your information, JPEG2000 is the data format chosen by the European Space Agency (ESA) to distribute the images of the Sentinel-2 and Sentinel-2A satellites. Data from these satellites is one of the major source of information about the Earth’s land surfaces.

Thanks and best regards
Gilberto 
 

> On 3 Dec 2020, at 23:05, Simon Urbanek <simon.urbanek using R-project.org> wrote:
> 
> Gilberto,
> 
> well, that one is easy and has nothing to do with tools nor with BigSur - no one asked for OpenJPEG support so that's why OpenJPEG is not included on the build machine. There are several dozen of optional features in GDAL and I have no idea which are useful, which are not and which are legacy, so it's up to the rgdal maintainers or users to signal if there is anything useful missing. Given the complexity of GDAL dependencies and corresponding continuous build issues I'm relying on them to tell me when it's safe to update/change settings. Simply adding the OpenJPEG library seems sufficient:
> 
>> grep("JP2", rgdal::gdalDrivers()$name, value=TRUE)
> [1] "JP2OpenJPEG"
> 
> so that is now available in the CRAN binary.
> 
> Cheers,
> Simon
> 
> 
> 
>> On 3/12/2020, at 9:50 PM, Gilberto Camara <gilberto.camara.inpe using gmail.com> wrote:
>> 
>> Dear Simon, 
>> 
>> To test the error, please do the following on a Mac with Big Sur:
>> 
>> 1. Install GDAL
>>  The available binaries I could find out are Mac OS Frameworks from https://www.kyngchaos.com/software/frameworks/ and “homebrew”.
>> 
>> 2. Install rgdal binary from CRAN
>> I am using R version 4.0.3 Patched (2020-11-25 r79505) and the rgdal version is 1.5-18 (SVN revision 1082), the latest ones available on CRAN.
>> 
>> 3. Do the following R commands:
>>> drvs <- rgdal::gdalDrivers()$name; drvs[grep("JP2", drvs)]
>> character(0)
>> 
>> The response indicates that the JPEG2000 driver is not installed.
>> 
>> After installing both GDAL and rgdal from source with Apple clang 12.0, we get
>> 
>>> drvs <- rgdal::gdalDrivers()$name; drvs[grep("JP2", drvs)]
>> “JP2OpenJPEG"
>> 
>> More details on the rgdal version compiled from source with the new GDAL: 
>> 
>>> library(rgdal)
>> Loading required package: sp
>> rgdal: version: 1.5-18, (SVN revision 1082)
>> Geospatial Data Abstraction Library extensions to R successfully loaded
>> Loaded GDAL runtime: GDAL 3.2.0, released 2020/10/26
>> Path to GDAL shared files: /Users/gilbertocamara/Library/R/4.0/library/sf/gdal
>> GDAL binary built with GEOS: TRUE 
>> Loaded PROJ runtime: Rel. 7.1.1, September 1st, 2020, [PJ_VERSION: 711]
>> Path to PROJ shared files: /Users/gilbertocamara/Library/Application Support/proj:/usr/local/share/proj:/usr/local/share/proj
>> PROJ CDN enabled: FALSE
>> Linking to sp version:1.4-4
>> 
>> Hope this helps
>> Gilberto
>> 
>> 
>>> On 3 Dec 2020, at 01:07, Simon Urbanek <simon.urbanek using R-project.org> wrote:
>>> 
>>> Gilberto,
>>> 
>>> can you provide details, please? (code, file to test etc.) The claim was that this is specific to BigSur which I doubt from your description. Please start a new thread about that at it seem entirely unrelated to the discussion at hand. There was an incompatibility with BigSur, but that was addressed.
>>> 
>>> Thanks,
>>> Simon
>>> 
>>> 
>>> 
>>> 
>>>> On 3/12/2020, at 11:11 AM, GilbertoCamara <gilberto.camara.inpe using gmail.com> wrote:
>>>> 
>>>> Dear Simon 
>>>> 
>>>> Unfortunately, the problems reported are real. The rgdal version from CRAN fails to recognise images in JPEG2000 format used by Copernicus Sentinel-2 satellite. I have investigated the matter carefully with Roger Bivand. 
>>>> 
>>>> I tried many combinations on Mac OS Big Sur: 
>>>> (a) rgdal from CRAN (binary) + GDAL from homebrew.
>>>> (b) rgdal from CRAN (source) + GDAL from homebrew.
>>>> (c) rgdal from CRAN (binary) + GDAL Mac OS Frameworks from https://www.kyngchaos.com/software/frameworks/
>>>> (d) rgdal from CRAN (source) + GDAL from kyngchaos. 
>>>> (e) rgdal from CRAN (source) + GDAL/GEOS/etc (source) with Apple clang 11.0
>>>> (f) rgdal from CRAN (source) + GDAL/GEOS/etc (source) with Apple clang 12.0
>>>> 
>>>> Only option (f) works correctly. 
>>>> 
>>>> Best regards
>>>> Gilberto 
>>>> 
>>>>> On 2 Dec 2020, at 22:44, Simon Urbanek <simon.urbanek using R-project.org> wrote:
>>>>> 
>>>>> Are you chasing a red herring here? Switching tools won help you - in fact they cause more issues since you'd need R-devel version of R to avoid breakage with the most recent tools or extra flags. I'm not sure which issue you are trying to solve. For gdal et al - make sure you install the latest version for CRAN - it is compatible with BigSur, older versions (before BigSur) were not.
>>>>> 
>>>>> Please start a new thread about questions unrelated to the original post as it's unclear if there are any open questions in the thread. Also please only answer if your have anything useful to say - if someone asks a question about the GUI it's rather pointless to post when you say you don't use it ... And if you ask a question, please be specific so we can help.
>>>>> 
>>>>> There is a lot of misinformation around, so be careful with random suggestions, they are very often plain wrong.
>>>>> 
>>>>> Cheers,
>>>>> Simon
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>>> On 3/12/2020, at 6:52 AM, Spencer Graves <spencer.graves using prodsyse.com> wrote:
>>>>>> 
>>>>>> 	  Sorry:  Typo:  I tried "sudo xcode-select --install" and got the same error.  This was in Terminal where "The default interactive shell is now zsh."  It's not critical, because "I was able to update Xcode from App Store."
>>>>>> 
>>>>>> 
>>>>>> 	  Thanks for the reply.  sg
>>>>>> 
>>>>>> 
>>>>>> On 2020-12-02 11:48, Dr Eberhard W Lisse wrote:
>>>>>>> sudo xcode-select --install
>>>>>>> el
>>>>>>> On 2020-12-02 18:52 , Spencer Graves wrote:
>>>>>>>> $ clang --version
>>>>>>>> Apple clang version 11.0.3 (clang-1103.0.32.59)
>>>>>>>> Target: x86_64-apple-darwin20.1.0
>>>>>>>> Thread model: posix
>>>>>>>> InstalledDir:
>>>>>>>> /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>>  ** "sudo rm -rf /Library/Developer/CommandLineTools" deleted
>>>>>>>> "CommandLineTools".  Then "sudo xcode-select —install" gave me "invalid
>>>>>>>> argument '—install'".  "sudo xcode-select —install" gave me the same.
>>>>>>>> 
>>>>>>>> 
>>>>>>>>  ** Fortunately, I was able to update Xcode from App Store.  Now I
>>>>>>>> get:
>>>>>>>> 
>>>>>>>> 
>>>>>>>> $ clang --version
>>>>>>>> Apple clang version 12.0.0 (clang-1200.0.32.27)
>>>>>>>> Target: x86_64-apple-darwin20.1.0
>>>>>>>> Thread model: posix
>>>>>>>> InstalledDir:
>>>>>>>> /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>>  If you also have a problem with "sudo xcode-select —install", you
>>>>>>>> might try the App Store.
>>>>>>>> 
>>>>>>>> 
>>>>>>>>  Thanks, Spencer
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On 2020-12-02 08:18, Gilberto Camara wrote:
>>>>>>>>> For me, it worked as follows
>>>>>>>>> 
>>>>>>>>> sudo rm -rf /Library/Developer/CommandLineTools
>>>>>>>>> sudo xcode-select —install
>>>>>>>>> 
>>>>>>>>> Best
>>>>>>>>> Gilberto
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> On 2 Dec 2020, at 15:15, Spencer Graves <spencer.graves using prodsyse.com>
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>  What should I do to "make sure to have installed the latest
>>>>>>>>>> version of Apple's development environment, including version 12.0 of
>>>>>>>>>> the compiler"?
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>  Thanks,
>>>>>>>>>>  Spencer Graves
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On 2020-12-02 07:36, GilbertoCamara wrote:
>>>>>>>>>>> Dear all,
>>>>>>>>>>> I am running R 4.0.3 on BigSur without major performance issues. I
>>>>>>>>>>> am using RStudio instead of the R GUI.
>>>>>>>>>>> I had to build all spatial packages (rgdal, sf, terra and raster)
>>>>>>>>>>> from source using the Apple clang version 12.0.0 compiler. I also
>>>>>>>>>>> had to build GDAL from scratch, since neither the “homebrew” version
>>>>>>>>>>> nor the “kyngchaos” version work well with BigSur.
>>>>>>>>>>> If anyone is building packages from source, please make sure to have
>>>>>>>>>>> installed the latest version of Apple’s development environment,
>>>>>>>>>>> including version 12.0 of the compiler as earlier versions do not
>>>>>>>>>>> work well in BigSur.
>>>>>>>>>>> Best
>>>>>>>>>>> Gilberto
>>>>>>>>>>> ===========================
>>>>>>>>>>> Prof Dr Gilberto Camara
>>>>>>>>>>> Secretariat Director
>>>>>>>>>>> GEO - Group on Earth Observations
>>>>>>>>>>>> On 2 Dec 2020, at 14:15, Bob Rudis <bob using rud.is> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> After a bit of git and google poking it seems this is happening to
>>>>>>>>>>>> users
>>>>>>>>>>>> of MacVim (https://github.com/macvim-dev/macvim/issues/1114) and a few
>>>>>>>>>>>> other FOSS projects.
>>>>>>>>>>>> 
>>>>>>>>>>>> In various GH issues (like that one) the issue claims to be
>>>>>>>>>>>> resolved by
>>>>>>>>>>>> using the latest Xcode & SDK but that’s not likely to be a solution
>>>>>>>>>>>> for R
>>>>>>>>>>>> GUI (which I’m assuming you're using and where this error is coming
>>>>>>>>>>>> up).
>>>>>>>>>>>> 
>>>>>>>>>>>> Some of the threads I saw made it seem like it was happening
>>>>>>>>>>>> especially on
>>>>>>>>>>>> Touch Bar-equipped MacBook Pros (but that’s not confirmed).
>>>>>>>>>>>> 
>>>>>>>>>>>> When I try R GUI on Big Sur (on the new M1 Mini) — either just plain R
>>>>>>>>>>>> scripts or plotting — I do not see this, nor do I see this on my Intel
>>>>>>>>>>>> MacBook Pro (both are on the latest Big Sur beta and R 4.0.3).
>>>>>>>>>>>> 
>>>>>>>>>>>> On Dec 2, 2020 at 7:53:26 AM, Spencer Graves
>>>>>>>>>>>> <spencer.graves using prodsyse.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> I didn't get this with R 4.0.3 under macOS Big Sur 11.0.1 with "time
>>>>>>>>>>>>> R CMD build" / check.  I have not tried it under RStudio.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> sg
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On 2020-12-02 05:01, Dr Eberhard W Lisse wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I don't get this in 4.0.3 (using the command line version of R) and I
>>>>>>>>>>>>> 
>>>>>>>>>>>>> don't get this in RStudio 1.3.1093 either.  I use the homebrew
>>>>>>>>>>>>> versions
>>>>>>>>>>>>> 
>>>>>>>>>>>>> without the R GUI even installed.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I am sure more detail will be helpful to the developers.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> el
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On 02/12/2020 11:09, Maria-Angeles Casares-de-Cal via R-SIG-Mac
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> To those in charge of R for Mac OS X:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I have installed the new Big Sur operating system and now I
>>>>>>>>>>>>>> always get
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> a warning similar to:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 2020-12-02 09:52:00.647 R[703:11786] Warning: Expected min height of
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> view: (<NSPopoverTouchBarItemButton: 0x7f9c6bb3df50>) to be less
>>>>>>>>>>>>>> than
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> or equal to 30 but got a height of 32.000000.  This error will be
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> logged once per view in violation.
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> and the computer becomes very slow.
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Can I do something to fix it, or will we have to wait for an R
>>>>>>>>>>>>>> update?
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Thank you very much for your attention.
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Sincerely,
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> María-Ángeles Casares-de-Cal
>>>>>>>>>>>>> 
>>>>>>>>>>>>> [...]
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> R-SIG-Mac mailing list
>>>>>>>>>>>>> R-SIG-Mac using r-project.org
>>>>>>>>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> [[alternative HTML version deleted]]
>>>>>>>>>>>> 
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> R-SIG-Mac mailing list
>>>>>>>>>>>> R-SIG-Mac using r-project.org
>>>>>>>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> _______________________________________________
>>>>>>>>>> R-SIG-Mac mailing list
>>>>>>>>>> R-SIG-Mac using r-project.org
>>>>>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> _______________________________________________
>>>>>>>> R-SIG-Mac mailing list
>>>>>>>> R-SIG-Mac using r-project.org
>>>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> R-SIG-Mac mailing list
>>>>>> R-SIG-Mac using r-project.org
>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> R-SIG-Mac mailing list
>>>>> R-SIG-Mac using r-project.org
>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>>>> 
>>>> _______________________________________________
>>>> R-SIG-Mac mailing list
>>>> R-SIG-Mac using r-project.org
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>>> 
>>> _______________________________________________
>>> R-SIG-Mac mailing list
>>> R-SIG-Mac using r-project.org
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>> 
>> _______________________________________________
>> R-SIG-Mac mailing list
>> R-SIG-Mac using r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
> 



===========================
Prof Dr Gilberto Camara
Secretariat Director
GEO - Group on Earth Observations
www.earthobservations.org

More information about the R-SIG-Mac mailing list