[R-pkg-devel] Errors in make check in Windows 10 with OpenBLAS
Tomas Kalibera
tom@@@k@||ber@ @end|ng |rom gm@||@com
Sat Mar 21 17:29:05 CET 2020
Maybe the created Riconv.dll is copied over by libiconv at some point
during your custom build.
Tomas
On 3/21/20 3:58 PM, Erin Hodgess wrote:
> Hello again.
>
> So I'm trying to find the place where the GNU iconv comes from.
> However, I looked at output from both the regular build and the
> custom, and I'm not seeing any differences in how win_iconv is generated.
>
> Any suggestions, please?
>
> Custom:
> c:/Rtools/mingw_64/bin/ar crs libtz.a localtime.o registryTZ.o strftime.o
> installing zoneinfo
> making win_iconv.d from win_iconv.c
> c:/Rtools/mingw_64/bin/gcc -std=gnu99 -I../../include -O3 -Wall
> -pedantic -march=native -pipe -c win_iconv.c -o win_iconv.o
> c:/Rtools/mingw_64/bin/gcc -std=gnu99 -shared -o Riconv.dll
> Riconv.def win_iconv.o
>
> Regular:
> c:/Rtools/mingw_64/bin/ar crs libtz.a localtime.o registryTZ.o strftime.o
> installing zoneinfo
> making win_iconv.d from win_iconv.c
> c:/Rtools/mingw_64/bin/gcc -std=gnu99 -I../../include -O3 -Wall
> -pedantic -mtune=core2 -c win_iconv.c -o win_iconv.o
> c:/Rtools/mingw_64/bin/gcc -std=gnu99 -shared -o Riconv.dll
> Riconv.def win_iconv.o
>
> Just the -march=native -pipe
>
> Erin Hodgess, PhD
> mailto: erinm.hodgess using gmail.com <mailto:erinm.hodgess using gmail.com>
>
>
> On Fri, Mar 20, 2020 at 9:01 AM Tomas Kalibera
> <tomas.kalibera using gmail.com <mailto:tomas.kalibera using gmail.com>> wrote:
>
> Thanks, so the problem is that your custom build uses GNU
> libiconv, which does not support codepage 28605. win_iconv, the
> one shipped with R, supports that code page. win_iconv is a
> wrapper for the Windows API. Unless you had a special reason not
> to, I would just recommend to use win_iconv. The "experience" will
> match the C library conversion calls, which may be used directly
> in some code at some places, and it is most tested as people
> normally use it. Still, the problem is not specific to strtrim,
> just to the example.
>
> Best
> Tomas
>
>
> On 3/20/20 3:27 PM, Erin Hodgess wrote:
>> Here are the results.
>>
>> Looks like enc2native and the iconv settings are different.
>>
>>
>> > #output from custom build
>> > Sys.getlocale()
>> [1] "LC_COLLATE=English_United
>> States.1252;LC_CTYPE=English_United
>> States.1252;LC_MONETARY=English_United
>> States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252"
>> > enc2native("\uff21")
>> [1] "<U+FF21>"
>> > enc2native("\u4e2d")
>> [1] "<U+4E2D>"
>> > Sys.setlocale("LC_CTYPE", "English_United States.28605")
>> [1] "English_United States.28605"
>> > enc2native("\uff21")
>> Error: unsupported conversion from 'UTF-8' in codepage 28605
>> > enc2native("\u4e2d")
>> Error: unsupported conversion from 'UTF-8' in codepage 28605
>> > extSoftVersion()[c("iconv","BLAS")]
>> iconv BLAS
>> "GNU libiconv 1.14" ""
>>
>>
>> > #output from regular build
>> > Sys.getlocale()
>> [1] "LC_COLLATE=English_United
>> States.1252;LC_CTYPE=English_United
>> States.1252;LC_MONETARY=English_United
>> States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252"
>> > enc2native("\uff21")
>> [1] "A"
>> > enc2native("\u4e2d")
>> [1] "<U+4E2D>"
>> > Sys.setlocale("LC_CTYPE", "English_United States.28605")
>> [1] "English_United States.28605"
>> > enc2native("\uff21")
>> [1] "A"
>> > enc2native("\u4e2d")
>> [1] "<U+4E2D>"
>> > extSoftVersion()[c("iconv","BLAS")]
>> iconv BLAS
>> "win_iconv" ""
>> >
>>
>> Thanks!
>>
>>
>> Erin Hodgess, PhD
>> mailto: erinm.hodgess using gmail.com <mailto:erinm.hodgess using gmail.com>
>>
>>
>> On Fri, Mar 20, 2020 at 8:03 AM Tomas Kalibera
>> <tomas.kalibera using gmail.com <mailto:tomas.kalibera using gmail.com>> wrote:
>>
>> Hi Erin,
>>
>> when you are building from sources, you can always uncomment
>> the example in str.Rd. But it would be good first to find out
>> why it is failing on your system/build.
>>
>> Please run the example in Rgui of the official R 3.6.3 build
>> on your machine. Does it work there?
>>
>> Please run the following commands in Rgui of the official R
>> 3.6.3 build on the machine and on your custom build. What do
>> you get?
>>
>> Sys.getlocale()
>> enc2native("\uff21")
>> enc2native("\u4e2d")
>> Sys.setlocale("LC_CTYPE", "English_United States.28605")
>> enc2native("\uff21")
>> enc2native("\u4e2d")
>> extSoftVersion()[c("iconv","BLAS")]
>>
>> Thanks,
>> Tomas
>>
>> On 3/20/20 2:35 PM, Erin Hodgess wrote:
>>> I was wondering if there is a way to either skip the
>>> utilities check and continuing, in order to test the other
>>> packages, please.
>>>
>>> Thanks!
>>>
>>>
>>> On Fri, Mar 20, 2020 at 5:01 AM Martin Maechler
>>> <maechler using stat.math.ethz.ch
>>> <mailto:maechler using stat.math.ethz.ch>> wrote:
>>>
>>> >>>>> Erin Hodgess on Thu, 19 Mar 2020 22:44:39 -0600
>>> writes:
>>>
>>> > Hi Tomas and others:
>>> > Here is the session Info. I also used the str
>>> example both without and
>>> > with changing the locale.
>>>
>>> > sI <- sessionInfo()
>>> >> str(sI)
>>> > List of 10
>>> > $ R.version :List of 14
>>> > ..$ platform : chr "x86_64-w64-mingw32"
>>> > ..$ arch : chr "x86_64"
>>> > ..$ os : chr "mingw32"
>>> > ..$ system : chr "x86_64, mingw32"
>>> > ..$ status : chr ""
>>> > ..$ major : chr "3"
>>> > ..$ minor : chr "6.3"
>>> > ..$ year : chr "2020"
>>> > ..$ month : chr "02"
>>> > ..$ day : chr "29"
>>> > ..$ svn rev : chr "77875"
>>> > ..$ language : chr "R"
>>> > ..$ version.string: chr "R version 3.6.3
>>> (2020-02-29)"
>>> > ..$ nickname : chr "Holding the Windsock"
>>> > $ platform : chr "x86_64-w64-mingw32/x64 (64-bit)"
>>> > $ locale : chr "LC_COLLATE=English_United
>>> > States.1252;LC_CTYPE=English_United
>>> States.1252;LC_MONETARY=English_United
>>> > States.125"| __truncated__
>>> > $ running : chr "Windows 10 x64 (build 18362)"
>>> > $ RNGkind : chr [1:3] "Mersenne-Twister"
>>> "Inversion" "Rejection"
>>> > $ basePkgs : chr [1:7] "stats" "graphics"
>>> "grDevices" "utils" ...
>>> > $ loadedOnly:List of 1
>>> > ..$ compiler:List of 9
>>> > .. ..$ Package : chr "compiler"
>>> > .. ..$ Version : chr "3.6.3"
>>> > .. ..$ Priority : chr "base"
>>> > .. ..$ Title : chr "The R Compiler Package"
>>> > .. ..$ Author : chr "Luke Tierney
>>> <luke-tierney using uiowa.edu <mailto:luke-tierney using uiowa.edu>>"
>>> > .. ..$ Maintainer : chr "R Core Team
>>> <R-core using r-project.org <mailto:R-core using r-project.org>>"
>>> > .. ..$ Description: chr "Byte code compiler for R."
>>> > .. ..$ License : chr "Part of R 3.6.3"
>>> > .. ..$ Built : chr "R 3.6.3; ; 2020-03-20
>>> 04:29:00 UTC; windows"
>>> > .. ..- attr(*, "class")= chr "packageDescription"
>>> > .. ..- attr(*, "file")= chr
>>> >
>>> "c:/newtimeR/R-3.6.3/library/compiler/Meta/package.rds"
>>> > $ matprod : chr "default"
>>> > $ BLAS : chr ""
>>> > $ LAPACK : chr ""
>>> > - attr(*, "class")= chr "sessionInfo"
>>>
>>>
>>> > Running the str example WITHOUT changing the locale:
>>> >> ## Truncation behavior (<-> correct width
>>> measurement) for "long"
>>> > non-ASCII:
>>> >> idx <- c(65313:65338, 65345:65350)
>>> >> fwch <- intToUtf8(idx) # full width character
>>> string: each has width 2
>>> >> ch <- strtrim(paste(LETTERS, collapse="._"), 64)
>>> >> (ncc <- c(c.ch <http://c.ch> = nchar(ch), w.ch
>>> <http://w.ch> = nchar(ch, "w"),
>>> > + c.fw = nchar(fwch), w.fw =
>>> nchar(fwch, "w")))
>>> > c.ch <http://c.ch> w.ch <http://w.ch> c.fw w.fw
>>> > 64 64 32 64
>>> >> stopifnot(unname(ncc) == c(64,64, 32, 64))
>>> >> ## nchar.max: 1st line needs an increase of 2
>>> in order to see 1 (in
>>> > UTF-$
>>> >> invisible(lapply(60:66, function(N) str(fwch,
>>> nchar.max = N)))
>>> > chr
>>> "<U+FF21><U+FF22><U+FF23><U+FF24><U+FF25><U+"| __truncated__
>>> > chr
>>> "<U+FF21><U+FF22><U+FF23><U+FF24><U+FF25><U+F"|
>>> __truncated__
>>> > chr
>>> "<U+FF21><U+FF22><U+FF23><U+FF24><U+FF25><U+FF"|
>>> __truncated__
>>> > chr
>>> "<U+FF21><U+FF22><U+FF23><U+FF24><U+FF25><U+FF2"|
>>> __truncated__
>>> > chr
>>> "<U+FF21><U+FF22><U+FF23><U+FF24><U+FF25><U+FF26"|
>>> __truncated__
>>> > chr
>>> "<U+FF21><U+FF22><U+FF23><U+FF24><U+FF25><U+FF26>"|
>>> __truncated__
>>> > chr
>>> "<U+FF21><U+FF22><U+FF23><U+FF24><U+FF25><U+FF26><"|
>>> __truncated__
>>> >> invisible(lapply(60:66, function(N) str( ch ,
>>> nchar.max = N))) # "1 is 1"
>>> > he$
>>> > chr
>>> "A._B._C._D._E._F._G._H._I._J._K._L._M._N._O"| __truncated__
>>> > chr
>>> "A._B._C._D._E._F._G._H._I._J._K._L._M._N._O."|
>>> __truncated__
>>> > chr
>>> "A._B._C._D._E._F._G._H._I._J._K._L._M._N._O._"|
>>> __truncated__
>>> > chr
>>> "A._B._C._D._E._F._G._H._I._J._K._L._M._N._O._P"|
>>> __truncated__
>>> > chr
>>> "A._B._C._D._E._F._G._H._I._J._K._L._M._N._O._P."|
>>> __truncated__
>>> > chr
>>> "A._B._C._D._E._F._G._H._I._J._K._L._M._N._O._P._"|
>>> __truncated__
>>> > chr
>>> "A._B._C._D._E._F._G._H._I._J._K._L._M._N._O._P._Q._R._S._T._U._V"
>>> >>
>>>
>>> > Running the str example WITH changing the locale:
>>>
>>> > oloc <- Sys.getlocale("LC_CTYPE")
>>> >> mbyte.lc <http://mbyte.lc> <-
>>> if(.Platform$OS.type == "windows")
>>> > + "English_United States.28605" else "en_GB.UTF-8"
>>> >> try(Sys.setlocale("LC_CTYPE", mbyte.lc
>>> <http://mbyte.lc>))
>>> > [1] "English_United States.28605"
>>> >> ## Truncation behavior (<-> correct width
>>> measurement) for "long"
>>> > non-ASCII:
>>> >> idx <- c(65313:65338, 65345:65350)
>>> >> fwch <- intToUtf8(idx) # full width character
>>> string: each has width 2
>>> >> ch <- strtrim(paste(LETTERS, collapse="._"), 64)
>>> >> (ncc <- c(c.ch <http://c.ch> = nchar(ch), w.ch
>>> <http://w.ch> = nchar(ch, "w"),
>>> > + c.fw = nchar(fwch), w.fw =
>>> nchar(fwch, "w")))
>>> > c.ch <http://c.ch> w.ch <http://w.ch> c.fw w.fw
>>> > 64 64 32 64
>>> >> stopifnot(unname(ncc) == c(64,64, 32, 64))
>>> >> ## nchar.max: 1st line needs an increase of 2
>>> in order to see 1 (in
>>> > UTF-$
>>> >> invisible(lapply(60:66, function(N) str(fwch,
>>> nchar.max = N)))
>>> > Error in strtrim(x.lrg, nchar.max - nc) :
>>> > unsupported conversion from 'UTF-8' in codepage 28605
>>> >> invisible(lapply(60:66, function(N) str( ch ,
>>> nchar.max = N))) # "1 is 1"
>>> > he$
>>> > chr
>>> "A._B._C._D._E._F._G._H._I._J._K._L._M._N._O"| __truncated__
>>> > chr
>>> "A._B._C._D._E._F._G._H._I._J._K._L._M._N._O."|
>>> __truncated__
>>> > chr
>>> "A._B._C._D._E._F._G._H._I._J._K._L._M._N._O._"|
>>> __truncated__
>>> > chr
>>> "A._B._C._D._E._F._G._H._I._J._K._L._M._N._O._P"|
>>> __truncated__
>>> > chr
>>> "A._B._C._D._E._F._G._H._I._J._K._L._M._N._O._P."|
>>> __truncated__
>>> > chr
>>> "A._B._C._D._E._F._G._H._I._J._K._L._M._N._O._P._"|
>>> __truncated__
>>> > chr
>>> "A._B._C._D._E._F._G._H._I._J._K._L._M._N._O._P._Q._R._S._T._U._V"
>>>
>>> > This is pretty strange.
>>>
>>> It is strange indeed that strtrim() fails in this
>>> situation.
>>>
>>> From what you wrote previously, this only happens when
>>> you link
>>> with OpenBLAS but not when you use R's own BLAS.
>>> Are you sure that that was the only difference, but not
>>> rather
>>> the difference in the locales you used in your two
>>> versions of R
>>> on Windows?
>>>
>>> Martin
>>>
>>> > Thanks for any help.
>>>
>>> > Sincerely,
>>> > Erin
>>>
>>> > Erin Hodgess, PhD
>>> > mailto: erinm.hodgess using gmail.com
>>> <mailto:erinm.hodgess using gmail.com>
>>>
>>>
>>> > On Thu, Mar 19, 2020 at 2:29 AM Tomas Kalibera
>>> <tomas.kalibera using gmail.com <mailto:tomas.kalibera using gmail.com>>
>>> > wrote:
>>>
>>> >> Hi Erin,
>>> >>
>>> >> this comes from examples in ?str. Could you
>>> please try to find a minimal
>>> >> example to reproduce on your system based on
>>> that? I would look for
>>> >> "line needs" in ?str and experiment with reducing
>>> the example there
>>> >> while still getting the error. Once it is
>>> minimal, could you please try
>>> >> also on the official build of R on your machine?
>>> >>
>>> >> Also it would be nice to have a session info. If
>>> it works in the
>>> >> official build but not your custom build, then
>>> one would have to look
>>> >> also into how the custom build is done.
>>> >>
>>> >> Thanks
>>> >> Tomas
>>> >>
>>> >>
>>> >> On 3/19/20 8:02 AM, Erin Hodgess wrote:
>>> >> > Hello everyone:
>>> >> >
>>> >> > I have built the latest version of the OpenBLAS
>>> on my Windows 10 machine.
>>> >> > When I run make distribution in R, everything
>>> is fine.
>>> >> >
>>> >> > However, when I run make check all, I get the
>>> following error:
>>> >> >
>>> >> > C:\newtimeR\R-3.6.3\src\gnuwin32>make check-all
>>> >> > Testing examples for package 'base'
>>> >> > Testing examples for package 'tools'
>>> >> > comparing 'tools-Ex.Rout' to
>>> 'tools-Ex.Rout.save' ... OK
>>> >> > Testing examples for package 'utils'
>>> >> > Error: testing 'utils' failed
>>> >> > Execution halted
>>> >> > make[3]: *** [Makefile.win:29:
>>> test-Examples-Base] Error 1
>>> >> > make[2]: *** [Makefile.common:185:
>>> test-Examples] Error 2
>>> >> > make[1]: *** [Makefile.common:171:
>>> test-all-basics] Error 1
>>> >> > make: *** [Makefile:324: check-all] Error 2
>>> >> >
>>> >> > And here is the section from the .Rout.fail for
>>> utils:
>>> >> >
>>> >> >> ## Multibyte characters in strings (in
>>> multibyte locales):
>>> >> >> oloc <- Sys.getlocale("LC_CTYPE")
>>> >> >> mbyte.lc <http://mbyte.lc> <-
>>> if(.Platform$OS.type == "windows")
>>> >> > + "English_United States.28605" else "en_GB.UTF-8"
>>> >> >> try(Sys.setlocale("LC_CTYPE", mbyte.lc
>>> <http://mbyte.lc>))
>>> >> > [1] "English_United States.28605"
>>> >> >> ## Truncation behavior (<-> correct width
>>> measurement) for "long"
>>> >> > non-ASCII:
>>> >> >> idx <- c(65313:65338, 65345:65350)
>>> >> >> fwch <- intToUtf8(idx) # full width character
>>> string: each has width 2
>>> >> >> ch <- strtrim(paste(LETTERS, collapse="._"), 64)
>>> >> >> (ncc <- c(c.ch <http://c.ch> = nchar(ch), w.ch
>>> <http://w.ch> = nchar(ch, "w"),
>>> >> > + c.fw = nchar(fwch), w.fw =
>>> nchar(fwch, "w")))
>>> >> > c.ch <http://c.ch> w.ch <http://w.ch> c.fw w.fw
>>> >> > 64 64 32 64
>>> >> >> stopifnot(unname(ncc) == c(64,64, 32, 64))
>>> >> >> ## nchar.max: 1st line needs an increase of
>>> 2 in order to see 1 (in
>>> >> > UTF-8!):
>>> >> >> invisible(lapply(60:66, function(N) str(fwch,
>>> nchar.max = N)))
>>> >> > Error in strtrim(x.lrg, nchar.max - nc) :
>>> >> > unsupported conversion from 'UTF-8' in
>>> codepage 28605
>>> >> > Calls: lapply ... FUN -> str -> str.default ->
>>> maybe_truncate -> strtrim
>>> >> > Execution halted
>>> >> >
>>> >> > Has anyone run into this before, please?
>>> >> >
>>> >> > If I run this line-by-line starting at the "idx
>>> <-", things are fine.
>>> >> >
>>> >> > I also compiled R from source without the
>>> OpenBLAS, and it was fine.
>>> >> >
>>> >> > Thanks for any suggestions.
>>> >> >
>>> >> > Sincerely,
>>> >> > Erin
>>> >> >
>>> >> >
>>> >> >
>>> >> > Erin Hodgess, PhD
>>> >> > mailto: erinm.hodgess using gmail.com
>>> <mailto:erinm.hodgess using gmail.com>
>>>
>>> --
>>> Erin Hodgess, PhD
>>> mailto: erinm.hodgess using gmail.com <mailto:erinm.hodgess using gmail.com>
>>
>>
>
[[alternative HTML version deleted]]
More information about the R-package-devel
mailing list