[Rd] Unicode characters in ISO8859-15 locale

[Prof Brian Ripley]

On 17/12/2020 12:28, Jeroen Ooms wrote:
> The hunspell package uses the code below to replace curly quotes (aka

officially these are directional (or right/left) quotes.

> fancyquotes) with a regular ascii quotes that are needed for check
> spelling:
> 
>      chartr("\u2019", "'", input)
> 
> As of last week this stopped working on CRAN in the Linux server that
> runs in ISO8859-15 locale. From the error message, it seems that R no
> longer parses the escaped unique string, which gets turned into
> "<U+2019>".

You need to distinguish the string and what gets printed.  I see (for 
the CRAN log from a 2-day-old R)

  Error in chartr("<U+2019>", "'", as.character(add_words)) :

which is what I would expect to be printed in that locale:

> x <- "\u2019"
> Encoding(x)
[1] "UTF-8"
> x
[1] "<U+2019>"

> Is this expected?

Changes are expected, as this is an area being worked on (to try to 
remove some system-dependent behaviour).  However, I cannot reproduce 
this easily:

> chartr("\u2019", "'", "abc\u2019")
[1] "abc'"

As I say, work in progress.

-- 
Brian D. Ripley,                  ripley using stats.ox.ac.uk
Emeritus Professor of Applied Statistics, University of Oxford