[R-pkg-devel] Possible false negative for compiled C++ code in CRAN checks

Tue Nov 12 05:02:17 CET 2024

The issue persists, and I really appreciate the help. The last comment in Stack Overflow (https://stackoverflow.com/questions/79171799/addresssanitizer-error-alloc-dealloc-mismatch-operator-new-vs-free-in-r-packa?noredirect=1#comment139621299_79171799) suggests a false positive, and if that is the case, should I add that as a note when resubmitting?

For v2.0.3 I implemented a full refactor, where I simplified the code considerably while trying to solve what looks to be a false positive.

Now I implemented a "patch" based on the suggestion:

  vector<pair<size_t, size_t>> ret;

  // ADDED //
  if (entities.empty() || entities.size() == 0) {
    return ret;
  }
  ////////////

  for (size_t i = 0; i < entities.size() - 1; ++i) {
    ret.push_back(
        {entities[i].GetBounds().second, entities[i + 1].GetBounds().first});
  }

After testing with the R-Hub image "Clang Asan" with this code:

  # Define the package directory and Docker image
  PACKAGE_DIR=$(pwd)
  DOCKER_IMAGE="ghcr.io/r-hub/containers/clang-asan:latest"

  # Pull the Docker image
  docker pull $DOCKER_IMAGE

  # Run the R CMD check inside the Docker container
  docker run --rm -v "$PACKAGE_DIR":/workspace -w /workspace $DOCKER_IMAGE bash -c "
   Rscript -e 'if (!requireNamespace(\"pak\", quietly = TRUE)) { install.packages(\"pak\", repos = \"https://r-lib.github.io/p/pak/dev//") }'
   Rscript -e 'pak::pkg_install(\"decor\", dependencies = TRUE)'
   Rscript -e 'pak::pkg_install(\"rcmdcheck\", dependencies = TRUE)'
   Rscript -e 'pak::pkg_install(\"pkgbuild\", dependencies = TRUE)'
   Rscript -e 'pak::pkg_install(\".\", dependencies = TRUE)'
   Rscript -e 'pkgbuild::check_build_tools(debug = TRUE)'
   Rscript -e 'rcmdcheck::rcmdcheck(args = c(\"--no-manual\", \"--as-cran\"), build_args = \"--no-manual\", error_on = \"error\")'
  "

I get the following output:

    #2 0x7bc709b3e119 in RedatamLib::ByteArrayReader::MovePos(int) readers/ByteArrayReader.cpp:99:44
    #3 0x7bc709b3e119 in RedatamLib::ByteArrayReader::ReadByte() readers/ByteArrayReader.cpp:172:3
    #4 0x7bc709b3d2a6 in RedatamLib::ByteArrayReader::ReadInt16LE() readers/ByteArrayReader.cpp:178:32
    #5 0x7bc709b3d2a6 in RedatamLib::ByteArrayReader::TryReadStr(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>*, bool) readers/ByteArrayReader.cpp:110:20
    #6 0x7bc709b4778a in RedatamLib::FuzzyEntityParser::TryGetEntity() readers/FuzzyEntityParser.cpp:44:17
    #7 0x7bc709b4601b in RedatamLib::FuzzyEntityParser::ParseEntities() readers/FuzzyEntityParser.cpp:18:14
    #8 0x7bc709baaaa7 in RedatamLib::RedatamDatabase::OpenDictionary(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) entities/RedatamDatabase.cpp:31:25
    #9 0x7bc709baa876 in RedatamLib::RedatamDatabase::RedatamDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) entities/RedatamDatabase.cpp:18:3
    #10 0x7bc709bb76f8 in export_redatam_to_list_(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>) /tmp/RtmpEHQaSu/file136708ed8/redatam.Rcheck/00_pkg_src/redatam/src/main.cpp:13:33
    #11 0x7bc709bb8438 in _redatam_export_redatam_to_list_ src/cpp11.cpp:12:27
    #12 0x7bc757f3a700 in R_doDotCall /tmp/R-devel/src/main/dotcode.c:754:11

   SUMMARY: AddressSanitizer: alloc-dealloc-mismatch (/opt/R/devel-asan/lib/R/bin/exec/R+0xc6306) (BuildId: 178e357df79b1589a38c1949da5e5f022d4bb535) in free
   ==4218==HINT: if you don't care about these errors you may set ASAN_OPTIONS=alloc_dealloc_mismatch=0
   ==4218==ABORTING

  2 errors ✖ | 0 warnings ✔ | 2 notes ✖
  Error: R CMD check found ERRORs

——
Mauricio "Pachá" Vargas Sepúlveda
PhD Student, Political Science
University of Toronto

________________________________________
From: Mauricio Vargas Sepulveda <m.sepulveda using mail.utoronto.ca>
Sent: November 11, 2024 7:44 PM
To: Ivan Krylov <ikrylov using disroot.org>
Cc: r-package-devel using r-project.org <r-package-devel using r-project.org>
Subject: Re: [R-pkg-devel] Possible false negative for compiled C++ code in CRAN checks

Dear Prof. Dr.  Ivan Krylov,

I hope you are doing well.

Thanks for your prompt reply. I shall try to implement a fix and resumit based on this. It is a valuable suggestion.
——
Mauricio "Pachá" Vargas Sepúlveda
PhD Student, Political Science
University of Toronto

________________________________________
From: Ivan Krylov <ikrylov using disroot.org>
Sent: November 11, 2024 4:48 AM
To: Mauricio Vargas Sepulveda <m.sepulveda using mail.utoronto.ca>
Cc: r-package-devel using r-project.org <r-package-devel using r-project.org>
Subject: Re: [R-pkg-devel] Possible false negative for compiled C++ code in CRAN checks

[You don't often get email from ikrylov using disroot.org. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

Dear Mauricio Vargas Sepulveda,

Welcome to R-package-devel!

В Sat, 9 Nov 2024 17:34:07 +0000
Mauricio Vargas Sepulveda <m.sepulveda using mail.utoronto.ca> пишет:

> CRAN reported memory leaks for:
>
> CLAN/ASAN:
> https://www.stats.ox.ac.uk/pub/bdr/memtests/clang-ASAN/redatam/00check.log
> CLANG/UBSAN:
> https://www.stats.ox.ac.uk/pub/bdr/memtests/gcc-UBSAN/redatam/00check.log

These include container-overflow at
src/redatamlib/ByteArrayReader.cpp:170:23 and method calls with `this`
being a null pointer at src/redatamlib/FuzzyVariableParser.cpp:47 and
src/redatamlib/Entity.cpp:52.

A container-overflow in ByteArrayReader::ReadByte is dangerous because
the code is reading past the end of the vector populated with known
data. While the code doesn't crash, the remainder of the vector could
contain anything. This could invalidate the conclusions of a scientific
work if they end up being based on random contents of uninitialised
memory. You need to find out why m_currPos exceeds the number of bytes
in the array and prevent it from happening.

Calling methods on a null pointer is also an error in the code. This is
quite confusing, but I have a guess for what might be causing this:

  for (size_t i = 0; i < entities.size() - 1; ++i) {
    ret.push_back(
        {entities[i].GetBounds().second, entities[i + 1].GetBounds().first});
  }

If entities.size() is 0, entities.size() - 1 will overflow to a large
positive number, due to std::vector<T>::size_type being unsigned. The
loop will proceed accessing out-of-bounds elements of 'entities' until
something will cause it to crash. The guess could be completely wrong,
unfortunately.

Have you fixed these problems since v2.0.0?

> After asking on Stack Overflow
> (https://stackoverflow.com/q/79171799/3720258), it was suggested that
> I set 'CXXFLAGS="-stdlib=libc++"' in 'configure'. The question is
> very long and provides all the details that I skip here.

I am not seeing alloc-dealloc-mismatch in the CRAN checks, neither in
the links you provided, nor in the latest pretest results at
<https://win-builder.r-project.org/incoming_pretest/redatam_2.0.3_20241107_154750/>.
I think that these indicate a problem with the way your GitHub actions
are set up. Have you received a CRAN check report with
alloc-dealloc-mismatch in it?

I've tried to compare the versions visible in the CRAN archive but
couldn't get far because there was a lot of formatting changes. What
exactly are the problems for which your latest submission of v2.0.3 has
been archived? Do you need help reproducing the container-overflow
and/or null object pointer errors?

--
Best regards,
Ivan