[R-pkg-devel] Segmentation Fault cause 'memory not mapped' on Debian only

[Dirk Eddelbuettel]

On 22 April 2023 at 14:43, D Z wrote:
| Thank you Dirk and Joshua,
| Using Valgrind, I have found several memory leaks in the program that required
| some smaller rewrites.
| Valgrind (with Makevars flags -O0 and --ggdb) now reports no "definitely" and
| no "indirectly" lost bytes so that's a small win already.

Very nice and good news!
 
| But two questions remain:
| 
|  1. I have 960 bytes "possibly lost", where the callstack does not touch any of
|     my code but seems to originate from subset.c (base R?) and libgomp, is this
|     ok/expected, or am I misreading the output?
|  2. For all tests that I ran, there are still 105mln bytes still reachable. I
|     read online that this can be problematic, but usually isn't. Is this
|     something that I should invest more time in or can I ignore this?

It is difficult to say something _definite_ with a language like R that
manages memory dynamically. This is fundamentally different from C/C++ which
valgrind was written for.  So _ex ante_ you are in good news territory.  When
I am in doubt I sometimes compare to valgrind output (using the same
parameters) on a known-good package. You could try xts or zoo, for example,
or another package equally focused on creating data objects from binary files
as RITCH is.

The log below mentions GOMP. Might be worthwhile opening another branch in
your repo and throwing OpenMP out, or forcing a single thread, or ... for
another test. 

The additional ASAN/UBSAN tests mentioned by Josh are a good idea too -- and
CRAN runs them anyway (as part of the add-on test suites running from a
well-known university town in southern England...)

Best,  Dirk

| Thank you very much!
| David
| 
| 
| To reproduce the valgrind output using bash (this is using the latest code
| commit https://github.com/DavZim/RITCH/commit/
| 13ea3f478717edf24f8db709f0cac1e22b501afc)
| 
| echo "PKG_CXXFLAGS=-O0 -ggdb\nPKG_LIBS=-O0 -ggdb -lz" >> src/Makevars.win
| echo "PKG_CXXFLAGS=-O0 -ggdb\nPKG_LIBS=-O0 -ggdb -lz" >> src/Makevars
| 
| R CMD build .
| R CMD INSTALL RITCH_0.1.15.tar.gz
| 
| R -d "valgrind --tool=memcheck --leak-check=full" -e "tinytest::test_package
| ('RITCH')"
| # omitted for brevity...
| tinytest::test_package('RITCH')
| test_filename_helpers.R.......   13 tests OK 1.1s
| test_filter_itch.R............   60 tests OK 30.1s
| test_gz_functions.R...........    5 tests OK 6.0s
| test_read_functions.R.........  122 tests OK 14.1s
| test_write_itch.R.............   18 tests OK 33.0s
| All ok, 218 results (1m 24.5s)
| >
| ==1535==
| ==1535== HEAP SUMMARY:
| ==1535==     in use at exit: 105,616,149 bytes in 18,292 blocks
| ==1535==   total heap usage: 562,268 allocs, 543,976 frees, 6,834,040,088 bytes
| allocated
| ==1535==
| ==1535== 960 bytes in 3 blocks are possibly lost in loss record 199 of 2,000
| ==1535==    at 0x483DD99: calloc (in /usr/lib/x86_64-linux-gnu/valgrind/
| vgpreload_memcheck-amd64-linux.so)
| ==1535==    by 0x40149DA: allocate_dtv (dl-tls.c:286)
| ==1535==    by 0x40149DA: _dl_allocate_tls (dl-tls.c:532)
| ==1535==    by 0x5741322: allocate_stack (allocatestack.c:622)
| ==1535==    by 0x5741322: pthread_create@@GLIBC_2.2.5 (pthread_create.c:660)
| ==1535==    by 0x5710DEA: ??? (in /usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0)
| ==1535==    by 0x57088E0: GOMP_parallel (in /usr/lib/x86_64-linux-gnu/
| libgomp.so.1.0.0)
| ==1535==    by 0x10882EB9: subsetVectorRaw (subset.c:54)
| ==1535==    by 0x10883B50: subsetDT (subset.c:309)
| ==1535==    by 0x494253D: ??? (in /usr/lib/R/lib/libR.so)
| ==1535==    by 0x49861A6: ??? (in /usr/lib/R/lib/libR.so)
| ==1535==    by 0x4999D77: Rf_eval (in /usr/lib/R/lib/libR.so)
| ==1535==    by 0x499BC2E: ??? (in /usr/lib/R/lib/libR.so)
| ==1535==    by 0x499CB21: Rf_applyClosure (in /usr/lib/R/lib/libR.so)
| ==1535==
| ==1535== LEAK SUMMARY:
| ==1535==    definitely lost: 0 bytes in 0 blocks
| ==1535==    indirectly lost: 0 bytes in 0 blocks
| ==1535==      possibly lost: 960 bytes in 3 blocks
| ==1535==    still reachable: 105,615,189 bytes in 18,289 blocks
| ==1535==                       of which reachable via heuristic:
| ==1535==                         newarray           : 4,264 bytes in 1 blocks
| ==1535==         suppressed: 0 bytes in 0 blocks
| ==1535== Reachable blocks (those to which a pointer was found) are not shown.
| ==1535== To see them, rerun with: --leak-check=full --show-leak-kinds=all
| ==1535==
| ==1535== For lists of detected and suppressed errors, rerun with: -s
| ==1535== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

-- 
dirk.eddelbuettel.com | @eddelbuettel | edd using debian.org