[Rd] RODBC Close Memory Leak Question

[Tom McCallum]

Hi everyone,

In relation to the RODBC odbcClose bug which was fixed back in the  
changelog below:

Version: 1.2-3 (2008-01-24, released)

	* Plug a memory leak in inRODBCClose (closing a connection),
	reported by Stephan Henne.

	* Use translateChar() on character data sent in.

Background:
I am running some data from a SQL Server database, through unixODBC  
(freetds) into R via the RODBC package.  If I make many calls to  
odbcConnect, perform a query and then close the connection over a period  
of time the memory attached appears never to be freed and the process  
eventually - say 4 hours into the run - dies due to lack of memory.

Research:
I have created a dummy piece of code which shows the growing memory  
allocation shown below.   Each call brings back between 20-100 pieces of  
data which fits into a one column vector - so not a lot of data.  The  
varible x is normally returns from a function and used subsequently but  
for the purposes of this test I have kept it simple.

library("RODBC")
for( i in 1:100 ) {
	conn <- odbcConnect(************)					<-- anonymised
	x <- sqlQuery(conn, "SELECT * FROM table_name");			<-- anonymised
	odbcClose(conn);
}
quit()

I am not sure whether the leak is related to this previous bug or if it is  
related to the way the garbage collection works, but either way if I run  
this piece of code the memory attached to the database handle is never  
freed so as I bring back more and more data over a large number of runs my  
memory slowly fills up and the application crashes, unable to allocate any  
more memory.

Just to make sure I was running the latest and greatest RODBC I installed  
it again from source, but the initial installation was only done last  
month so I presume .install.packages would have got this latest version  
anyway.

Any advice on where to start searching to patch this as it is a real  
pain.   I have included below version information and the valgrind  
output.  If this is a new bug I will add it the bug tracker but I wanted  
to make sure I was not missing anything blindingly obvious to someone in  
the know.

Thanks for your help,

Tom


> version
                _
platform       i686-pc-linux-gnu
arch           i686
os             linux-gnu
system         i686, linux-gnu
status
major          2
minor          7.0
year           2008
month          04
day            22
svn rev        45424
language       R
version.string R version 2.7.0 (2008-04-22)

==8682== Memcheck, a memory error detector.
==8682== Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward et al.
==8682== Using LibVEX rev 1471, a library for dynamic binary translation.
==8682== Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP.
==8682== Using valgrind-3.1.0, a dynamic binary instrumentation framework.
==8682== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward et al.
==8682== For more details, rerun with: -v
==8682==

R version 2.7.0 (2008-04-22)
Copyright (C) 2008 The R Foundation for Statistical Computing
ISBN 3-900051-07-0

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

   Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library("RODBC")
--8682-- WARNING: unhandled syscall: 311
--8682-- You may be able to write your own handler.
--8682-- Read the file README_MISSING_SYSCALL_OR_IOCTL.
> for( i in 1:100 ) {
+ conn <- odbcConnect(************)					<-- anonymised
+ x <- sqlQuery(conn, "SELECT * FROM table_name");			<-- anonymised
+ odbcClose(conn);
+ }
> quit()
==8682==
==8682== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 255 from 1)
==8682== malloc/free: in use at exit: 13,470,816 bytes in 6,739 blocks.
==8682== malloc/free: 121,121 allocs, 114,382 frees, 134,084,145 bytes  
allocated.
==8682== For counts of detected errors, rerun with: -v
==8682== searching for pointers to 6,739 not-freed blocks.
==8682== checked 13,332,140 bytes.
==8682==
==8682== 56 bytes in 1 blocks are possibly lost in loss record 15 of 55
==8682==    at 0x40045EB: calloc (vg_replace_malloc.c:279)
==8682==    by 0x805AA60: R_chk_calloc (memory.c:2368)
==8682==    by 0x57C89D6: RODBCDriverConnect (RODBC.c:244)
==8682==    by 0x8160A68: do_dotcall (dotcode.c:863)
==8682==    by 0x81857CE: Rf_eval (eval.c:489)
==8682==    by 0x81895B7: do_set (eval.c:1420)
==8682==    by 0x81855B0: Rf_eval (eval.c:463)
==8682==    by 0x818631E: do_begin (eval.c:1172)
==8682==    by 0x81855B0: Rf_eval (eval.c:463)
==8682==    by 0x81879B4: Rf_applyClosure (eval.c:669)
==8682==    by 0x8185469: Rf_eval (eval.c:507)
==8682==    by 0x818631E: do_begin (eval.c:1172)
==8682==
==8682==
==8682== 56 bytes in 14 blocks are definitely lost in loss record 16 of 55
==8682==    at 0x40051F9: malloc (vg_replace_malloc.c:149)
==8682==    by 0x4005271: realloc (vg_replace_malloc.c:306)
==8682==    by 0x80B47EC: parse_expression (regex.c:5202)
==8682==    by 0x80B4934: parse_branch (regex.c:4714)
==8682==    by 0x80B49BF: parse_reg_exp (regex.c:4666)
==8682==    by 0x80B5317: Rf_regcomp (regex.c:4635)
==8682==    by 0x812A38F: do_gsub (character.c:1356)
==8682==    by 0x806650A: do_internal (names.c:1129)
==8682==    by 0x81855B0: Rf_eval (eval.c:463)
==8682==    by 0x818631E: do_begin (eval.c:1172)
==8682==    by 0x81855B0: Rf_eval (eval.c:463)
==8682==    by 0x81879B4: Rf_applyClosure (eval.c:669)
==8682==
==8682==
==8682== 237 bytes in 9 blocks are definitely lost in loss record 27 of 55
==8682==    at 0x40051F9: malloc (vg_replace_malloc.c:149)
==8682==    by 0x4D6BFC51: xmalloc (in /usr/lib/libreadline.so.5.0)
==8682==    by 0x4D6AA4B9: readline_internal_teardown (in  
/usr/lib/libreadline.so.5.0)
==8682==    by 0x4D6BC1DA: rl_callback_read_char (in  
/usr/lib/libreadline.so.5.0)
==8682==    by 0x80F9A8F: Rstd_ReadConsole (sys-std.c:902)
==8682==    by 0x8058949: Rf_ReplIteration (main.c:206)
==8682==    by 0x8058A89: R_ReplConsole (main.c:306)
==8682==    by 0x8058D67: run_Rmainloop (main.c:967)
==8682==    by 0x8056670: main (Rmain.c:35)
==8682==
==8682==
==8682== 5,544 bytes in 99 blocks are definitely lost in loss record 46 of  
55
==8682==    at 0x40045EB: calloc (vg_replace_malloc.c:279)
==8682==    by 0x805AA60: R_chk_calloc (memory.c:2368)
==8682==    by 0x57C89D6: RODBCDriverConnect (RODBC.c:244)
==8682==    by 0x8160A68: do_dotcall (dotcode.c:863)
==8682==    by 0x81857CE: Rf_eval (eval.c:489)
==8682==    by 0x81895B7: do_set (eval.c:1420)
==8682==    by 0x81855B0: Rf_eval (eval.c:463)
==8682==    by 0x818631E: do_begin (eval.c:1172)
==8682==    by 0x81855B0: Rf_eval (eval.c:463)
==8682==    by 0x81879B4: Rf_applyClosure (eval.c:669)
==8682==    by 0x8185469: Rf_eval (eval.c:507)
==8682==    by 0x818631E: do_begin (eval.c:1172)
==8682==
==8682== LEAK SUMMARY:
==8682==    definitely lost: 5,837 bytes in 122 blocks.
==8682==      possibly lost: 56 bytes in 1 blocks.
==8682==    still reachable: 13,464,923 bytes in 6,616 blocks.
==8682==         suppressed: 0 bytes in 0 blocks.
==8682== Reachable blocks (those to which a pointer was found) are not  
shown.
==8682== To see them, rerun with: --show-reachable=yes