[R] segfault reading large BLOB from SQL Server
jeffrey.j.hallman at frb.gov
jeffrey.j.hallman at frb.gov
Fri Dec 21 21:51:37 CET 2012
I am getting segfaults when I try to read a large binary object from a SQL
Server database via RODBC. I am using the FreeTDS ODBC driver, and it has
been working fine when reading from this same database. I have included
relevant parts of the session below.
Each row of the v_MAFiles view holds a text, html or pdf version of a
document. The FileType entry is one of TXT, PDF, or HTML. If it's PDF,
the FileString variable is empty and the FileBLOB holds the (binary)
contents of the pdf. If FileType is TXT or HTML, then FileBLOB is empty
and the file contents are in FileString.
Note that the COLUMN_SIZE for FileBLOB and FileString are both 2 GB.
However, the actual strings and blobs in there are actually only about 60
KB.
I can read the FileString column with no problem. But trying to read a
FileBLOB entry segfaults, apparently when R is calling malloc(). I suspect
that R is trying and failing to allocate 2 GB of memory to hold something
with the reported COLUMN_SIZE. But it doesn't barf when it loads a
FileString entry of about the same 60 KB size. Perhaps the RODBC code is
somehow discovering that the string in FileString is not really 2 GB, but
it's not finding that out for a FileBLOB?
Any help with this would be much appreciated.
> Sys.info()
sysname release
version
"Linux" "2.6.18-308.24.1.el5"
"#1 SMP Wed Nov 21 11:42:14 EST 2012"
nodename machine
login
"mralx1.rsma.frb.gov" "x86_64"
"unknown"
user effective_user
"m1jjh00" "m1jjh00"
> sessionInfo()
R version 2.15.0 (2012-03-30)
Platform: x86_64-redhat-linux-gnu (64-bit)
locale:
[1] LC_CTYPE=en_US LC_NUMERIC=C LC_TIME=en_US
LC_COLLATE=en_US LC_MONETARY=en_US
[6] LC_MESSAGES=en_US LC_PAPER=C LC_NAME=C
LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] mra_1.0 frb_3.9 fame_2.19 tis_1.23
RODBC_1.3-6
[6] RObjectTables_0.3-1
loaded via a namespace (and not attached):
[1] data.table_1.8.2 XML_3.9-4
> conn <- h41Connect(devl = T)
> odbcGetInfo(conn)
DBMS_Name DBMS_Ver Driver_ODBC_Ver
Data_Source_Name Driver_Name
"Microsoft SQL Server" "10.50.4000" "03.50"
"h41_devl" "libtdsodbc.so"
Driver_Ver ODBC_Ver Server_Name
"0.91" "03.52" "MSSDEV1"
> sqlColumns(conn, "v_MAFiles")
TABLE_CAT TABLE_SCHEM TABLE_NAME COLUMN_NAME DATA_TYPE TYPE_NAME
COLUMN_SIZE BUFFER_LENGTH DECIMAL_DIGITS
1 H41 dbo v_MAFiles ReleaseId 1 char
3 3 NA
2 H41 dbo v_MAFiles AsOfDate -9 date
10 20 NA
3 H41 dbo v_MAFiles GeneratedDate 93 datetime
23 16 3
4 H41 dbo v_MAFiles FileType 12 varchar
4 4 NA
5 H41 dbo v_MAFiles FileBLOB -4 image
2147483647 2147483647 NA
6 H41 dbo v_MAFiles FileString -1 text
2147483647 2147483647 NA
7 H41 dbo v_MAFiles UpdatedBy 12 varchar
60 60 NA
8 H41 dbo v_MAFiles UpdatedDateTime 93 datetime
23 16 3
NUM_PREC_RADIX NULLABLE REMARKS COLUMN_DEF SQL_DATA_TYPE SQL_DATETIME_SUB
CHAR_OCTET_LENGTH ORDINAL_POSITION
1 NA 0 <NA> <NA> 1 NA
3 1
2 NA 0 <NA> <NA> -9 NA
NA 2
3 NA 0 <NA> <NA> 9 3
NA 3
4 NA 0 <NA> <NA> 12 NA
4 4
5 NA 1 <NA> <NA> -4 NA
2147483647 5
6 NA 1 <NA> <NA> -1 NA
2147483647 6
7 NA 0 <NA> <NA> 12 NA
60 7
8 NA 0 <NA> <NA> 9 3
NA 8
IS_NULLABLE SS_DATA_TYPE
1 NO 47
2 NO 0
3 NO 61
4 NO 39
5 YES 37
6 YES 39
7 NO 39
8 NO 61
> z <- odbcQuery(conn, "select FileBLOB from v_MAFiles where ReleaseId =
'L61' and AsOfDate = '2012-12-19'")
> z
[1] 1
> sqlGetResults(conn)
*** glibc detected *** /opt/R/R-2.15.0/lib64/R/bin/exec/R: free(): invalid
next size (fast): 0x00000000110fde60 ***
======= Backtrace: =========
/lib64/libc.so.6[0x35efa711df]
/lib64/libc.so.6(cfree+0x4b)[0x35efa7163b]
/lcl/mra/prod1/files/R/library/RODBC/libs/RODBC.so(RODBCFetchRows
+0x401)[0x2ae891623a71]
/opt/R/R-2.15.0/lib64/R/lib/libR.so[0x2ae88aeb6d39]
/opt/R/R-2.15.0/lib64/R/lib/libR.so(Rf_eval+0x696)[0x2ae88aeecc96]
/opt/R/R-2.15.0/lib64/R/lib/libR.so[0x2ae88aeee389]
/opt/R/R-2.15.0/lib64/R/lib/libR.so(Rf_eval+0x474)[0x2ae88aeeca74]
/opt/R/R-2.15.0/lib64/R/lib/libR.so(Rf_applyClosure+0x308)[0x2ae88aef11e8]
/opt/R/R-2.15.0/lib64/R/lib/libR.so(Rf_eval+0x323)[0x2ae88aeec923]
/opt/R/R-2.15.0/lib64/R/lib/libR.so[0x2ae88aeef5e7]
/opt/R/R-2.15.0/lib64/R/lib/libR.so(Rf_eval+0x474)[0x2ae88aeeca74]
/opt/R/R-2.15.0/lib64/R/lib/libR.so[0x2ae88aeee389]
/opt/R/R-2.15.0/lib64/R/lib/libR.so(Rf_eval+0x474)[0x2ae88aeeca74]
/opt/R/R-2.15.0/lib64/R/lib/libR.so(Rf_applyClosure+0x308)[0x2ae88aef11e8]
/opt/R/R-2.15.0/lib64/R/lib/libR.so(Rf_eval+0x323)[0x2ae88aeec923]
/opt/R/R-2.15.0/lib64/R/lib/libR.so(Rf_ReplIteration+0x18d)[0x2ae88af2b9cd]
/opt/R/R-2.15.0/lib64/R/lib/libR.so(run_Rmainloop+0xc0)[0x2ae88af2bd20]
/opt/R/R-2.15.0/lib64/R/bin/exec/R(main+0x1b)[0x40081b]
/lib64/libc.so.6(__libc_start_main+0xf4)[0x35efa1d994]
/opt/R/R-2.15.0/lib64/R/bin/exec/R[0x400719]
======= Memory map: ========
00400000-00401000 r-xp 00000000 00:1b
1305930 /opt/R/R-2.15.0/lib64/R/bin/exec/R
00600000-00601000 rw-p 00000000 00:1b
1305930 /opt/R/R-2.15.0/lib64/R/bin/exec/R
10a5b000-13192000 rw-p 10a5b000 00:00 0
[heap]
35ef600000-35ef61c000 r-xp 00000000 fd:07
131076 /lib64/ld-2.5.so
35ef81c000-35ef81d000 r--p 0001c000 fd:07
131076 /lib64/ld-2.5.so
35ef81d000-35ef81e000 rw-p 0001d000 fd:07
131076 /lib64/ld-2.5.so
35efa00000-35efb4e000 r-xp 00000000 fd:07
131079 /lib64/libc-2.5.so
35efb4e000-35efd4d000 ---p 0014e000 fd:07
131079 /lib64/libc-2.5.so
35efd4d000-35efd51000 r--p 0014d000 fd:07
131079 /lib64/libc-2.5.so
35efd51000-35efd52000 rw-p 00151000 fd:07
131079 /lib64/libc-2.5.so
35efd52000-35efd57000 rw-p 35efd52000 00:00 0
35efe00000-35efe82000 r-xp 00000000 fd:07
131083 /lib64/libm-2.5.so
35efe82000-35f0081000 ---p 00082000 fd:07
131083 /lib64/libm-2.5.so
35f0081000-35f0082000 r--p 00081000 fd:07
131083 /lib64/libm-2.5.so
35f0082000-35f0083000 rw-p 00082000 fd:07
131083 /lib64/libm-2.5.so
35f0200000-35f020d000 r-xp 00000000 fd:0a
817044 /usr/lib64/libgomp.so.1.0.0
35f020d000-35f040c000 ---p 0000d000 fd:0a
817044 /usr/lib64/libgomp.so.1.0.0
35f040c000-35f040d000 rw-p 0000c000 fd:0a
817044 /usr/lib64/libgomp.so.1.0.0
35f0600000-35f0616000 r-xp 00000000 fd:07
131104 /lib64/libpthread-2.5.so
35f0616000-35f0815000 ---p 00016000 fd:07
131104 /lib64/libpthread-2.5.so
35f0815000-35f0816000 r--p 00015000 fd:07
131104 /lib64/libpthread-2.5.so
35f0816000-35f0817000 rw-p 00016000 fd:07
131104 /lib64/libpthread-2.5.so
35f0817000-35f081b000 rw-p 35f0817000 00:00 0
35f0a00000-35f0a5c000 r-xp 00000000 fd:0a
816812 /usr/lib64/libodbc.so.1.0.0
35f0a5c000-35f0c5c000 ---p 0005c000 fd:0a
816812 /usr/lib64/libodbc.so.1.0.0
35f0c5c000-35f0c64000 rw-p 0005c000 fd:0a
816812 /usr/lib64/libodbc.so.1.0.0
35f1a00000-35f1a35000 r-xp 00000000 fd:0a
812139 /usr/lib64/libreadline.so.5.1
35f1a35000-35f1c34000 ---p 00035000 fd:0a
812139 /usr/lib64/libreadline.so.5.1
35f1c34000-35f1c3c000 rw-p 00034000 fd:0a
812139 /usr/lib64/libreadline.so.5.1
35f1c3c000-35f1c3d000 rw-p 35f1c3c000 00:00 0
35f4200000-35f4207000 r-xp 00000000 fd:07 131111
Process R aborted at Fri Dec 21 14:57:23 2012
More information about the R-help
mailing list