Date: 2013-02-25 11:56:49 +0100
From: @swingbit
To: MonetDB5 devs <>
Version: 11.15.1 (Feb2013)
CC: @mlkersten, @drstmane

Last updated: 2013-03-07 12:41:20 +0100

Comment 18553

Date: 2013-02-25 11:56:49 +0100
From: @swingbit

User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 Safari/537.17
Build Identifier:

The sql file in attachment, ran on an empty database, crashes with a SEGFAULT.

The debugger suggests the crash happens in optimizer.reduce().
Indeed, the same SQL dos not crash when simply removing optimizer.reduce() from the default_pipe.

gdb output:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fe983d4c700 (LWP 12246)]
0x00007fe98d2128ca in runMALsequence (cntxt=0x173c768, mb=0x7fe974075b50, startpc=1, stoppc=49, stk=0x7fe9741fe540, env=0x7fe97423f0e0, pcicaller=0x7fe974232d90)
at /opt/spinque/MonetDBServer/MonetDB.Spinque_Feb2013/src/monetdb5/mal/mal_interpreter.c:801
801 if (isaBatType(getArgType(mb, pci, i))) {
(gdb) bt
0 0x00007fe98d2128ca in runMALsequence (cntxt=0x173c768, mb=0x7fe974075b50, startpc=1, stoppc=49, stk=0x7fe9741fe540, env=0x7fe97423f0e0, pcicaller=0x7fe974232d90)
at /opt/spinque/MonetDBServer/MonetDB.Spinque_Feb2013/src/monetdb5/mal/mal_interpreter.c:801
1 0x00007fe98d2122b3 in runMALsequence (cntxt=0x173c768, mb=0x7fe974075fb0, startpc=1, stoppc=0, stk=0x7fe97423f0e0, env=0x0, pcicaller=0x0) at /opt/spinque/MonetDBServer/MonetDB.Spinque_Feb2013/src/monetdb5/mal/mal_interpreter.c:720
2 0x00007fe98d211335 in callMAL (cntxt=0x173c768, mb=0x7fe974075fb0, env=0x7fe983d4bba0, argv=0x7fe983d4bc20, debug=0 '\000') at /opt/spinque/MonetDBServer/MonetDB.Spinque_Feb2013/src/monetdb5/mal/mal_interpreter.c:469
3 0x00007fe984fc5f56 in SQLexecutePrepared (c=0x173c768, be=0x7fe974074910, q=0x7fe97408a370) at /opt/spinque/MonetDBServer/MonetDB.Spinque_Feb2013/src/sql/backends/monet5/sql_scenario.c:1840
4 0x00007fe984fc6345 in SQLengineIntern (c=0x173c768, be=0x7fe974074910) at /opt/spinque/MonetDBServer/MonetDB.Spinque_Feb2013/src/sql/backends/monet5/sql_scenario.c:1907
5 0x00007fe984fc68ba in SQLengine (c=0x173c768) at /opt/spinque/MonetDBServer/MonetDB.Spinque_Feb2013/src/sql/backends/monet5/sql_scenario.c:2008
6 0x00007fe98d23ea95 in runPhase (c=0x173c768, phase=4) at /opt/spinque/MonetDBServer/MonetDB.Spinque_Feb2013/src/monetdb5/mal/mal_scenario.c:522
7 0x00007fe98d23ec82 in runScenarioBody (c=0x173c768) at /opt/spinque/MonetDBServer/MonetDB.Spinque_Feb2013/src/monetdb5/mal/mal_scenario.c:567
8 0x00007fe98d23edb4 in runScenario (c=0x173c768) at /opt/spinque/MonetDBServer/MonetDB.Spinque_Feb2013/src/monetdb5/mal/mal_scenario.c:586
9 0x00007fe98d23fe4a in MSserveClient (dummy=0x173c768) at /opt/spinque/MonetDBServer/MonetDB.Spinque_Feb2013/src/monetdb5/mal/mal_session.c:431
10 0x0000003599007761 in start_thread () from /lib64/libpthread.so.0
11 0x0000003598ce098d in clone () from /lib64/libc.so.6
(gdb) p *pci
$3 = {token = 54 '6', barrier = 0 '\000', typechk = 2 '\002', gc = 3 '\003', polymorphic = 0 '\000', varargs = 0 '\000', recycle = 0, jump = 0, fcn = 0x7fe98d49391c , blk = 0x290b6e0, modname = 0x1d500a0 "optimizer",
fcnname = 0x290b540 "reduce", argc = 1, retc = 1, maxarg = 4, argv = 0x7fe9741f4c40}

Reproducible: Always

Steps to Reproduce:

1.run the SQL in attachment on an empty database
2.
3.

Actual Results:

SEGFAULT

MonetDB 5 server v11.15.2 (64-bit, 64-bit oids)
This is an unreleased version
Copyright (c) 1993-July 2008 CWI
Copyright (c) August 2008-2013 MonetDB B.V., all rights reserved
Visit http://www.monetdb.org/ for further information
Found 35.5GiB available memory, 8 available cpu cores
Libraries:
libpcre: 7.8 2008-09-05 (compiled with 7.8)
openssl: OpenSSL 1.0.0d 8 Feb 2011 (compiled with OpenSSL 1.0.0d-fips 8 Feb 2011)
libxml2: 2.7.7 (compiled with 2.7.7)
Compiled by: roberto@spinque01.ins.cwi.nl (x86_64-unknown-linux-gnu)
Compilation: gcc -g -Werror -Wall -Wextra -W -Werror-implicit-function-declaration -Wpointer-arith -Wdeclaration-after-statement -Wformat=2 -Wno-format-nonliteral -Winit-self -Winvalid-pch -Wmissing-declarations -Wmissing-format-attribute -Wmissing-prototypes -Wold-style-definition -Wpacked -Wunknown-pragmas -Wvariadic-macros -fstack-protector-all -Wstack-protector -Wpacked-bitfield-compat -Wsync-nand -Wmissing-include-dirs
Linking : /usr/bin/ld -m elf_x86_64

Comment 18554

Date: 2013-02-25 12:03:11 +0100
From: @swingbit

The file turns out to be too big to be stored as an attachment.
I made it available (for limited time) here:

~roberto/tmp/bug-3241.sql

Comment 18558

Date: 2013-02-25 12:42:02 +0100
From: @drstmane

This seems to be a MAL interpreter / optimizer problem.

Here's a first gdb trace:

[...]
[ 2441, "start", "12:27:48.745072", 3, 0, "optimizer.reduce();", ]
[New Thread 0x7fffee7ff700 (LWP 10890)]
[New Thread 0x7fffee5fe700 (LWP 10891)]
[New Thread 0x7fffee3fd700 (LWP 10928)]
[New Thread 0x7fffee1fc700 (LWP 10929)]
[New Thread 0x7fffedffb700 (LWP 10930)]
[New Thread 0x7fffeddfa700 (LWP 10931)]
[New Thread 0x7fffedbf9700 (LWP 10932)]
[New Thread 0x7fffed9f8700 (LWP 10933)]
[New Thread 0x7fffed7f7700 (LWP 10934)]
[New Thread 0x7fffed5f6700 (LWP 10935)]
[New Thread 0x7fffed3f5700 (LWP 10936)]
[New Thread 0x7fffec9f4700 (LWP 10937)]
[New Thread 0x7fffec7f3700 (LWP 10938)]
[New Thread 0x7fffec5f2700 (LWP 10939)]
[Thread 0x7fffec5f2700 (LWP 10939) exited]
[Thread 0x7fffec7f3700 (LWP 10938) exited]
[Thread 0x7fffec9f4700 (LWP 10937) exited]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffee3fd700 (LWP 10928)]
0x00007ffff7a4cec2 in instruction2str (mb=0x7fffe00c7c60, stk=0x7fffe023db00, p=0x7fffe00c6440, flg=330) at .../Feb2013/source/MonetDB/monetdb5/mal/mal_listing.c:316
316 if (p->argc > 0 && isTmpVar(mb, getArg(p, 0))) {
Missing separate debuginfos, use: debuginfo-install bzip2-libs-1.0.6-3.fc15.x86_64 cfitsio-3.280-2.fc16.x86_64 cyrus-sasl-lib-2.1.23-27.fc16.x86_64 geos-3.3.1-1.fc16.x86_64 glibc-2.14.90-24.fc16.9.x86_64 keyutils-libs-1.5.2-1.fc16.x86_64 krb5-libs-1.9.4-3.fc16.x86_64 libcom_err-1.41.14-2.fc15.x86_64 libcurl-7.21.7-8.fc16.x86_64 libgcc-4.6.3-2.fc16.x86_64 libidn-1.22-3.fc16.x86_64 libselinux-2.1.6-6.fc16.x86_64 libssh2-1.2.7-4.fc16.x86_64 libstdc++-4.6.3-2.fc16.x86_64 libuuid-2.20.1-2.3.fc16.x86_64 libxml2-2.7.8-8.fc16.x86_64 ncurses-libs-5.9-2.20110716.fc16.x86_64 nspr-4.9.4-1.fc16.x86_64 nss-3.14.1-3.fc16.x86_64 nss-softokn-freebl-3.14.1-3.fc16.x86_64 nss-util-3.14.1-1.fc16.x86_64 openldap-2.4.26-8.fc16.x86_64 openssl-1.0.0j-1.fc16.x86_64 pcre-8.12-9.fc16.x86_64 readline-6.2-2.fc16.x86_64 zlib-1.2.5-7.fc16.x86_64
(gdb) bt
0 0x00007ffff7a4cec2 in instruction2str (mb=0x7fffe00c7c60, stk=0x7fffe023db00, p=0x7fffe00c6440, flg=330) at .../Feb2013/source/MonetDB/monetdb5/mal/mal_listing.c:316
1 0x00007ffff7a5dea7 in offlineProfilerEvent (idx=1, mb=0x7fffe00c7c60, stk=0x7fffe023db00, pc=48, start=0) at .../Feb2013/source/MonetDB/monetdb5/mal/mal_profiler.c:450
2 0x00007ffff7a5bd41 in profilerEvent (idx=1, mb=0x7fffe00c7c60, stk=0x7fffe023db00, pc=48, start=0) at .../Feb2013/source/MonetDB/monetdb5/mal/mal_profiler.c:218
3 0x00007ffff7a3b068 in runtimeProfileExit (cntxt=0x62d088, mb=0x7fffe00c7c60, stk=0x7fffe023db00, prof=0x7fffee3fc360) at .../Feb2013/source/MonetDB/monetdb5/mal/mal_runtime.c:101
4 0x00007ffff7a45fb9 in runMALsequence (cntxt=0x62d088, mb=0x7fffe00c7c60, startpc=1, stoppc=49, stk=0x7fffe023db00, env=0x7fffe00eac80, pcicaller=0x7fffe00c8be0) at .../Feb2013/source/MonetDB/monetdb5/mal/mal_interpreter.c:761
5 0x00007ffff7a45d29 in runMALsequence (cntxt=0x62d088, mb=0x7fffe014ef40, startpc=1, stoppc=0, stk=0x7fffe00eac80, env=0x0, pcicaller=0x0) at .../Feb2013/source/MonetDB/monetdb5/mal/mal_interpreter.c:720
6 0x00007ffff7a44dd2 in callMAL (cntxt=0x62d088, mb=0x7fffe014ef40, env=0x7fffee3fcb78, argv=0x7fffee3fcc20, debug=0 '\000') at .../Feb2013/source/MonetDB/monetdb5/mal/mal_interpreter.c:469
7 0x00007fffef411eb3 in SQLexecutePrepared (c=0x62d088, be=0x7fffe002b470, q=0x7fffe00beea0) at .../Feb2013/source/MonetDB/sql/backends/monet5/sql_scenario.c:1840
8 0x00007fffef41229a in SQLengineIntern (c=0x62d088, be=0x7fffe002b470) at .../Feb2013/source/MonetDB/sql/backends/monet5/sql_scenario.c:1907
9 0x00007fffef4127fb in SQLengine (c=0x62d088) at .../Feb2013/source/MonetDB/sql/backends/monet5/sql_scenario.c:2008
10 0x00007ffff7a7334b in runPhase (c=0x62d088, phase=4) at .../Feb2013/source/MonetDB/monetdb5/mal/mal_scenario.c:522
11 0x00007ffff7a73534 in runScenarioBody (c=0x62d088) at .../Feb2013/source/MonetDB/monetdb5/mal/mal_scenario.c:566
12 0x00007ffff7a73657 in runScenario (c=0x62d088) at .../Feb2013/source/MonetDB/monetdb5/mal/mal_scenario.c:586
13 0x00007ffff7a746d0 in MSserveClient (dummy=0x62d088) at .../Feb2013/source/MonetDB/monetdb5/mal/mal_session.c:431
14 0x0000003cf3c07d90 in start_thread () from /lib64/libpthread.so.0
15 0x0000003cf30f119d in clone () from /lib64/libc.so.6
(gdb) li
311 }
312 if (flg & LIST_MAL_LNR){
313 snprintf(t,len-1,"%3d ",getPC(mb,p));
314 advance(t,base,len);
315 }
316 if (p->argc > 0 && isTmpVar(mb, getArg(p, 0))) {
317 if (isVarUsed(mb, getDestVar(p))) {
318 snprintf(nmebuf, PATHLENGTH, "%c%d", TMPMARKER, getVarTmp(mb, getArg(p, 0)));
319 } else
320 nmebuf[0] = 0;
(gdb) p p
$1 = (InstrPtr) 0x7fffe00c6440
(gdb) p *p
$2 = {token = 54 '6', barrier = 0 '\000', typechk = 2 '\002', gc = 3 '\003', polymorphic = 0 '\000', varargs = 0 '\000', recycle = 0, jump = 0, fcn = 0x7ffff7ccefa0 , blk = 0x17fc5c0, modname = 0xc3f040 "optimizer", fcnname = 0x17fc420 "reduce", argc = 1, retc = 1, maxarg = 4,
(gdb) p p->argc
$3 = 1
(gdb) p mb
$4 = (MalBlkPtr) 0x7fffe00c7c60
(gdb) p *mb
$5 = {binding = 0x0, help = 0x0, alternative = 0x0, vtop = 51, vsize = 64, var = 0x7fffe00e57a0, stop = 48, ssize = 81, stmt = 0x7fffe00eb600, ptop = 2, psize = 32, prps = 0x7fffe00d6100, errors = 0, typefixed = 0, flowfixed = 1, profiler = 0x7fffe0245bd0, history = 0x0, keephistory = 0,
dotfile = 0, marker = 0x0, maxarg = 8, replica = 0x0, recycle = 0, recid = 0, legid = -4774451407313060419, trap = 0, starttime = 34743987}
(gdb) p getArg(p, 0)
No symbol "getArg" in current context.
(gdb) p p->argv[0]
$6 = 51
(gdb) p isTmpVar(mb, 51)
No symbol "isTmpVar" in current context.
(gdb) p mb->var[51]->tmpindex
Cannot access memory at address 0x10
(gdb) p mb->var[51]
$7 = (VarRecord *) 0x0
(gdb) quit

Comment 18559

Date: 2013-02-25 12:54:22 +0100
From: @sjoerdmullender

The crash happens after a call to optimizer.reduce(). The interpreter looks at the return value (which should be str), but it looks like the code doesn't fill in a return value.

Martin, what's the deal here?

Comment 18560

Date: 2013-02-25 13:06:35 +0100
From: @mlkersten

The optimizer.reduce() is called by the optimizer wrapper, not directly from MAL unless it is part of a MAL test case. The wrapper returns the (exception) result from the program check.

Comment 18561

Date: 2013-02-25 14:02:00 +0100
From: @mlkersten

Splitting the script into three phases shows that
you need the copy.sql part to trigger the error.
If you reduce the copy.sql to a single record,
it works without problems.

monetdb destroy bug -f; monetdb create bug; monetdb release bug; mclient -d bug create.sql ; mclient -d bug copy.sql ;mclient -d bug fcn.sql

Comment 18562

Date: 2013-02-25 15:03:22 +0100
From: @sjoerdmullender

You only need 6 elements in the test table. 5 is not enough.

An execution trace is interesting. After each return gettype, the next optimizer.something() instruction is executed, until reduce hits and causes a crash.

Run mserver5 and execute
profiler.openStream("console");
profiler.setAll();
profiler.activate("event","time","thread","ticks","stmt","start");
profiler.start();

on the console before running the mclient.

Comment 18564

Date: 2013-02-25 15:32:19 +0100
From: @sjoerdmullender

Changeset 335efc7b71a1 made by Sjoerd Mullender sjoerd@acm.org in the MonetDB repo, refers to this bug.

For complete details, see http//devmonetdborg/hg/MonetDB?cmd=changeset;node=335efc7b71a1

Changeset description:

Added test for bug #3241.

Comment 18568

Date: 2013-02-26 11:16:57 +0100
From: @sjoerdmullender

Changeset be7612616e15 made by Sjoerd Mullender sjoerd@acm.org in the MonetDB repo, refers to this bug.

For complete details, see http//devmonetdborg/hg/MonetDB?cmd=changeset;node=be7612616e15

Changeset description:

Off by one error.  This fixes bug #3241.
Also approve test output that now works.

Comment 18569

Date: 2013-02-26 14:16:55 +0100
From: @sjoerdmullender

If this wasn't fixed, please reopen.

Comment 18587

Date: 2013-03-07 12:41:20 +0100
From: @sjoerdmullender

Feb2013-SP1 has been released.