Date: 2016-08-01 14:20:17 +0200
From: anthonin.bonnefoy
To: SQL devs <>
Version: 11.23.7 (Jun2016-SP1)
CC: frederic.jolliton+monetdb, @njnes

Last updated: 2016-12-21 13:07:38 +0100

Comment 22272

Date: 2016-08-01 14:20:17 +0200
From: anthonin.bonnefoy

User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0
Build Identifier:

On parallel drop and update of tables, a segfault can occur with the given traceback

0 0x00007ffa546934b8 in count_col (tr=0x1ee7330, c=0xf7db730, all=1) at bat_storage.c:826
1 0x00007ffa5458e59b in SQLgetStatistics (cntxt=0x20cc320, m=0xf7eac10, mb=0x3989c10) at sql_optimizer.c:168
2 0x00007ffa5458eadd in addOptimizers (c=0x20cc320, mb=0x3989c10, pipe=0xecfd930 "default_pipe") at sql_optimizer.c:251
3 0x00007ffa5458ec98 in addQueryToCache (c=0x20cc320) at sql_optimizer.c:293
4 0x00007ffa5458cd1f in backend_dumpproc (be=0x8a69190, c=0x20cc320, cq=0x4434390, s=0xbf35940) at sql_gencode.c:2815
5 0x00007ffa5455d31a in SQLparser (c=0x20cc320) at sql_scenario.c:1216
6 0x00007ffa5b9e33d6 in runPhase (c=0x20cc320, phase=1) at mal_scenario.c:515
7 0x00007ffa5b9e3521 in runScenarioBody (c=0x20cc320) at mal_scenario.c:550
8 0x00007ffa5b9e36c4 in runScenario (c=0x20cc320) at mal_scenario.c:579
9 0x00007ffa5b9e479a in MSserveClient (dummy=0x20cc320) at mal_session.c:439
10 0x00007ffa5b9e4309 in MSscheduleClient (command=0xf7eac10 "", challenge=0x7ffa52ee1e30 "LBlKWgIa", fin=0x43f3080, fout=0x1ee7a70) at mal_session.c:319
11 0x00007ffa5baa1f72 in doChallenge (data=0x43f3090) at mal_mapi.c:184
12 0x00007ffa5b459ea5 in thread_starter (arg=0xcfa3f00) at gdk_system.c:459
13 0x00007ffa59f480a4 in start_thread (arg=0x7ffa52ee2700) at pthread_create.c:309
14 0x00007ffa59c7d87d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Reproducible: Always

Steps to Reproduce:

Start a database (preferably on a tmpfs)

Launch the given script:

!/bin/bash
set -e

mclient nova -s "create schema sact;" 2> /dev/null || true

mserver_pid=$(pidof mserver5)

line="9GJ3152\t1467287703373954\n1467287703759937\t3\t3\t10\t\t62438190489824\t116350668306\t3\t0\t3232238295\t\t3223098188\t\t\t\t55460\t443\t0\t0\t0\t0\t52\t52\t\t46\tEthernet/IPv4/TCP\t21ae5e54-637e-405f-99f5-41b93d9d769a\t258\t132\t4\t2\t0\t0\t1\t1\t0\t0\t0\t0\t0\t1\t1\t0\t0\t0\t1\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\t1\t274\t75076\t1\t150529\t22658979841\t0\t0\t0\t0\t0\t0\t0\t0\t0\t1\t605119\t366169004161\t\n"
rm -f /tmp/data
for (( i = 0; i < 10; i++ )); do
echo -e -n $line >> /tmp/data
done

function drop_tables() {
echo "Launch drop of tables"
tables=$(mclient nova -f csv -s "select name from _tables where schema_id > 5000 ;")
drop_query=""
local i=0
for table in $tables ; do
drop_query="$drop_query drop table sact.$table;"
i=$((i + 1))
if [[ $i == 25 ]]; then
break
fi
done
mclient nova -a -s "$drop_query; commit; rollback;" > /dev/null
}

function update_tables() {
echo "Launch update"
tables=$(mclient nova -f csv -s "select name from _tables where schema_id > 5000 order by id desc;")
local i=0
for table in $tables ; do
mclient nova -s "update sact.$table set toto1 = 'C8CCF6D5-A7B7-449E-911E-5D9082D73640' where toto1 = '9GJ3152';" > /dev/null
if [[ $i == 10 ]]; then
break
fi
i=$((i + 1))
done
echo "Update finished"
}

drop_tables

for (( i = 0; i < 10000; i++ )); do
table="sact.test_$i"

 create_query="$create_query CREATE TABLE $table (toto1 TEXT, toto2 BIGINT, toto3 BIGINT, toto4 SMALLINT, toto5 SMALLINT, toto6 INT, toto7 INT, toto8 BIGINT, toto9 BIGINT, toto10 INT, toto11 INT, toto12 BIGINT, toto13 HUGEINT, toto14 BIGINT, toto15 HUGEINT, toto16 BIGINT, toto17 HUGEINT, toto18 INT, toto19 INT, toto20 SMALLINT, toto21 SMALLINT, toto22 SMALLINT, toto23 SMALLINT, toto24 INT, toto25 INT, toto26 TEXT, toto27 INT, toto28 TEXT, toto29 UUID, toto30 BIGINT, toto31 BIGINT, toto32 BIGINT, toto33 BIGINT, toto34 BIGINT, toto35 BIGINT, toto36 BIGINT, toto37 BIGINT, toto38 BIGINT, toto39 BIGINT, toto40 BIGINT, toto41 BIGINT, toto42 BIGINT, toto43 BIGINT, toto44 BIGINT, toto45 BIGINT, toto46 BIGINT, toto47 BIGINT, toto48 BIGINT, toto49 BIGINT, toto50 BIGINT, toto51 INT, toto52 INT, toto53 BIGINT, toto54 BIGINT, toto55 HUGEINT, toto56 BIGINT, toto57 BIGINT, toto58 HUGEINT, toto59 BIGINT, toto60 BIGINT, toto61 HUGEINT, toto62 BIGINT, toto63 BIGINT, toto64 HUGEINT, toto65 BIGINT, toto66 BIGINT, toto67 HUGEINT, toto68 BIGINT, toto69 BIGINT, toto70 HUGEINT, toto71 BIGINT, toto72 BIGINT, toto73 HUGEINT, toto74 BIGINT, toto75 BIGINT, toto76 HUGEINT, toto77 UUID);"
 create_query="$create_query COPY INTO $table FROM '/home/sora/notes/monet/schema_absent/reproduce/lite_tcp_v29.csv' DELIMITERS '\t','\n','\"' NULL AS '<NULL>';"

 if [[ $mserver_pid != $(pidof mserver5) ]]; then
     echo "Got a crash"
     exit 1
 fi

 if [[ $((i % 4)) == 0 ]]; then
     echo "Launching create at $i"
     mclient nova -a -s "$create_query; commit; rollback;" > /dev/null
     create_query=""
 fi

 if [[ $((i % 10)) == 0 ]]; then
     update_tables &
 fi

 if [[ $((i % 20)) == 0 ]]; then
     drop_tables
 fi

done

Actual Results:

I usually get the segfault near test_600 table creation.

Comment 22273

Date: 2016-08-01 14:22:51 +0200
From: anthonin.bonnefoy

Can't edit comment, the 'COPY INTO' in the script should be:

create_query="$create_query COPY INTO $table FROM '/tmp/data' DELIMITERS '\t','\n','"' NULL AS '';"

Comment 22274

Date: 2016-08-01 14:40:10 +0200
From: anthonin.bonnefoy

And

line="9GJ3152\t1467287703373954\n1467287703759937...

should be

line="9GJ3152\t1467287703373954\t1467287703759937...

Comment 22312

Date: 2016-08-19 17:01:15 +0200
From: @njnes

is this still the case on Jun2016?

Comment 22332

Date: 2016-08-29 15:14:30 +0200
From: Frédéric Jolliton <<frederic.jolliton+monetdb>>

Running the provided script, with the two changes indicated by Anthonin, we see that the Jun2016 version also crashes with a SIGSEGV at some point, at the exact same place (in count_col).

The version tested:

MonetDB 5 server v11.23.8 (64-bit, 64-bit oids, 128-bit integers)
This is an unreleased version
Copyright (c) 1993-July 2008 CWI
Copyright (c) August 2008-2016 MonetDB B.V., all rights reserved
Visit http://www.monetdb.org/ for further information
Found 15.6GiB available memory, 4 available cpu cores
Libraries:
libpcre: 8.39 2016-06-14 (compiled with 8.39)
openssl: OpenSSL 1.0.2h 3 May 2016 (compiled with OpenSSL 1.0.2h 3 May 2016)
libxml2: 2.9.4 (compiled with 2.9.4)
Compiled by: fjolliton@localhost (x86_64-pc-linux-gnu)
Compilation: gcc -O3 -fomit-frame-pointer -pipe -Werror -Wall -Wextra -W -Werror-implicit-function-declaration -Wpointer-arith -Wdeclaration-after-statement -Wundef -Wformat=2 -Wno-format-nonliteral -Winit-self -Winvalid-pch -Wmissing-declarations -Wmissing-format-attribute -Wmissing-prototypes -Wold-style-definition -Wpacked -Wunknown-pragmas -Wvariadic-macros -fstack-protector-all -Wstack-protector -Wpacked-bitfield-compat -Wsync-nand -Wjump-misses-init -Wmissing-include-dirs -Wlogical-op -Wunreachable-code -D_FORTIFY_SOURCE=2
Linking : /usr/bin/ld -m elf_x86_64

Comment 22333

Date: 2016-08-29 15:15:20 +0200
From: Frédéric Jolliton <<frederic.jolliton+monetdb>>

Changing version.

Comment 22359

Date: 2016-09-08 08:12:00 +0200
From: @njnes

on both the jun branch and jun sp1 I had a successfull run of 10K creates. They both use about 12G of memory. The bigger difference is in the used virtual memory,
the branch is down to 15G, while sp1 take 45G. This was run on 2 16G machines.

Comment 22363

Date: 2016-09-12 20:06:17 +0200
From: @njnes

also with the mount -t tmfs I'm unable to reproduce the problem. We run on a plain fedora (24) system.

Comment 22392

Date: 2016-09-29 12:57:40 +0200
From: MonetDB Mercurial Repository <>

Changeset 975111071fcb made by Niels Nes niels@cwi.nl in the MonetDB repo, refers to this bug.

For complete details, see http//devmonetdborg/hg/MonetDB?cmd=changeset;node=975111071fcb

Changeset description:

added test for bug #4047

Comment 24764

Date: 2016-12-08 10:48:40 +0100
From: @sjoerdmullender

This was apparently fixed.