Date: 2017-07-17 15:35:14 +0200
From: @kutsurak
To: MonetDB5 devs <>
Version: 11.25.23 (Dec2016-SP5)

Last updated: 2017-08-02 10:21:20 +0200

Comment 25493

Date: 2017-07-17 15:35:14 +0200
From: @kutsurak

User-Agent: Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:54.0) Gecko/20100101 Firefox/54.0
Build Identifier:

When the MAL profiler creates the JSON objects, if the textual representation is larger than 8192 bytes, the output gets truncated. This creates incomplete JSON objects that create problems for profiler consumers.

Reproducible: Always

Steps to Reproduce:

1. mserver5 --dbpath=/tmp/foo
2. stethoscope -d foo -j
3. python -c "print("select '" + 'a'*8192 + "';")" | mclient -d foo

Actual Results:

The JSON object with PC=1 is incomplete.

Expected Results:

The output should be a stream of complete JSON objects.

Comment 25494

Date: 2017-07-21 18:25:52 +0200
From: MonetDB Mercurial Repository <>

Changeset 7ba8ed6d4865 made by Panagiotis Koutsourakis kutsurak@monetdbsolutions.com in the MonetDB repo, refers to this bug.

For complete details, see https//devmonetdborg/hg/MonetDB?cmd=changeset;node=7ba8ed6d4865

Changeset description:

Add support for long SQL texts in the profiler

This is related to Bug #6375, but is still incomplete.

The problem is that the profiler truncates JSON objects to 8KB. There are two
solutions to this problem:

  1. Since the location of the very big parts of the MAL plan are
  known (statement text, short statement text, and value in argument list) we
  can truncate only these parts, and thus produce a complete JSON object. The
  problem with this approach is that we do not know in advance how much we need
  to truncate, because other elements are to follow and everything needs to fit
  in 8KB.

  2. Transmit more than 8KB chunks. The obvious downside to this is that we will
  need more IO and since the query itself can in principle be unbounded, this
  can be very expensive.

The solution is the combination of the above. We truncate the big parts and so
effectively we make the amount of IO bounded but also recognizing that they
contain useful information, and try to minimize the information loss.

Still TODO:
The concatenation and transmission of big objects is buggy. The problem can be
reproduced with the command:

python -c "print(\"select * from _tables where name='\" + 'a'*8192 + \"';\")" | mclient -d

Comment 25498

Date: 2017-07-24 15:00:07 +0200
From: @kutsurak

The last missing piece is fixing the function shortStmtRendering.

For some reason it cuts off the last few chars of really big arguments, and this causes some quote characters to be dropped leading to malformed JSON objects in the end.

Comment 25499

Date: 2017-07-26 12:47:07 +0200
From: MonetDB Mercurial Repository <>

Changeset 3584cfa2625e made by Panagiotis Koutsourakis kutsurak@monetdbsolutions.com in the MonetDB repo, refers to this bug.

For complete details, see https//devmonetdborg/hg/MonetDB?cmd=changeset;node=3584cfa2625e

Changeset description:

Try to reallocate buffers when terms are too long

Part of the fix for bug #6375

Comment 25500

Date: 2017-07-26 12:47:14 +0200
From: MonetDB Mercurial Repository <>

Changeset 55a75e48e3fc made by Panagiotis Koutsourakis kutsurak@monetdbsolutions.com in the MonetDB repo, refers to this bug.

For complete details, see https//devmonetdborg/hg/MonetDB?cmd=changeset;node=55a75e48e3fc

Changeset description:

Error handling in case we cannot allocate strings

Part of the fix for bug #6375

Comment 25501

Date: 2017-07-26 12:47:22 +0200
From: MonetDB Mercurial Repository <>

Changeset 9c5b374c3886 made by Panagiotis Koutsourakis kutsurak@monetdbsolutions.com in the MonetDB repo, refers to this bug.

For complete details, see https//devmonetdborg/hg/MonetDB?cmd=changeset;node=9c5b374c3886

Changeset description:

Make sure stethoscope outputs each received character only once

This completes the fix for bug #6375

Comment 25533

Date: 2017-08-02 10:21:20 +0200
From: @sjoerdmullender

Fixed in Jul2017-SP1 release.