Date: 2015-02-05 08:02:39 +0100
From: sorear
To: MonetDB5 devs <>
Version: 11.19.7 (Oct2014-SP1)
CC: @njnes

Last updated: 2015-05-07 12:37:28 +0200

Comment 20630

Date: 2015-02-05 08:02:39 +0100
From: sorear

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.94 Safari/537.36
Build Identifier:

If I run a long query in one terminal session, and a very short one in another, MonetDB seems to wait for the long query to finish before running the short query. In particular, I can't get useful results from sys.queue() because by the time sys.queue starts, it's the only running query and reports only on itself(?).

Reproducible: Always

Steps to Reproduce:

1. start two mclients in different terminals. In the first:

Welcome to mclient, the MonetDB/SQL interactive terminal (Oct2014-SP1)
Database: MonetDB v11.19.7 (Oct2014-SP1), '...'
Type \q to quit, ? for a list of available commands
auto commit mode: on
sql>create table blah as select * from sys.generate_series(10000,13000) with data;
operation successful (2.098ms)

second:

Welcome to mclient, the MonetDB/SQL interactive terminal (Oct2014-SP1)
Database: MonetDB v11.19.7 (Oct2014-SP1), 'mapi:monetdb://sorear1.dev.gudtech.com:50000/gt'
Type \q to quit, ? for a list of available commands
auto commit mode: on

first, issue a long-running command:

sql>select count(distinct a.value * b.value) from blah a, blah b;

second attempt to see this command:

sql>select * from sys.queue();

several seconds later, this appears in the first:

+---------+
| L1 |
+=========+
| 4072685 |
+---------+
1 tuple (9.0s)

simultaneously in the second:

+------+---------+----------------------------+----------------------------+----------+---------+------------+----------------------------+
| qtag | user | started | estimate | progress | status | tag | query |
+======+=========+============================+============================+==========+=========+============+============================+
| 40 | monetdb | 2015-02-05 06:55:21.000000 | null | null | running | 12783204@0 | select * from sys.queue(); |
+------+---------+----------------------------+----------------------------+----------+---------+------------+----------------------------+
1 tuple (2.5s)

when not busy, queue requests finish instantly:

sql>select * from sys.queue();
+------+---------+----------------------------+----------------------------+----------+---------+------------+----------------------------+
| qtag | user | started | estimate | progress | status | tag | query |
+======+=========+============================+============================+==========+=========+============+============================+
| 41 | monetdb | 2015-02-05 06:55:24.000000 | null | null | running | 12783204@0 | select * from sys.queue(); |
+------+---------+----------------------------+----------------------------+----------+---------+------------+----------------------------+
1 tuple (0.638ms)

Expected Results:

I'm not certain this is truly a bug, but it'd be very very useful if I could observe concurrently running queries with select * from sys.queue() or something functionally equivalent.

Comment 20631

Date: 2015-02-05 08:04:52 +0100
From: sorear

This particular example is on a VM with 2 cores; I suspect that makes a difference here.

Comment 20714

Date: 2015-03-13 14:08:52 +0100
From: @sjoerdmullender

The problem is caused by changeset e79b2edeba71 from January 2014.

That changeset introduced a semaphore to limit concurrency. But this was done in such a way that no new queries enter the fray if the maximum level of concurrency (currently set at half the number of cores) is reached.

Comment 20715

Date: 2015-03-13 14:34:24 +0100
From: @sjoerdmullender

(In reply to comment 2)

The problem is caused by changeset e79b2edeba71 from January 2014.

Make that Jan 2013.

Comment 20716

Date: 2015-03-13 18:14:32 +0100
From: sorear

Interesting. We've been seeing behavior on a couple of 16 core production machines where the system is under sustained MonetDB load (mserver5 consistently > 500% CPU), but when I run "select * from sys.queue" it waits several seconds and then reports that the select is the only thing running. It sounds like the example I included may not have been an accurate reproduction then?

Comment 20717

Date: 2015-03-16 14:51:31 +0100
From: MonetDB Mercurial Repository <>

Changeset 68156d38a8c4 made by Sjoerd Mullender sjoerd@acm.org in the MonetDB repo, refers to this bug.

For complete details, see http//devmonetdborg/hg/MonetDB?cmd=changeset;node=68156d38a8c4

Changeset description:

Don't limit incoming queries based on number of cores.
This fixes bug #3665.

Comment 20794

Date: 2015-04-14 12:35:56 +0200
From: @sjoerdmullender

The limiting semaphore has been removed. The problem should not occur anymore.