Date: 2012-12-17 22:53:48 +0100
From: Percy Wegmann <>
To: SQL devs <>
Version: 11.13.5 (Oct2012-SP1)
CC: @njnes

Last updated: 2014-02-20 15:03:01 +0100

Comment 18258

Date: 2012-12-17 22:53:48 +0100
From: Percy Wegmann <>

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.97 Safari/537.11
Build Identifier:

When loading two tables concurrently and at the same time running a COUNT(*) query against one of those tables, one occasionally gets a COMMIT: failed message.

The strange thing is that this requires this exact scenario. Concurrent loads into two tables work fine. Concurrent loads while running another aggregate query (like AVG(*)) work fine. Concurrent loads while running a non-aggregate query work fine.

I'm able to reproduce this using the JDBC client with multiple threads. Note - I'm only able to reproduce the problem when using prepared statements with batching. Non-batched inserts don't seem to suffer from this problem.

I haven't had a chance to dig into it yet, but is it possible that COUNT() may be using some cached counts from a system table somewhere, and that both the act of querying for COUNT() and INSERTING may side-effect that table and create a concurrency problem?

Reproducible: Always

Steps to Reproduce:

I'll attach a JUnit test for this.

Actual Results:

INSERT fails with COMMIT: failed

Expected Results:

All INSERTs succeed.

This is problematic for my app as we are continuously loading multiple tables at the same time as users are interactively querying the database, including asking for COUNT(*).

Comment 18259

Date: 2012-12-17 23:01:44 +0100
From: Percy Wegmann <>

Created attachment 165
JUnit test case for reproducing problem

One or the other of these tests fails fairly reliably:

testConcurrentLoadsWithCountQuery
testConcurrentLoadsWithCountQueryAgain

None of the other tests ever fail.

Note - this test expects a database on server percy-sandbox named ctest with user/pass monetdb/monetdb. You can edit these values on line 368.

Attached file: MultiThreadedMonetTest.java (text/java, 12611 bytes)
Description: JUnit test case for reproducing problem

Comment 18262

Date: 2012-12-18 21:00:41 +0100
From: Percy Wegmann <>

So, we're getting a commit failure at line 2810 of store.c (inside the validate_tables() function).

/* commited write before t's read */
if (c->base.rtime && c->base.rtime < oc->base.wtime) {

Comment 18264

Date: 2012-12-18 21:25:54 +0100
From: @njnes

is the count statement failing with this commit message?

Comment 18265

Date: 2012-12-18 21:41:55 +0100
From: Percy Wegmann <>

Created attachment 166
Test that shows using setAutoCommit(false) and rolling back the transaction works

Attached file: MultiThreadedMonetTest.java (text/java, 6673 bytes)
Description: Test that shows using setAutoCommit(false) and rolling back the transaction works

Comment 18266

Date: 2012-12-18 21:44:48 +0100
From: Percy Wegmann <>

Niels - it's actually the INSERT statement that's failing on that line. I added some logging and found that the check is failing on column "id" on "table_a". The question is, why would a SELECT be updating the wtime on a column?

Comment 18315

Date: 2012-12-29 19:14:26 +0100
From: Percy Wegmann <>

I've done a little more digging, and it appears that simply preparing a SELECT query (not even executing it) causes this problem.

Comment 18337

Date: 2013-01-15 18:47:23 +0100
From: @njnes

did you try SP3?

Comment 18357

Date: 2013-01-21 22:18:11 +0100
From: Percy Wegmann <>

I just tried it with SP3 (11.13.9) on Ubuntu 12 64-bit, and I'm still getting a commit failure.

(In reply to comment 7)

did you try SP3?

Comment 18470

Date: 2013-02-13 13:44:36 +0100
From: @njnes

this has been fixed in both the Oct2012 and Feb2013 branches. The Feb2013 is about to be released, ie closing with nextfeaturerelease

Comment 19618

Date: 2014-02-20 15:03:01 +0100
From: @sjoerdmullender

Jan2014 has been released.