Date: 2017-12-14 20:33:45 +0100
From: Anton Kravchenko <<kravchenko.anton86>>
To: SQL devs <>
Version: 11.27.11 (Jul2017-SP3)

Last updated: 2018-02-12 16:12:09 +0100

Comment 26008

Date: 2017-12-14 20:33:45 +0100
From: Anton Kravchenko <<kravchenko.anton86>>

User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36
Build Identifier:

'select count(v_blob) from blob_table' produces a virtual memory spike ~800Gb
[on the second run within same session]

Reproducible: Always

Steps to Reproduce:

Steps to Reproduce:

1. to generate data by using Python
   hex_blob='47'*2000
   hex_blob_chunk=(hex_blob+'\n')*1000

f=open('blob_data.txt','w')
for ichunk in range(10000):
f.write(hex_blob_chunk)
f.close()

2. to load data into Monet blob_table
   create table blob_table(v_blob blob);
   COPY 10000000 RECORDS INTO blob_table FROM '/home/blob_data.txt' using
   delimiters ',','\n' NULL AS '' LOCKED;

select count(v_blob) from blob_table; 1st run: OK
select count(v_blob) from blob_table; 2nd run: very slow [~30s] and memory spike of ~800 GB

Actual Results:

virtual memory spike of ~800GB

Expected Results:

virtual memory of ~25GB [MonetDB storage size of the blob column]

MonetDB 5 server v11.27.11 "Jul2017-SP3" (64-bit, 128-bit integers)
Copyright (c) 1993-July 2008 CWI
Copyright (c) August 2008-2017 MonetDB B.V., all rights reserved
Visit https://www.monetdb.org/ for further information
Found 503.6GiB available memory, 32 available cpu cores
Libraries:
libpcre: 8.32 2012-11-30 (compiled with 8.32)
openssl: OpenSSL 1.0.2k 26 Jan 2017 (compiled with OpenSSL 1.0.2l 25 May 2017)
libxml2: 2.9.1 (compiled with 2.9.1)
Compiled by: akravchenko@cent7-1 (x86_64-pc-linux-gnu)
Compilation: gcc -O3 -fomit-frame-pointer -pipe -D_FORTIFY_SOURCE=2
Linking : /usr/bin/ld -m elf_x86_64

Comment 26009

Date: 2017-12-15 13:25:57 +0100
From: MonetDB Mercurial Repository <>

Changeset a2f70a576335 made by Sjoerd Mullender sjoerd@acm.org in the MonetDB repo, refers to this bug.

For complete details, see https//devmonetdborg/hg/MonetDB?cmd=changeset;node=a2f70a576335

Changeset description:

Save no_mitosis flag in query cache.
This should fix bug #6498.

Comment 26010

Date: 2017-12-15 13:30:07 +0100
From: @sjoerdmullender

I couldn't really reproduce the big slow down, nor did I see the large memory spike, but I did find something that could still explain your observations.

We save query plans in a cache. The first query that you did was executed and saved, the second one was retrieved from the cache, but the optimizations that happened on the first and second queries were different (after the plan was stored into / retrieved from the cache). I have now fixed this.

Asuming this was indeed the problem, I'm closing this bug. Please reopen if this doesn't fix the problem.

Comment 26014

Date: 2017-12-15 18:59:42 +0100
From: Anton Kravchenko <<kravchenko.anton86>>

thanks Sjoerd, it works as expected now.