Date: 2012-07-03 13:47:14 +0200
From: info
To: Merovingian devs <>
Version: 11.9.7 (Apr2012-SP2) [obsolete]

Last updated: 2012-07-17 13:57:41 +0200

Comment 17402

Date: 2012-07-03 13:47:14 +0200
From: info

User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:13.0) Gecko/20100101 Firefox/13.0.1
Build Identifier: Ubuntu 10.04 LTS

Dear MonetDB team,
For our project we are using monetdb, we want to make a jump from our current version:
monetdb release monetdb-hg-april_11.3.4-1_bed1350dc1be to the MonetDB (Apr2012-SP2)
release.
To make this a succesful “jump” we are currently testing the MonetDB (Apr2012-SP2) release
on Ubuntu 10.04.2 LTS
, 64 bits. To get more insight information about the problems we did run the tests with Valgrind, the following issues where reported when using the "monetdb set" commando:
==31739== HEAP SUMMARY:
==31739==     in use at exit: 21,276 bytes in 239 blocks
==31739==   total heap usage: 6,158 allocs, 5,920 frees, 6,465,213 bytes allocated
==31739==
==31739== Searching for pointers to 239 not-freed blocks
==31739== Checked 456,600 bytes
==31739==
==31739== 4,018 bytes in 98 blocks are definitely lost in loss record 13 of 14
==31739==    at 0x4C274A8: malloc (vg_replace_malloc.c:236)
==31739==    by 0x54EAD81: strdup (strdup.c:43)
==31739==    by 0x50467F7: mcrypt_hashPassword (in /data/programs/MonetDB/lib/libmapi.so.6.0.1)
==31739==    by 0x40A855: control_authorise (in /data/programs/MonetDB/bin/monetdbd)
==31739==    by 0x409A11: ??? (in /data/programs/MonetDB/bin/monetdbd)
==31739==    by 0x409EA2: acceptConnections (in /data/programs/MonetDB/bin/monetdbd)
==31739==    by 0x40752A: main (in /data/programs/MonetDB/bin/monetdbd)
==31739==
==31739== 11,760 bytes in 49 blocks are definitely lost in loss record 14 of 14
==31739==    at 0x4C274A8: malloc (vg_replace_malloc.c:236)
==31739==    by 0x4141CD: getDefaultProps (in /data/programs/MonetDB/bin/monetdbd)
==31739==    by 0x41463D: setProp (in /data/programs/MonetDB/bin/monetdbd)
==31739==    by 0x40B61A: ??? (in /data/programs/MonetDB/bin/monetdbd)
==31739==    by 0x409A28: ??? (in /data/programs/MonetDB/bin/monetdbd)
==31739==    by 0x409EA2: acceptConnections (in /data/programs/MonetDB/bin/monetdbd)
==31739==    by 0x40752A: main (in /data/programs/MonetDB/bin/monetdbd)
==31739==
==31739== LEAK SUMMARY:
==31739==    definitely lost: 15,778 bytes in 147 blocks
==31739==    indirectly lost: 0 bytes in 0 blocks
==31739==      possibly lost: 0 bytes in 0 blocks
==31739==    still reachable: 5,498 bytes in 92 blocks
==31739==         suppressed: 0 bytes in 0 blocks
==31739== Reachable blocks (those to which a pointer was found) are not shown.
==31739== To see them, rerun with: --leak-check=full --show-reachable=yes

Using the "monetdb set" command multiple times shows an increase in the “definitely lost” memory blocks. Investigating the monetdb sources we found the following:
In monetdb.c the value of the variable props (assigned via confkeyval *props = getDefaultProps();) allocates memory that IS free-ed as expected.
But the variable defprops (assigned via confkeyval *defprops = getDefaultProps();) allocs memory that IS NOT free-ed. If we understand it correctly than there seems to be a problem in this piece of code.

Something similar is happening when we use the "monetdb get" command, below the Valgrind report for this:
==32049== HEAP SUMMARY:
==32049==     in use at exit: 6,895 bytes in 101 blocks
==32049==   total heap usage: 563 allocs, 463 frees, 533,667 bytes allocated
==32049==
==32049== Searching for pointers to 101 not-freed blocks
==32049== Checked 456,600 bytes
==32049==
==32049== 202 bytes in 1 blocks are definitely lost in loss record 4 of 15
==32049==    at 0x4C274A8: malloc (vg_replace_malloc.c:236)
==32049==    by 0x4143ED: writePropsBuf (in /data/programs/MonetDB/bin/monetdbd)
==32049==    by 0x40BDA8: ??? (in /data/programs/MonetDB/bin/monetdbd)
==32049==    by 0x40C884: controlRunner (in /data/programs/MonetDB/bin/monetdbd)
==32049==    by 0x52519C9: start_thread (pthread_create.c:300)
==32049==    by 0x7E4E6FF: ???
==32049==
==32049== 1,195 bytes in 8 blocks are definitely lost in loss record 15 of 15
==32049==    at 0x4C274A8: malloc (vg_replace_malloc.c:236)
==32049==    by 0x4143ED: writePropsBuf (in /data/programs/MonetDB/bin/monetdbd)
==32049==    by 0x40C29A: ??? (in /data/programs/MonetDB/bin/monetdbd)
==32049==    by 0x40C884: controlRunner (in /data/programs/MonetDB/bin/monetdbd)
==32049==    by 0x52519C9: start_thread (pthread_create.c:300)
==32049==    by 0x7E4E6FF: ???
==32049==
==32049== LEAK SUMMARY:
==32049==    definitely lost: 1,397 bytes in 9 blocks
==32049==    indirectly lost: 0 bytes in 0 blocks
==32049==      possibly lost: 0 bytes in 0 blocks
==32049==    still reachable: 5,498 bytes in 92 blocks
==32049==         suppressed: 0 bytes in 0 blocks
==32049== Reachable blocks (those to which a pointer was found) are not shown.
==32049== To see them, rerun with: --leak-check=full --show-reachable=yes

It seems that in module controlrunner.c the claimed memory for the variable pbuf IS NOT free-ed after calling:
writePropsBuf(_mero_db_props, &pbuf);
writePropsBuf(props, &pbuf);
inside function:
static void ctl_handle_client(
        const char *origin,
        int msgsock,
        stream *fdin,
        stream *fout)

We hope this information is useful for monetdb, unfortunately for our project it seems that using the current release is not (yet) suitable to us.

regards and keep up the good work !

Reproducible: Always

Comment 17403

Date: 2012-07-03 13:48:21 +0200
From: info

sorry for the messy formatting ;)

Comment 17404

Date: 2012-07-03 13:49:25 +0200
From: @grobian

thank you! this is a blocker for the Jul2012 release

Comment 17418

Date: 2012-07-06 10:31:04 +0200
From: @grobian

Changeset 87e8d77d05df made by Fabian Groffen fabian@cwi.nl in the MonetDB repo, refers to this bug.

For complete details, see http//devmonetdborg/hg/MonetDB?cmd=changeset;node=87e8d77d05df

Changeset description:

monetdbd: fix misc memory leaks

- the result from getDefaultProps() is malloced and needs to be freed
- the result from writePropsBuf() is malloced and needs to be freed

This keeps the memory usage of monetdbd constant over several monetdb
get, set, status and inherit calls.  Thanks Patchupedia in Bug #3111.

Comment 17480

Date: 2012-07-17 13:57:41 +0200
From: @grobian

Fix released in Jul2012