You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In cases where Free Heap Memory in JVM approaches 0 bytes, the cacheSize in MonetConnection$ResultSetResponse gets assigned to 0, causing a divide-by-zero condition when computing the block size in getLine(). Here is a sample stack trace:
Caused by: java.lang.ArithmeticException: divide by zero
at nl.cwi.monetdb.jdbc.MonetConnection$ResultSetResponse.getLine(MonetConnection.java:1580)
at nl.cwi.monetdb.jdbc.MonetResultSet.absolute(MonetResultSet.java:194)
at nl.cwi.monetdb.jdbc.MonetResultSet.relative(MonetResultSet.java:2149)
at nl.cwi.monetdb.jdbc.MonetResultSet.next(MonetResultSet.java:2116)
This condition is rare, but occurs often enough to be quite disruptive. Please note: this is NOT an out-of-memory condition. Although Free Memory is momentarily exhausted, the Max Memory available to the JVM is still several gigabytes away. What would normally happen is GC would run to reclaim memory and / or the Total Memory would be increased (IE: The Java Heap would grow).
I am running a memory-intensive application, so it is not unusual for the Free Memory in the Heap to be exhausted (several times) until the Total Memory has expanded to its peak size.
Reproducible: Sometimes
Steps to Reproduce:
Start the JVM (only tested on IBM Java 6.0) with very conservative min heap size, but a very large max heap size.
Run several queries through JDBC client (in a loop).
Concurrently, run a process that leaks memory until Free Memory reaches is threshold.
Actual Results:
If you are (un)lucky, MonetConnection$ResultSetResponse.getLine() will be invoked around the time Free Memory reaches 0, which will cause a divide-by-zero. It may take many runs before this occurs (it is NOT deterministic).
Expected Results:
No error should have happened.
Recommend enforcing a min cacheSize (never let it go below some threshold) to avoid divide-by-zero. You can always expect Free Memory to reach 0 at some point, although this is rare.
Created attachment 131
jdbc-cache-size-no-freemem.patch
The free memory thing was completely bogus, as it was counting how much memory was free, but used it to check how many rows of data we could hold. A row obviously requires more than just 2 bytes (UTF-16), so this is flawed. Instead, I think it's better just to rely on the JVM to do the right thing with memory management, and hence not to keep on increasing the cacheSize until memory seems to be short. Instead I now just use a fixed increment of the cacheSize, to fetch bigger blocks, this should gain the efficiency aimed at, while not blowing up the memory requirements to a rediculous amount either.
Can you try this patch, or would you prefer a JDBC jar instead?
ResultsetResponse.getLine: fix possible divide by zero error
cacheSize can turn 0 when the amount of free memory becomes 0, with a
divide by zero as a result.
The free memory thing was completely bogus, as it was counting how much
memory was free, but used it to check how many rows of data we could
hold. A row obviously requires more than just 2 bytes (UTF-16), so this
is flawed. Instead, it's better just to rely on the JVM to do the right
thing with memory management, and hence not to keep on increasing the
cacheSize until memory seems to be short. Instead we now just use a
fixed increment of the cacheSize, to fetch bigger blocks, this should
gain the efficiency aimed at, while not blowing up the memory
requirements to a rediculous amount either.
This fixes bug #3119.
Date: 2012-07-12 20:47:19 +0200
From: Joseph <>
To: clients devs <>
Version: 11.11.5 (Jul2012)
Last updated: 2012-08-23 10:14:37 +0200
Comment 17454
Date: 2012-07-12 20:47:19 +0200
From: Joseph <>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0.4) Gecko/20100101 Firefox/10.0.4
Build Identifier:
In cases where Free Heap Memory in JVM approaches 0 bytes, the cacheSize in MonetConnection$ResultSetResponse gets assigned to 0, causing a divide-by-zero condition when computing the block size in getLine(). Here is a sample stack trace:
Caused by: java.lang.ArithmeticException: divide by zero
at nl.cwi.monetdb.jdbc.MonetConnection$ResultSetResponse.getLine(MonetConnection.java:1580)
at nl.cwi.monetdb.jdbc.MonetResultSet.absolute(MonetResultSet.java:194)
at nl.cwi.monetdb.jdbc.MonetResultSet.relative(MonetResultSet.java:2149)
at nl.cwi.monetdb.jdbc.MonetResultSet.next(MonetResultSet.java:2116)
This condition is rare, but occurs often enough to be quite disruptive. Please note: this is NOT an out-of-memory condition. Although Free Memory is momentarily exhausted, the Max Memory available to the JVM is still several gigabytes away. What would normally happen is GC would run to reclaim memory and / or the Total Memory would be increased (IE: The Java Heap would grow).
I am running a memory-intensive application, so it is not unusual for the Free Memory in the Heap to be exhausted (several times) until the Total Memory has expanded to its peak size.
Reproducible: Sometimes
Steps to Reproduce:
Actual Results:
If you are (un)lucky, MonetConnection$ResultSetResponse.getLine() will be invoked around the time Free Memory reaches 0, which will cause a divide-by-zero. It may take many runs before this occurs (it is NOT deterministic).
Expected Results:
No error should have happened.
Recommend enforcing a min cacheSize (never let it go below some threshold) to avoid divide-by-zero. You can always expect Free Memory to reach 0 at some point, although this is rare.
Comment 17467
Date: 2012-07-16 14:08:19 +0200
From: @grobian
thanks for the report, I'll look into this for Jul2012-SP1
Comment 17502
Date: 2012-07-18 11:11:47 +0200
From: @grobian
Which JDBC driver do you use? Your line numbers don't match with the latest (Apr2012-SP2/Jul2012) code.
Comment 17505
Date: 2012-07-18 11:48:06 +0200
From: @grobian
Created attachment 131
jdbc-cache-size-no-freemem.patch
The free memory thing was completely bogus, as it was counting how much memory was free, but used it to check how many rows of data we could hold. A row obviously requires more than just 2 bytes (UTF-16), so this is flawed. Instead, I think it's better just to rely on the JVM to do the right thing with memory management, and hence not to keep on increasing the cacheSize until memory seems to be short. Instead I now just use a fixed increment of the cacheSize, to fetch bigger blocks, this should gain the efficiency aimed at, while not blowing up the memory requirements to a rediculous amount either.
Can you try this patch, or would you prefer a JDBC jar instead?
Comment 17506
Date: 2012-07-18 13:51:54 +0200
From: Joseph <>
Thanks. I would prefer a JAR, please.
Comment 17507
Date: 2012-07-18 13:55:25 +0200
From: @grobian
Created attachment 132
monetdb-jdbc-3119.jar
Here it is, compiled with Java 7, hope that's ok for you
Comment 17517
Date: 2012-07-19 21:07:03 +0200
From: Joseph <>
I don't suppose you can compile it against Java 6 for me to test it out?
Comment 17523
Date: 2012-07-20 10:16:41 +0200
From: @grobian
Created attachment 135
monetdb-jdbc-3119.jar
Here is the 1.6 built version
Comment 17530
Date: 2012-07-20 16:04:31 +0200
From: Joseph <>
Thanks! This client appears to work good.
Comment 17531
Date: 2012-07-20 17:03:29 +0200
From: @grobian
Changeset 84ad1fe6eac3 made by Fabian Groffen fabian@cwi.nl in the MonetDB repo, refers to this bug.
For complete details, see http//devmonetdborg/hg/MonetDB?cmd=changeset;node=84ad1fe6eac3
Changeset description:
Comment 17532
Date: 2012-07-20 17:04:02 +0200
From: @grobian
ok, thanks for testing!
Comment 17638
Date: 2012-08-23 10:14:37 +0200
From: @sjoerdmullender
Jul2012-SP1 has been released.
The text was updated successfully, but these errors were encountered: