Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Fixed
-
1.5.9
-
None
Description
Hello,
My topic is related to this task https://jira.mariadb.org/browse/CONJ-440.
When I'm trying to store BLOB file, which size is 1GB and using useServerPrepStmts=true, then I get the following exception:
_java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Unknown Source)
at java.io.ByteArrayOutputStream.toByteArray(Unknown Source)
at org.mariadb.jdbc.internal.packet.dao.parameters.StreamParameter.toString(StreamParameter.java:172)
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.throwErrorWithQuery(AbstractQueryProtocol.java:976)
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executePreparedQuery(AbstractQueryProtocol.java:620)
at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:401)
at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:362)
at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeUpdate(MariaDbServerPreparedStatement.java:351)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at oracle.ucp.jdbc.proxy.StatementProxyFactory.invoke(StatementProxyFactory.java:272)
at oracle.ucp.jdbc.proxy.PreparedStatementProxyFactory.invoke(PreparedStatementProxyFactory.java:138)
at com.sun.proxy.$Proxy11.executeUpdate(Unknown Source)
at TransferDataTableAccess.storeBinaryData(TransferDataTableAccess.java:596)
at TransferDataTableAccessTest.testTransferDataTableAccess(TransferDataTableAccessTest.java:155)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)_
The similar is happening when useServerPrepStmts=false.
_java.lang.OutOfMemoryError: Requested array size exceeds VM limit
at java.nio.HeapByteBuffer.<init>(Unknown Source)
at java.nio.ByteBuffer.allocate(Unknown Source)
at org.mariadb.jdbc.internal.stream.PacketOutputStream.increase(PacketOutputStream.java:111)
at org.mariadb.jdbc.internal.stream.PacketOutputStream.assureBufferCapacity(PacketOutputStream.java:598)
at org.mariadb.jdbc.internal.stream.PacketOutputStream.write(PacketOutputStream.java:376)
at org.mariadb.jdbc.internal.packet.dao.parameters.ParameterWriter.writeBytesEscaped(ParameterWriter.java:78)
at org.mariadb.jdbc.internal.packet.dao.parameters.ParameterWriter.write(ParameterWriter.java:142)
at org.mariadb.jdbc.internal.packet.dao.parameters.StreamParameter.writeTo(StreamParameter.java:94)
at org.mariadb.jdbc.internal.packet.ComExecute.sendSubCmd(ComExecute.java:89)
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:200)
at org.mariadb.jdbc.MariaDbClientPreparedStatement.executeInternal(MariaDbClientPreparedStatement.java:224)
at org.mariadb.jdbc.MariaDbClientPreparedStatement.executeUpdate(MariaDbClientPreparedStatement.java:197)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at oracle.ucp.jdbc.proxy.StatementProxyFactory.invoke(StatementProxyFactory.java:272)
at oracle.ucp.jdbc.proxy.PreparedStatementProxyFactory.invoke(PreparedStatementProxyFactory.java:138)
at com.sun.proxy.$Proxy11.executeUpdate(Unknown Source)
at TransferDataTableAccess.storeBinaryData(TransferDataTableAccess.java:596)
at TransferDataTableAccessTest.testTransferDataTableAccess(TransferDataTableAccessTest.java:155)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)_
My.ini is the following:
sql_mode="STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION"
default_storage_engine=innodb
innodb_buffer_pool_size=4G
innodb_log_file_size=4G
innodb_file_format=barracuda
innodb_strict_mode=on
innodb_large_prefix=on
innodb_file_per_table=1
character-set-server=utf8
max_allowed_packet=1G
innodb_flush_method=ALL_O_DIRECT
I have also tried to put -Xmx6144m -ea -Djava.awt.headless=true for execution.
In the new version of MariaDB connector (1.5.9) the speed of storing of BLOB objects has been decreased comparing to the version 1.5.7 (approximately 2x slower).
Attachments
Activity
Field | Original Value | New Value |
---|---|---|
Description |
Hello,
My topic is related to this task https://jira.mariadb.org/browse/CONJ-440. When I try to store BLOB file, which size is 1GB and using useServerPrepStmts=true, then I get the following exception: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Unknown Source) at java.io.ByteArrayOutputStream.toByteArray(Unknown Source) at org.mariadb.jdbc.internal.packet.dao.parameters.StreamParameter.toString(StreamParameter.java:172) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.throwErrorWithQuery(AbstractQueryProtocol.java:976) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executePreparedQuery(AbstractQueryProtocol.java:620) at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:401) at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:362) at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeUpdate(MariaDbServerPreparedStatement.java:351) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at oracle.ucp.jdbc.proxy.StatementProxyFactory.invoke(StatementProxyFactory.java:272) at oracle.ucp.jdbc.proxy.PreparedStatementProxyFactory.invoke(PreparedStatementProxyFactory.java:138) at com.sun.proxy.$Proxy11.executeUpdate(Unknown Source) at TransferDataTableAccess.storeBinaryData(TransferDataTableAccess.java:596) at TransferDataTableAccessTest.testTransferDataTableAccess(TransferDataTableAccessTest.java:155) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) In the new version of MariaDB connector (1.5.9) the speed of storing of BLOB objects has been decreased comparing to the version 1.5.7 (approximately 2x slower). |
Hello,
My topic is related to this task https://jira.mariadb.org/browse/CONJ-440. When I'm trying to store BLOB file, which size is 1GB and using useServerPrepStmts=true, then I get the following exception: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Unknown Source) at java.io.ByteArrayOutputStream.toByteArray(Unknown Source) at org.mariadb.jdbc.internal.packet.dao.parameters.StreamParameter.toString(StreamParameter.java:172) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.throwErrorWithQuery(AbstractQueryProtocol.java:976) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executePreparedQuery(AbstractQueryProtocol.java:620) at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:401) at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:362) at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeUpdate(MariaDbServerPreparedStatement.java:351) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at oracle.ucp.jdbc.proxy.StatementProxyFactory.invoke(StatementProxyFactory.java:272) at oracle.ucp.jdbc.proxy.PreparedStatementProxyFactory.invoke(PreparedStatementProxyFactory.java:138) at com.sun.proxy.$Proxy11.executeUpdate(Unknown Source) at TransferDataTableAccess.storeBinaryData(TransferDataTableAccess.java:596) at TransferDataTableAccessTest.testTransferDataTableAccess(TransferDataTableAccessTest.java:155) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) In the new version of MariaDB connector (1.5.9) the speed of storing of BLOB objects has been decreased comparing to the version 1.5.7 (approximately 2x slower). |
Description |
Hello,
My topic is related to this task https://jira.mariadb.org/browse/CONJ-440. When I'm trying to store BLOB file, which size is 1GB and using useServerPrepStmts=true, then I get the following exception: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Unknown Source) at java.io.ByteArrayOutputStream.toByteArray(Unknown Source) at org.mariadb.jdbc.internal.packet.dao.parameters.StreamParameter.toString(StreamParameter.java:172) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.throwErrorWithQuery(AbstractQueryProtocol.java:976) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executePreparedQuery(AbstractQueryProtocol.java:620) at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:401) at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:362) at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeUpdate(MariaDbServerPreparedStatement.java:351) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at oracle.ucp.jdbc.proxy.StatementProxyFactory.invoke(StatementProxyFactory.java:272) at oracle.ucp.jdbc.proxy.PreparedStatementProxyFactory.invoke(PreparedStatementProxyFactory.java:138) at com.sun.proxy.$Proxy11.executeUpdate(Unknown Source) at TransferDataTableAccess.storeBinaryData(TransferDataTableAccess.java:596) at TransferDataTableAccessTest.testTransferDataTableAccess(TransferDataTableAccessTest.java:155) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) In the new version of MariaDB connector (1.5.9) the speed of storing of BLOB objects has been decreased comparing to the version 1.5.7 (approximately 2x slower). |
Hello,
My topic is related to this task https://jira.mariadb.org/browse/CONJ-440. When I'm trying to store BLOB file, which size is 1GB and using useServerPrepStmts=true, then I get the following exception: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Unknown Source) at java.io.ByteArrayOutputStream.toByteArray(Unknown Source) at org.mariadb.jdbc.internal.packet.dao.parameters.StreamParameter.toString(StreamParameter.java:172) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.throwErrorWithQuery(AbstractQueryProtocol.java:976) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executePreparedQuery(AbstractQueryProtocol.java:620) at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:401) at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:362) at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeUpdate(MariaDbServerPreparedStatement.java:351) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at oracle.ucp.jdbc.proxy.StatementProxyFactory.invoke(StatementProxyFactory.java:272) at oracle.ucp.jdbc.proxy.PreparedStatementProxyFactory.invoke(PreparedStatementProxyFactory.java:138) at com.sun.proxy.$Proxy11.executeUpdate(Unknown Source) at TransferDataTableAccess.storeBinaryData(TransferDataTableAccess.java:596) at TransferDataTableAccessTest.testTransferDataTableAccess(TransferDataTableAccessTest.java:155) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) The same is happening when useServerPrepStmts=false. In the new version of MariaDB connector (1.5.9) the speed of storing of BLOB objects has been decreased comparing to the version 1.5.7 (approximately 2x slower). |
Description |
Hello,
My topic is related to this task https://jira.mariadb.org/browse/CONJ-440. When I'm trying to store BLOB file, which size is 1GB and using useServerPrepStmts=true, then I get the following exception: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Unknown Source) at java.io.ByteArrayOutputStream.toByteArray(Unknown Source) at org.mariadb.jdbc.internal.packet.dao.parameters.StreamParameter.toString(StreamParameter.java:172) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.throwErrorWithQuery(AbstractQueryProtocol.java:976) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executePreparedQuery(AbstractQueryProtocol.java:620) at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:401) at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:362) at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeUpdate(MariaDbServerPreparedStatement.java:351) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at oracle.ucp.jdbc.proxy.StatementProxyFactory.invoke(StatementProxyFactory.java:272) at oracle.ucp.jdbc.proxy.PreparedStatementProxyFactory.invoke(PreparedStatementProxyFactory.java:138) at com.sun.proxy.$Proxy11.executeUpdate(Unknown Source) at TransferDataTableAccess.storeBinaryData(TransferDataTableAccess.java:596) at TransferDataTableAccessTest.testTransferDataTableAccess(TransferDataTableAccessTest.java:155) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) The same is happening when useServerPrepStmts=false. In the new version of MariaDB connector (1.5.9) the speed of storing of BLOB objects has been decreased comparing to the version 1.5.7 (approximately 2x slower). |
Hello,
My topic is related to this task https://jira.mariadb.org/browse/CONJ-440. When I'm trying to store BLOB file, which size is 1GB and using useServerPrepStmts=true, then I get the following exception: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Unknown Source) at java.io.ByteArrayOutputStream.toByteArray(Unknown Source) at org.mariadb.jdbc.internal.packet.dao.parameters.StreamParameter.toString(StreamParameter.java:172) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.throwErrorWithQuery(AbstractQueryProtocol.java:976) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executePreparedQuery(AbstractQueryProtocol.java:620) at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:401) at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:362) at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeUpdate(MariaDbServerPreparedStatement.java:351) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at oracle.ucp.jdbc.proxy.StatementProxyFactory.invoke(StatementProxyFactory.java:272) at oracle.ucp.jdbc.proxy.PreparedStatementProxyFactory.invoke(PreparedStatementProxyFactory.java:138) at com.sun.proxy.$Proxy11.executeUpdate(Unknown Source) at TransferDataTableAccess.storeBinaryData(TransferDataTableAccess.java:596) at TransferDataTableAccessTest.testTransferDataTableAccess(TransferDataTableAccessTest.java:155) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) The similar is happening when useServerPrepStmts=false. java.lang.OutOfMemoryError: Requested array size exceeds VM limit at java.nio.HeapByteBuffer.<init>(Unknown Source) at java.nio.ByteBuffer.allocate(Unknown Source) at org.mariadb.jdbc.internal.stream.PacketOutputStream.increase(PacketOutputStream.java:111) at org.mariadb.jdbc.internal.stream.PacketOutputStream.assureBufferCapacity(PacketOutputStream.java:598) at org.mariadb.jdbc.internal.stream.PacketOutputStream.write(PacketOutputStream.java:376) at org.mariadb.jdbc.internal.packet.dao.parameters.ParameterWriter.writeBytesEscaped(ParameterWriter.java:78) at org.mariadb.jdbc.internal.packet.dao.parameters.ParameterWriter.write(ParameterWriter.java:142) at org.mariadb.jdbc.internal.packet.dao.parameters.StreamParameter.writeTo(StreamParameter.java:94) at org.mariadb.jdbc.internal.packet.ComExecute.sendSubCmd(ComExecute.java:89) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:200) at org.mariadb.jdbc.MariaDbClientPreparedStatement.executeInternal(MariaDbClientPreparedStatement.java:224) at org.mariadb.jdbc.MariaDbClientPreparedStatement.executeUpdate(MariaDbClientPreparedStatement.java:197) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at oracle.ucp.jdbc.proxy.StatementProxyFactory.invoke(StatementProxyFactory.java:272) at oracle.ucp.jdbc.proxy.PreparedStatementProxyFactory.invoke(PreparedStatementProxyFactory.java:138) at com.sun.proxy.$Proxy11.executeUpdate(Unknown Source) at TransferDataTableAccess.storeBinaryData(TransferDataTableAccess.java:596) at TransferDataTableAccessTest.testTransferDataTableAccess(TransferDataTableAccessTest.java:155) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) In the new version of MariaDB connector (1.5.9) the speed of storing of BLOB objects has been decreased comparing to the version 1.5.7 (approximately 2x slower). |
Description |
Hello,
My topic is related to this task https://jira.mariadb.org/browse/CONJ-440. When I'm trying to store BLOB file, which size is 1GB and using useServerPrepStmts=true, then I get the following exception: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Unknown Source) at java.io.ByteArrayOutputStream.toByteArray(Unknown Source) at org.mariadb.jdbc.internal.packet.dao.parameters.StreamParameter.toString(StreamParameter.java:172) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.throwErrorWithQuery(AbstractQueryProtocol.java:976) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executePreparedQuery(AbstractQueryProtocol.java:620) at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:401) at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:362) at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeUpdate(MariaDbServerPreparedStatement.java:351) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at oracle.ucp.jdbc.proxy.StatementProxyFactory.invoke(StatementProxyFactory.java:272) at oracle.ucp.jdbc.proxy.PreparedStatementProxyFactory.invoke(PreparedStatementProxyFactory.java:138) at com.sun.proxy.$Proxy11.executeUpdate(Unknown Source) at TransferDataTableAccess.storeBinaryData(TransferDataTableAccess.java:596) at TransferDataTableAccessTest.testTransferDataTableAccess(TransferDataTableAccessTest.java:155) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) The similar is happening when useServerPrepStmts=false. java.lang.OutOfMemoryError: Requested array size exceeds VM limit at java.nio.HeapByteBuffer.<init>(Unknown Source) at java.nio.ByteBuffer.allocate(Unknown Source) at org.mariadb.jdbc.internal.stream.PacketOutputStream.increase(PacketOutputStream.java:111) at org.mariadb.jdbc.internal.stream.PacketOutputStream.assureBufferCapacity(PacketOutputStream.java:598) at org.mariadb.jdbc.internal.stream.PacketOutputStream.write(PacketOutputStream.java:376) at org.mariadb.jdbc.internal.packet.dao.parameters.ParameterWriter.writeBytesEscaped(ParameterWriter.java:78) at org.mariadb.jdbc.internal.packet.dao.parameters.ParameterWriter.write(ParameterWriter.java:142) at org.mariadb.jdbc.internal.packet.dao.parameters.StreamParameter.writeTo(StreamParameter.java:94) at org.mariadb.jdbc.internal.packet.ComExecute.sendSubCmd(ComExecute.java:89) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:200) at org.mariadb.jdbc.MariaDbClientPreparedStatement.executeInternal(MariaDbClientPreparedStatement.java:224) at org.mariadb.jdbc.MariaDbClientPreparedStatement.executeUpdate(MariaDbClientPreparedStatement.java:197) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at oracle.ucp.jdbc.proxy.StatementProxyFactory.invoke(StatementProxyFactory.java:272) at oracle.ucp.jdbc.proxy.PreparedStatementProxyFactory.invoke(PreparedStatementProxyFactory.java:138) at com.sun.proxy.$Proxy11.executeUpdate(Unknown Source) at TransferDataTableAccess.storeBinaryData(TransferDataTableAccess.java:596) at TransferDataTableAccessTest.testTransferDataTableAccess(TransferDataTableAccessTest.java:155) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) In the new version of MariaDB connector (1.5.9) the speed of storing of BLOB objects has been decreased comparing to the version 1.5.7 (approximately 2x slower). |
Hello,
My topic is related to this task https://jira.mariadb.org/browse/CONJ-440. When I'm trying to store BLOB file, which size is 1GB and using useServerPrepStmts=true, then I get the following exception: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Unknown Source) at java.io.ByteArrayOutputStream.toByteArray(Unknown Source) at org.mariadb.jdbc.internal.packet.dao.parameters.StreamParameter.toString(StreamParameter.java:172) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.throwErrorWithQuery(AbstractQueryProtocol.java:976) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executePreparedQuery(AbstractQueryProtocol.java:620) at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:401) at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:362) at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeUpdate(MariaDbServerPreparedStatement.java:351) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at oracle.ucp.jdbc.proxy.StatementProxyFactory.invoke(StatementProxyFactory.java:272) at oracle.ucp.jdbc.proxy.PreparedStatementProxyFactory.invoke(PreparedStatementProxyFactory.java:138) at com.sun.proxy.$Proxy11.executeUpdate(Unknown Source) at TransferDataTableAccess.storeBinaryData(TransferDataTableAccess.java:596) at TransferDataTableAccessTest.testTransferDataTableAccess(TransferDataTableAccessTest.java:155) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) The similar is happening when useServerPrepStmts=false. java.lang.OutOfMemoryError: Requested array size exceeds VM limit at java.nio.HeapByteBuffer.<init>(Unknown Source) at java.nio.ByteBuffer.allocate(Unknown Source) at org.mariadb.jdbc.internal.stream.PacketOutputStream.increase(PacketOutputStream.java:111) at org.mariadb.jdbc.internal.stream.PacketOutputStream.assureBufferCapacity(PacketOutputStream.java:598) at org.mariadb.jdbc.internal.stream.PacketOutputStream.write(PacketOutputStream.java:376) at org.mariadb.jdbc.internal.packet.dao.parameters.ParameterWriter.writeBytesEscaped(ParameterWriter.java:78) at org.mariadb.jdbc.internal.packet.dao.parameters.ParameterWriter.write(ParameterWriter.java:142) at org.mariadb.jdbc.internal.packet.dao.parameters.StreamParameter.writeTo(StreamParameter.java:94) at org.mariadb.jdbc.internal.packet.ComExecute.sendSubCmd(ComExecute.java:89) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:200) at org.mariadb.jdbc.MariaDbClientPreparedStatement.executeInternal(MariaDbClientPreparedStatement.java:224) at org.mariadb.jdbc.MariaDbClientPreparedStatement.executeUpdate(MariaDbClientPreparedStatement.java:197) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at oracle.ucp.jdbc.proxy.StatementProxyFactory.invoke(StatementProxyFactory.java:272) at oracle.ucp.jdbc.proxy.PreparedStatementProxyFactory.invoke(PreparedStatementProxyFactory.java:138) at com.sun.proxy.$Proxy11.executeUpdate(Unknown Source) at TransferDataTableAccess.storeBinaryData(TransferDataTableAccess.java:596) at TransferDataTableAccessTest.testTransferDataTableAccess(TransferDataTableAccessTest.java:155) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) I have also tried to put -Xmx6144m -ea -Djava.awt.headless=true for execution. In the new version of MariaDB connector (1.5.9) the speed of storing of BLOB objects has been decreased comparing to the version 1.5.7 (approximately 2x slower). |
Description |
Hello,
My topic is related to this task https://jira.mariadb.org/browse/CONJ-440. When I'm trying to store BLOB file, which size is 1GB and using useServerPrepStmts=true, then I get the following exception: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Unknown Source) at java.io.ByteArrayOutputStream.toByteArray(Unknown Source) at org.mariadb.jdbc.internal.packet.dao.parameters.StreamParameter.toString(StreamParameter.java:172) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.throwErrorWithQuery(AbstractQueryProtocol.java:976) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executePreparedQuery(AbstractQueryProtocol.java:620) at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:401) at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:362) at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeUpdate(MariaDbServerPreparedStatement.java:351) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at oracle.ucp.jdbc.proxy.StatementProxyFactory.invoke(StatementProxyFactory.java:272) at oracle.ucp.jdbc.proxy.PreparedStatementProxyFactory.invoke(PreparedStatementProxyFactory.java:138) at com.sun.proxy.$Proxy11.executeUpdate(Unknown Source) at TransferDataTableAccess.storeBinaryData(TransferDataTableAccess.java:596) at TransferDataTableAccessTest.testTransferDataTableAccess(TransferDataTableAccessTest.java:155) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) The similar is happening when useServerPrepStmts=false. java.lang.OutOfMemoryError: Requested array size exceeds VM limit at java.nio.HeapByteBuffer.<init>(Unknown Source) at java.nio.ByteBuffer.allocate(Unknown Source) at org.mariadb.jdbc.internal.stream.PacketOutputStream.increase(PacketOutputStream.java:111) at org.mariadb.jdbc.internal.stream.PacketOutputStream.assureBufferCapacity(PacketOutputStream.java:598) at org.mariadb.jdbc.internal.stream.PacketOutputStream.write(PacketOutputStream.java:376) at org.mariadb.jdbc.internal.packet.dao.parameters.ParameterWriter.writeBytesEscaped(ParameterWriter.java:78) at org.mariadb.jdbc.internal.packet.dao.parameters.ParameterWriter.write(ParameterWriter.java:142) at org.mariadb.jdbc.internal.packet.dao.parameters.StreamParameter.writeTo(StreamParameter.java:94) at org.mariadb.jdbc.internal.packet.ComExecute.sendSubCmd(ComExecute.java:89) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:200) at org.mariadb.jdbc.MariaDbClientPreparedStatement.executeInternal(MariaDbClientPreparedStatement.java:224) at org.mariadb.jdbc.MariaDbClientPreparedStatement.executeUpdate(MariaDbClientPreparedStatement.java:197) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at oracle.ucp.jdbc.proxy.StatementProxyFactory.invoke(StatementProxyFactory.java:272) at oracle.ucp.jdbc.proxy.PreparedStatementProxyFactory.invoke(PreparedStatementProxyFactory.java:138) at com.sun.proxy.$Proxy11.executeUpdate(Unknown Source) at TransferDataTableAccess.storeBinaryData(TransferDataTableAccess.java:596) at TransferDataTableAccessTest.testTransferDataTableAccess(TransferDataTableAccessTest.java:155) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) I have also tried to put -Xmx6144m -ea -Djava.awt.headless=true for execution. In the new version of MariaDB connector (1.5.9) the speed of storing of BLOB objects has been decreased comparing to the version 1.5.7 (approximately 2x slower). |
Hello,
My topic is related to this task https://jira.mariadb.org/browse/CONJ-440. When I'm trying to store BLOB file, which size is 1GB and using useServerPrepStmts=true, then I get the following exception: _java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Unknown Source) at java.io.ByteArrayOutputStream.toByteArray(Unknown Source) at org.mariadb.jdbc.internal.packet.dao.parameters.StreamParameter.toString(StreamParameter.java:172) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.throwErrorWithQuery(AbstractQueryProtocol.java:976) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executePreparedQuery(AbstractQueryProtocol.java:620) at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:401) at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:362) at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeUpdate(MariaDbServerPreparedStatement.java:351) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at oracle.ucp.jdbc.proxy.StatementProxyFactory.invoke(StatementProxyFactory.java:272) at oracle.ucp.jdbc.proxy.PreparedStatementProxyFactory.invoke(PreparedStatementProxyFactory.java:138) at com.sun.proxy.$Proxy11.executeUpdate(Unknown Source) at TransferDataTableAccess.storeBinaryData(TransferDataTableAccess.java:596) at TransferDataTableAccessTest.testTransferDataTableAccess(TransferDataTableAccessTest.java:155) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)_ The similar is happening when useServerPrepStmts=false. _java.lang.OutOfMemoryError: Requested array size exceeds VM limit at java.nio.HeapByteBuffer.<init>(Unknown Source) at java.nio.ByteBuffer.allocate(Unknown Source) at org.mariadb.jdbc.internal.stream.PacketOutputStream.increase(PacketOutputStream.java:111) at org.mariadb.jdbc.internal.stream.PacketOutputStream.assureBufferCapacity(PacketOutputStream.java:598) at org.mariadb.jdbc.internal.stream.PacketOutputStream.write(PacketOutputStream.java:376) at org.mariadb.jdbc.internal.packet.dao.parameters.ParameterWriter.writeBytesEscaped(ParameterWriter.java:78) at org.mariadb.jdbc.internal.packet.dao.parameters.ParameterWriter.write(ParameterWriter.java:142) at org.mariadb.jdbc.internal.packet.dao.parameters.StreamParameter.writeTo(StreamParameter.java:94) at org.mariadb.jdbc.internal.packet.ComExecute.sendSubCmd(ComExecute.java:89) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:200) at org.mariadb.jdbc.MariaDbClientPreparedStatement.executeInternal(MariaDbClientPreparedStatement.java:224) at org.mariadb.jdbc.MariaDbClientPreparedStatement.executeUpdate(MariaDbClientPreparedStatement.java:197) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at oracle.ucp.jdbc.proxy.StatementProxyFactory.invoke(StatementProxyFactory.java:272) at oracle.ucp.jdbc.proxy.PreparedStatementProxyFactory.invoke(PreparedStatementProxyFactory.java:138) at com.sun.proxy.$Proxy11.executeUpdate(Unknown Source) at TransferDataTableAccess.storeBinaryData(TransferDataTableAccess.java:596) at TransferDataTableAccessTest.testTransferDataTableAccess(TransferDataTableAccessTest.java:155) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)_ I have also tried to put -Xmx6144m -ea -Djava.awt.headless=true for execution. In the new version of MariaDB connector (1.5.9) the speed of storing of BLOB objects has been decreased comparing to the version 1.5.7 (approximately 2x slower). |
Description |
Hello,
My topic is related to this task https://jira.mariadb.org/browse/CONJ-440. When I'm trying to store BLOB file, which size is 1GB and using useServerPrepStmts=true, then I get the following exception: _java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Unknown Source) at java.io.ByteArrayOutputStream.toByteArray(Unknown Source) at org.mariadb.jdbc.internal.packet.dao.parameters.StreamParameter.toString(StreamParameter.java:172) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.throwErrorWithQuery(AbstractQueryProtocol.java:976) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executePreparedQuery(AbstractQueryProtocol.java:620) at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:401) at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:362) at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeUpdate(MariaDbServerPreparedStatement.java:351) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at oracle.ucp.jdbc.proxy.StatementProxyFactory.invoke(StatementProxyFactory.java:272) at oracle.ucp.jdbc.proxy.PreparedStatementProxyFactory.invoke(PreparedStatementProxyFactory.java:138) at com.sun.proxy.$Proxy11.executeUpdate(Unknown Source) at TransferDataTableAccess.storeBinaryData(TransferDataTableAccess.java:596) at TransferDataTableAccessTest.testTransferDataTableAccess(TransferDataTableAccessTest.java:155) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)_ The similar is happening when useServerPrepStmts=false. _java.lang.OutOfMemoryError: Requested array size exceeds VM limit at java.nio.HeapByteBuffer.<init>(Unknown Source) at java.nio.ByteBuffer.allocate(Unknown Source) at org.mariadb.jdbc.internal.stream.PacketOutputStream.increase(PacketOutputStream.java:111) at org.mariadb.jdbc.internal.stream.PacketOutputStream.assureBufferCapacity(PacketOutputStream.java:598) at org.mariadb.jdbc.internal.stream.PacketOutputStream.write(PacketOutputStream.java:376) at org.mariadb.jdbc.internal.packet.dao.parameters.ParameterWriter.writeBytesEscaped(ParameterWriter.java:78) at org.mariadb.jdbc.internal.packet.dao.parameters.ParameterWriter.write(ParameterWriter.java:142) at org.mariadb.jdbc.internal.packet.dao.parameters.StreamParameter.writeTo(StreamParameter.java:94) at org.mariadb.jdbc.internal.packet.ComExecute.sendSubCmd(ComExecute.java:89) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:200) at org.mariadb.jdbc.MariaDbClientPreparedStatement.executeInternal(MariaDbClientPreparedStatement.java:224) at org.mariadb.jdbc.MariaDbClientPreparedStatement.executeUpdate(MariaDbClientPreparedStatement.java:197) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at oracle.ucp.jdbc.proxy.StatementProxyFactory.invoke(StatementProxyFactory.java:272) at oracle.ucp.jdbc.proxy.PreparedStatementProxyFactory.invoke(PreparedStatementProxyFactory.java:138) at com.sun.proxy.$Proxy11.executeUpdate(Unknown Source) at TransferDataTableAccess.storeBinaryData(TransferDataTableAccess.java:596) at TransferDataTableAccessTest.testTransferDataTableAccess(TransferDataTableAccessTest.java:155) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)_ I have also tried to put -Xmx6144m -ea -Djava.awt.headless=true for execution. In the new version of MariaDB connector (1.5.9) the speed of storing of BLOB objects has been decreased comparing to the version 1.5.7 (approximately 2x slower). |
Hello,
My topic is related to this task https://jira.mariadb.org/browse/CONJ-440. When I'm trying to store BLOB file, which size is 1GB and using useServerPrepStmts=true, then I get the following exception: _java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Unknown Source) at java.io.ByteArrayOutputStream.toByteArray(Unknown Source) at org.mariadb.jdbc.internal.packet.dao.parameters.StreamParameter.toString(StreamParameter.java:172) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.throwErrorWithQuery(AbstractQueryProtocol.java:976) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executePreparedQuery(AbstractQueryProtocol.java:620) at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:401) at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:362) at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeUpdate(MariaDbServerPreparedStatement.java:351) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at oracle.ucp.jdbc.proxy.StatementProxyFactory.invoke(StatementProxyFactory.java:272) at oracle.ucp.jdbc.proxy.PreparedStatementProxyFactory.invoke(PreparedStatementProxyFactory.java:138) at com.sun.proxy.$Proxy11.executeUpdate(Unknown Source) at TransferDataTableAccess.storeBinaryData(TransferDataTableAccess.java:596) at TransferDataTableAccessTest.testTransferDataTableAccess(TransferDataTableAccessTest.java:155) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)_ The similar is happening when useServerPrepStmts=false. _java.lang.OutOfMemoryError: Requested array size exceeds VM limit at java.nio.HeapByteBuffer.<init>(Unknown Source) at java.nio.ByteBuffer.allocate(Unknown Source) at org.mariadb.jdbc.internal.stream.PacketOutputStream.increase(PacketOutputStream.java:111) at org.mariadb.jdbc.internal.stream.PacketOutputStream.assureBufferCapacity(PacketOutputStream.java:598) at org.mariadb.jdbc.internal.stream.PacketOutputStream.write(PacketOutputStream.java:376) at org.mariadb.jdbc.internal.packet.dao.parameters.ParameterWriter.writeBytesEscaped(ParameterWriter.java:78) at org.mariadb.jdbc.internal.packet.dao.parameters.ParameterWriter.write(ParameterWriter.java:142) at org.mariadb.jdbc.internal.packet.dao.parameters.StreamParameter.writeTo(StreamParameter.java:94) at org.mariadb.jdbc.internal.packet.ComExecute.sendSubCmd(ComExecute.java:89) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:200) at org.mariadb.jdbc.MariaDbClientPreparedStatement.executeInternal(MariaDbClientPreparedStatement.java:224) at org.mariadb.jdbc.MariaDbClientPreparedStatement.executeUpdate(MariaDbClientPreparedStatement.java:197) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at oracle.ucp.jdbc.proxy.StatementProxyFactory.invoke(StatementProxyFactory.java:272) at oracle.ucp.jdbc.proxy.PreparedStatementProxyFactory.invoke(PreparedStatementProxyFactory.java:138) at com.sun.proxy.$Proxy11.executeUpdate(Unknown Source) at TransferDataTableAccess.storeBinaryData(TransferDataTableAccess.java:596) at TransferDataTableAccessTest.testTransferDataTableAccess(TransferDataTableAccessTest.java:155) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)_ My.ini is the following: sql_mode="STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION" default_storage_engine=innodb innodb_buffer_pool_size=4G innodb_log_file_size=4G innodb_file_format=barracuda innodb_strict_mode=on innodb_large_prefix=on innodb_file_per_table=1 character-set-server=utf8 max_allowed_packet=1G innodb_flush_method=ALL_O_DIRECT I have also tried to put -Xmx6144m -ea -Djava.awt.headless=true for execution. In the new version of MariaDB connector (1.5.9) the speed of storing of BLOB objects has been decreased comparing to the version 1.5.7 (approximately 2x slower). |
Affects Version/s | 1.5.7 [ 22500 ] |
Priority | Major [ 3 ] | Critical [ 2 ] |
Status | Open [ 1 ] | In Progress [ 3 ] |
Fix Version/s | 1.5.10 [ 22523 ] |
Fix Version/s | 2.0.0 [ 22119 ] | |
Fix Version/s | 1.5.10 [ 22523 ] |
Component/s | Other [ 12201 ] | |
Resolution | Fixed [ 1 ] | |
Status | In Progress [ 3 ] | Closed [ 6 ] |
Workflow | MariaDB v3 [ 79910 ] | MariaDB v4 [ 134961 ] |
Implementation is been reviewed to never use a buffer bigger than 16mb, streaming to socket.
The development snapshot are available on sonatype nexus repository using :
<repositories>
<repository>
<id>sonatype-nexus-snapshots</id>
<name>Sonatype Nexus Snapshots</name>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>org.mariadb.jdbc</groupId>
<artifactId>mariadb-java-client</artifactId>
</dependency>
</dependencies>
Benchmarks show performance results similar to 1.5.8 without using lot of memory that can lead to OOM when dealing with large stream.