[CONJ-315] Cannot cancel Streaming result sets Created: 2016-06-19  Updated: 2017-09-27  Resolved: 2017-04-08

Status: Closed
Project: MariaDB Connector/J
Component/s: Other
Affects Version/s: 1.4.2, 1.4.6
Fix Version/s: 2.0.0-RC

Type: Bug Priority: Major
Reporter: Olivier Bertrand Assignee: Diego Dupin
Resolution: Fixed Votes: 0
Labels: None
Environment:

Windows 7


Attachments: Java Source File JdbcCancel.java     Text File diagsamp.txt    
Issue Links:
Problem/Incident
causes CONJ-531 Cancel of result set is incorrect/exc... Closed

 Description   

The MariaDB J connector documentation says:

Streaming result sets
By default, Statement.executeQuery() will read the full result set from the server before returning. With large result sets, this will require large amounts of memory. Better behavior in this case would be reading row-by-row, with ResultSet.next(), so called "streaming". This is activated using Statement.setFetchSize(Integer.MIN_VALUE)

The problem is that if an application decides to stop reading the result set, the connector still insists to fetch all the rows of the result set before closing it. This can make very long queries with big tables even a small part of it is useful.

The here attached application JdbcCancel.java demonstrates this. It is written to read and dispay the first rows of a big table (the original one has 10 millions rows) until a row containing a "-" in the rln column is met.

In previous versions of the MySQL J connector (for instance version 5.1.17) a turnaround was to cancel the statement before closing it. Unfortunately later versions (for instance 6.0.2) as well as the MariaDB J connector (I tested versions 1.4.2 and 1.4.6) fail to stop the reading. Even if a cancel command is executed the reading continues.

Here is the console output of the program execution:

Connection org.mariadb.jdbc.MariaDbConnection@20c684 established
recordnb, diagnb, rln, diag, dp
339858, 5, A49U9MQ94, 0, 0
985921, 1, LFFXXDGOL, 0, 0
1441640, 1, 8W887WRWG, 0, 0
1760557, 0, 2MMXOF1JU, 0, 0
1760558, 0, 5DDDSJDTD, 0, 0
1760559, 0, 7UQQS7SKK, 0, 0
Canceling query
Closing statement
Statement closed

All connectors display the same output, the difference is that between the last two lines, more than 3 minutes are spent reading the rest of the file by the new versions.

Trying the cancel the statement from another thread does not work either.

Attached is the java application and the first 100 rows of the file that was used the make the diag table.



 Comments   
Comment by Olivier Bertrand [ 2016-08-11 ]

This is not addressed by the new version 1.5.0. Any chance to have it considered for the next one?

Comment by Diego Dupin [ 2017-04-07 ]

Not sure this can be done.
There is some protocol solution for that (COM_STMT_FETCH), but this solution is a lot slower than actual implementation, and does break in 5.7 and 10.2.

Best solution would be that application handle that with LIMIT and OFFSET. See also pagination

Comment by Olivier Bertrand [ 2017-04-07 ]

The same problem occurs when limit is used.

Comment by Olivier Bertrand [ 2017-04-07 ]

Not sure this can be done
Come on! Other JDBC drivers do it and older version of the MySQL JDBC driver also did it. Just do the same. It is not normal to continue reading a table when the query is stopped...

Comment by Diego Dupin [ 2017-04-08 ]

Right,
I was reluctant to set up a solution with "kill query", but it is definitely the right solution for this case.
this can definitively be done : Commit 0d6e4db
will be released in 2.0.0

Generated at Thu Feb 08 03:14:46 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.