[CONJ-550] Losing streaming ability in statement after executing Optimize Created: 2017-11-14  Updated: 2017-11-20  Resolved: 2017-11-15

Status: Closed
Project: MariaDB Connector/J
Component/s: Other
Affects Version/s: 2.1.1
Fix Version/s: 2.2.1, 1.7.1

Type: Bug Priority: Minor
Reporter: Rafael Lopez Fernández Assignee: Diego Dupin
Resolution: Fixed Votes: 0
Labels: None


 Description   

When issuing a query that returns a lot of rows after issuing an Optimize or Analyze
in a Statement with streaming active (fecthSize = Integer.MIN_VALUE) it loads
all results in memory producing an OOME

stmt = connection.createStatement();
stmt.setFetchSize(Integer.MIN_VALUE);            
stmt.executeUpdate("DROP TEMPORARY TABLE IF EXISTS tmp.a");               
stmt.executeUpdate("CREATE TEMPORARY TABLE tmp.a ( a INT )");
stmt.executeUpdate("INSERT INTO tmp.a VALUES(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1)");
//optimize internally returns a ResultSet that I do not read
stmt.executeUpdate("OPTIMIZE TABLE tmp.a");
final ResultSet rs = stmt.executeQuery("SELECT a from tmp.a");                
int count = 0;
//this resultset iteration is not streamed,it is fully loaded on memory
while (rs.next()) {
    count++;
}
System.out.println(count + " rows read");
rs.close();          

I am not sure if the workaround is to change fetchSize reset in this two places:

org.mariadb.jdbc.internal.com.read.dao.Results (303) loadFully methd:

 public void loadFully(boolean skip, Protocol protocol) throws SQLException {
        if (fetchSize != 0) {           
            //fetchSize = 0; comenting this to avoid reading 0 in the next statement
            if (resultSet != null) {
          ....

and here also is reset

org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol (1282) removeActiveStreamingResult()

removes fetch size also in loadFully -> resultSet.fetchRemaining()
org.mariadb.jdbc.internal.com.read.dao.Results.removeFetchSize()



 Comments   
Comment by Diego Dupin [ 2017-11-15 ]

workaround is to set "stmt.setFetchSize(1);" just before "final ResultSet rs = stmt.executeQuery("SELECT a from tmp.a");"

Fetch size has been reset to 0 (=no fetch) when skipping previous streaming resultset, that will be corrected in next version.

Comment by Diego Dupin [ 2017-11-15 ]

correction done.
available in snapshot :

<repositories>
    <repository>
        <id>sonatype-nexus-snapshots</id>
        <name>Sonatype Nexus Snapshots</name>
        <url>https://oss.sonatype.org/content/repositories/snapshots</url>
    </repository>
</repositories>
 
<dependencies>
    <dependency>
        <groupId>org.mariadb.jdbc</groupId>
        <artifactId>mariadb-java-client</artifactId>
        <version>2.2.1-SNAPSHOT</version>
    </dependency>
</dependencies>

Generated at Thu Feb 08 03:16:35 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.