Uploaded image for project: 'MariaDB Connector/J'
  1. MariaDB Connector/J
  2. CONJ-550

Losing streaming ability in statement after executing Optimize

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Minor
    • Resolution: Fixed
    • 2.1.1
    • 2.2.1, 1.7.1
    • Other
    • None

    Description

      When issuing a query that returns a lot of rows after issuing an Optimize or Analyze
      in a Statement with streaming active (fecthSize = Integer.MIN_VALUE) it loads
      all results in memory producing an OOME

      stmt = connection.createStatement();
      stmt.setFetchSize(Integer.MIN_VALUE);            
      stmt.executeUpdate("DROP TEMPORARY TABLE IF EXISTS tmp.a");               
      stmt.executeUpdate("CREATE TEMPORARY TABLE tmp.a ( a INT )");
      stmt.executeUpdate("INSERT INTO tmp.a VALUES(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1)");
      //optimize internally returns a ResultSet that I do not read
      stmt.executeUpdate("OPTIMIZE TABLE tmp.a");
      final ResultSet rs = stmt.executeQuery("SELECT a from tmp.a");                
      int count = 0;
      //this resultset iteration is not streamed,it is fully loaded on memory
      while (rs.next()) {
          count++;
      }
      System.out.println(count + " rows read");
      rs.close();          
      

      I am not sure if the workaround is to change fetchSize reset in this two places:

      org.mariadb.jdbc.internal.com.read.dao.Results (303) loadFully methd:

       public void loadFully(boolean skip, Protocol protocol) throws SQLException {
              if (fetchSize != 0) {           
                  //fetchSize = 0; comenting this to avoid reading 0 in the next statement
                  if (resultSet != null) {
                ....
      

      and here also is reset

      org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol (1282) removeActiveStreamingResult()

      removes fetch size also in loadFully -> resultSet.fetchRemaining()
      org.mariadb.jdbc.internal.com.read.dao.Results.removeFetchSize()

      Attachments

        Activity

          People

            diego dupin Diego Dupin
            rafalopez79 Rafael Lopez Fernández
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.