Uploaded image for project: 'MariaDB Connector/J'
  1. MariaDB Connector/J
  2. CONJ-550

Losing streaming ability in statement after executing Optimize

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.1.1
    • Fix Version/s: 2.2.1, 1.7.1
    • Component/s: Other
    • Labels:
      None

      Description

      When issuing a query that returns a lot of rows after issuing an Optimize or Analyze
      in a Statement with streaming active (fecthSize = Integer.MIN_VALUE) it loads
      all results in memory producing an OOME

      stmt = connection.createStatement();
      stmt.setFetchSize(Integer.MIN_VALUE);            
      stmt.executeUpdate("DROP TEMPORARY TABLE IF EXISTS tmp.a");               
      stmt.executeUpdate("CREATE TEMPORARY TABLE tmp.a ( a INT )");
      stmt.executeUpdate("INSERT INTO tmp.a VALUES(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1)");
      //optimize internally returns a ResultSet that I do not read
      stmt.executeUpdate("OPTIMIZE TABLE tmp.a");
      final ResultSet rs = stmt.executeQuery("SELECT a from tmp.a");                
      int count = 0;
      //this resultset iteration is not streamed,it is fully loaded on memory
      while (rs.next()) {
          count++;
      }
      System.out.println(count + " rows read");
      rs.close();          
      

      I am not sure if the workaround is to change fetchSize reset in this two places:

      org.mariadb.jdbc.internal.com.read.dao.Results (303) loadFully methd:

       public void loadFully(boolean skip, Protocol protocol) throws SQLException {
              if (fetchSize != 0) {           
                  //fetchSize = 0; comenting this to avoid reading 0 in the next statement
                  if (resultSet != null) {
                ....
      

      and here also is reset

      org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol (1282) removeActiveStreamingResult()

      removes fetch size also in loadFully -> resultSet.fetchRemaining()
      org.mariadb.jdbc.internal.com.read.dao.Results.removeFetchSize()

        Attachments

          Activity

            People

            Assignee:
            diego dupin Diego Dupin
            Reporter:
            rafalopez79 Rafael Lopez Fernández
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Git Integration