Uploaded image for project: 'MariaDB Connector/J'
  1. MariaDB Connector/J
  2. CONJ-715

OOM happening when querying large datasets

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Blocker
    • Resolution: Not a Bug
    • Affects Version/s: 2.2.1
    • Fix Version/s: N/A
    • Component/s: performance
    • Labels:
      None

      Description

      I am using the following

      • AWS Aurora MySQL 5.6
      • mariadb-java-client 2.2.1
      • springframework 4.3.16

      I am currently using fetchsize of 1000 in JdbcTemplate. For large size tables, when using select queries, I am experiencing OOM. I could confirm that Spring is able to propagate the fetchSize to the driver.

      When I started debugging, I found the following observation in SelectResultSet in the driver code. When fetchSize > 0, the execution flow proceeds like

      • nextStreamingValue
      • addStreamingValue
      • readNextValue
      • growDataArray

      So essentially, SelectResultSet data variable keeps growing when next is called on the ResultSet. So OOM is ought to happen eventually. BTW, I am using the default TYPE_FORWARD_ONLY scroll type.

      Could anyone confirm if my understanding is correct? Is there a way to query large datasets without OOM?

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              diego dupin Diego Dupin
              Reporter:
              chaitan64arun Arun Chaitanya Miriappalli
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.