Uploaded image for project: 'MariaDB Connector/J'
  1. MariaDB Connector/J
  2. CONJ-125

Optimize cached ResultSet memory footprint

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Closed (View Workflow)
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.1.7
    • Fix Version/s: 1.4.0
    • Component/s: Other
    • Labels:
      None
    • Sprint:
      Sprint connector/j 1.3.0

      Description

      How to reproduce:

      1. Download the attached test case BigResultSetMemoryOptimization.java
      2. Use Oracle Java (perhaps also the JDK is required, I’m not sure)
      3. Run the test case (for example in Eclipse) as is to get a heap dump for 100K rows of ResultSet using MariaDB JDBC
      4. Edit the test case to use Mysql JDBC (see instructions in the comment in the file)
      5. Run the test case to get a heap dump using Mysql JDBC
      6. Open the two heap dumps side by side in Eclipse Memory Analyzer https://eclipse.org/mat/ (I recommend to download the standalone version).
      7. Run the Leak Suspects Report to get values for the “Retained Heap” column
      8. Open Histogram view (the second icon).
      9. Sort on the “Retained Heap” column.
      10. Select the row that represents the main java object holding the memory In MariaDB it is org.mariadb.jdbc.internal.common.queryresults.CachedSelectResult, in Mysql it is com.mysql.jdbc.JDBC4ResultSet.

      Actual:
      MariaDB is using 41 226 960 bytes distributed on about 1,5 M objects
      Mysql is using 26 829 400 bytes distributed on about 0,9 M objects

      In other words: MariaDB is using 53 % more memory and 67 % more objects for this specific table pattern.

      Expected:
      For MariaDB JDBC to be a drop in replacement for Mysql I would expect that the memory foot print (in terms of bytes and objects) to be about equivalent. Would it be possible to optimize this?

      I have run into memory problems for this exact table structure but with 5,7M rows. I know about the stream ResultSet feature and that might be a workaround for me in this case, but so far I have not gotten it to work, perhaps due to some bug in either my code or in MariaDB JDBC. If I find a bug in MariaDB ResultSet stream feature I will write a separate ticket for this.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              diego dupin Diego Dupin
              Reporter:
              lennartschedin Lennart Schedin
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.