Uploaded image for project: 'MariaDB Connector/J'
  1. MariaDB Connector/J
  2. CONJ-125

Optimize cached ResultSet memory footprint

    XMLWordPrintable

Details

    • Task
    • Status: Closed (View Workflow)
    • Minor
    • Resolution: Fixed
    • 1.1.7
    • 1.4.0
    • Other
    • None
    • Sprint connector/j 1.3.0

    Description

      How to reproduce:

      1. Download the attached test case BigResultSetMemoryOptimization.java
      2. Use Oracle Java (perhaps also the JDK is required, I’m not sure)
      3. Run the test case (for example in Eclipse) as is to get a heap dump for 100K rows of ResultSet using MariaDB JDBC
      4. Edit the test case to use Mysql JDBC (see instructions in the comment in the file)
      5. Run the test case to get a heap dump using Mysql JDBC
      6. Open the two heap dumps side by side in Eclipse Memory Analyzer https://eclipse.org/mat/ (I recommend to download the standalone version).
      7. Run the Leak Suspects Report to get values for the “Retained Heap” column
      8. Open Histogram view (the second icon).
      9. Sort on the “Retained Heap” column.
      10. Select the row that represents the main java object holding the memory In MariaDB it is org.mariadb.jdbc.internal.common.queryresults.CachedSelectResult, in Mysql it is com.mysql.jdbc.JDBC4ResultSet.

      Actual:
      MariaDB is using 41 226 960 bytes distributed on about 1,5 M objects
      Mysql is using 26 829 400 bytes distributed on about 0,9 M objects

      In other words: MariaDB is using 53 % more memory and 67 % more objects for this specific table pattern.

      Expected:
      For MariaDB JDBC to be a drop in replacement for Mysql I would expect that the memory foot print (in terms of bytes and objects) to be about equivalent. Would it be possible to optimize this?

      I have run into memory problems for this exact table structure but with 5,7M rows. I know about the stream ResultSet feature and that might be a workaround for me in this case, but so far I have not gotten it to work, perhaps due to some bug in either my code or in MariaDB JDBC. If I find a bug in MariaDB ResultSet stream feature I will write a separate ticket for this.

      Attachments

        Issue Links

          Activity

            People

              diego dupin Diego Dupin
              lennartschedin Lennart Schedin
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.