Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-35049

fix innodb-adaptive-hash-index scalability with multiple threads

Details

    Description

      InnoDB adaptive indexes has performance problems when running queries in parallel that uses InnoDB hash indexes (at least on the same tables).

      Here is an example of running the same SELECT query with 10-256 concurrent threads.
      (The query normally takes about 0.36 seconds):

      Total execution with 10 threads time: 1 seconds
      Total execution with 32 threads time: 1 seconds
      Total execution with 64 threads time: 3 seconds
      Total execution with 128 threads time: 5 seconds
      Total execution with 256 threads time: 9 seconds

      When using --innodb-adaptive-hash-index=1 we get:

      Total execution with 10 threads time: 1 seconds
      Total execution with 32 threads time: 5 seconds
      Total execution with 64 threads time: 9 seconds
      Total execution with 128 threads time: 18 seconds
      Total execution with 256 threads time: 50 seconds

      MySQL 5.7 does not have this issue. It has roughly the same speed with and without the hash indexes.

      In the above query, there was a driving table with a 400 rows doing lookups in a child table using 200 lookups on a secondary keys for every row from the driving table.
      Using innodb-adaptive-hash-index=1 speeds up the query with 37% when running a query in one thread but it slows down drastically when running 32-256 concurrent threads.

      Attachments

        1. ahi.sh
          1 kB
          Marko Mäkelä
        2. innodb-hashtest.sh
          1 kB
          Michael Widenius

        Issue Links

          Activity

            Transition Time In Source Status Execution Times
            Marko Mäkelä made transition -
            Open In Progress
            19h 5m 1
            Marko Mäkelä made transition -
            In Progress In Testing
            1d 3h 42m 1
            Marko Mäkelä made transition -
            Stalled In Progress
            8s 1
            Marko Mäkelä made transition -
            In Progress In Review
            30d 23h 40m 1
            Vladislav Lesin made transition -
            In Review Stalled
            9d 16h 3m 1
            Sergei Golubchik made transition -
            Stalled In Testing
            27d 12h 17m 2
            Matthias Leich made transition -
            In Testing Stalled
            29d 18h 38m 3
            Marko Mäkelä made transition -
            Stalled Closed
            2d 5h 47m 1

            People

              marko Marko Mäkelä
              monty Michael Widenius
              Votes:
              1 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.