Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-26476

InnoDB is missing futex support on some platforms

Details

    Description

      In MDEV-20612, MDEV-24142 and related tasks, the InnoDB synchronization primitives were rewritten. On Linux, Microsoft Windows, and OpenBSD, we make use of futex-like system calls. (On Microsoft Windows, srw_mutex and srw_lock are simple wrappers of SRWLOCK; on other platforms they are implemented with futexes.)

      According to https://shift.click/blog/futex-like-apis/ there are a few more platforms that we should be able to support:

      • FreeBSD: _umtx_op (UMTX_OP_WAIT_UINT_PRIVATE, UMTX_OP_WAKE_PRIVATE)
      • DragonflyBSD: umtx_sleep, umtx_wakeup
      • Apple macOS: __ulock_wait, __ulock_wake

      When futex support is missing, we will fall back to the SUX_LOCK_GENERIC implementation that is based on native mutexes, condition variables and native rw-locks when available. Not only will this increase the memory footprint, but lock_sys_t::hash_latch::release() could be a scalability bottleneck, because it uses a single mutex and condition variable to ‘emulate’ futex so that we can avoid increasing the memory footprint of lock_sys.rec_hash.

      To be determined: Are any futex-like operations available on other operating systems where MariaDB Server might be able to run?

      • AIX
      • HP-UX
      • Solaris or its derivatives, such as Illumos or OpenIndiana

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-26781 InnoDB hangs on SPATIAL INDEX when using SUX_LOCK_GENERIC

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-32788 CMAKE_BUILD_TYPE=Debug build failure with SUX_LOCK_GENERIC

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-34678 pthread_mutex_init() without pthread_mutex_destroy() if SUX_LOCK_GENERIC

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. MDEV-20612 Improve InnoDB lock_sys scalability

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-25404 read-only performance regression in 10.6

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-27222 FreeBSD 13.0-p4 build error mariadb-10.6.5

          • Major - Major loss of function.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            marko Marko Mäkelä created issue -
            marko Marko Mäkelä made changes -
            Field Original Value New Value
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 124574 ] MariaDB v4 [ 143116 ]
            marko Marko Mäkelä made changes -
            Component/s Platform FreeBSD [ 10139 ]
            Assignee Daniel Black [ danblack ] Marko Mäkelä [ marko ]
            marko Marko Mäkelä made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            marko Marko Mäkelä added a comment -

            On DragonFly BSD v6.2.1, I had to apply the following patch to get MariaDB Server to compile:

            diff --git a/client/mysql.cc b/client/mysql.cc
            		 
            index 37f506a99cd..ad8fb9ccfa6 100644
            		 
            --- a/client/mysql.cc
            		 
            +++ b/client/mysql.cc
            		 
            @@ -104,7 +104,7 @@ extern "C" {
            		 
             #define USE_POPEN
            		 
             }
            		 
             
            -#ifdef HAVE_VIDATTR
            		 
            +#if defined HAVE_VIDATTR && !defined __DragonFly__
            		 
             static int have_curses= 0;
            		 
             static void my_vidattr(chtype attrs)
            		 
             {

            On FreeBSD 13, the compilation worked out of the box. On both systems, I successfully tested the implementation with

            storage/innobase/unittest/innodb_sync-t
            		 
            cd mysql-test
            		 
            ./mtr --parallel=8 --suite=innodb_gis --force

            Without futex support, the test suite would have failed due to MDEV-26781.

            marko Marko Mäkelä added a comment - On DragonFly BSD v6.2.1, I had to apply the following patch to get MariaDB Server to compile: diff --git a/client/mysql.cc b/client/mysql.cc index 37f506a99cd..ad8fb9ccfa6 100644 --- a/client/mysql.cc +++ b/client/mysql.cc @@ -104,7 +104,7 @@ extern "C" { #define USE_POPEN } -#ifdef HAVE_VIDATTR +#if defined HAVE_VIDATTR && !defined __DragonFly__ static int have_curses= 0; static void my_vidattr(chtype attrs) { On FreeBSD 13, the compilation worked out of the box. On both systems, I successfully tested the implementation with storage/innobase/unittest/innodb_sync-t cd mysql-test ./mtr --parallel=8 --suite=innodb_gis --force Without futex support, the test suite would have failed due to MDEV-26781 .
            marko Marko Mäkelä made changes -
            issue.field.resolutiondate 2022-02-18 13:32:41.0 2022-02-18 13:32:41.884
            marko Marko Mäkelä made changes -
            Fix Version/s 10.9.0 [ 27113 ]
            Fix Version/s 10.6.8 [ 27506 ]
            Fix Version/s 10.7.4 [ 27504 ]
            Fix Version/s 10.8.3 [ 27502 ]
            Fix Version/s 10.6 [ 24028 ]
            Resolution Fixed [ 1 ]
            Status In Progress [ 3 ] Closed [ 6 ]
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -

            People

              marko Marko Mäkelä
              marko Marko Mäkelä
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.