Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-21452

Use condition variables and normal mutexes instead of InnoDB os_event and mutex

    XMLWordPrintable

Details

    Description

      investigation suggested by marko on zulip after reading http://smalldatum.blogspot.com/2020/01/it-is-all-about-constant-factors.html

      no patches - built just straight from 10.4.10 release tag. Built with cmake -DMUTEXTYPE=$type -DCMAKE_PREFIX_INSTALL=/scratch/mariadb-10.4.10-$type $HOME/mariadb-10.4.10
      Distro ubuntu-18.04 compiler.

      TPCCRunner test:

      POWER8, altivec supported - 20 core, 8 thread/core

      		 
      $ tail  fullrun-master-fstn4-mariadb-10.4.10-futex-28444.txt   fullrun-master-fstn4-mariadb-10.4.10-event-48215.txt  fullrun-master-fstn4-mariadb-10.4.10-sys-60112.txt
      		 
      ==> fullrun-master-fstn4-mariadb-10.4.10-futex-28444.txt <==
      		 
       
      		 
                    timestamp          tpm      avg_rt      max_rt   avg_db_rt   max_db_rt
      		 
                      average  2519939.03       50.01         687       50.00         687
      		 
       
      		 
         All phase Transactions: 100508512
      		 
      Warmup phase Transactions: 24910341
      		 
         Run phase Transactions: 75598171
      		 
       
      		 
      Waiting slaves to terminate users.
      		 
      All slaves disconnected.
      		 
       
      		 
      ==> fullrun-master-fstn4-mariadb-10.4.10-event-48215.txt <==
      		 
       
      		 
                    timestamp          tpm      avg_rt      max_rt   avg_db_rt   max_db_rt
      		 
                      average  1944470.28       63.97         782       63.96         782
      		 
       
      		 
         All phase Transactions: 466885487
      		 
      Warmup phase Transactions: 350217270
      		 
         Run phase Transactions: 116668217
      		 
       
      		 
      Waiting slaves to terminate users.
      		 
      All slaves disconnected.
      		 
       
      		 
      ==> fullrun-master-fstn4-mariadb-10.4.10-sys-60112.txt <==
      		 
       
      		 
                    timestamp          tpm      avg_rt      max_rt   avg_db_rt   max_db_rt
      		 
                      average  2412875.70       51.72         846       51.71         846
      		 
       
      		 
         All phase Transactions: 579124495
      		 
      Warmup phase Transactions: 434351953
      		 
         Run phase Transactions: 144772542
      		 
       
      		 
      Waiting slaves to terminate users.
      		 
      All slaves disconnected.

      Note: while futex was run for much less time - innodb_buffer_pool_dump_pct=100 from the last run and the 30 minutes was receiving consistent throughput.

      Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz - 22 core, 4 thread/core

      		 
      ==> fullrun-master-ka4-mariadb-10.4.10-event-rr-50070.txt <==
      		 
       
      		 
                    timestamp          tpm      avg_rt      max_rt   avg_db_rt   max_db_rt
      		 
                      average  3354020.40       34.35         487       34.32         487
      		 
       
      		 
         All phase Transactions: 132952152
      		 
      Warmup phase Transactions: 32331540
      		 
         Run phase Transactions: 100620612
      		 
       
      		 
      Waiting slaves to terminate users.
      		 
      All slaves disconnected.
      		 
       
      		 
      ==> fullrun-master-ka4-mariadb-10.4.10-futex-rr-63543.txt <==
      		 
       
      		 
                    timestamp          tpm      avg_rt      max_rt   avg_db_rt   max_db_rt
      		 
                      average  3362135.83       33.50         604       33.48         604
      		 
       
      		 
         All phase Transactions: 131218680
      		 
      Warmup phase Transactions: 30354605
      		 
         Run phase Transactions: 100864075
      		 
       
      		 
      Waiting slaves to terminate users.
      		 
      All slaves disconnected.
      		 
       
      		 
      ==> fullrun-master-ka4-mariadb-10.4.10-sys-rr-56865.txt <==
      		 
       
      		 
                    timestamp          tpm      avg_rt      max_rt   avg_db_rt   max_db_rt
      		 
                      average  3363324.87       34.13         996       34.11         996
      		 
       
      		 
         All phase Transactions: 132642637
      		 
      Warmup phase Transactions: 31742891
      		 
         Run phase Transactions: 100899746
      		 
       
      		 
      Waiting slaves to terminate users.
      		 
      All slaves disconnected.

      Attachments

        1. master.properties.60
          0.8 kB
        2. MDEV-21452.ods
          80 kB
        3. MDEV-21452.ods
          60 kB
        4. MDEV-21452-nbl.ods
          66 kB
        5. my.cnf
          2 kB
        6. Screenshot from 2020-03-26 12-08-54.png
          Screenshot from 2020-03-26 12-08-54.png
          88 kB

        Issue Links

          blocks

          Task - A task that needs to be done. MDEV-23169 Optimize InnoDB code around mutexes assuming InnoDB locks are uncontended

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          causes

          Bug - A problem which impairs or prevents the functions of the product. MDEV-24637 fts_slots is being accessed after it gets freed.

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-26779 reduce lock_sys.wait_mutex contention by using spinloop construct

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-29896 mariadb-backup got stuck with --throttle option doing incremental backup

          • Major - Major loss of function.
          • Closed
          is blocked by

          Task - A task that needs to be done. MDEV-16232 Use fewer mini-transactions

          • Major - Major loss of function.
          • Stalled

          Task - A task that needs to be done. MDEV-16406 Refactor the InnoDB record locks

          • Major - Major loss of function.
          • Open

          Task - A task that needs to be done. MDEV-20612 Improve InnoDB lock_sys scalability

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22871 Contention on the buf_pool.page_hash

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-23888 Potential server hang on replication with InnoDB

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-24142 rw_lock_t has unnecessarily complex wait logic

          • Major - Major loss of function.
          • Closed
          relates to

          Task - A task that needs to be done. MDEV-7109 Add support for INFORMATION_SCHEMA.INNODB_SEMAPHORE_WAITS

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-15653 [Draft] Assertion `lock_word <= 0x20000000' failed in rw_lock_get_writer

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-18250 [Draft] Server crashed in dirname_length / innobase_basename / sync_arr_fill_sys_semphore_waits_table

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-24426 fil_crypt_thread keep spinning even if innodb_encryption_rotate_key_age=0

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-24449 Corruption of system tablespace or last recovered page

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-24845 Oddities around innodb_fatal_semaphore_wait_threshold and global.innodb_disallow_writes

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-24973 Performance schema duplicates rarely executed code for mutex operations

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-25267 Reported latching order violation in ibuf_remove_free_page()

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-25890 Trying to lock mutex ... when the mutex was already locked | SIGABRT in safe_mutex_lock | Hangs and cross-mysqld interaction

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-27985 buf_flush_freed_pages() causes InnoDB to hang

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-28157 SAFE_MUTEX and DBUG corrupt memory | SIGABRT in __libc_message

          • Major - Major loss of function.
          • Confirmed

          Task - A task that needs to be done. MDEV-15706 Remove information_schema.innodb_metrics, innodb_monitor_enable, innodb_monitor_disable

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MDEV-15752 Possible race between DDL and accessing I_S.INNODB_TABLESPACES_ENCRYPTION

          • Major - Major loss of function.
          • Confirmed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-21330 Lock monitor doesn't print a semaphore's last reserved thread in non-debug builds and INFORMATION_SCHEMA.INNODB_SYS_SEMAPHORE_WAITS is totally broken

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22782 SUMMARY: AddressSanitizer: unknown-crash storage/innobase/trx/trx0trx.cc:566 in trx_t::commit_state()

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-23399 10.5 performance regression with IO-bound tpcc

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-23472 ASAN use-after-poison in LatchCounter::Count::reset

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-24630 MY_RELAX_CPU assembly instruction upgrade/research for memory barrier on ARM

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-30951 Make small facelift to Innotop perl script

          • Major - Major loss of function.
          • Open

          Task - A task that needs to be done. MDEV-32065 Always check whether lock is free at first to optimize InnoDB mutexes

          • Major - Major loss of function.
          • Open

          Activity

            People

              marko Marko Mäkelä
              danblack Daniel Black
              Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.