Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-27774

Reduce scalability bottlenecks in mtr_t::commit()

Details

    Description

      MDEV-14425 improved write performance less than it was hoped. We are still seeing log_sys.mutex and buf_pool.mutex dominating the wait times.

      A prominent bottleneck was identified to be log_sys.mutex between log_sys.append_prepare() and log_close(). To protect the copying of data to log_sys.buf, we only need a shared log_sys.latch, which can replace log_sys.mutex and log_sys.flush_order_mutex. The log_sys.lsn and the position of log_sys.buf needs to be protected by a new log_sys.lsn_lock. Writes of log_sys.buf to the file system will have to acquire exclusive log_sys.latch.

      While working on this code, we can also make mtr_t::commit() invoke buf_pool.page_cleaner_wakeup() and update Innodb_buffer_pool_write_requests at most once per call.

      Attachments

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. MDEV-27868 buf_pool.flush_list is in the wrong order

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-27876 Server building fails on MacOS caused by srw_lock_impl<false> template specialization

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-27969 buf_page_cleaner not getting signaled event if dirty_pct > innodb_max_dirty_pages_pct_lwm

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-28043 Race condition between mtr_t::commit() and checkpoint

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-32374 log_sys.lsn_lock is a performance hog

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-34689 Redo log corruption at high load

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-27866 Switching log_sys.latch to use spin based variant

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-27910 Internal compiler error on CentOS 7 ARMv8 GCC 4.8.5

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-28111 Redo log writes are being buffered on Linux for no good reason

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. MDEV-14425 Change the InnoDB redo log format to reduce write amplification

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-14462 Confusing error message: ib_logfiles are too small for innodb_thread_concurrency=0

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-21923 LSN allocation is a bottleneck

          • Critical - Crashes, loss of data, severe memory leak.
          • In Progress

          Bug - A problem which impairs or prevents the functions of the product. MDEV-33670 Redo log overwritten/corrupted

          • Critical - Crashes, loss of data, severe memory leak.
          • Confirmed

          Activity

            Ascending order - Click to sort in descending order
            marko Marko Mäkelä added a comment -

            Thanks to wlad for the idea, and for further optimization suggestions.

            marko Marko Mäkelä added a comment - Thanks to wlad for the idea, and for further optimization suggestions.
            marko Marko Mäkelä added a comment -

            greenman, please note that with this change, the minimum value of innodb_log_file_size will increase from 1M to 4M. That was done in order to remove a very unlikely startup error condition, as well as a very unlikely error message output from log_sys.append().

            marko Marko Mäkelä added a comment - greenman , please note that with this change, the minimum value of innodb_log_file_size will increase from 1M to 4M. That was done in order to remove a very unlikely startup error condition, as well as a very unlikely error message output from log_sys.append() .
            greenman Ian Gilfillan added a comment -

            Thanks, updated

            greenman Ian Gilfillan added a comment - Thanks, updated

            People

              marko Marko Mäkelä
              marko Marko Mäkelä
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.