Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-12288

Reset DB_TRX_ID when the history is removed, to speed up MVCC

    XMLWordPrintable

Details

    • 10.3.1-2

    Description

      The InnoDB clustered index record system columns DB_TRX_ID,DB_ROLL_PTR are used by multi-versioning and for determining if a record is implicitly locked. After the history is no longer needed, these columns can safely be reset to 0 and 1<<55 (to indicate a fresh insert).

      When a reader sees 0 in the DB_TRX_ID column, it can instantly determine that the record is present the read view. There is no need to acquire the transaction system mutex to check if the transaction exists, because writes can never be conducted by a transaction whose ID is 0.

      The persistent InnoDB undo log is split into two parts: insert_undo and update_undo. The insert_undo log is discarded at transaction commit or rollback, and the update_undo log is processed by the purge subsystem. As part of this change, we must merge the two types of undo logs into one, and the purge subsystem will reset the DB_TRX_ID whenever a clustered index record is ‘touched’.

      Upgrade considerations

      This will change the persistent InnoDB file formats, not only in the undo log and redo log, but also in the data files. There are some debug assertions that would not allow any record to contain DB_TRX_ID=0.

      A new redo log format tag must be introduced so that the writes of the system columns can be properly redo-logged. (See MDEV-11432, MDEV-11782.) This will prevent a startup of an older version with the new-version redo logs. We may also prevent a crash recovery of MariaDB 10.2 files with the newer version. (Crash recovery of files from 10.1 or earlier versions is already prevented in 10.2.)

      The undo log format will be changed as well. To be able to get rid of legacy code, InnoDB startup should detect if any old-format undo logs are present. If yes, startup will be refused, and the user must perform a slow shutdown (SET GLOBAL innodb_fast_shutdown=0) with the old server in order to empty the undo logs.

      A proof-of-concept implementation for 10.2 consists of 4 consecutive commits. It is missing any of the above-mentioned upgrade logic.

      Attachments

        Issue Links

          blocks

          Bug - A problem which impairs or prevents the functions of the product. MDEV-13697 DB_TRX_ID is not always reset when the history is removed

          • Major - Major loss of function.
          • Closed
          causes

          Bug - A problem which impairs or prevents the functions of the product. MDEV-13475 InnoDB: Failing assertion: lsn == log_sys->lsn || srv_force_recovery == SRV_FORCE_NO_LOG_REDO

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-13476 TRX_UNDO_PAGE_TYPE mismatch when writing undo log after upgrade

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-13495 Crash in rollback of a recovered INSERT transaction after upgrade

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-13536 DB_TRX_ID is not actually being reset when the history is removed

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-13654 Various crashes due to DB_TRX_ID mismatch in table-rebuilding ALTER TABLE…LOCK=NONE

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-13820 trx_id_check() fails during row_log_table_apply()

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-15370 Upgrade fails when both insert_undo and update_undo exist for recovered transactions

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-15912 InnoDB: Failing assertion: purge_sys.tail.commit <= purge_sys.rseg->last_commit upon upgrade from 10.0 or 10.1 to 10.3

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-16017 galera mtr tests fail with sst_xtrabackup-v2

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-16273 innodb.alter_kill fails in buildbot with Unknown storage engine 'InnoDB'

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-18966 Transaction recovery may be broken after upgrade to 10.3

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-21158 trx_undo_seg_free() is never redo-logged

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-13542 Crashing on a corrupted page is unhelpful

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-14848 MariaDB 10.3 refuses InnoDB crash-upgrade from MariaDB 10.2

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-17745 innodb.innodb_stats_persistent failed in buildbot with wrong result

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-20804 Use the sequence engine in main.index_merge_innodb and main.index_merge_myisam

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-26475 InnoDB must not lock delete-marked records

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MDEV-27437 Galera snapshot transfer fails to upgrade between some major versions

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Task - A task that needs to be done. MDEV-515 innodb bulk insert

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-8139 Fix scrubbing

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. MDEV-11432 Change the informational redo log format tag to say "MariaDB 10.2.3"

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. MDEV-11658 Simpler, faster IMPORT of InnoDB tables

          • Critical - Crashes, loss of data, severe memory leak.
          • Open

          Task - A task that needs to be done. MDEV-11782 Redefine the innodb_encrypt_log format

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-12258 InnoDB: Fix the bogus debug assertion introduced in MDEV-12219

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. MDEV-13269 Upgrade tests for InnoDB undo log

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. MDEV-15522 Change galera suite MTR tests to use mariabackup instead of xtrabackup

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-16168 Performance regression on sysbench write benchmarks from 10.2 to 10.3

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-17044 mysql_upgrade is causing crash of mysqld

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-23017 range query performance regression in 10.5.4

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-24153 innodb assertion - trx_undo_t *trx_undo_mem_create_at_db_start(trx_rseg_t *, ulint, ulint, mtr_t *): Assertion `type == 2' faile

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. MDEV-33067 SCN(Sequence Commit Number) based MVCC

          • Major - Major loss of function.
          • Open
          mentioned in

          Page Loading...

          Activity

            People

              marko Marko Mäkelä
              marko Marko Mäkelä
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.