Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-12353

Efficient InnoDB redo log record format

    XMLWordPrintable

Details

    Description

      The InnoDB crash recovery performance can be improved a little while not changing the file format. MDEV-12699 removed unnecessary reads of pages that can be initialized via the redo log. MDEV-19586 will make recovery read the pages in more sequential order.

      We should fix some fundamental issues that exist with the current InnoDB redo log record format:

      • Records do not contain their length. When buffering records, we must painstakingly parse entire records in order to determine the length. This idea was mentioned in MySQL Bug#82937.
      • For B-tree operations, we are writing a lot of redundant data for mlog_parse_index(). We should use a lower-level format and lower-level apply functions. MySQL Bug#82176 merely speeds up the code around mlog_parse_index().
      • If a mini-transaction is writing multiple records to a page, the page identifier is being repeated for every record. We should omit the page identifier if multiple consecutive records are modifying the same page.

      In this task, we will only improve the redo log record format. Format changes to the redo log blocks and files will be covered by MDEV-14425.

      Attachments

        1. prt
          60 kB
          Matthias Leich
        2. simp_page_rec_set_n_owned.test
          0.7 kB
          Matthias Leich
        3. simp_page_rec_set_n_owned-master.opt
          0.1 kB
          Matthias Leich

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. MDEV-21724 Optimize page_cur_insert_rec_low() redo logging

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-21725 Optimize btr_page_reorganize_low() redo logging

          • Major - Major loss of function.
          • Stalled

          Bug - A problem which impairs or prevents the functions of the product. MDEV-21727 Optimize redo logging for ROW_FORMAT=COMPRESSED

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MDEV-21744 Assertion `!rec_offs_nth_sql_null(offsets, n)' failed in btr_cur_upd_rec_in_place

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-21748 ASAN use-after-poison in PageBulk::insertPage on table rebuild

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-21749 storage/innobase/page/page0cur.cc:1306: rec_t* page_cur_insert_rec_low(const page_cur_t*, dict_index_t*, const rec_t*, offset_t*, mtr_t*): Assertion `rdm - rd + bd <= insert_buf + rec_size' failed.

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-21779 Assertion `!memcmp(insert_buf, rec - extra_size, rec_size)' failed in page_cur_insert_rec_low

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-21850 ASAN use-after-poison in page_cur_insert_rec_low

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-21892 Assertion `index != clust_index || row_get_rec_trx_id(rec, index, offsets)' failed in row_search_mvcc

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-21893 Assertion `log_block_get_start_lsn(lsn, log_block_get_hdr_no(buf)) == lsn' failed upon attempt to start on encrypted datadir from previous versions

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-21945 Assertion w==OPT failed in trx_purge_add_undo_to_history()

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-21949 key rotation for innodb_encrypt_log is not working in 10.5

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22107 Restore accidentally orphaned MTR_MEMO_MODIFY

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22108 Crash recovery fails with [ERROR] InnoDB: Malformed log record

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22242 B-trees can become extremely skewed

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-23806 Undo page corruption on recovery

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-24196 WITH_UBSAN runtime error: member call on null pointer of type 'struct log_phys_t'

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-24412 Mariadb 10.5: InnoDB: Upgrade after a crash is not supported

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-24652 mtr fails while reusing the cached undo log block

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-24848 Assertion `rlen < llen' failed in log_phys_t::apply_status log_phys_t::apply upon MariaBackup prepare

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-27059 page_zip_dir_insert() may corrupt ROW_FORMAT=COMPRESSED tables

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-27444 Perform backup prepare using mariabbackup 10.4 version when performing rolling upgrade on joiner node with 10.5

          • Major - Major loss of function.
          • Stalled

          Bug - A problem which impairs or prevents the functions of the product. MDEV-27486 Refuse Galera SST if major version of donor and joiner are different

          • Critical - Crashes, loss of data, severe memory leak.
          • Stalled

          Bug - A problem which impairs or prevents the functions of the product. MDEV-28731 Race condition on log checkpoint

          • Major - Major loss of function.
          • Closed
          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. MDEV-19747 Deprecate and ignore innodb_log_optimize_ddl

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. MDEV-21174 Refactor mlog_write_ulint, mlog_memset, mlog_write_string

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-21351 Replace recv_sys.heap with list of buf_block_t*

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-21674 purge_sys.stop() no longer waits for purge workers to complete

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-12699 Improve crash recovery of corrupted data pages

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. MDEV-14425 Change the InnoDB redo log format to reduce write amplification

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Task - A task that needs to be done. MDEV-18976 Implement a CHECKSUM redo log record for improved validation

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. MDEV-19586 Replace recv_sys_t::addr_hash with a std::map

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-20562 btr_cur_open_at_rnd_pos() fails to return error for corrupted page

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-20584 Clearing delete-mark on R-tree pages does not appear crash-safe

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-20636 Potential SPATIAL INDEX corruption with ROW_FORMAT=COMPRESSED

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MDEV-21024 InnoDB is issuing redundant writes to redo log

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22126 Rename confusing constant mtr_t::OPT

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-23136 InnoDB init fail after upgrade from 10.4 to 10.5

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-23986 [ERROR] [FATAL] InnoDB: Page ... name ... page_type ... key_version 1 lsn ... compressed_len ...

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-27437 Galera snapshot transfer fails to upgrade between some major versions

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-28256 Galera SST fails with 'InnoDB: Upgrade after a crash is not supported'

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-29153 2022-07-22 18:12:56 0x7f3319fff700 InnoDB: Assertion failure in file /home/buildbot/buildbot/build/mariadb-10.4.25/storage/innobase/page/page0cur.cc line 11

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-32144 Debug assertion failure w == MAYBE_NOP in mtr_t::memcpy(), trx_undo_write_xid()

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-32445 InnoDB may corrupt its log before upgrading it on startup

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-33274 The test encryption.innodb-redo-nokeys often fails

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-15274 innodb_gis.rtree_recovery failed in buildbot with error on check table

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-19747 Deprecate and ignore innodb_log_optimize_ddl

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-20608 innodb_log_optimize_ddl=OFF may omit some redo log

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-21899 INSERT into a secondary index with zero-data-length key is not crash-safe

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22097 Not applying DELETE_ROW_FORMAT_REDUNDANT due to corruption

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-23474 InnoDB fails to start with Invalid log block checksum after unsetting innodb_log_checksums dynamically

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Task - A task that needs to be done. MDEV-24719 backport MDEV-24705 from 10.5 to 10.2, 10.3, 10.4

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-26322 Last binlog file and position are "empty" in mariabackup --prepare output

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-31042 Certain tables are missing in innodb storage engine of a TSE enabled database after an incremental backup restore

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          mentioned in

          Page Loading...

          Page Loading...

          Page Loading...

          Page Loading...

          Page Loading...

          Page Loading...

          Activity

            People

              marko Marko Mäkelä
              marko Marko Mäkelä
              Votes:
              4 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.