Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-8139

Fix scrubbing

Details

    • 10.1.6-1, 10.1.6-2, 10.1.21

    Description

      Scrubbing tests somehow got encryption enabled in their opt files. After removing --innodb-encrypt-tables=ON and --innodb-encrypt-log=ON from innodb_scrub.opt, innodb_scrub_background.opt, innodb_scrub_compressed.opt, these tests start to fail.

      Now that the problematic ‘background scrubbing’ and all scrubbing code that incorrectly skipped redo logging has been removed from MariaDB Server 10.5.2, we should continue where MDEV-15528 left off, and finally enable the scrubbing tests to prove that the theoretically sound way of scrubbing actually works:

      1. Remove the invalidate parameter of btr_free_root()
      2. Introduce a data structure that allows us to mark a range of data pages of a tablespace as freed, even when those pages are not in the buffer pool. This could be attached to fil_space_t.
      3. Ensure that crash recovery will replay the FREE_PAGE records (extend the above mentioned ranges of freed pages), and punch holes in page_compressed tables, or overwrite blocks with zeroes if innodb_immediate_scrub_data_uncompressed is set.
      4. On page flush, try to combine large punch_hole or zero-initialization requests to a small one.
      5. Optional: when freeing the last pages of a tablespace, write a TRIM_PAGES record, and on page flush, invoke ftruncate() to shrink the file. (Do this on any file, when neither scrubbing nor page_compressed is being used.)

      Attachments

        Issue Links

          blocks

          Bug - A problem which impairs or prevents the functions of the product. MDEV-17596 Assertion `block->page.flush_observer == __null || block->page.flush_observer == observer' failed in buf_flush_note_modification

          • Major - Major loss of function.
          • Closed
          causes

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22911 Fix the Valgrind & MSAN instrumentation of MDEV-8139

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22931 mtr_t::mtr_t() allocates some memory

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22970 Possible corruption of page_compressed tables, or when scrubbing is enabled

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-24445 Using innodb_undo_tablespaces corrupts system tablespace

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-24765 fseg_free_extent fails to call buf_page_free() for the whole segment

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-31253 Freed data pages are not always being scrubbed

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-35494 fil_space_t::fil_space_t() is potentially unsafe with GCC -flifetime-dse

          • Major - Major loss of function.
          • Closed
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. MDEV-20476 Assertion `!"wrong page type"' failed in row_log_table_low or ER_NOT_KEYFILE upon concurrent ALTER and UPDATE

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-11802 innodb.innodb_bug14676111 fails in buildbot due to InnoDB purge failing to start when there is work to do

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. MDEV-12288 Reset DB_TRX_ID when the history is removed, to speed up MVCC

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-13536 DB_TRX_ID is not actually being reset when the history is removed

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. MDEV-14425 Change the InnoDB redo log format to reduce write amplification

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-15528 Avoid writing freed InnoDB pages

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-32151 innodb_immediate_scrub_data_uncompressed=ON may fail to scrub temporary tablespace pages

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. MDEV-11068 Review which innodb_compression_algorithm to support in binary packages

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-15949 InnoDB: Failing assertion: space->n_pending_ops == 0 in fil_delete_tablespace upon DROP TABLE

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-20813 Assertion `!srv_safe_truncate || !newest_lsn || fil_page_get_type(page)' or `!newest_lsn || fil_page_get_type(page)' failed in buf_flush_init_for_writing

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-24780 [FATAL] InnoDB: Trying to read page number <...> with encryption and innodb-background-scrub-data-uncompressed

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Activity

            Transition Time In Source Status Execution Times
            Jan Lindström (Inactive) made transition -
            Open In Progress
            		14d 17h 3m 1
            Jan Lindström (Inactive) made transition -
            In Progress Stalled
            		36d 6h 11m 1
            Marko Mäkelä made transition -
            Stalled Confirmed
            		1h 44m 1
            Thirunarayanan Balathandayuthapani made transition -
            Confirmed In Progress
            		1167d 2h 1
            Thirunarayanan Balathandayuthapani made transition -
            In Progress Closed
            		2d 17h 1
            Marko Mäkelä made transition -
            Closed Stalled
            		10d 3h 31m 1
            Thirunarayanan Balathandayuthapani made transition -
            Stalled In Progress
            		572d 5h 6m 3
            Thirunarayanan Balathandayuthapani made transition -
            In Progress In Review
            		69d 4h 20m 3
            Marko Mäkelä made transition -
            In Review Stalled
            		21d 11h 59m 3
            Thirunarayanan Balathandayuthapani made transition -
            Stalled Closed
            		3d 1h 57m 1

            People

              thiru Thirunarayanan Balathandayuthapani
              serg Sergei Golubchik
              Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.