Details

    • 10.1.6-1, 10.1.6-2, 10.1.21

    Description

      Scrubbing tests somehow got encryption enabled in their opt files. After removing --innodb-encrypt-tables=ON and --innodb-encrypt-log=ON from innodb_scrub.opt, innodb_scrub_background.opt, innodb_scrub_compressed.opt, these tests start to fail.

      Now that the problematic ‘background scrubbing’ and all scrubbing code that incorrectly skipped redo logging has been removed from MariaDB Server 10.5.2, we should continue where MDEV-15528 left off, and finally enable the scrubbing tests to prove that the theoretically sound way of scrubbing actually works:

      1. Remove the invalidate parameter of btr_free_root()
      2. Introduce a data structure that allows us to mark a range of data pages of a tablespace as freed, even when those pages are not in the buffer pool. This could be attached to fil_space_t.
      3. Ensure that crash recovery will replay the FREE_PAGE records (extend the above mentioned ranges of freed pages), and punch holes in page_compressed tables, or overwrite blocks with zeroes if innodb_immediate_scrub_data_uncompressed is set.
      4. On page flush, try to combine large punch_hole or zero-initialization requests to a small one.
      5. Optional: when freeing the last pages of a tablespace, write a TRIM_PAGES record, and on page flush, invoke ftruncate() to shrink the file. (Do this on any file, when neither scrubbing nor page_compressed is being used.)

      Attachments

        Issue Links

          Activity

            The feature was disabled in the MariaDB 10.5.4 release due to concerns over data corruption (MDEV-22970).

            marko Marko Mäkelä added a comment - The feature was disabled in the MariaDB 10.5.4 release due to concerns over data corruption ( MDEV-22970 ).

            Bad effect 3. from above replays on actual 10.5 too.

            mleich Matthias Leich added a comment - Bad effect 3. from above replays on actual 10.5 too.

            Patch is in bb-10.5-thiru. Please take a look

            thiru Thirunarayanan Balathandayuthapani added a comment - Patch is in bb-10.5-thiru. Please take a look

            I posted some review comments. It looks OK, except that crash recovery fails to scrub freed pages in page_compressed tables when innodb_immediate_scrub_data_uncompressed=OFF (which is the default setting).

            OK to push after addressing my comments.

            marko Marko Mäkelä added a comment - I posted some review comments . It looks OK, except that crash recovery fails to scrub freed pages in page_compressed tables when innodb_immediate_scrub_data_uncompressed=OFF (which is the default setting). OK to push after addressing my comments.

            The tree
            commit 4a47938d9c36605aa9fcbc7d3bbd34bea4e7f5f7 (HEAD, origin/bb-10.5-thiru)
            Author: Thirunarayanan Balathandayuthapani <thiru@mariadb.com>
            Date:   Fri Jul 17 21:03:50 2020 +0530
                - check empty container for range_set_t::iterator::find
            commit 90c308c5e795ceda9c0f644bbde693507b597b74
            Author: Thirunarayanan Balathandayuthapani <thiru@mariadb.com>
            Date:   Fri Jul 17 19:56:33 2020 +0530
                MDEV-22970 Possible corruption of page_compressed tables, or
                           when scrubbing is enabled
            performed well during RQG testing.  The issues seen are in actual 10.5 too.
            

            mleich Matthias Leich added a comment - The tree commit 4a47938d9c36605aa9fcbc7d3bbd34bea4e7f5f7 (HEAD, origin/bb-10.5-thiru) Author: Thirunarayanan Balathandayuthapani <thiru@mariadb.com> Date: Fri Jul 17 21:03:50 2020 +0530 - check empty container for range_set_t::iterator::find commit 90c308c5e795ceda9c0f644bbde693507b597b74 Author: Thirunarayanan Balathandayuthapani <thiru@mariadb.com> Date: Fri Jul 17 19:56:33 2020 +0530 MDEV-22970 Possible corruption of page_compressed tables, or when scrubbing is enabled performed well during RQG testing. The issues seen are in actual 10.5 too.

            People

              thiru Thirunarayanan Balathandayuthapani
              serg Sergei Golubchik
              Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.