Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-18644

Support FULL_CRC32 for compressed pages.

Details

    Description

      Allow compressed page in full_crc32 format. The format of full_crc32 is also follows:

      • Set the MSB of FIL_PAGE_TYPE as 1 for compressed page
      • Allocate the 10 bits for storing compressed (length + (4 + 63)) / 64. The 4 extra bytes are for storing the checksum over the compressed (and optionally encrypted) stream. The checksum would be stored in the last 4 bytes of a 64-byte aligned block.

      FSP_SPACE_FLAGS already contains COMPRESSED_PAGE_ALGORITHM. So that InnoDB shouldn't allow the tablespace to open if particular algorithm is not supported.

      The following should be fixed as part of this task:

      • ha_innobase::check_if_supported_inplace_alter() must allow instant change of page_compressed=1
      • buf_dblwr_check_page_lsn() must evaluate fil_space_t::full_crc32()
      • More use of #define FIL_* or #define FSP_ should be replaced with inline member functions of fil_space_t

      Attachments

        Issue Links

          blocks

          Task - A task that needs to be done. MDEV-11916 Page compression - use smaller writes, avoid trimming/zeroing rest of the page if possible

          • Major - Major loss of function.
          • Open
          causes

          Bug - A problem which impairs or prevents the functions of the product. MDEV-23986 [ERROR] [FATAL] InnoDB: Page ... name ... page_type ... key_version 1 lsn ... compressed_len ...

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-18958 InnoDB buffer pool load is opening an inaccessible data file

          • Major - Major loss of function.
          • Stalled

          Task - A task that needs to be done. MDEV-19534 Make innodb_checksum_algorithm=full_crc32 by default, and remove innodb_checksums

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-20588 Mariabackup incorrectly thinks a compressed table is corrupted

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-23199 page_compression flag is missing for full_crc32 tablespace

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-27091 log_phys_t::apply_status log_phys_t::apply(const buf_block_t&, uint16_t&) const: Assertion `mach_read_from_4(frame + 34U) == block.page.id().space()' failed during CrashRecovery

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MDEV-28168 Compressed page is read for non-compressed space during recovery

          • Major - Major loss of function.
          • Confirmed

          Task - A task that needs to be done. MDEV-12026 Support encrypted SPATIAL INDEX

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            thiru Thirunarayanan Balathandayuthapani created issue -
            marko Marko Mäkelä made changes -
            Field Original Value New Value
            Description Allow compressed page in full_crc32 format. The format of full_crc32 is also follows:
            i) Set the MSB of FIL_PAGE_TYPE as 1 for compressed page
            ii) Allocate the 10 bits for storing compressed length

            FSP_SPACE_FLAGS already contains COMPRESSED_PAGE_ALGORITHM. So that
            InnoDB shouldn't allow the tablespace to open if particular algorithm is not supported.             		Allow compressed page in full_crc32 format. The format of full_crc32 is also follows:
            * Set the MSB of FIL_PAGE_TYPE as 1 for compressed page
            * Allocate the 10 bits for storing compressed {{(length + (4 + 63)) / 64}}. The 4 extra bytes are for storing the checksum over the compressed (and optionally encrypted) stream. The checksum would be stored in the last 4 bytes of a 64-byte aligned block.

            {{FSP_SPACE_FLAGS}} already contains {{COMPRESSED_PAGE_ALGORITHM}}. So that InnoDB shouldn't allow the tablespace to open if particular algorithm is not supported.

            The following should be fixed as part of this task:
            * {{ha_innobase::check_if_supported_inplace_alter()}} must allow instant change of {{page_compressed=1}}
            * {{buf_dblwr_check_page_lsn()}} must evaluate {{fil_space_t::full_crc32()}}
            * More use of {{#define FIL_*}} or {{#define FSP_}} should be replaced with inline member functions of {{fil_space_t}}
            serg Sergei Golubchik made changes -
            Fix Version/s 10.4 [ 22408 ]
            Fix Version/s 10.4.4 [ 23310 ]
            thiru Thirunarayanan Balathandayuthapani made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            thiru Thirunarayanan Balathandayuthapani added a comment -

            patch is in bb-10.4-MDEV-12026.

            thiru Thirunarayanan Balathandayuthapani added a comment - patch is in bb-10.4- MDEV-12026 .
            thiru Thirunarayanan Balathandayuthapani made changes -
            Assignee Thirunarayanan Balathandayuthapani [ thiru ] Marko Mäkelä [ marko ]
            Status In Progress [ 3 ] In Review [ 10002 ]
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä added a comment -

            I pushed some fixes and cleanup to the branch. Once we get QA approval, I will squash to 1 commit and push to 10.4.

            marko Marko Mäkelä added a comment - I pushed some fixes and cleanup to the branch. Once we get QA approval, I will squash to 1 commit and push to 10.4.
            marko Marko Mäkelä made changes -
            Status In Review [ 10002 ] Stalled [ 10000 ]
            marko Marko Mäkelä made changes -
            issue.field.resolutiondate 2019-03-18 12:10:12.0 2019-03-18 12:10:12.075
            marko Marko Mäkelä made changes -
            Fix Version/s 10.4.4 [ 23310 ]
            Fix Version/s 10.4 [ 22408 ]
            Resolution Fixed [ 1 ]
            Status Stalled [ 10000 ] Closed [ 6 ]
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 92699 ] MariaDB v4 [ 133870 ]
            marko Marko Mäkelä made changes -

            People

              marko Marko Mäkelä
              thiru Thirunarayanan Balathandayuthapani
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.