Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-26762

Compressed table remains accessible without provider after unclean restart

Details

    Description

      It isn't clear yet whether this can cause a real problem, but it appears somewhat unexpected and vaguely disturbing in the context of provider plugins, so I think at least it is worth thinking what possible side effects it may have.

      The test case is derived from plugins.compression, with only important difference that it sets shutdown_timeout to zero, thus disallowing the server to shut down normally.

      Then, even though after the server is restarted without the provider at all, SELECT from the compressed table succeeds instead of failing with ER_NO_SUCH_TABLE_IN_ENGINE.

      --source include/have_innodb.inc
      		 
       
      		 
      let $restart_parameters = --plugin-load-add=provider_snappy;
      		 
      source include/restart_mysqld.inc;
      		 
       
      		 
      set global innodb_compression_algorithm = snappy;
      		 
      create table t1 (a int, b text) engine=InnoDB page_compressed=1;
      		 
       
      		 
      insert t1 (a, b) values (0, repeat("abc", 100));
      		 
       
      		 
      let $restart_parameters =;
      		 
      --let $shutdown_timeout= 0
      		 
      source include/restart_mysqld.inc;
      		 
       
      		 
      #error ER_NO_SUCH_TABLE_IN_ENGINE;
      		 
      select a, left(b, 9), length(b) from t1;
      		 
       
      		 
      drop table t1;

      preview-10.7-MDEV-12933-provider-plugins 67a8fca9b

      		 
      # restart
      		 
      select a, left(b, 9), length(b) from t1;
      		 
      a	left(b, 9)	length(b)
      		 
      0	abcabcabc	300
      		 
      drop table t1;
      		 
      bug.t 'innodb'                           [ pass ]   2017
      		 
      ***Warnings generated in error logs during shutdown after running tests: bug.t
      		 
       
      		 
      2021-10-04 21:46:14 0 [Warning] InnoDB: Page compression errors: 3

      The warning on shutdown also hints that something has gone not quite as expected.

      Attachments

        Issue Links

          is caused by

          Task - A task that needs to be done. MDEV-12933 sort out the compression library chaos

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-26773 MariaBackup backup does not work with compression providers

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            elenst Elena Stepanova added a comment -

            I have no objections against demoting it and removing it from 10.7 RC scope once it has been looked at.

            elenst Elena Stepanova added a comment - I have no objections against demoting it and removing it from 10.7 RC scope once it has been looked at.
            serg Sergei Golubchik added a comment - - edited

            Likely it's the same thing that caused mariabackup tests to pass without mariabackup actually supporting compression (MDEV-26773). Perhaps fresh table data are read from the uncompressed log before they're written to a compressed table space.

            I don't see what can we do about it

            serg Sergei Golubchik added a comment - - edited Likely it's the same thing that caused mariabackup tests to pass without mariabackup actually supporting compression ( MDEV-26773 ). Perhaps fresh table data are read from the uncompressed log before they're written to a compressed table space. I don't see what can we do about it
            marko Marko Mäkelä added a comment -

            Crash recovery must read the first page of any .ibd file that is referred to by the redo log. At that point, if the file is in the innodb_checksum_algorithm=full_crc32 format, we will know the compression algorithm. If the compression library is not available, we can (and in my opinion should) refuse to start up the server unless innodb_force_recovery has been set to a nonzero value.

            marko Marko Mäkelä added a comment - Crash recovery must read the first page of any .ibd file that is referred to by the redo log. At that point, if the file is in the innodb_checksum_algorithm=full_crc32 format, we will know the compression algorithm. If the compression library is not available, we can (and in my opinion should) refuse to start up the server unless innodb_force_recovery has been set to a nonzero value.
            serg Sergei Golubchik added a comment -

            I don't think it even needs to be fixed, if InnoDB reads uncompressed pages from the redo log — well good for them, why should we introduce artificial limitations and prevent it from doing that?

            serg Sergei Golubchik added a comment - I don't think it even needs to be fixed, if InnoDB reads uncompressed pages from the redo log — well good for them, why should we introduce artificial limitations and prevent it from doing that?
            serg Sergei Golubchik added a comment -

            on a related matter, we can do a warning on purge, to make the user aware that some pages are lost.

            serg Sergei Golubchik added a comment - on a related matter, we can do a warning on purge, to make the user aware that some pages are lost.
            marko Marko Mäkelä added a comment -

            I do not think that it is easy to issue a meaningful warning for tables that are unaccessible to purge. We cannot add a table to cache if we cannot load the tablespace. If we do not cache the table, how can we avoid issuing umpteen warnings, one per to-be-purged undo log record?

            It is not ‘some pages are lost’ but ‘some history was not purged‘. That is, some garbage may remain in the table, should the table be made available at some later point. The situation could be rectified by executing OPTIMIZE TABLE or ALTER TABLE…FORCE. It is only a matter of failing to reclaim some space, or failing to reset transaction identifiers (MDEV-12288). Purging undo log records without cleaning up the index pages should not affect correctness, only performance.

            marko Marko Mäkelä added a comment - I do not think that it is easy to issue a meaningful warning for tables that are unaccessible to purge. We cannot add a table to cache if we cannot load the tablespace. If we do not cache the table, how can we avoid issuing umpteen warnings, one per to-be-purged undo log record? It is not ‘some pages are lost’ but ‘some history was not purged‘. That is, some garbage may remain in the table, should the table be made available at some later point. The situation could be rectified by executing OPTIMIZE TABLE or ALTER TABLE…FORCE . It is only a matter of failing to reclaim some space, or failing to reset transaction identifiers ( MDEV-12288 ). Purging undo log records without cleaning up the index pages should not affect correctness, only performance.
            serg Sergei Golubchik added a comment -

            It's not a bug that InnoDB can read some pages of the compressed table even if the compression isn't available. Recently modified pages reside in the uncompressed redo log, it's normal that that are accessible even if compression is disabled.

            serg Sergei Golubchik added a comment - It's not a bug that InnoDB can read some pages of the compressed table even if the compression isn't available. Recently modified pages reside in the uncompressed redo log, it's normal that that are accessible even if compression is disabled.

            People

              serg Sergei Golubchik
              elenst Elena Stepanova
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.