Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-12114

install_db shows corruption for rest encryption and innodb_checksum_algorithm=strict_none

Details

    • 10.1.24

    Description

      When trying to install database with rest encryption and innodb_checksum_algorithm=strict_none following errors are shown:

      2017-02-23 12:16:40 139675033013184 [Warning] InnoDB: New log files created, LSN=45883
      		 
      2017-02-23 12:16:40 139675033013184 [Note] InnoDB: Doublewrite buffer not found: creating new
      		 
      2017-02-23 12:16:40 139675033013184 [Note] InnoDB: Doublewrite buffer created
      		 
      2017-02-23 12:16:40 139675033013184 [ERROR] InnoDB:  Page 2 in space N/A (0) maybe corrupted. Post encryption checksum 3735928559 stored [3735928559:3735928559] key_version 1
      		 
      2017-02-23 12:16:40 139675033013184 [ERROR] InnoDB:  Page 2 in space ./ibdata1 (0) maybe corrupted. Post encryption checksum 3735928559 stored [3735928559:3735928559] key_version 1
      		 
      2017-02-23 12:16:40 7f08a0ba0bc0  InnoDB: Assertion failure in thread 139675033013184 in file fsp0fsp.cc line 1765
      		 
      InnoDB: Failing assertion: n != ULINT_UNDEFINED
      		 
      ...
      		 
      mysys/stacktrace.c:268(my_print_stacktrace)[0x5629310fdf86]
      		 
      sql/signal_handler.cc:168(handle_fatal_signal)[0x5629311003d4]
      		 
      srv/srv0start.cc:2837(innobase_start_or_create_for_mysql())[0x5629310e810a]
      		 
      handler/ha_innodb.cc:4439(innobase_init(void*))[0x56293102633d]
      		 
      sql/handler.cc:513(ha_initialize_handlerton(st_plugin_int*))[0x562930defc75]
      		 
      sql/sql_plugin.cc:1408(plugin_initialize(st_mem_root*, st_plugin_int*, int*, char**, bool))[0x562930c943ab]
      		 
      sql/sql_plugin.cc:1678(plugin_init(int*, char**, int))[0x562930c952c2]
      		 
      sql/mysqld.cc:5148(init_server_components())[0x562930bffbad]
      		 
      sql/mysqld.cc:5739(mysqld_main(int, char**))[0x562930c04aa2]

      Other values for innodb_checksum_algorithm are handled properly. As well no problem happens for strict_none without rest encryption.

      Attachments

        1. error.log
          5 kB
          Andrii Nikitin
        2. out
          2.03 MB
          Jan Lindström

        Issue Links

          is caused by

          Bug - A problem which impairs or prevents the functions of the product. MDEV-12113 install_db shows corruption for rest encryption with innodb_data_file_path=ibdata1:3M;

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-21441 Assertion: index->id == btr_page_get_index_id(page) failed in btr_cur_search_to_nth_level

          • Major - Major loss of function.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            View 5 older comments
            jplindst Jan Lindström (Inactive) added a comment -

            Made additional changes that require review in:

            https://github.com/MariaDB/server/commit/d4c8a8b420688b3f63944a2132a6965bb0b79d46

            jplindst Jan Lindström (Inactive) added a comment - Made additional changes that require review in: https://github.com/MariaDB/server/commit/d4c8a8b420688b3f63944a2132a6965bb0b79d46
            jplindst Jan Lindström (Inactive) added a comment - 

            jan@jan-laptop-asus:~/mysql/10.1/mysql-test$ ./mtr --vardir=/dev/shm innodb-compressed-blob --valgrind
            		 
            Logging: ./mtr  --vardir=/dev/shm innodb-compressed-blob --valgrind
            		 
            Turning on valgrind for all executables
            		 
            Running valgrind with options " --tool=memcheck --show-reachable=yes --leak-check=yes --num-callers=16 --quiet --suppressions=/home/jan/mysql/10.1/mysql-test/valgrind.supp "
            		 
            vardir: /dev/shm
            		 
            Checking leftover processes...
            		 
            Removing old var directory...
            		 
            couldn't chmod(0777, /dev/shm): Operation not permitted at /usr/share/perl/5.24/File/Find.pm line 511.
            		 
            Couldn't remove directory '/dev/shm': Permission denied at /usr/share/perl/5.24/File/Find.pm line 511.
            		 
            Creating var directory '/dev/shm'...
            		 
            Checking supported features...
            		 
            MariaDB Version 10.1.24-MariaDB-debug
            		 
             - SSL connections supported
            		 
             - binaries are debug compiled
            		 
            Sphinx 'indexer' binary not found, sphinx suite will be skipped
            		 
            Collecting tests...
            		 
            Installing system database...
            		 
             
            		 
            ==============================================================================
            		 
             
            		 
            TEST                                      RESULT   TIME (ms) or COMMENT
            		 
            --------------------------------------------------------------------------
            		 
             
            		 
            worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
            		 
            encryption.innodb-compressed-blob 'cbc,crc32,innodb_plugin' [ pass ]  75139
            		 
            encryption.innodb-compressed-blob 'cbc,innodb_plugin,none' [ pass ]  77155
            		 
            encryption.innodb-compressed-blob 'cbc,crc32,xtradb' [ pass ]  99573
            		 
            encryption.innodb-compressed-blob 'cbc,none,xtradb' [ pass ]  108279
            		 
            encryption.innodb-compressed-blob 'crc32,ctr,innodb_plugin' [ pass ]  102948
            		 
            encryption.innodb-compressed-blob 'ctr,innodb_plugin,none' [ pass ]  102571
            		 
            encryption.innodb-compressed-blob 'crc32,ctr,xtradb' [ pass ]  90882
            		 
            encryption.innodb-compressed-blob 'ctr,none,xtradb' [ pass ]  88621
            		 
            valgrind_report                          [ pass ]       
            --------------------------------------------------------------------------
            		 
            The servers were restarted 7 times
            		 
            Spent 745.168 of 1024 seconds executing testcases
            		 
             
            		 
            Completed: All 9 tests were successful.

            jplindst Jan Lindström (Inactive) added a comment - jan@jan-laptop-asus:~/mysql/10.1/mysql-test$ ./mtr --vardir=/dev/shm innodb-compressed-blob --valgrind Logging: ./mtr --vardir=/dev/shm innodb-compressed-blob --valgrind Turning on valgrind for all executables Running valgrind with options " --tool=memcheck --show-reachable=yes --leak-check=yes --num-callers=16 --quiet --suppressions=/home/jan/mysql/10.1/mysql-test/valgrind.supp " vardir: /dev/shm Checking leftover processes... Removing old var directory... couldn't chmod(0777, /dev/shm): Operation not permitted at /usr/share/perl/5.24/File/Find.pm line 511. Couldn't remove directory '/dev/shm': Permission denied at /usr/share/perl/5.24/File/Find.pm line 511. Creating var directory '/dev/shm'... Checking supported features... MariaDB Version 10.1.24-MariaDB-debug - SSL connections supported - binaries are debug compiled Sphinx 'indexer' binary not found, sphinx suite will be skipped Collecting tests... Installing system database...   ==============================================================================   TEST RESULT TIME (ms) or COMMENT --------------------------------------------------------------------------   worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019 encryption.innodb-compressed-blob 'cbc,crc32,innodb_plugin' [ pass ] 75139 encryption.innodb-compressed-blob 'cbc,innodb_plugin,none' [ pass ] 77155 encryption.innodb-compressed-blob 'cbc,crc32,xtradb' [ pass ] 99573 encryption.innodb-compressed-blob 'cbc,none,xtradb' [ pass ] 108279 encryption.innodb-compressed-blob 'crc32,ctr,innodb_plugin' [ pass ] 102948 encryption.innodb-compressed-blob 'ctr,innodb_plugin,none' [ pass ] 102571 encryption.innodb-compressed-blob 'crc32,ctr,xtradb' [ pass ] 90882 encryption.innodb-compressed-blob 'ctr,none,xtradb' [ pass ] 88621 valgrind_report [ pass ] -------------------------------------------------------------------------- The servers were restarted 7 times Spent 745.168 of 1024 seconds executing testcases   Completed: All 9 tests were successful.
            marko Marko Mäkelä added a comment -

            Please address my additional review comments and also manually test an upgrade from innodb_checksum_algorithm=none: encrypted tables should be unreadable (invalid checksum)

            Add mysql-test for the following:

            • restart after changing the setting innodb_checksum_algorithm=innodb to innodb_checksum_algorithm=strict_crc32: must get reports of checksum mismatch, for both encrypted and unencrypted tables, also for ROW_FORMAT=COMPRESSED
            • restart after changing the setting innodb_checksum_algorithm=crc32 to strict_innodb: No warnings for encrypted tables, also not for ROW_FORMAT=COMPRESSED (because encrypted tables will always use crc32)
            marko Marko Mäkelä added a comment - Please address my additional review comments and also manually test an upgrade from innodb_checksum_algorithm=none: encrypted tables should be unreadable (invalid checksum) Add mysql-test for the following: restart after changing the setting innodb_checksum_algorithm=innodb to innodb_checksum_algorithm=strict_crc32: must get reports of checksum mismatch, for both encrypted and unencrypted tables, also for ROW_FORMAT=COMPRESSED restart after changing the setting innodb_checksum_algorithm=crc32 to strict_innodb: No warnings for encrypted tables, also not for ROW_FORMAT=COMPRESSED (because encrypted tables will always use crc32)
            jplindst Jan Lindström (Inactive) added a comment -

            As fil_crypt_verify_crypt_checksum still supports stored none checksum to allow upgrade from earlier versions it will not report any invalid checksum:

            jan@jan-laptop-asus:~/mysql/10.1-bugs/scripts/mysql_install_db --no-defaults --innodb-data-file-path="ibdata1:3M:autoextend" --basedir=/usr/local/mysql --plugin-load-add=file_key_management --file-key-management-filename=/home/jan/mysql/10.1//mysql-test/std_data/keys.txt --innodb-encrypt-tables --innodb-encrypt-log --innodb-checksum-algorithm=none
            		 
             
            		 
            jan@jan-laptop-asus:~/mysql/10.1$ sql/mysqld --innodb-data-file-path="ibdata1:3M:autoextend" --basedir=/usr/local/mysql --plugin-load-add=file_key_management --file-key-management-filename=/home/jan/mysql/10.1//mysql-test/std_data/keys.txt --innodb-encrypt-tables --innodb-encrypt-log --datadir=/home/jan/jpl/data
            		 
             
            		 
            2017-06-01 13:23:27 140387115673088 [Note] sql/mysqld (mysqld 10.1.24-MariaDB-debug) starting as process 26428 ...
            		 
            2017-06-01 13:23:27 140387115673088 [Note] InnoDB: innodb_empty_free_list_algorithm has been changed to legacy because of small buffer pool size. In order to use backoff, increase buffer pool at least up to 20MB.
            		 
             
            		 
            2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Using mutexes to ref count buffer pool pages
            		 
            2017-06-01 13:23:27 140387115673088 [Note] InnoDB:  InnoDB: !!!!!!!! UNIV_DEBUG switched on !!!!!!!!!
            		 
            2017-06-01 13:23:27 140387115673088 [Note] InnoDB:  InnoDB: !!!!!!!! UNIV_SYNC_DEBUG switched on !!!!!!!!!
            		 
            2017-06-01 13:23:27 140387115673088 [Note] InnoDB: The InnoDB memory heap is disabled
            		 
            2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
            		 
            2017-06-01 13:23:27 140387115673088 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
            		 
            2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Compressed tables use zlib 1.2.8
            		 
            2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Using Linux native AIO
            		 
            2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Using SSE crc32 instructions
            		 
            2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Initializing buffer pool, size = 128.0M
            		 
            2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Completed initialization of buffer pool
            		 
            2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Created tablespace for space 0 name ./ibdata1 key_id 1 encryption 0.
            		 
            2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Created tablespace for space 4294967280 name ./ib_logfile0 key_id 0 encryption 0.
            		 
            2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Created tablespace for space 4294967281 name arch_log_space key_id 0 encryption 0.
            		 
            2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Highest supported file format is Barracuda.
            		 
            2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Created tablespace for space 3 name mysql/gtid_slave_pos key_id 0 encryption 0.
            		 
            2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Created tablespace for space 2 name mysql/innodb_index_stats key_id 0 encryption 0.
            		 
            2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Created tablespace for space 1 name mysql/innodb_table_stats key_id 0 encryption 0.
            		 
            2017-06-01 13:23:29 140387115673088 [Note] InnoDB: 128 rollback segment(s) are active.
            		 
            2017-06-01 13:23:29 140387115673088 [Note] InnoDB: Waiting for purge to start
            		 
            2017-06-01 13:23:29 140387115673088 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.36-82.0 started; log sequence number 1617148
            		 
            2017-06-01 13:23:29 140386362193664 [Note] InnoDB: Dumping buffer pool(s) not yet started
            		 
            2017-06-01 13:23:29 140387115673088 [Note] Plugin 'FEEDBACK' is disabled.
            		 
            2017-06-01 13:23:29 140387115673088 [Note] Server socket created on IP: '::'.
            		 
            2017-06-01 13:23:29 140387115673088 [Note] sql/mysqld: ready for connections.
            		 
            Version: '10.1.24-MariaDB-debug'  socket: '/tmp/mysql.sock'  port: 3306  Source distribution
            		 
            2017-06-01 13:25:04 140387114973952 [Note] sql/mysqld: Normal shutdown
            		 
             
            		 
            2017-06-01 13:25:04 140387114973952 [Note] Event Scheduler: Purging the queue. 0 events
            		 
            2017-06-01 13:25:04 140386341213952 [Note] InnoDB: FTS optimize thread exiting.
            		 
            2017-06-01 13:25:04 140387114973952 [Note] InnoDB: Starting shutdown...
            		 
            2017-06-01 13:25:05 140387114973952 [Note] InnoDB: Waiting for page_cleaner to finish flushing of buffer pool
            		 
            2017-06-01 13:25:06 140387114973952 [Note] InnoDB: Shutdown completed; log sequence number 1617158
            		 
            2017-06-01 13:25:06 140387114973952 [Note] sql/mysqld: Shutdown complete

            jplindst Jan Lindström (Inactive) added a comment - As fil_crypt_verify_crypt_checksum still supports stored none checksum to allow upgrade from earlier versions it will not report any invalid checksum: jan@jan-laptop-asus:~/mysql/10.1-bugs/scripts/mysql_install_db --no-defaults --innodb-data-file-path="ibdata1:3M:autoextend" --basedir=/usr/local/mysql --plugin-load-add=file_key_management --file-key-management-filename=/home/jan/mysql/10.1//mysql-test/std_data/keys.txt --innodb-encrypt-tables --innodb-encrypt-log --innodb-checksum-algorithm=none   jan@jan-laptop-asus:~/mysql/10.1$ sql/mysqld --innodb-data-file-path="ibdata1:3M:autoextend" --basedir=/usr/local/mysql --plugin-load-add=file_key_management --file-key-management-filename=/home/jan/mysql/10.1//mysql-test/std_data/keys.txt --innodb-encrypt-tables --innodb-encrypt-log --datadir=/home/jan/jpl/data   2017-06-01 13:23:27 140387115673088 [Note] sql/mysqld (mysqld 10.1.24-MariaDB-debug) starting as process 26428 ... 2017-06-01 13:23:27 140387115673088 [Note] InnoDB: innodb_empty_free_list_algorithm has been changed to legacy because of small buffer pool size. In order to use backoff, increase buffer pool at least up to 20MB.   2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Using mutexes to ref count buffer pool pages 2017-06-01 13:23:27 140387115673088 [Note] InnoDB: InnoDB: !!!!!!!! UNIV_DEBUG switched on !!!!!!!!! 2017-06-01 13:23:27 140387115673088 [Note] InnoDB: InnoDB: !!!!!!!! UNIV_SYNC_DEBUG switched on !!!!!!!!! 2017-06-01 13:23:27 140387115673088 [Note] InnoDB: The InnoDB memory heap is disabled 2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2017-06-01 13:23:27 140387115673088 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier 2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Compressed tables use zlib 1.2.8 2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Using Linux native AIO 2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Using SSE crc32 instructions 2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Initializing buffer pool, size = 128.0M 2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Completed initialization of buffer pool 2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Created tablespace for space 0 name ./ibdata1 key_id 1 encryption 0. 2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Created tablespace for space 4294967280 name ./ib_logfile0 key_id 0 encryption 0. 2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Created tablespace for space 4294967281 name arch_log_space key_id 0 encryption 0. 2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Highest supported file format is Barracuda. 2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Created tablespace for space 3 name mysql/gtid_slave_pos key_id 0 encryption 0. 2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Created tablespace for space 2 name mysql/innodb_index_stats key_id 0 encryption 0. 2017-06-01 13:23:27 140387115673088 [Note] InnoDB: Created tablespace for space 1 name mysql/innodb_table_stats key_id 0 encryption 0. 2017-06-01 13:23:29 140387115673088 [Note] InnoDB: 128 rollback segment(s) are active. 2017-06-01 13:23:29 140387115673088 [Note] InnoDB: Waiting for purge to start 2017-06-01 13:23:29 140387115673088 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.36-82.0 started; log sequence number 1617148 2017-06-01 13:23:29 140386362193664 [Note] InnoDB: Dumping buffer pool(s) not yet started 2017-06-01 13:23:29 140387115673088 [Note] Plugin 'FEEDBACK' is disabled. 2017-06-01 13:23:29 140387115673088 [Note] Server socket created on IP: '::'. 2017-06-01 13:23:29 140387115673088 [Note] sql/mysqld: ready for connections. Version: '10.1.24-MariaDB-debug' socket: '/tmp/mysql.sock' port: 3306 Source distribution 2017-06-01 13:25:04 140387114973952 [Note] sql/mysqld: Normal shutdown   2017-06-01 13:25:04 140387114973952 [Note] Event Scheduler: Purging the queue. 0 events 2017-06-01 13:25:04 140386341213952 [Note] InnoDB: FTS optimize thread exiting. 2017-06-01 13:25:04 140387114973952 [Note] InnoDB: Starting shutdown... 2017-06-01 13:25:05 140387114973952 [Note] InnoDB: Waiting for page_cleaner to finish flushing of buffer pool 2017-06-01 13:25:06 140387114973952 [Note] InnoDB: Shutdown completed; log sequence number 1617158 2017-06-01 13:25:06 140387114973952 [Note] sql/mysqld: Shutdown complete
            jplindst Jan Lindström (Inactive) added a comment -

            MDEV-12114: install_db shows corruption for rest encryption and innodb_checksum_algorithm=strict_none

            Problem was that checksum check resulted false positives that page is
            both not encrypted and encryted when checksum_algorithm was
            strict_none.

            Encrypton checksum will use only crc32 regardless of setting.

            buf_zip_decompress: If compression fails report a error message
            containing the space name if available (not available during import).
            And note if space could be encrypted.

            buf_page_get_gen: Do not assert if decompression fails,
            instead unfix the page and return NULL to upper layer.

            fil_crypt_calculate_checksum: Use only crc32 method.

            fil_space_verify_crypt_checksum: Here we need to check
            crc32, innodb and none method for old datafiles.

            fil_space_release_for_io: Allow null space.

            encryption.innodb-compressed-blob is now run with crc32 and none
            combinations.

            Note that with none and strict_none method there is not really
            a way to detect page corruptions and page corruptions after
            decrypting the page with incorrect key.

            New test innodb-checksum-algorithm to test different checksum
            algorithms with encrypted, row compressed and page compressed
            tables.

            jplindst Jan Lindström (Inactive) added a comment - MDEV-12114 : install_db shows corruption for rest encryption and innodb_checksum_algorithm=strict_none Problem was that checksum check resulted false positives that page is both not encrypted and encryted when checksum_algorithm was strict_none. Encrypton checksum will use only crc32 regardless of setting. buf_zip_decompress: If compression fails report a error message containing the space name if available (not available during import). And note if space could be encrypted. buf_page_get_gen: Do not assert if decompression fails, instead unfix the page and return NULL to upper layer. fil_crypt_calculate_checksum: Use only crc32 method. fil_space_verify_crypt_checksum: Here we need to check crc32, innodb and none method for old datafiles. fil_space_release_for_io: Allow null space. encryption.innodb-compressed-blob is now run with crc32 and none combinations. Note that with none and strict_none method there is not really a way to detect page corruptions and page corruptions after decrypting the page with incorrect key. New test innodb-checksum-algorithm to test different checksum algorithms with encrypted, row compressed and page compressed tables.

            People

              jplindst Jan Lindström (Inactive)
              anikitin Andrii Nikitin (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.