Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-13559

encryption.innodb-redo-badkey failed in buildbot

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • 10.3(EOL)
    • 10.2.9, 10.3.1
    • Encryption, Tests
    • None
    • kvm-bintar-quantal-amd64

    Description

      http://buildbot.askmonty.org/buildbot/builders/kvm-bintar-quantal-amd64/builds/6902/steps/test/logs/stdio

      encryption.innodb-redo-badkey 'cbc,innodb' w4 [ fail ]  Found warnings/errors in server log file!
              Test ended at 2017-08-17 10:04:15
      line
      2017-08-17 10:04:12 140553760634624 [ERROR] InnoDB: The page [page id: space=7, page number=3] in file './test/t4.ibd' cannot be decrypted.
      2017-08-17 10:04:12 140553760634624 [ERROR] InnoDB: The page [page id: space=6, page number=3] in file './test/t3.ibd' cannot be decrypted.
      2017-08-17 10:04:12 140553760634624 [ERROR] InnoDB: Unable to decompress ./test/t2.ibd[page id: space=5, page number=3]
      2017-08-17 10:04:12 140553760634624 [ERROR] InnoDB: Database page corruption on disk or a failed file read of tablespace test/t2 page [page id: space=5, page number=3]. You may have to recover from a backup.
      2017-08-17 10:04:12 140553760634624 [ERROR] InnoDB: Unable to decompress ./test/t1.ibd[page id: space=4, page number=3]
      2017-08-17 10:04:12 140553760634624 [ERROR] InnoDB: Database page corruption on disk or a failed file read of tablespace test/t1 page [page id: space=4, page number=3]. You may have to recover from a backup.
      ^ Found warnings in /usr/local/mariadb-10.3.1-linux-x86_64/mysql-test/var/4/log/mysqld.1.err
      
      

      Attachments

        Issue Links

          Activity

            I believe I observed a similar failure when testing MDEV-13536.

            I did touch this test in my work-around for MDEV-13542, maybe in vain, because this test is abruptly killing the server, not doing a clean shutdown and restart.

            Looking at the server error log
            http://buildbot.askmonty.org/buildbot/builders/kvm-bintar-quantal-amd64/builds/6902/steps/test/logs/mysqld.1.err.4
            I determined that this crash occurs on the first restart, after no_checkpoint_end.inc killed the server. This server startup is expected to fail, and it did fail:

            --source ../../suite/innodb/include/no_checkpoint_end.inc
             
            --echo # restart
            --error 1
            -- source include/start_mysqld.inc
            

            The only problem here is that we got an unexpected message about decompression failure. There is only a suppression about a decryption failure:

            call mtr.add_suppression("InnoDB: The page \[page id: space=[1-9][0-9]*, page number=[0-9]+\] in file test/t[1-4] cannot be decrypted");
            

            The proper solution would seem to be to add the missing suppression.

            The reason why this message is being issued should be that with MDEV-13536 (and MDEV-12288), purge now has more work to do, and thus more redo log records can be emitted before the crash. Purge would clear the DB_TRX_ID columns of each INSERT. Before MDEV-12288, purge did not have any access to any undo log generated by INSERT to an empty table.

            I was not able to repeat the failure locally, even with some small tricks.

            marko Marko Mäkelä added a comment - I believe I observed a similar failure when testing MDEV-13536 . I did touch this test in my work-around for MDEV-13542 , maybe in vain, because this test is abruptly killing the server, not doing a clean shutdown and restart. Looking at the server error log http://buildbot.askmonty.org/buildbot/builders/kvm-bintar-quantal-amd64/builds/6902/steps/test/logs/mysqld.1.err.4 I determined that this crash occurs on the first restart, after no_checkpoint_end.inc killed the server. This server startup is expected to fail, and it did fail: --source ../../suite/innodb/include/no_checkpoint_end.inc   --echo # restart --error 1 -- source include/start_mysqld.inc The only problem here is that we got an unexpected message about decompression failure. There is only a suppression about a decryption failure: call mtr.add_suppression("InnoDB: The page \[page id: space=[1-9][0-9]*, page number=[0-9]+\] in file test/t[1-4] cannot be decrypted"); The proper solution would seem to be to add the missing suppression. The reason why this message is being issued should be that with MDEV-13536 (and MDEV-12288 ), purge now has more work to do, and thus more redo log records can be emitted before the crash. Purge would clear the DB_TRX_ID columns of each INSERT. Before MDEV-12288 , purge did not have any access to any undo log generated by INSERT to an empty table. I was not able to repeat the failure locally, even with some small tricks.

            I hope bb-10.2-marko will run fine with the added suppressions.
            The test is injecting faults, and it is not completely unplausible to get decompression or read errors in addition to decryption errors.

            marko Marko Mäkelä added a comment - I hope bb-10.2-marko will run fine with the added suppressions. The test is injecting faults, and it is not completely unplausible to get decompression or read errors in addition to decryption errors.

            People

              marko Marko Mäkelä
              alice Alice Sherepa
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.