Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-11222

encryption.encrypt_and_grep failed in buildbot on P8

Details

    • Bug
    • Status: Closed (View Workflow)
    • Minor
    • Resolution: Fixed
    • 10.1(EOL)
    • 10.1.23
    • Tests
    • None

    Description

      http://buildbot.askmonty.org/buildbot/builders/p8-rhel7-bintar-debug/builds/1784/steps/test/logs/stdio

      encryption.encrypt_and_grep 'xtradb'     w4 [ fail ]  Found warnings/errors in server log file!
              Test ended at 2016-10-29 10:55:02
      line
      InnoDB: Error: trying to do an operation on a dropped tablespace 4
      ^ Found warnings in /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysql-test/var/4/log/mysqld.1.err
      ok
       
       - saving '/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysql-test/var/4/log/encryption.encrypt_and_grep-xtradb/' to '/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysql-test/var/log/encryption.encrypt_and_grep-xtradb/'
      

      Attachments

        Issue Links

          Activity

            Once MDEV-11738 simplifies and corrects the concurrent access protection of fil_space_t::crypt_data this issue should be fixed. Specifically, the interlocking between fil_space_t::n_pending_ops and fil_space_t::stop_ios should prevent concurrent access to a tablespace that is being dropped, and the dropping of a tablespace while concurrent access is in progress.

            marko Marko Mäkelä added a comment - Once MDEV-11738 simplifies and corrects the concurrent access protection of fil_space_t::crypt_data this issue should be fixed. Specifically, the interlocking between fil_space_t::n_pending_ops and fil_space_t::stop_ios should prevent concurrent access to a tablespace that is being dropped, and the dropping of a tablespace while concurrent access is in progress.

            commit 765a43605a42c069ede604826ede2d93d72c4fdd
            Author: Jan Lindström <jan.lindstrom@mariadb.com>
            Date: Wed Apr 26 15:19:16 2017 +0300

            MDEV-12253: Buffer pool blocks are accessed after they have been freed

            Problem was that bpage was referenced after it was already freed
            from LRU. Fixed by adding a new variable encrypted that is
            passed down to buf_page_check_corrupt() and used in
            buf_page_get_gen() to stop processing page read.

            This patch should also address following test failures and
            bugs:

            MDEV-12419: IMPORT should not look up tablespace in
            PageConverter::validate(). This is now removed.

            MDEV-10099: encryption.innodb_onlinealter_encryption fails
            sporadically in buildbot

            MDEV-11420: encryption.innodb_encryption-page-compression
            failed in buildbot

            MDEV-11222: encryption.encrypt_and_grep failed in buildbot on P8

            Removed dict_table_t::is_encrypted and dict_table_t::ibd_file_missing
            and replaced these with dict_table_t::file_unreadable. Table
            ibd file is missing if fil_get_space(space_id) returns NULL
            and encrypted if not. Removed dict_table_t::is_corrupted field.

            Ported FilSpace class from 10.2 and using that on buf_page_check_corrupt(),
            buf_page_decrypt_after_read(), buf_page_encrypt_before_write(),
            buf_dblwr_process(), buf_read_page(), dict_stats_save_defrag_stats().

            Added test cases when enrypted page could be read while doing
            redo log crash recovery. Also added test case for row compressed
            blobs.

            btr_cur_open_at_index_side_func(),
            btr_cur_open_at_rnd_pos_func(): Avoid referencing block that is
            NULL.

            buf_page_get_zip(): Issue error if page read fails.

            buf_page_get_gen(): Use dberr_t for error detection and
            do not reference bpage after we hare freed it.

            buf_mark_space_corrupt(): remove bpage from LRU also when
            it is encrypted.

            buf_page_check_corrupt(): @return DB_SUCCESS if page has
            been read and is not corrupted,
            DB_PAGE_CORRUPTED if page based on checksum check is corrupted,
            DB_DECRYPTION_FAILED if page post encryption checksum matches but
            after decryption normal page checksum does not match. In read
            case only DB_SUCCESS is possible.

            buf_page_io_complete(): use dberr_t for error handling.

            buf_flush_write_block_low(),
            buf_read_ahead_random(),
            buf_read_page_async(),
            buf_read_ahead_linear(),
            buf_read_ibuf_merge_pages(),
            buf_read_recv_pages(),
            fil_aio_wait():
            Issue error if page read fails.

            btr_pcur_move_to_next_page(): Do not reference page if it is
            NULL.

            Introduced dict_table_t::is_readable() and dict_index_t::is_readable()
            that will return true if tablespace exists and pages read from
            tablespace are not corrupted or page decryption failed.
            Removed buf_page_t::key_version. After page decryption the
            key version is not removed from page frame. For unencrypted
            pages, old key_version is removed at buf_page_encrypt_before_write()

            dict_stats_update_transient_for_index(),
            dict_stats_update_transient()
            Do not continue if table decryption failed or table
            is corrupted.

            dict0stats.cc: Introduced a dict_stats_report_error function
            to avoid code duplication.

            fil_parse_write_crypt_data():
            Check that key read from redo log entry is found from
            encryption plugin and if it is not, refuse to start.

            PageConverter::validate(): Removed access to fil_space_t as
            tablespace is not available during import.

            Fixed error code on innodb.innodb test.

            Merged test cased innodb-bad-key-change5 and innodb-bad-key-shutdown
            to innodb-bad-key-change2. Removed innodb-bad-key-change5 test.
            Decreased unnecessary complexity on some long lasting tests.

            Removed fil_inc_pending_ops(), fil_decr_pending_ops(),
            fil_get_first_space(), fil_get_next_space(),
            fil_get_first_space_safe(), fil_get_next_space_safe()
            functions.

            fil_space_verify_crypt_checksum(): Fixed bug found using ASAN
            where FIL_PAGE_END_LSN_OLD_CHECKSUM field was incorrectly
            accessed from row compressed tables. Fixed out of page frame
            bug for row compressed tables in
            fil_space_verify_crypt_checksum() found using ASAN. Incorrect
            function was called for compressed table.

            Added new tests for discard, rename table and drop (we should allow them
            even when page decryption fails). Alter table rename is not allowed.
            Added test for restart with innodb-force-recovery=1 when page read on
            redo-recovery cant be decrypted. Added test for corrupted table where
            both page data and FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION is corrupted.

            Adjusted the test case innodb_bug14147491 so that it does not anymore
            expect crash. Instead table is just mostly not usable.

            fil0fil.h: fil_space_acquire_low is not visible function
            and fil_space_acquire and fil_space_acquire_silent are
            inline functions. FilSpace class uses fil_space_acquire_low
            directly.

            recv_apply_hashed_log_recs() does not return anything.

            jplindst Jan Lindström (Inactive) added a comment - commit 765a43605a42c069ede604826ede2d93d72c4fdd Author: Jan Lindström <jan.lindstrom@mariadb.com> Date: Wed Apr 26 15:19:16 2017 +0300 MDEV-12253 : Buffer pool blocks are accessed after they have been freed Problem was that bpage was referenced after it was already freed from LRU. Fixed by adding a new variable encrypted that is passed down to buf_page_check_corrupt() and used in buf_page_get_gen() to stop processing page read. This patch should also address following test failures and bugs: MDEV-12419 : IMPORT should not look up tablespace in PageConverter::validate(). This is now removed. MDEV-10099 : encryption.innodb_onlinealter_encryption fails sporadically in buildbot MDEV-11420 : encryption.innodb_encryption-page-compression failed in buildbot MDEV-11222 : encryption.encrypt_and_grep failed in buildbot on P8 Removed dict_table_t::is_encrypted and dict_table_t::ibd_file_missing and replaced these with dict_table_t::file_unreadable. Table ibd file is missing if fil_get_space(space_id) returns NULL and encrypted if not. Removed dict_table_t::is_corrupted field. Ported FilSpace class from 10.2 and using that on buf_page_check_corrupt(), buf_page_decrypt_after_read(), buf_page_encrypt_before_write(), buf_dblwr_process(), buf_read_page(), dict_stats_save_defrag_stats(). Added test cases when enrypted page could be read while doing redo log crash recovery. Also added test case for row compressed blobs. btr_cur_open_at_index_side_func(), btr_cur_open_at_rnd_pos_func(): Avoid referencing block that is NULL. buf_page_get_zip(): Issue error if page read fails. buf_page_get_gen(): Use dberr_t for error detection and do not reference bpage after we hare freed it. buf_mark_space_corrupt(): remove bpage from LRU also when it is encrypted. buf_page_check_corrupt(): @return DB_SUCCESS if page has been read and is not corrupted, DB_PAGE_CORRUPTED if page based on checksum check is corrupted, DB_DECRYPTION_FAILED if page post encryption checksum matches but after decryption normal page checksum does not match. In read case only DB_SUCCESS is possible. buf_page_io_complete(): use dberr_t for error handling. buf_flush_write_block_low(), buf_read_ahead_random(), buf_read_page_async(), buf_read_ahead_linear(), buf_read_ibuf_merge_pages(), buf_read_recv_pages(), fil_aio_wait(): Issue error if page read fails. btr_pcur_move_to_next_page(): Do not reference page if it is NULL. Introduced dict_table_t::is_readable() and dict_index_t::is_readable() that will return true if tablespace exists and pages read from tablespace are not corrupted or page decryption failed. Removed buf_page_t::key_version. After page decryption the key version is not removed from page frame. For unencrypted pages, old key_version is removed at buf_page_encrypt_before_write() dict_stats_update_transient_for_index(), dict_stats_update_transient() Do not continue if table decryption failed or table is corrupted. dict0stats.cc: Introduced a dict_stats_report_error function to avoid code duplication. fil_parse_write_crypt_data(): Check that key read from redo log entry is found from encryption plugin and if it is not, refuse to start. PageConverter::validate(): Removed access to fil_space_t as tablespace is not available during import. Fixed error code on innodb.innodb test. Merged test cased innodb-bad-key-change5 and innodb-bad-key-shutdown to innodb-bad-key-change2. Removed innodb-bad-key-change5 test. Decreased unnecessary complexity on some long lasting tests. Removed fil_inc_pending_ops(), fil_decr_pending_ops(), fil_get_first_space(), fil_get_next_space(), fil_get_first_space_safe(), fil_get_next_space_safe() functions. fil_space_verify_crypt_checksum(): Fixed bug found using ASAN where FIL_PAGE_END_LSN_OLD_CHECKSUM field was incorrectly accessed from row compressed tables. Fixed out of page frame bug for row compressed tables in fil_space_verify_crypt_checksum() found using ASAN. Incorrect function was called for compressed table. Added new tests for discard, rename table and drop (we should allow them even when page decryption fails). Alter table rename is not allowed. Added test for restart with innodb-force-recovery=1 when page read on redo-recovery cant be decrypted. Added test for corrupted table where both page data and FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION is corrupted. Adjusted the test case innodb_bug14147491 so that it does not anymore expect crash. Instead table is just mostly not usable. fil0fil.h: fil_space_acquire_low is not visible function and fil_space_acquire and fil_space_acquire_silent are inline functions. FilSpace class uses fil_space_acquire_low directly. recv_apply_hashed_log_recs() does not return anything.

            People

              jplindst Jan Lindström (Inactive)
              elenst Elena Stepanova
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.