Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-30157

Galera SST doesn't properly handle undo* files from innodb

Details

    Description

      Galera SST doesn't handle undo* files from innodb correctly - as a result, SST may terminate incorrectly when --innodb-undo-tablespaces is set to 3 or more (and possibly other settings related to undo tablespaces):

      Dec  1 09:52:47 debian-buster-amd64 mariadbd[3996]: 2022-12-01  9:52:47 0 [Note] InnoDB: Completed initialization of buffer pool
      		 
      Dec  1 09:52:47 debian-buster-amd64 mariadbd[3996]: 2022-12-01  9:52:47 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes)
      		 
      Dec  1 09:52:47 debian-buster-amd64 mariadbd[3996]: 2022-12-01  9:52:47 0 [ERROR] InnoDB: Expected to open innodb_undo_tablespaces=3 but was able to find only 0
      		 
      Dec  1 09:52:47 debian-buster-amd64 mariadbd[3996]: 2022-12-01  9:52:47 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
      		 
      Dec  1 09:52:47 debian-buster-amd64 mariadbd[3996]: 2022-12-01  9:52:47 0 [Note] InnoDB: Starting shutdown...
      		 
      Dec  1 09:52:47 debian-buster-amd64 mariadbd[3996]: 2022-12-01  9:52:47 0 [ERROR] Plugin 'InnoDB' init function returned error.

      Attachments

        Issue Links

          blocks

          Task - A task that needs to be done. MDEV-29986 Set innodb_undo_tablespaces=3 by default

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed
          duplicates

          Bug - A problem which impairs or prevents the functions of the product. MDEV-28669 Cannot join second node to new cluster WSREP: Application received wrong state

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          split to

          Task - A task that needs to be done. MDEV-30383 improvements to mtr to properly process datadir and other paths

          • Major - Major loss of function.
          • Stalled

          Activity

            Ascending order - Click to sort in descending order
            marko Marko Mäkelä added a comment -

            A fairly large number of tests is affected by this, as illustrated by this work-around.

            marko Marko Mäkelä added a comment - A fairly large number of tests is affected by this, as illustrated by this work-around .
            marko Marko Mäkelä added a comment -

            I noticed one more related test failure:

            CURRENT_TEST: galera_3nodes.galera_garbd_backup
            		 
            …
            		 
            2022-12-12  9:23:53 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
            		 
            2022-12-12  9:23:53 2 [ERROR] InnoDB: Transaction id 0x1e associated with recordCOMPACT RECORD(info_bits=32, 7 fields): {[36]a847d38c-79fe-11ed-900d-3a9f991f9ce5(0x61383437643338632D373966652D313165642D393030642D336139663939316639636535),[6]      (0x00000000001E),[7]    1  (0x0A0000003101D4),[8]        (0x8000000000000002),[8]        (0x8000000000000002),[4]    (0x80000004),[4]    (0x8002D17F)} in index `PRIMARY` of table `mysql`.`wsrep_cluster` is greater than the global counter 21! The table is corrupted.
            		 
            2022-12-12  9:23:53 2 [ERROR] InnoDB: Transaction id 0x1e associated with recordCOMPACT RECORD(info_bits=32, 7 fields): {[36]a847d38c-79fe-11ed-900d-3a9f991f9ce5(0x61383437643338632D373966652D313165642D393030642D336139663939316639636535),[6]      (0x00000000001E),[7]    1  (0x0A0000003101D4),[8]        (0x8000000000000002),[8]        (0x8000000000000002),[4]    (0x80000004),[4]    (0x8002D17F)} in index `PRIMARY` of table `mysql`.`wsrep_cluster` is greater than the global counter 21! The table is corrupted.
            		 
            mariadbd: /home/buildbot/buildbot/build/storage/innobase/btr/btr0cur.cc:3233: dberr_t btr_cur_upd_rec_sys(buf_block_t*, rec_t*, dict_index_t*, const rec_offs*, const trx_t*, roll_ptr_t, mtr_t*): Assertion `roll_ptr == 0 || lock_check_trx_id_sanity(trx_read_trx_id(rec + offset), rec, index, offsets)' failed.

            marko Marko Mäkelä added a comment - I noticed one more related test failure: CURRENT_TEST: galera_3nodes.galera_garbd_backup … 2022-12-12 9:23:53 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2022-12-12 9:23:53 2 [ERROR] InnoDB: Transaction id 0x1e associated with recordCOMPACT RECORD(info_bits=32, 7 fields): {[36]a847d38c-79fe-11ed-900d-3a9f991f9ce5(0x61383437643338632D373966652D313165642D393030642D336139663939316639636535),[6] (0x00000000001E),[7] 1 (0x0A0000003101D4),[8] (0x8000000000000002),[8] (0x8000000000000002),[4] (0x80000004),[4] (0x8002D17F)} in index `PRIMARY` of table `mysql`.`wsrep_cluster` is greater than the global counter 21! The table is corrupted. 2022-12-12 9:23:53 2 [ERROR] InnoDB: Transaction id 0x1e associated with recordCOMPACT RECORD(info_bits=32, 7 fields): {[36]a847d38c-79fe-11ed-900d-3a9f991f9ce5(0x61383437643338632D373966652D313165642D393030642D336139663939316639636535),[6] (0x00000000001E),[7] 1 (0x0A0000003101D4),[8] (0x8000000000000002),[8] (0x8000000000000002),[4] (0x80000004),[4] (0x8002D17F)} in index `PRIMARY` of table `mysql`.`wsrep_cluster` is greater than the global counter 21! The table is corrupted. mariadbd: /home/buildbot/buildbot/build/storage/innobase/btr/btr0cur.cc:3233: dberr_t btr_cur_upd_rec_sys(buf_block_t*, rec_t*, dict_index_t*, const rec_offs*, const trx_t*, roll_ptr_t, mtr_t*): Assertion `roll_ptr == 0 || lock_check_trx_id_sanity(trx_read_trx_id(rec + offset), rec, index, offsets)' failed.
            jplindst Jan Lindström (Inactive) added a comment -

            ok to push.

            Please make 10.4-10.6 versions in one branch (all of these in same).

            jplindst Jan Lindström (Inactive) added a comment - ok to push. Please make 10.4-10.6 versions in one branch (all of these in same).
            sysprg Julius Goryavsky added a comment -

            Fixed, https://github.com/MariaDB/server/commit/b84f3fa769228c4ffd367bc8f2426e6e912325a5 (and https://github.com/MariaDB/server/commit/e4a4aad7cf5a8cdb3ae4d97527f489cc590146a1)

            sysprg Julius Goryavsky added a comment - Fixed, https://github.com/MariaDB/server/commit/b84f3fa769228c4ffd367bc8f2426e6e912325a5 (and https://github.com/MariaDB/server/commit/e4a4aad7cf5a8cdb3ae4d97527f489cc590146a1 )

            People

              sysprg Julius Goryavsky
              sysprg Julius Goryavsky
              Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.