Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-12773

xtrabackup --prepare may hang with 'Waiting for 1 active transactions to finish'

    XMLWordPrintable

Details

    Description

      Xtrabackup 2.4.7 may hang in --prepare phase with MariaDB 10.2 in this test
      https://github.com/percona/percona-xtrabackup/blob/2.4/storage/innobase/xtrabackup/test/t/undo_tablespaces.sh

      /usr/bin/innobackupex version 2.4.7 based on MySQL server 5.7.13 Linux (x86_64) (revision id: 6f7a799)
      xtrabackup: cd to /dev/shm/__var0/var1/backup/
      xtrabackup: This target seems to be not prepared yet.
      InnoDB: Number of pools: 1
      xtrabackup: xtrabackup_logfile detected: size=10092544, start_lsn=(1504807)
      xtrabackup: using the following InnoDB configuration for recovery:
      xtrabackup:   innodb_data_home_dir = .
      xtrabackup:   innodb_data_file_path = ibdata1:12M:autoextend
      xtrabackup:   innodb_log_group_home_dir = .
      xtrabackup:   innodb_log_files_in_group = 1
      xtrabackup:   innodb_log_file_size = 10092544
      xtrabackup: using the following InnoDB configuration for recovery:
      xtrabackup:   innodb_data_home_dir = .
      xtrabackup:   innodb_data_file_path = ibdata1:12M:autoextend
      xtrabackup:   innodb_log_group_home_dir = .
      xtrabackup:   innodb_log_files_in_group = 1
      xtrabackup:   innodb_log_file_size = 10092544
      xtrabackup: Starting InnoDB instance for recovery.
      xtrabackup: Using 104857600 bytes for buffer pool (set by --use-memory parameter)
      InnoDB: PUNCH HOLE support available
      InnoDB: Mutexes and rw_locks use GCC atomic builtins
      InnoDB: Uses event mutexes
      InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
      InnoDB: Compressed tables use zlib 1.2.8
      InnoDB: Number of pools: 1
      InnoDB: Using CPU crc32 instructions
      InnoDB: Initializing buffer pool, total size = 100M, instances = 1, chunk size = 100M
      InnoDB: Completed initialization of buffer pool
      InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
      InnoDB: Opened 4 undo tablespaces
      InnoDB: 4 undo tablespaces made active
      InnoDB: Highest supported file format is Barracuda.
      InnoDB: Log scan progressed past the checkpoint lsn 1504807
      InnoDB: Doing recovery: scanned up to log sequence number 6682112 (57%)
      InnoDB: Doing recovery: scanned up to log sequence number 10416535 (99%)
      InnoDB: Database was not shutdown normally!
      InnoDB: Starting crash recovery.
      InnoDB: 1 transaction(s) which must be rolled back or cleaned up in total 16049 row operations to undo
      InnoDB: Trx id counter is 1792
      InnoDB: Starting an apply batch of log records to the database...
      InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 
      InnoDB: Apply batch completed
      InnoDB: xtrabackup: Last MySQL binlog file position 3503344, file name ./mysql-bin.000002
      InnoDB: Creating shared tablespace for temporary commit
      InnoDB: Creating shared tablespace for temporary commit
      InnoDB: Rolling back trx with id 1466, 16049 rows to undo
       
      InnoDB: Progress in percents: 1InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
       2 3InnoDB: File './ibtmp1' size is now 12 MB.
      InnoDB: 96 redo rollback segment(s) found. 1 redo rollback segment(s) are active.
      InnoDB: 32 non-redo rollback segment(s) are active.
       4InnoDB: 5.7.13 started; log sequence number 10416535
      InnoDB: xtrabackup: Last MySQL binlog file position 3503344, file name ./mysql-bin.000002
       
      xtrabackup: starting shutdown with innodb_fast_shutdown = 1
      InnoDB: FTS optimize thread exiting.
      InnoDB: Starting shutdown...
      InnoDB: Waiting for 1 active transactions to finish
      InnoDB: Waiting for 1 active transactions to finish
      

      The problem is not stable. Sometimes I can repeat it 1/10, sometimes it is 9/10 .
      10.1.23 and 5.7.18 are not affected. 10.2.5 is affected.

      There is consideration that the problem is incompatibility introduced by MDEV-12289, after which InnoDB 5.7 is not expecting TRX_SYS page slots 1..32 to contain transactions.

      There is consideration that --prepare is not needed for simplest backup / restore scenario, and Server may be started directly on backup.

      Attachments

        1. data.zip
          7.14 MB
        2. MDEV-12773.zip
          8.26 MB

        Issue Links

          Activity

            People

              anikitin Andrii Nikitin (Inactive)
              anikitin Andrii Nikitin (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.