Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-11976

incremental backup may crash when KEY_BLOCK_SIZE=1 and innodb-track-changed-pages

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Cannot Reproduce
    • 10.1
    • N/A
    • Backup
    • None

    Description

      Attached mtr test :
      1. crashes reliably in virtual Ubuntu 16.04 on Windows 10 host (with -DWITH_LIBARCHIVE=1)
      2. sometimes crashes in virtual Centos 7 on Ubuntu 16.04 host (with -DWITH_LIBARCHIVE=1)
      3. Doesn't crash in the same virtual Centos 7 on Ubuntu 16.04 host when binaries built without libarchive
      4. doesn't crash in native Ubuntu 16.04 with and without DWITH_LIBARCHIVE=1
      5. 'Similar' shell script on the same virtual Ubuntu 16.04 on Windows 10 host (with -DWITH_LIBARCHIVE=1) shows little different crash
      6. Never crashed for me with KEY_BLOCK_SIZE=2
      7. Other small changes to test case in 1. or 5. (like using different database name for test table or inserting data different way or reducing number of rows or do no restart or disable innodb-track-changed-pages) usually makes the test pass without crash.

      Stack trace of mtr test for virtual ubuntu 16.04 (case 1):
      (Note that stack traces are missing and it continues printing messages after the crash and test case aborts on timeout)

      [01] ./incremental_sample/test.ibd is compressed with page size = 1024 bytes
      170202 13:29:15 [01] Copying ./incremental_sample/test.ibd to /farm/m0-bb-10.1-wlad-xtrabackup/build/mysql-test/var/tmp/backup_inc1/incremental_sample/test.ibd.
      delta
      170202 13:29:15 [01]        ...done
      170202 13:29:15 [ERROR] mysqld got signal 11 ;
      This could be because you hit a bug. It is also possible that this binary
      ...
      Thread pointer: 0x0x0
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      170202 13:29:15 >> log scanned up to (2532997)
      170202 13:29:16 >> log scanned up to (2532997)
      ...
      170202 13:29:41 >> log scanned up to (2532997)
      170202 13:29:42 >> log scanned up to (2532997)
      

      Stack trace for virtual Centos 7 (case 2)

      [01] ./incremental_sample/test.ibd is compressed with page size = 1024 bytes
      170202 13:53:02 [01] Copying ./incremental_sample/test.ibd to /farm/m0-bb-10.1-wlad-xtrabackup/build/mysql-test/var/tmp/backup_inc1/incremental_sample/test.ibd.delta
      170202 13:53:02 [ERROR] mysqld got signal 11 ;
      ...
      Thread pointer: 0x0x0
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x0 thread_stack 0x48400
      /farm/m0-bb-10.1-wlad-xtrabackup/build/extra/xtrabackup/xtrabackup(my_print_stacktrace+0x29)[0x55d2a8e81a69]
      /farm/m0-bb-10.1-wlad-xtrabackup/build/extra/xtrabackup/xtrabackup(handle_fatal_signal+0x305)[0x55d2a8a42d35]
      /lib64/libpthread.so.0(+0xf370)[0x7fd8f078e370]
      /lib64/libc.so.6(+0x1493cb)[0x7fd8ef6953cb]
      mysys/stacktrace.c:268(my_print_stacktrace)[0x55d2a882f889]
      xtrabackup/xtrabackup.cc:2424(xtrabackup_copy_datafile)[0x55d2a8824799]
      xtrabackup/xtrabackup.cc:2948(data_copy_thread_func)[0x55d2a88249f2]
      /lib64/libpthread.so.0(+0x7dc5)[0x7fd8f0786dc5]
      /lib64/libc.so.6(clone+0x6d)[0x7fd8ef64373d]
      

      Stack trace for bash script (Ubuntu 16.04 - case 5)

      [01] ./incremental_sample/test.ibd is compressed with page size = 1024 bytes
      170202 14:20:13 [01] Copying ./incremental_sample/test.ibd to /farm/x1-2.4.4/var/var1/data/delta/incremental_sample/test.ibd.delta
      170202 14:20:13 [ERROR] mysqld got signal 11 ;
      ...
      stack_bottom = 0x0 thread_stack 0x48400
      xtrabackup(my_print_stacktrace+0x29)[0x5641fd75f519]
      xtrabackup(handle_fatal_signal+0x2f5)[0x5641fd349395]
      /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7f62f9232390]
      xtrabackup(ds_write+0x4)[0x5641fd12eda4]
      xtrabackup(+0x39634c)[0x5641fd13234c]
      mysys/stacktrace.c:268(my_print_stacktrace)[0x5641fd127ae3]
      xtrabackup.cc:2948(data_copy_thread_func(void*))[0x5641fd127cb2]
      /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba)[0x7f62f92286ba]
      /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f62f779782d]
      

      To repeat: copy attached mtr test into xtrabackup suite and run with --repeat=100
      PS: I did run case 1. with percona-xtrabackup binary and it didn't crash

      Attachments

        1. case1.log
          24 kB
        2. case2.log
          191 kB
        3. case5.log
          316 kB
        4. case5.sh
          2 kB
        5. incremental_compressed_bitmap_1kb.opt
          0.1 kB
        6. incremental_compressed_bitmap_1kb.test
          1 kB

        Activity

          People

            wlad Vladislav Vaintroub
            anikitin Andrii Nikitin (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.