Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-21109

Table corruption not detected with CHECK TABLE or innochecksum, only with mariabackup

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Critical
    • Resolution: Incomplete
    • 10.3.17, 10.3.21, 10.3.23
    • N/A
    • mariabackup

    Description

      It is observed that, while taking the backup using mariabackup(10.3.17) it's keep failing with following error:

      Backup command used:

      /var/lib/mysql/bin/mariabackup --defaults-file=/etc/my.cnf --user=**** --password=******* --backup --skip-encrypted-backup --compress --ftwrl-wait-timeout=5 --ftwrl-wait-threshold=300 --ftwrl-wait-query-type=all --target-dir=/tmp/backup/full_backup_2019

      mariabackup output:

      [00] 2019-11-16 06:29:24 Connecting to MySQL server host: localhost, user: xxx, password: xxx, port: 3306, socket: /var/lib/mysql/mysql.sock
      		 
      [00] 2019-11-16 06:29:24 Using server version 10.3.17-MariaDB-log
      		 
      /var/lib/mysql/bin/mariabackup based on MariaDB server 10.3.17-MariaDB Linux (x86_64)
      		 
      ....
      		 
      ....
      		 
      [01] 2019-11-16 06:29:27 Compressing ./foo/bar.ibd to /tmp/backup/full_backup_2019/foo/bar.ibd.qp
      		 
      [01] 2019-11-16 06:29:27         ...done
      		 
      [01] 2019-11-16 06:29:27 Compressing ./foo/foobar.ibd to /tmp/backup/full_backup_2019/foo/foobar.ibd.qp
      		 
      [01] 2019-11-16 06:29:27 Database page corruption detected at page 5, retrying...
      		 
      [01] 2019-11-16 06:29:27 Database page corruption detected at page 5, retrying...
      		 
      [01] 2019-11-16 06:29:27 Database page corruption detected at page 5, retrying...
      		 
      [01] 2019-11-16 06:29:27 Database page corruption detected at page 5, retrying...
      		 
      [01] 2019-11-16 06:29:27 Database page corruption detected at page 5, retrying...
      		 
      [01] 2019-11-16 06:29:27 Database page corruption detected at page 5, retrying...
      		 
      [01] 2019-11-16 06:29:27 Database page corruption detected at page 5, retrying...
      		 
      [01] 2019-11-16 06:29:27 Database page corruption detected at page 5, retrying...
      		 
      [01] 2019-11-16 06:29:28 Database page corruption detected at page 5, retrying...
      		 
      [00] 2019-11-16 06:29:28 >> log scanned up to (30851334774)
      		 
      [01] 2019-11-16 06:29:28 Error: failed to read page after 10 retries. File ./foo/foobar.ibd seems to be corrupted.
      		 
      2019-11-16 6:29:28 0 [Note] InnoDB: Page dump in ascii and hex (16384 bytes):
      		 
      .......
      		 
      .......
      		 
      InnoDB: End of page dump
      		 
      2019-11-16 6:29:28 0 [Note] InnoDB: Uncompressed page, stored checksum in field1 3454859770, calculated checksums for field1: crc32 3454859770, innodb 3654618756, page type 17855 == INDEX.none 3735928559, stored checksum in field2 3454859770, calculated checksums for field2: crc32 3454859770, innodb 4252287317, none 3735928559, page LSN 7 450897511, low 4 bytes of LSN at page end 450897511, page number (if stored to page already) 7, space id (if created with >= MySQL-4.1.1 and stored already) 1962
      		 
      2019-11-16 6:29:28 0 [Note] InnoDB: Page may be an index page where index id is 5185
      		 
      [01] 2019-11-16 06:29:28 mariabackup: xtrabackup_copy_datafile() failed.
      		 
      [00] FATAL ERROR: 2019-11-16 06:29:28 failed to copy datafile.

      Did tried, fixing the table (pointed as corrupted) with following, but the same issue is occurring.

      set OLD_ALTER_TABLE=1
      Alter table table_name engine=InnoDB
      Alter table table_name FORCE
      Take mysqldump of table and restore it to Database

      Attachments

        1. break_down_rate.ddl
          0.8 kB
          Juan
        2. break_down_rate.ibd
          112 kB
          Juan
        3. corrupt20200921.tgz.enc
          10 kB
          Juan
        4. cs0093460_20200807_core.21139.gza
          10.00 MB
          Juan
        5. cs0093460_20200807_core.21139.gzb
          10.00 MB
          Juan
        6. cs0093460_20200807_core.21139.gzc
          10.00 MB
          Juan
        7. cs0093460_20200807_core.21139.gzd
          10.00 MB
          Juan
        8. cs0093460_20200807_core.21139.gze
          10.00 MB
          Juan
        9. cs0093460_20200807_core.21139.gzf
          10.00 MB
          Juan
        10. cs0093460_20200807_core.21139.gzg
          10.00 MB
          Juan
        11. cs0093460_20200807_core.21139.gzh
          10.00 MB
          Juan
        12. cs0093460_20200807_core.21139.gzi
          10.00 MB
          Juan
        13. cs0093460_20200807_core.21139.gzj
          10.00 MB
          Juan
        14. cs0093460_20200807_core.21139.gzk
          10.00 MB
          Juan
        15. cs0093460_20200807_core.21139.gzl
          10.00 MB
          Juan
        16. cs0093460_20200807_core.21139.gzm
          8.14 MB
          Juan
        17. cs0093460_20200807_error_mariadb_x01bofiddb1a.log
          78 kB
          Juan
        18. cs0093460_20200807_lib.tar.gza
          10.00 MB
          Juan
        19. cs0093460_20200807_lib.tar.gzb
          4.03 MB
          Juan
        20. cs0093460_20200917_error_mariadb_x01bstredb1a.log
          14 kB
          Juan
        21. cs0093460_20200917_full_backup_20200916_054536.log
          30 kB
          Juan
        22. cs0093460_20200917_full_backup_20200917_120913.log
          29 kB
          Juan
        23. mdev-21109-rr.sh
          4 kB
          Vladislav Lesin
        24. show_global_status_x07gisiddb3a.log
          62 kB
          Allen Lee
        25. show_global_variables_x07gisiddb3a.log
          475 kB
          Allen Lee
        26. t33914-202006252311.log
          104 kB
          Juan
        27. t33914-202006252313.tgz.enc.a
          9.00 MB
          Juan
        28. t33914-202006252313.tgz.enc.b
          9.00 MB
          Juan
        29. t33914-202006252313.tgz.enc.c
          9.00 MB
          Juan
        30. t33914-202006252313.tgz.enc.d
          8.09 MB
          Juan

        Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. MDEV-24260 mariabackup and innochecksum detects page faults but all ok in application

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-19871 Add page id matching check in innochecksum tool

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. MDEV-22929 MariaBackup option to report and/or continue when corruption is encountered

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Task - A task that needs to be done. MDEV-23971 add the ability to fix corrupted pages on --prepare

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-29938 InnoDB: Assertion failure in btr0pcur.cc line 532

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MDEV-25361 innochecksum must not report errors for freed pages

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            People

              marko Marko Mäkelä
              suresh.ramagiri@mariadb.com suresh ramagiri
              Votes:
              4 Vote for this issue
              Watchers:
              19 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.