Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-14420

[Draft] Assertion failure in file /data/src/10.3-bug/storage/innobase/log/log0recv.cc line 1186

Details

    Description

      10.3 31bd86c8df63a4d

      ...
      2017-11-16 20:25:04 0 [Note] InnoDB: Ignoring data file './test/t6.ibd' with space ID 4959, since the redo log references ./test/t6.ibd with space ID 4953.
      2017-11-16 20:25:04 0 [Note] InnoDB: Ignoring data file './test/#sql-160f_e.ibd' with space ID 4959. Another data file called ./test/t6.ibd exists with the same space ID.
      2017-11-16 20:25:05 0 [Note] InnoDB: Starting final batch to recover 3021 pages from redo log.
      2017-11-16 20:25:05 0x7f68cd4dc700  InnoDB: Assertion failure in file /data/src/10.3-bug/storage/innobase/log/log0recv.cc line 1186
      InnoDB: We intentionally generate a memory trap.
       
      #5  0x00007f1fcd8ad3fa in abort () from /lib/x86_64-linux-gnu/libc.so.6
      #6  0x000055dfcdab67cb in ut_dbg_assertion_failed (expr=0x0, file=0x55dfcdfdd3b8 "/data/src/10.3-bug/storage/innobase/log/log0recv.cc", line=1186) at /data/src/10.3-bug/storage/innobase/ut/ut0dbg.cc:61
      #7  0x000055dfcd9240c7 in recv_parse_or_apply_log_rec_body (type=MLOG_1BYTE, ptr=0x7f1fae032860 "", end_ptr=0x7f1fae032864 "", space_id=4608, page_no=0, apply=true, block=0x7f1fabc61a20, mtr=0x7f1fa877e590) at /data/src/10.3-bug/storage/innobase/log/log0recv.cc:1186
      #8  0x000055dfcd9264cd in recv_recover_page (just_read_in=true, block=0x7f1fabc61a20) at /data/src/10.3-bug/storage/innobase/log/log0recv.cc:1834
      #9  0x000055dfcdb2b540 in buf_page_io_complete (bpage=0x7f1fabc61a20, evict=false) at /data/src/10.3-bug/storage/innobase/buf/buf0buf.cc:6034
      #10 0x000055dfcdbc29f1 in fil_aio_wait (segment=2) at /data/src/10.3-bug/storage/innobase/fil/fil0fil.cc:5359
      #11 0x000055dfcda51006 in io_handler_thread (arg=0x55dfcf213110 <n+16>) at /data/src/10.3-bug/storage/innobase/srv/srv0start.cc:340
      #12 0x00007f1fcf57b494 in start_thread (arg=0x7f1fa877f700) at pthread_create.c:333
      #13 0x00007f1fcd96193f in clone () from /lib/x86_64-linux-gnu/libc.so.6
      

      perl ./runall-new.pl --grammar=replication-ddl_sql.yy --gendata=conf/replication/replication-ddl_data.zz --duration=600 --threads=6 --queries=100M --redefine=instant_add.yy --mysqld=--max-statement-time=30  --scenario=Restart --scenario-type=crash --engine=InnoDB --seed=time --basedir=/data/src/10.3-bug --vardir=/dev/shm/vardir6
      

      It's reproducible reasonably well, but not reducible easily. Probably not the highest priority, because the scenario is crash-restart during DML/DDL, which can't be good anyway, but maybe there is still something to solve here.

      I have the datadir, but it's too big to attach.

      Attachments

        1. instant_add.yy
          9 kB
        2. mysql.err
          2.21 MB
        3. threads1
          18 kB

        Issue Links

          Activity

            The problem is that a MLOG_1BYTE record is being applied to an unexpected byte offset to page 0 of tablespace 4608.

                                    switch (type) {
                                    default:
                                            ut_error;
            

            Can I get the local variables from recv_parse_or_apply_log_rec_body()?
            Something like this:

            thread 1
            frame 7
            info locals
            p/x ptr[-8]@16
            

            marko Marko Mäkelä added a comment - The problem is that a MLOG_1BYTE record is being applied to an unexpected byte offset to page 0 of tablespace 4608. switch (type) { default : ut_error; Can I get the local variables from recv_parse_or_apply_log_rec_body() ? Something like this: thread 1 frame 7 info locals p/x ptr[-8]@16
            elenst Elena Stepanova added a comment - - edited

            Happens on 10.4 in mariabackup: https://travis-ci.org/elenst/travis-tests/jobs/564697197 863 13 2

            10.4 83d368a

             
            /home/travis/server/bin/mariabackup: O_TMPFILE is not supported on /tmp (disabling future attempts)
            2019-07-29 14:09:18 0x7f716e6b7740  InnoDB: Assertion failure in file /home/travis/src/storage/innobase/log/log0recv.cc line 1434
             
             
            lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x7f716c32102a]
            /home/travis/server/bin/mariabackup(+0x10c6678)[0x5652dece4678]
            ut/ut0mem.cc:42(ut_strlcpy(char*, char const*, unsigned long))[0x5652deb74ab0]
            log/log0recv.cc:1439(recv_parse_or_apply_log_rec_body(mlog_id_t, unsigned char*, unsigned char*, unsigned long, unsigned long, bool, buf_block_t*, mtr_t*))[0x5652deb76f33]
            log/log0recv.cc:2038(recv_recover_page(buf_block_t*, mtr_t&, recv_addr_t*, mlog_init_t::init*))[0x5652deb7825e]
            log/log0recv.cc:2376(recv_apply_hashed_log_recs(bool))[0x5652dec8e81d]
            srv/srv0start.cc:1905(srv_start(bool))[0x5652de2ef078]
            mariabackup/xtrabackup.cc:1985(innodb_init())[0x5652de2f9540]
            mariabackup/xtrabackup.cc:5544(xtrabackup_prepare_func(char**))[0x5652de2fb7f1]
            mariabackup/xtrabackup.cc:6357(main_low(char**))[0x5652de2fb003]
            /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f716c30a830]
            

            elenst-jira-refs 00fcd15 Toolbox: 32d7437

            perl ./runall-new.pl --basedir=/home/travis/server --vardir=/home/travis/logs/vardir --duration=350 --threads=4 --seed=1564408990 --reporters=Backtrace,ErrorLog,Deadlock --skip-gendata --gendata-advanced --views --grammar=conf/mariadb/generic-dml.yy --redefine=conf/mariadb/bulk_insert.yy --mysqld=--log_output=FILE --mysqld=--max-statement-time=20 --mysqld=--lock-wait-timeout=10 --mysqld=--loose-innodb-lock-wait-timeout=5 --mysqld=--loose-debug_assert_on_not_freed_memory=0 --filter=/home/travis/mariadb-toolbox/travis/10.4-combo-filter.ff --mysqld=--innodb-compression-algorithm=none --mysqld=--innodb-page-size=32K --engine=InnoDB --scenario=MariaBackupIncremental
            

            or

            perl ./runall-new.pl --basedir=/home/travis/server --vardir=/home/travis/logs/vardir --duration=350 --threads=4 --seed=1564408377 --reporters=Backtrace,ErrorLog,Deadlock --skip-gendata --gendata-advanced --views --grammar=conf/mariadb/generic-dml.yy --redefine=conf/mariadb/bulk_insert.yy --mysqld=--log_output=FILE --mysqld=--max-statement-time=20 --mysqld=--lock-wait-timeout=10 --mysqld=--loose-innodb-lock-wait-timeout=5 --mysqld=--loose-debug_assert_on_not_freed_memory=0 --filter=/home/travis/mariadb-toolbox/travis/10.4-combo-filter.ff --mysqld=--innodb-compression-algorithm=none --mysqld=--innodb-page-size=32K --engine=InnoDB --scenario=MariaBackupFull
            

            elenst Elena Stepanova added a comment - - edited Happens on 10.4 in mariabackup: https://travis-ci.org/elenst/travis-tests/jobs/564697197 863 13 2 10.4 83d368a /home/travis/server/bin/mariabackup: O_TMPFILE is not supported on /tmp (disabling future attempts) 2019-07-29 14:09:18 0x7f716e6b7740 InnoDB: Assertion failure in file /home/travis/src/storage/innobase/log/log0recv.cc line 1434     lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x7f716c32102a] /home/travis/server/bin/mariabackup(+0x10c6678)[0x5652dece4678] ut/ut0mem.cc:42(ut_strlcpy(char*, char const*, unsigned long))[0x5652deb74ab0] log/log0recv.cc:1439(recv_parse_or_apply_log_rec_body(mlog_id_t, unsigned char*, unsigned char*, unsigned long, unsigned long, bool, buf_block_t*, mtr_t*))[0x5652deb76f33] log/log0recv.cc:2038(recv_recover_page(buf_block_t*, mtr_t&, recv_addr_t*, mlog_init_t::init*))[0x5652deb7825e] log/log0recv.cc:2376(recv_apply_hashed_log_recs(bool))[0x5652dec8e81d] srv/srv0start.cc:1905(srv_start(bool))[0x5652de2ef078] mariabackup/xtrabackup.cc:1985(innodb_init())[0x5652de2f9540] mariabackup/xtrabackup.cc:5544(xtrabackup_prepare_func(char**))[0x5652de2fb7f1] mariabackup/xtrabackup.cc:6357(main_low(char**))[0x5652de2fb003] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f716c30a830] elenst-jira-refs 00fcd15 Toolbox: 32d7437 perl ./runall-new.pl --basedir=/home/travis/server --vardir=/home/travis/logs/vardir --duration=350 --threads=4 --seed=1564408990 --reporters=Backtrace,ErrorLog,Deadlock --skip-gendata --gendata-advanced --views --grammar=conf/mariadb/generic-dml.yy --redefine=conf/mariadb/bulk_insert.yy --mysqld=--log_output=FILE --mysqld=--max-statement-time=20 --mysqld=--lock-wait-timeout=10 --mysqld=--loose-innodb-lock-wait-timeout=5 --mysqld=--loose-debug_assert_on_not_freed_memory=0 --filter=/home/travis/mariadb-toolbox/travis/10.4-combo-filter.ff --mysqld=--innodb-compression-algorithm=none --mysqld=--innodb-page-size=32K --engine=InnoDB --scenario=MariaBackupIncremental or perl ./runall-new.pl --basedir=/home/travis/server --vardir=/home/travis/logs/vardir --duration=350 --threads=4 --seed=1564408377 --reporters=Backtrace,ErrorLog,Deadlock --skip-gendata --gendata-advanced --views --grammar=conf/mariadb/generic-dml.yy --redefine=conf/mariadb/bulk_insert.yy --mysqld=--log_output=FILE --mysqld=--max-statement-time=20 --mysqld=--lock-wait-timeout=10 --mysqld=--loose-innodb-lock-wait-timeout=5 --mysqld=--loose-debug_assert_on_not_freed_memory=0 --filter=/home/travis/mariadb-toolbox/travis/10.4-combo-filter.ff --mysqld=--innodb-compression-algorithm=none --mysqld=--innodb-page-size=32K --engine=InnoDB --scenario=MariaBackupFull

            People

              Unassigned Unassigned
              elenst Elena Stepanova
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.