Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-12023

Assertion failure sym_node->table != NULL on startup

Details

    Description

      When I use .sh test from https://bugs.launchpad.net/percona-xtrabackup/+bug/1399471 and ignore_builtin_innodb
      plugin_load_add=ha_innodb
      in my.cnf - I get reliable crash :

      InnoDB: Apply batch completed
      2017-02-08 09:34:38 7f4f19145780  InnoDB: Assertion failure in thread 139977699907456 in file pars0pars.cc line 865
      InnoDB: Failing assertion: sym_node->table != NULL
      InnoDB: We intentionally generate a memory trap.
      InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
      InnoDB: If you get repeated assertion failures or crashes, even
      InnoDB: immediately after the mysqld startup, there may be
      InnoDB: corruption in the InnoDB tablespace. Please refer to
      InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
      InnoDB: about forcing recovery.
      170208  9:34:38 [ERROR] mysqld got signal 6 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.
       
      To report this bug, see https://mariadb.com/kb/en/reporting-bugs
       
      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed,
      something is definitely wrong and this may fail.
       
      Server version: 10.1.21-MariaDB
      key_buffer_size=0
      read_buffer_size=131072
      max_used_connections=0
      max_threads=1
      thread_count=0
      It is possible that mysqld could use up to
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 5297 K  bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
       
      Thread pointer: 0x0x0
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x0 thread_stack 0x48400
      /farm/m0-bb-10.1-wlad-xtrabackup/build/extra/xtrabackup//xtrabackup(my_print_stacktrace+0x29)[0x55b98f6df519]
      /farm/m0-bb-10.1-wlad-xtrabackup/build/extra/xtrabackup//xtrabackup(handle_fatal_signal+0x2f5)[0x55b98f2c9395]
      /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7f4f18d23390]
      /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x7f4f171b7428]
      /lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x7f4f171b902a]
      mysys/stacktrace.c:268(my_print_stacktrace)[0x55b98f612ca6]
      pars/pars0pars.cc:1282(pars_update_statement(upd_node_t*, sym_node_t*, void*))[0x55b98f614536]
      xtradb/pars0grm.y:443(yyparse())[0x55b98f6a9bdb]
      pars/pars0pars.cc:2247(pars_sql(pars_info_t*, char const*))[0x55b98f615be0]
      que/que0que.cc:1260(que_eval_sql(pars_info_t*, char const*, unsigned long, trx_t*))[0x55b98f619163]
      row/row0mysql.cc:4957(row_drop_table_for_mysql(char const*, trx_t*, bool, unsigned long, bool))[0x55b98f63e9c9]
      row/row0mysql.cc:5211(row_mysql_drop_temp_tables())[0x55b98f6402b5]
      log/log0recv.cc:3611(recv_recovery_rollback_active())[0x55b98f5efb3e]
      srv/srv0start.cc:2715(innobase_start_or_create_for_mysql())[0x55b98f66a37c]
      xtrabackup/xtrabackup.cc:1847(innodb_init())[0x55b98f08263a]
      xtrabackup/xtrabackup.cc:6369(xtrabackup_prepare_func)[0x55b98f08edce]
      /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f4f171a2830]
      /farm/m0-bb-10.1-wlad-xtrabackup/build/extra/xtrabackup//xtrabackup(_start+0x29)[0x55b98f0a2959]
      The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
      information that should help you find out what is causing the crash.
      inc/common.sh: line 102:  2296 Aborted                 "$@"
      

      I couldn't reproduce the same with manual testing nor with 'identical' mtr tests (which i run with "-mysqld=ignore_builtin_innodb --mysqld=-plugin_load_add=ha_innodb" ) like below:

      START TRANSACTION;
      CREATE TEMPORARY TABLE t(id INT) ENGINE=InnoDB;
      let $targetdir=$MYSQLTEST_VARDIR/tmp/backup;
      exec $XTRABACKUP --defaults-file=$MYSQLTEST_VARDIR/my.cnf --backup --target-dir=$targetdir;
      exec $XTRABACKUP --defaults-file=$MYSQLTEST_VARDIR/my.cnf --prepare --target-dir=$targetdir;
      

      Thus I am still filing this bug to let developer review if XtraDB patches from https://bugs.launchpad.net/percona-xtrabackup/+bug/1399471 should be applied to ha_innodb as well.

      Attachments

        Issue Links

          Activity

            anikitin Andrii Nikitin (Inactive) created issue -
            anikitin Andrii Nikitin (Inactive) made changes -
            Field Original Value New Value
            Description When I use .sh test from https://bugs.launchpad.net/percona-xtrabackup/+bug/1399471 and ignore_builtin_innodb
            plugin_load_add=ha_innodb
            in my.cnf - I get reliable crash :


            {noformat}
            InnoDB: Apply batch completed
            2017-02-08 09:34:38 7f4f19145780 InnoDB: Assertion failure in thread 139977699907456 in file pars0pars.cc line 865
            InnoDB: Failing assertion: sym_node->table != NULL
            InnoDB: We intentionally generate a memory trap.
            InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
            InnoDB: If you get repeated assertion failures or crashes, even
            InnoDB: immediately after the mysqld startup, there may be
            InnoDB: corruption in the InnoDB tablespace. Please refer to
            InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
            InnoDB: about forcing recovery.
            170208 9:34:38 [ERROR] mysqld got signal 6 ;
            This could be because you hit a bug. It is also possible that this binary
            or one of the libraries it was linked against is corrupt, improperly built,
            or misconfigured. This error can also be caused by malfunctioning hardware.

            To report this bug, see https://mariadb.com/kb/en/reporting-bugs

            We will try our best to scrape up some info that will hopefully help
            diagnose the problem, but since we have already crashed,
            something is definitely wrong and this may fail.

            Server version: 10.1.21-MariaDB
            key_buffer_size=0
            read_buffer_size=131072
            max_used_connections=0
            max_threads=1
            thread_count=0
            It is possible that mysqld could use up to
            key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 5297 K bytes of memory
            Hope that's ok; if not, decrease some variables in the equation.

            Thread pointer: 0x0x0
            Attempting backtrace. You can use the following information to find out
            where mysqld died. If you see no messages after this, something went
            terribly wrong...
            stack_bottom = 0x0 thread_stack 0x48400
            /farm/m0-bb-10.1-wlad-xtrabackup/build/extra/xtrabackup//xtrabackup(my_print_stacktrace+0x29)[0x55b98f6df519]
            /farm/m0-bb-10.1-wlad-xtrabackup/build/extra/xtrabackup//xtrabackup(handle_fatal_signal+0x2f5)[0x55b98f2c9395]
            /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7f4f18d23390]
            /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x7f4f171b7428]
            /lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x7f4f171b902a]
            mysys/stacktrace.c:268(my_print_stacktrace)[0x55b98f612ca6]
            pars/pars0pars.cc:1282(pars_update_statement(upd_node_t*, sym_node_t*, void*))[0x55b98f614536]
            xtradb/pars0grm.y:443(yyparse())[0x55b98f6a9bdb]
            pars/pars0pars.cc:2247(pars_sql(pars_info_t*, char const*))[0x55b98f615be0]
            que/que0que.cc:1260(que_eval_sql(pars_info_t*, char const*, unsigned long, trx_t*))[0x55b98f619163]
            row/row0mysql.cc:4957(row_drop_table_for_mysql(char const*, trx_t*, bool, unsigned long, bool))[0x55b98f63e9c9]
            row/row0mysql.cc:5211(row_mysql_drop_temp_tables())[0x55b98f6402b5]
            log/log0recv.cc:3611(recv_recovery_rollback_active())[0x55b98f5efb3e]
            srv/srv0start.cc:2715(innobase_start_or_create_for_mysql())[0x55b98f66a37c]
            xtrabackup/xtrabackup.cc:1847(innodb_init())[0x55b98f08263a]
            xtrabackup/xtrabackup.cc:6369(xtrabackup_prepare_func)[0x55b98f08edce]
            /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f4f171a2830]
            /farm/m0-bb-10.1-wlad-xtrabackup/build/extra/xtrabackup//xtrabackup(_start+0x29)[0x55b98f0a2959]
            The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
            information that should help you find out what is causing the crash.
            inc/common.sh: line 102: 2296 Aborted "$@"
            {noformat}

            I couldn't reproduce the same with manual testing not with 'identical' mtr tests (which i run with "--mysqld=--ignore_builtin_innodb --mysqld=--plugin_load_add=ha_innodb" ) like below:

            {code:sql}
            START TRANSACTION;
            CREATE TEMPORARY TABLE t(id INT) ENGINE=InnoDB;
            let $targetdir=$MYSQLTEST_VARDIR/tmp/backup;
            exec $XTRABACKUP --defaults-file=$MYSQLTEST_VARDIR/my.cnf --backup --target-dir=$targetdir;
            exec $XTRABACKUP --defaults-file=$MYSQLTEST_VARDIR/my.cnf --prepare --target-dir=$targetdir;
            {code}

            Thus I am still filing this bug to let developer review if XtraDB patches from https://bugs.launchpad.net/percona-xtrabackup/+bug/1399471 should be applied to ha_innodb as well.
            When I use .sh test from https://bugs.launchpad.net/percona-xtrabackup/+bug/1399471 and ignore_builtin_innodb
            plugin_load_add=ha_innodb
            in my.cnf - I get reliable crash :


            {noformat}
            InnoDB: Apply batch completed
            2017-02-08 09:34:38 7f4f19145780 InnoDB: Assertion failure in thread 139977699907456 in file pars0pars.cc line 865
            InnoDB: Failing assertion: sym_node->table != NULL
            InnoDB: We intentionally generate a memory trap.
            InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
            InnoDB: If you get repeated assertion failures or crashes, even
            InnoDB: immediately after the mysqld startup, there may be
            InnoDB: corruption in the InnoDB tablespace. Please refer to
            InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
            InnoDB: about forcing recovery.
            170208 9:34:38 [ERROR] mysqld got signal 6 ;
            This could be because you hit a bug. It is also possible that this binary
            or one of the libraries it was linked against is corrupt, improperly built,
            or misconfigured. This error can also be caused by malfunctioning hardware.

            To report this bug, see https://mariadb.com/kb/en/reporting-bugs

            We will try our best to scrape up some info that will hopefully help
            diagnose the problem, but since we have already crashed,
            something is definitely wrong and this may fail.

            Server version: 10.1.21-MariaDB
            key_buffer_size=0
            read_buffer_size=131072
            max_used_connections=0
            max_threads=1
            thread_count=0
            It is possible that mysqld could use up to
            key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 5297 K bytes of memory
            Hope that's ok; if not, decrease some variables in the equation.

            Thread pointer: 0x0x0
            Attempting backtrace. You can use the following information to find out
            where mysqld died. If you see no messages after this, something went
            terribly wrong...
            stack_bottom = 0x0 thread_stack 0x48400
            /farm/m0-bb-10.1-wlad-xtrabackup/build/extra/xtrabackup//xtrabackup(my_print_stacktrace+0x29)[0x55b98f6df519]
            /farm/m0-bb-10.1-wlad-xtrabackup/build/extra/xtrabackup//xtrabackup(handle_fatal_signal+0x2f5)[0x55b98f2c9395]
            /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7f4f18d23390]
            /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x7f4f171b7428]
            /lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x7f4f171b902a]
            mysys/stacktrace.c:268(my_print_stacktrace)[0x55b98f612ca6]
            pars/pars0pars.cc:1282(pars_update_statement(upd_node_t*, sym_node_t*, void*))[0x55b98f614536]
            xtradb/pars0grm.y:443(yyparse())[0x55b98f6a9bdb]
            pars/pars0pars.cc:2247(pars_sql(pars_info_t*, char const*))[0x55b98f615be0]
            que/que0que.cc:1260(que_eval_sql(pars_info_t*, char const*, unsigned long, trx_t*))[0x55b98f619163]
            row/row0mysql.cc:4957(row_drop_table_for_mysql(char const*, trx_t*, bool, unsigned long, bool))[0x55b98f63e9c9]
            row/row0mysql.cc:5211(row_mysql_drop_temp_tables())[0x55b98f6402b5]
            log/log0recv.cc:3611(recv_recovery_rollback_active())[0x55b98f5efb3e]
            srv/srv0start.cc:2715(innobase_start_or_create_for_mysql())[0x55b98f66a37c]
            xtrabackup/xtrabackup.cc:1847(innodb_init())[0x55b98f08263a]
            xtrabackup/xtrabackup.cc:6369(xtrabackup_prepare_func)[0x55b98f08edce]
            /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f4f171a2830]
            /farm/m0-bb-10.1-wlad-xtrabackup/build/extra/xtrabackup//xtrabackup(_start+0x29)[0x55b98f0a2959]
            The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
            information that should help you find out what is causing the crash.
            inc/common.sh: line 102: 2296 Aborted "$@"
            {noformat}

            I couldn't reproduce the same with manual testing nor with 'identical' mtr tests (which i run with "--mysqld=--ignore_builtin_innodb --mysqld=--plugin_load_add=ha_innodb" ) like below:

            {code:sql}
            START TRANSACTION;
            CREATE TEMPORARY TABLE t(id INT) ENGINE=InnoDB;
            let $targetdir=$MYSQLTEST_VARDIR/tmp/backup;
            exec $XTRABACKUP --defaults-file=$MYSQLTEST_VARDIR/my.cnf --backup --target-dir=$targetdir;
            exec $XTRABACKUP --defaults-file=$MYSQLTEST_VARDIR/my.cnf --prepare --target-dir=$targetdir;
            {code}

            Thus I am still filing this bug to let developer review if XtraDB patches from https://bugs.launchpad.net/percona-xtrabackup/+bug/1399471 should be applied to ha_innodb as well.
            anikitin Andrii Nikitin (Inactive) made changes -
            Attachment 1.sh.err [ 43315 ]
            wlad Vladislav Vaintroub made changes -
            Assignee Vladislav Vaintroub [ wlad ] Marko Mäkelä [ marko ]
            anikitin Andrii Nikitin (Inactive) made changes -
            Assignee Marko Mäkelä [ marko ] Andrii Nikitin [ anikitin ]
            anikitin Andrii Nikitin (Inactive) made changes -
            Fix Version/s N/A [ 14700 ]
            Resolution Cannot Reproduce [ 5 ]
            Status Open [ 1 ] Closed [ 6 ]
            marko Marko Mäkelä made changes -
            Assignee Andrii Nikitin [ anikitin ] Marko Mäkelä [ marko ]
            Resolution Cannot Reproduce [ 5 ]
            Status Closed [ 6 ] Stalled [ 10000 ]
            marko Marko Mäkelä made changes -
            Component/s Storage Engine - InnoDB [ 10129 ]
            Component/s Storage Engine - XtraDB [ 10135 ]
            Component/s Backup [ 13902 ]
            marko Marko Mäkelä made changes -
            Affects Version/s 10.0 [ 16000 ]
            Affects Version/s 10.2 [ 14601 ]
            Affects Version/s 10.3 [ 22126 ]
            marko Marko Mäkelä made changes -
            Fix Version/s 10.0 [ 16000 ]
            Fix Version/s 10.1 [ 16100 ]
            Fix Version/s 10.2 [ 14601 ]
            Fix Version/s 10.3 [ 22126 ]
            Fix Version/s N/A [ 14700 ]
            marko Marko Mäkelä made changes -
            Summary prepare crashes with ha_innodb and temporary table Assertion failure sym_node->table != NULL on
            marko Marko Mäkelä made changes -
            Summary Assertion failure sym_node->table != NULL on Assertion failure sym_node->table != NULL on startup
            marko Marko Mäkelä made changes -
            Assignee Marko Mäkelä [ marko ] Thirunarayanan Balathandayuthapani [ thiru ]
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            Assignee Thirunarayanan Balathandayuthapani [ thiru ] Marko Mäkelä [ marko ]
            marko Marko Mäkelä made changes -
            Status Stalled [ 10000 ] In Progress [ 3 ]
            marko Marko Mäkelä made changes -
            issue.field.resolutiondate 2018-10-30 15:22:01.0 2018-10-30 15:22:01.687
            marko Marko Mäkelä made changes -
            Fix Version/s 10.0.37 [ 22917 ]
            Fix Version/s 10.3.11 [ 23141 ]
            Fix Version/s 10.1.37 [ 23204 ]
            Fix Version/s 10.2.19 [ 23207 ]
            Fix Version/s 10.2 [ 14601 ]
            Fix Version/s 10.0 [ 16000 ]
            Fix Version/s 10.1 [ 16100 ]
            Fix Version/s 10.3 [ 22126 ]
            Resolution Fixed [ 1 ]
            Status In Progress [ 3 ] Closed [ 6 ]
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 79571 ] MariaDB v4 [ 151685 ]
            mariadb-jira-automation Jira Automation (IT) made changes -
            Zendesk Related Tickets 113492

            People

              marko Marko Mäkelä
              anikitin Andrii Nikitin (Inactive)
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.