Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-11742

[Draft] InnoDB: Failing assertion: mysql_table

Details

    Description

      Encountered during tests on bb-10.2-monty. Information in the report is related to that test run, but I couldn't reproduce it so far neither on the same tree, nor on 10.2.

      bb-10.2-monty 4ec3e60ae3734 of Jan 6

      2017-01-07 19:38:28 140022359500544 [Note] InnoDB: Read page 0 from tablespace forspace 418 name testdb_N/t1_base1_N key_id 0 encryption 0 handle 52
      2017-01-07 19:38:28 140022351107840 [ERROR] InnoDB: Failed to find tablespace for table `testdb_N`.`t1_merge2_N` in the cache. Attempting to load the tablespace with space id 474
      2017-01-07 19:38:28 140022351107840 [ERROR] InnoDB: Operating system error number 2 in a file operation.
      2017-01-07 19:38:28 140022351107840 [ERROR] InnoDB: The error means the system cannot find the path specified.
      2017-01-07 19:38:28 140022351107840 [ERROR] InnoDB: Cannot open datafile for read-only: './testdb_N/t1_merge2_N.ibd' OS error: 71
      2017-01-07 19:38:28 140022351107840 [ERROR] InnoDB: Operating system error number 2 in a file operation.
      2017-01-07 19:38:28 140022351107840 [ERROR] InnoDB: The error means the system cannot find the path specified.
      2017-01-07 19:38:28 140022351107840 [ERROR] InnoDB: Could not find a valid tablespace file for `testdb_N/t1_merge2_N`. Please refer to http://dev.mysql.com/doc/refman/5.7/en/innodb-troubleshooting-datadict.html for how to resolve the issue.
      2017-01-07 19:38:28 140022351107840 [Warning] InnoDB: Cannot calculate statistics for table `testdb_N`.`t1_merge2_N` because the .ibd file is missing. Please refer to http://dev.mysql.com/doc/refman/5.7/en/innodb-troubleshooting.html for how to resolve the issue.
      2017-01-07 19:38:28 0x7f597e7f9700  InnoDB: Assertion failure in file /home/elenst/git/bb-10.2-monty/storage/innobase/handler/ha_innodb.cc line 23607
      InnoDB: Failing assertion: mysql_table
      

      # 2017-01-07T19:38:51 [1224] #6  0x00007f59c1985df5 in ut_dbg_assertion_failed (expr=0x7f59c1ec6929 "mysql_table", file=0x7f59c1eb9718 "/home/elenst/git/bb-10.2-monty/storage/innobase/handler/ha_innodb.cc", line=23607) at /home/elenst/git/bb-10.2-monty/storage/innobase/ut/ut0dbg.cc:59
      # 2017-01-07T19:38:51 [1224] #7  0x00007f59c1777185 in innobase_init_vc_templ (table=0x7f597c870688) at /home/elenst/git/bb-10.2-monty/storage/innobase/handler/ha_innodb.cc:23607
      # 2017-01-07T19:38:51 [1224] #8  0x00007f59c18cdf26 in row_purge_parse_undo_rec (node=0x7f59be1f1688, undo_rec=0x7f597d57c630 "\035\330\016", updated_extern=0x7f597e7f8d2e, thr=0x7f59be206660) at /home/elenst/git/bb-10.2-monty/storage/innobase/row/row0purge.cc:886
      # 2017-01-07T19:38:51 [1224] #9  0x00007f59c18ce45c in row_purge (node=0x7f59be1f1688, undo_rec=0x7f597d57c630 "\035\330\016", thr=0x7f59be206660) at /home/elenst/git/bb-10.2-monty/storage/innobase/row/row0purge.cc:1028
      # 2017-01-07T19:38:51 [1224] #10 0x00007f59c18ce6d6 in row_purge_step (thr=0x7f59be206660) at /home/elenst/git/bb-10.2-monty/storage/innobase/row/row0purge.cc:1110
      # 2017-01-07T19:38:51 [1224] #11 0x00007f59c1858050 in que_thr_step (thr=0x7f59be206660) at /home/elenst/git/bb-10.2-monty/storage/innobase/que/que0que.cc:1056
      # 2017-01-07T19:38:51 [1224] #12 0x00007f59c185824f in que_run_threads_low (thr=0x7f59be206660) at /home/elenst/git/bb-10.2-monty/storage/innobase/que/que0que.cc:1118
      # 2017-01-07T19:38:51 [1224] #13 0x00007f59c185840f in que_run_threads (thr=0x7f59be206660) at /home/elenst/git/bb-10.2-monty/storage/innobase/que/que0que.cc:1158
      # 2017-01-07T19:38:51 [1224] #14 0x00007f59c191de80 in srv_task_execute () at /home/elenst/git/bb-10.2-monty/storage/innobase/srv/srv0srv.cc:2721
      # 2017-01-07T19:38:51 [1224] #15 0x00007f59c191dfff in srv_worker_thread (arg=0x0) at /home/elenst/git/bb-10.2-monty/storage/innobase/srv/srv0srv.cc:2771
      # 2017-01-07T19:38:51 [1224] #16 0x00007f59bf61fe9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
      # 2017-01-07T19:38:51 [1224] #17 0x00007f59beb36cbd in clone () from /lib/x86_64-linux-gnu/libc.so.6
      

      Trying to get some variables.
      Some pointers may be invalid and cause the dump to abort.
      Query (0x0): 
      Connection ID (thread ID): 5
      Status: NOT_KILLED
      

      RQG experimental 67b190f486

      perl /home/elenst/git/rqg/runall-new.pl --no-mask --seed=1483810215 --threads=8 --duration=600 --queries=100M --validators=TransformerNoComparator --transformers=ExecuteAsPreparedTwice,DisableOptimizations,EnableOptimizations,InlineVirtualColumns,ExecuteAsTrigger --redefine=conf/mariadb/general-workarounds.yy --redefine=conf/mariadb/10.0-features-redefine.yy --mysqld=--log_output=FILE --views --grammar=conf/runtime/WL5004_sql.yy --gendata=conf/runtime/WL5004_data.zz --reporters=QueryTimeout,Backtrace,ErrorLog,Deadlock,CrashRestart --restart-timeout=20 --mtr-build-thread=72 --basedir1=/home/elenst/git/bb-10.2-monty --vardir1=/home/elenst/test_results/monty-4/current1_1
      

      Attachments

        1. mysql.err
          618 kB
        2. mysql.log.gz
          233 kB
        3. threads
          37 kB

        Issue Links

          Activity

            elenst Elena Stepanova created issue -
            elenst Elena Stepanova made changes -
            Field Original Value New Value
            Fix Version/s 10.2 [ 14601 ]
            marko Marko Mäkelä made changes -

            This is a curious bug. While the stated revision is not available, the line number matches mariadb-10.2.3. I see that back then, row_purge_parse_undo_rec() attempted to open the table from the SQL layer even before checking if node->table->ibd_file_missing. This is guaranteed to fail, of course.

            Any race condition between RENAME TABLE and purge should be prevented by MDL.

            A likely root cause for the failure is that the server was previously killed during RENAME TABLE. This would be fixed by MDEV-14717.

            It turns out that I fixed this bug in MariaDB 10.2.6 when cleaning up an Oracle bug fix. In MySQL 5.7.20, the crash scenario is still possible, because node->table->ibd_file_missing is only being checked after the innobase_init_vc_templ() call.

            marko Marko Mäkelä added a comment - This is a curious bug. While the stated revision is not available, the line number matches mariadb-10.2.3. I see that back then, row_purge_parse_undo_rec() attempted to open the table from the SQL layer even before checking if node->table->ibd_file_missing . This is guaranteed to fail, of course. Any race condition between RENAME TABLE and purge should be prevented by MDL. A likely root cause for the failure is that the server was previously killed during RENAME TABLE . This would be fixed by MDEV-14717 . It turns out that I fixed this bug in MariaDB 10.2.6 when cleaning up an Oracle bug fix. In MySQL 5.7.20, the crash scenario is still possible, because node->table->ibd_file_missing is only being checked after the innobase_init_vc_templ() call.
            marko Marko Mäkelä made changes -
            issue.field.resolutiondate 2017-12-20 11:21:33.0 2017-12-20 11:21:33.884
            marko Marko Mäkelä made changes -
            Component/s Storage Engine - InnoDB [ 10129 ]
            Fix Version/s 10.3.1 [ 22532 ]
            Fix Version/s 10.2.6 [ 22527 ]
            Assignee Elena Stepanova [ elenst ] Marko Mäkelä [ marko ]
            Resolution Fixed [ 1 ]
            Status Open [ 1 ] Closed [ 6 ]
            marko Marko Mäkelä made changes -
            Labels upstream
            marko Marko Mäkelä made changes -
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 79124 ] MariaDB v4 [ 151501 ]

            People

              marko Marko Mäkelä
              elenst Elena Stepanova
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.