Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-29456

Spider hangs in 'Waiting for table metadata lock' state on CREATE TABLE after LOCK TABLES

Details

    Description

      This is a serious locking bug where Spider fails to see that table t was already created previously (and not locked?).
      Affects optimized and debug builds. Issue is not sporadic.

      INSTALL PLUGIN Spider SONAME 'ha_spider.so';
      CREATE USER Spider@localhost IDENTIFIED BY 'PWD123';
      CREATE SERVER srv FOREIGN DATA WRAPPER MYSQL OPTIONS (SOCKET '../socket.sock',DATABASE 'test',user 'Spider',PASSWORD 'PWD123');
      CREATE TABLE t (c INT) ENGINE=InnoDB;
      CREATE TABLE t1 (c INT) ENGINE=Spider;
      CREATE TABLE t2 (c INT) ENGINE=Spider COMMENT='WRAPPER "mysql",srv "srv",TABLE "t"';
      LOCK TABLES t2 WRITE;
      LOCK TABLES t1 READ;
      CREATE TABLE t (c INT) ENGINE=Spider;
      

      Leads to:

      10.11.0 bc563f1a4b0b38de3b41fd0f0d3d8b7f1aacbd8b (Optimized)

      10.11.0-dbg>LOCK TABLES t1 READ;
      ERROR 1429 (HY000): Unable to connect to foreign data source: localhost
      10.11.0-dbg>CREATE TABLE t(c INT) ENGINE=Spider;
      # Hang
      

      10.11.0 bc563f1a4b0b38de3b41fd0f0d3d8b7f1aacbd8b (Optimized)

      10.11.0-opt>SHOW FULL PROCESSLIST;
      +----+-------------+-----------+------+---------+------+----------------------------------------------------+-------------------------------------+----------+
      | Id | User        | Host      | db   | Command | Time | State                                              | Info                                | Progress |
      +----+-------------+-----------+------+---------+------+----------------------------------------------------+-------------------------------------+----------+
      |  5 | root        | localhost | test | Query   |   77 | Waiting for table metadata lock                    | CREATE TABLE t(c INT) ENGINE=Spider |    0.000 |
      |  6 | system user |           | NULL | Daemon  | NULL | Spider table background statistics action handler  | NULL                                |    0.000 |
      |  7 | system user |           | NULL | Daemon  | NULL | Spider table background cardinality action handler | NULL                                |    0.000 |
      |  8 | system user |           | NULL | Daemon  | NULL | Spider table background statistics action handler  | NULL                                |    0.000 |
      |  9 | system user |           | NULL | Daemon  | NULL | Spider table background cardinality action handler | NULL                                |    0.000 |
      | 10 | system user |           | NULL | Daemon  | NULL | Spider table background statistics action handler  | NULL                                |    0.000 |
      | 11 | system user |           | NULL | Daemon  | NULL | Spider table background statistics action handler  | NULL                                |    0.000 |
      | 12 | system user |           | NULL | Sleep   |   77 | Reset for next command                             | NULL                                |    0.000 |
      | 13 | system user |           | NULL | Daemon  | NULL | Spider table background statistics action handler  | NULL                                |    0.000 |
      | 14 | system user |           | NULL | Daemon  | NULL | Spider table background cardinality action handler | NULL                                |    0.000 |
      | 17 | system user |           | NULL | Daemon  | NULL | Spider table background cardinality action handler | NULL                                |    0.000 |
      | 18 | system user |           | NULL | Daemon  | NULL | Spider table background cardinality action handler | NULL                                |    0.000 |
      | 20 | system user |           | NULL | Daemon  | NULL | Spider table background cardinality action handler | NULL                                |    0.000 |
      | 19 | system user |           | NULL | Daemon  | NULL | Spider table background cardinality action handler | NULL                                |    0.000 |
      | 24 | system user |           | NULL | Daemon  | NULL | Spider table background cardinality action handler | NULL                                |    0.000 |
      | 15 | system user |           | NULL | Daemon  | NULL | Spider table background cardinality action handler | NULL                                |    0.000 |
      | 22 | system user |           | NULL | Daemon  | NULL | Spider table background statistics action handler  | NULL                                |    0.000 |
      | 16 | system user |           | NULL | Daemon  | NULL | Spider table background cardinality action handler | NULL                                |    0.000 |
      | 21 | system user |           | NULL | Daemon  | NULL | Spider table background statistics action handler  | NULL                                |    0.000 |
      | 23 | system user |           | NULL | Daemon  | NULL | Spider table background statistics action handler  | NULL                                |    0.000 |
      | 25 | system user |           | NULL | Daemon  | NULL | Spider table background statistics action handler  | NULL                                |    0.000 |
      | 26 | spider      | localhost | test | Sleep   |   77 |                                                    | NULL                                |    0.000 |
      | 27 | root        | localhost | test | Query   |    0 | starting                                           | SHOW FULL PROCESSLIST               |    0.000 |
      +----+-------------+-----------+------+---------+------+----------------------------------------------------+-------------------------------------+----------+
      23 rows in set (0.001 sec)
      

      Attachments

        Issue Links

          Activity

            Please note that SET GLOBAL spider_same_server_link=ON; is not used here.

            Roel Roel Van de Paar added a comment - Please note that SET GLOBAL spider_same_server_link=ON; is not used here.

            So if I understand you correctly, for items 1+3, you are saying that LOCK TABLES t2 WRITE; - executed by the root at the CLI - will trigger the Spider engine to automatically lock table t - in a background (and non-user) thread, which uses the Spider UserID, and is done by the server/the Spider SE - because t is being referenced/linked in the t2 table definition?

            Mostly yes, but there is no background thread. The thread handling the user CLI connection also handles the Spider (background) connection.

            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) added a comment - So if I understand you correctly, for items 1+3, you are saying that LOCK TABLES t2 WRITE; - executed by the root at the CLI - will trigger the Spider engine to automatically lock table t - in a background (and non-user) thread, which uses the Spider UserID, and is done by the server/the Spider SE - because t is being referenced/linked in the t2 table definition? Mostly yes, but there is no background thread . The thread handling the user CLI connection also handles the Spider (background) connection.

            I believe that the following test case found in the investigation for MDEV-29484 reproduces the present bug.

            --disable_query_log
            --disable_result_log
            --source ../../t/test_init.inc
            --enable_result_log
            --enable_query_log
             
            --connection child2_1
            CREATE DATABASE auto_test_remote;
            USE auto_test_remote;
            eval CREATE TABLE tbl_a (
                a INT
            ) $CHILD2_1_ENGINE $CHILD2_1_CHARSET;
             
            --connection master_1
            CREATE DATABASE auto_test_local;
            USE auto_test_local;
            eval CREATE TABLE tbl_a (
                a INT
            ) $MASTER_1_ENGINE $MASTER_1_CHARSET COMMENT='table "tbl_a", srv "s_2_1"';
            eval CREATE TABLE tbl_b (
                a INT
            ) $MASTER_1_ENGINE $MASTER_1_CHARSET;
             
            LOCK TABLES tbl_a WRITE;
            --error 1429
            LOCK TABLES tbl_b READ, tbl_a READ;
             
            --connection master_1
            DROP DATABASE IF EXISTS auto_test_local;
             
            --connection child2_1
            DROP DATABASE IF EXISTS auto_test_remote;
             
            --disable_query_log
            --disable_result_log
            --source ../t/test_deinit.inc
            --enable_query_log
            --enable_result_log
            

            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) added a comment - I believe that the following test case found in the investigation for MDEV-29484 reproduces the present bug. --disable_query_log --disable_result_log --source ../../t/test_init.inc --enable_result_log --enable_query_log --connection child2_1 CREATE DATABASE auto_test_remote; USE auto_test_remote; eval CREATE TABLE tbl_a ( a INT ) $CHILD2_1_ENGINE $CHILD2_1_CHARSET; --connection master_1 CREATE DATABASE auto_test_local; USE auto_test_local; eval CREATE TABLE tbl_a ( a INT ) $MASTER_1_ENGINE $MASTER_1_CHARSET COMMENT= 'table "tbl_a", srv "s_2_1"' ; eval CREATE TABLE tbl_b ( a INT ) $MASTER_1_ENGINE $MASTER_1_CHARSET; LOCK TABLES tbl_a WRITE; --error 1429 LOCK TABLES tbl_b READ , tbl_a READ ; --connection master_1 DROP DATABASE IF EXISTS auto_test_local; --connection child2_1 DROP DATABASE IF EXISTS auto_test_remote; --disable_query_log --disable_result_log --source ../t/test_deinit.inc --enable_query_log --enable_result_log
            ycp Yuchen Pei added a comment -

            I can reprod the hang at 11.1 (4e5b771e980edfdad5c5414aa62c81d409d585a4) when running the test case in the previous comment.

            ycp Yuchen Pei added a comment - I can reprod the hang at 11.1 (4e5b771e980edfdad5c5414aa62c81d409d585a4) when running the test case in the previous comment.
            ycp Yuchen Pei added a comment -

            fixed by MDEV-30014

            ycp Yuchen Pei added a comment - fixed by MDEV-30014

            People

              ycp Yuchen Pei
              Roel Roel Van de Paar
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.