Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-33731

Server crashes when deleting partitions from a table with spider engine

Details

    Description

      --source include/have_partition.inc
       
      INSTALL PLUGIN Spider SONAME 'ha_spider.so';
      CREATE TABLE t (a INT) ENGINE=Spider PARTITION BY LIST (a) PARTITIONS 2 (PARTITION p1 VALUES IN (0,1),PARTITION p2 VALUES IN (2,3));
      DELETE FROM t PARTITION (p2);
      

      Leads to:

      10.5.25 7d36919f4b73a2507555ed101ccd02cb266dcc52 (Optimized)

      Core was generated by `/test/MD200324-mariadb-10.5.25-linux-x86_64-opt/bin/mariadbd --no-defaults --ma'.
      Program terminated with signal SIGSEGV, Segmentation fault.
      #0  0x0000150314141452 in ha_spider::need_info_for_auto_inc (
          this=0x1502943217d0) at /test/server_opt/storage/spider/ha_spider.cc:9433
      9433	  DBUG_RETURN((
      [Current thread is 1 (Thread 0x150328078700 (LWP 3742349))]
      (gdb) bt
      #0  0x0000150314141452 in ha_spider::need_info_for_auto_inc (this=0x1502943217d0) at /test/server_opt/storage/spider/ha_spider.cc:9433
      #1  0x00005633cb323766 in ha_partition::need_info_for_auto_inc (this=0x1502942c8810) at /test/server_opt/sql/ha_partition.cc:10761
      #2  ha_partition::external_lock (this=0x1502942c8810, thd=<optimized out>, lock_type=1) at /test/server_opt/sql/ha_partition.cc:4189
      #3  0x00005633cb0b13c4 in handler::ha_external_lock (this=0x1502942c8810, thd=thd@entry=0x150294000c58, lock_type=lock_type@entry=1) at /test/server_opt/sql/handler.cc:6792
      #4  0x00005633cb1bfc60 in lock_external (count=<optimized out>, tables=0x150294010ec8, thd=0x150294000c58) at /test/server_opt/sql/lock.cc:396
      #5  mysql_lock_tables (thd=thd@entry=0x150294000c58, sql_lock=sql_lock@entry=0x150294010e88, flags=flags@entry=0) at /test/server_opt/sql/lock.cc:341
      #6  0x00005633cb1c09d7 in mysql_lock_tables (thd=thd@entry=0x150294000c58, tables=0x150294010e80, count=count@entry=1, flags=flags@entry=0) at /test/server_opt/sql/lock.cc:304
      #7  0x00005633cae0ee7f in lock_tables (thd=thd@entry=0x150294000c58, tables=0x1502940105c0, count=<optimized out>, flags=flags@entry=0) at /test/server_opt/sql/sql_base.cc:5497
      #8  0x00005633cae10bb2 in open_and_lock_tables (thd=thd@entry=0x150294000c58, options=<optimized out>, tables=<optimized out>, tables@entry=0x1502940105c0, derived=derived@entry=true, flags=flags@entry=0, prelocking_strategy=prelocking_strategy@entry=0x150328076580) at /test/server_opt/sql/sql_base.cc:5257
      #9  0x00005633cb2217ec in open_and_lock_tables (flags=0, derived=true, tables=0x1502940105c0, thd=0x150294000c58) at /test/server_opt/sql/sql_base.h:508
      #10 mysql_delete (thd=thd@entry=0x150294000c58, table_list=0x1502940105c0, conds=<optimized out>, order_list=order_list@entry=0x150294005738, limit=18446744073709551615, options=0, result=0x0) at /test/server_opt/sql/sql_delete.cc:343
      #11 0x00005633cae809ff in mysql_execute_command (thd=0x150294000c58) at /test/server_opt/sql/sql_limit.h:78
      #12 0x00005633cae6e3d6 in mysql_parse (thd=0x150294000c58, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>, is_com_multi=<optimized out>, is_next_command=<optimized out>) at /test/server_opt/sql/sql_parse.cc:8196
      #13 0x00005633cae7b605 in dispatch_command (command=COM_QUERY, thd=0x150294000c58, packet=<optimized out>, packet_length=<optimized out>, is_com_multi=<optimized out>, is_next_command=<optimized out>) at /test/server_opt/sql/sql_class.h:1329
      #14 0x00005633cae7dd73 in do_command (thd=0x150294000c58) at /test/server_opt/sql/sql_parse.cc:1375
      #15 0x00005633caf8b711 in do_handle_one_connection (connect=<optimized out>, connect@entry=0x5633cdcea0f8, put_in_cache=put_in_cache@entry=true) at /test/server_opt/sql/sql_connect.cc:1415
      #16 0x00005633caf8bb9d in handle_one_connection (arg=arg@entry=0x5633cdcea0f8) at /test/server_opt/sql/sql_connect.cc:1317
      #17 0x00005633cb338fe0 in pfs_spawn_thread (arg=0x5633cdbf6328) at /test/server_opt/storage/perfschema/pfs.cc:2201
      #18 0x0000150329c45609 in start_thread (arg=<optimized out>) at pthread_create.c:477
      #19 0x0000150329831133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
      

      Bug confirmed present in:
      MariaDB: 10.5.25 (dbg), 10.5.25 (opt), 10.6.18 (dbg), 10.6.18 (opt), 10.11.8 (dbg), 10.11.8 (opt), 11.0.6 (dbg), 11.0.6 (opt)

      Bug (or feature/syntax) confirmed not present in:
      MariaDB: 10.4.33 (dbg), 10.4.33 (opt), 10.4.34 (opt), 11.1.4 (dbg), 11.1.4 (opt), 11.2.4 (dbg), 11.2.4 (opt), 11.3.2 (dbg), 11.3.2 (opt), 11.4.2 (dbg), 11.4.2 (opt), 11.5.0 (dbg), 11.5.0 (opt)

      Attachments

        Issue Links

          Activity

            ycp Yuchen Pei added a comment -

            The problem is that during the DELETE statement, only the relevant partition is opened (p2):

            10.5 3247c401730cd6a5e244e33768f1ea2b7b26359e

            int ha_partition::open_read_partitions(char *name_buff, size_t name_buff_size)
            {
            //  [... 5 lines elided]
              file= m_file;
              m_file_sample= NULL;
              do
              {
                int n_file= (int)(file-m_file);
                int is_open= bitmap_is_set(&m_opened_partitions, n_file);
                int should_be_open= bitmap_is_set(&m_part_info->read_partitions, n_file);
             
            //  [... 13 lines elided]
                if (!is_open && should_be_open) // p1 is skipped here
                {
            //  [... 9 lines elided]
                  error= (*file)->ha_open(table, name_buff, m_mode,
                                          m_open_test_lock | HA_OPEN_NO_PSI_CALL);
            //  [... 5 lines elided]
                }
            //  [... 3 lines elided]
              } while (*(++file));
            //  [... 3 lines elided]
            }

            But then need_info_for_auto_inc() is called on the handler of both partitions:

            10.5 3247c401730cd6a5e244e33768f1ea2b7b26359e

            bool ha_partition::need_info_for_auto_inc()
            {
              handler **file= m_file;
              DBUG_ENTER("ha_partition::need_info_for_auto_inc");
             
              do
              {
                if ((*file)->need_info_for_auto_inc())
                {
                  /* We have to get new auto_increment values from handler */
                  part_share->auto_inc_initialized= FALSE;
                  DBUG_RETURN(TRUE);
                }
              } while (*(++file));
              DBUG_RETURN(FALSE);
            }

            holyfoot: So the question is: does this make sense, or is it a partition bug? If it makes sense, what should need_info_for_auto_inc() return for a partition that is not opened?

            ycp Yuchen Pei added a comment - The problem is that during the DELETE statement, only the relevant partition is opened (p2): 10.5 3247c401730cd6a5e244e33768f1ea2b7b26359e int ha_partition::open_read_partitions( char *name_buff, size_t name_buff_size) { // [... 5 lines elided] file= m_file; m_file_sample= NULL; do { int n_file= ( int )(file-m_file); int is_open= bitmap_is_set(&m_opened_partitions, n_file); int should_be_open= bitmap_is_set(&m_part_info->read_partitions, n_file);   // [... 13 lines elided] if (!is_open && should_be_open) // p1 is skipped here { // [... 9 lines elided] error= (*file)->ha_open(table, name_buff, m_mode, m_open_test_lock | HA_OPEN_NO_PSI_CALL); // [... 5 lines elided] } // [... 3 lines elided] } while (*(++file)); // [... 3 lines elided] } But then need_info_for_auto_inc() is called on the handler of both partitions: 10.5 3247c401730cd6a5e244e33768f1ea2b7b26359e bool ha_partition::need_info_for_auto_inc() { handler **file= m_file; DBUG_ENTER( "ha_partition::need_info_for_auto_inc" );   do { if ((*file)->need_info_for_auto_inc()) { /* We have to get new auto_increment values from handler */ part_share->auto_inc_initialized= FALSE; DBUG_RETURN(TRUE); } } while (*(++file)); DBUG_RETURN(FALSE); } holyfoot : So the question is: does this make sense, or is it a partition bug? If it makes sense, what should need_info_for_auto_inc() return for a partition that is not opened?
            holyfoot Alexey Botchkov added a comment - - edited

            That seems to be a bug.
            We need to loop over the m_locked_partitions only in the ::need_info_for_auto_inc().

            holyfoot Alexey Botchkov added a comment - - edited That seems to be a bug. We need to loop over the m_locked_partitions only in the ::need_info_for_auto_inc().
            ycp Yuchen Pei added a comment - - edited

            Thanks holyfoot, that does seem to fix it (see the commit below). Could you provide an explanation why ha_partition::need_info_for_auto_inc() should only iterate over m_locked_partitions so that I can add it to the commit message?

            upstream/bb-10.5-mdev-33731 upstream/bb-10.5-ycp 09fd15f220bf4ddeb7abbc3db07b7ffd9cf4e5b7
            MDEV-33731 Only iterate over m_locked_partitions in update_next_auto_inc_val()
            

            The reason this bug does not reproduce in 10.4 is because the following commit adds a call to ha_partition::need_info_for_auto_inc() in ha_partition::external_lock(). This commit can be applied cleanly to 10.4, so question for serg: why was this commit not applied to 10.4?

            commit c5904702cdae585203fb1b7948d9008a6994c8a4
            Author: Sergei Golubchik <serg@mariadb.org>
            Date:   Wed Nov 29 17:19:14 2023 +0100
             
                cleanup: remove partition-specific code around update_auto_increment()
             
             sql/ha_partition.cc | 31 ++-----------------------------
             sql/ha_partition.h  |  3 +--
             2 files changed, 3 insertions(+), 31 deletions(-)
            

            ycp Yuchen Pei added a comment - - edited Thanks holyfoot , that does seem to fix it (see the commit below). Could you provide an explanation why ha_partition::need_info_for_auto_inc() should only iterate over m_locked_partitions so that I can add it to the commit message? upstream/bb-10.5-mdev-33731 upstream/bb-10.5-ycp 09fd15f220bf4ddeb7abbc3db07b7ffd9cf4e5b7 MDEV-33731 Only iterate over m_locked_partitions in update_next_auto_inc_val() The reason this bug does not reproduce in 10.4 is because the following commit adds a call to ha_partition::need_info_for_auto_inc() in ha_partition::external_lock() . This commit can be applied cleanly to 10.4, so question for serg : why was this commit not applied to 10.4? commit c5904702cdae585203fb1b7948d9008a6994c8a4 Author: Sergei Golubchik <serg@mariadb.org> Date: Wed Nov 29 17:19:14 2023 +0100   cleanup: remove partition-specific code around update_auto_increment()   sql/ha_partition.cc | 31 ++----------------------------- sql/ha_partition.h | 3 +-- 2 files changed, 3 insertions(+), 31 deletions(-)

            it was part of a fix for MDEV-32839, which was fixed in 10.5

            serg Sergei Golubchik added a comment - it was part of a fix for MDEV-32839 , which was fixed in 10.5
            ycp Yuchen Pei added a comment -

            Hi holyfoot, ptal thanks

            bb-10.5-mdev-33731 bb-10.5-ycp 695bb64ec82250fe052ccadfb79f70aea1759db7
            MDEV-33731 Only iterate over m_locked_partitions in update_next_auto_inc_val()
             
            Only locked will participate in the query in this case.
            

            ycp Yuchen Pei added a comment - Hi holyfoot , ptal thanks bb-10.5-mdev-33731 bb-10.5-ycp 695bb64ec82250fe052ccadfb79f70aea1759db7 MDEV-33731 Only iterate over m_locked_partitions in update_next_auto_inc_val()   Only locked will participate in the query in this case.
            ycp Yuchen Pei added a comment -

            thanks for the review - pushed b7b58a231099e21513f7a60835759260106b81a3 to 10.5

            ycp Yuchen Pei added a comment - thanks for the review - pushed b7b58a231099e21513f7a60835759260106b81a3 to 10.5

            People

              ycp Yuchen Pei
              ramesh Ramesh Sivaraman
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.