Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-33285

Assertion `m_table' failed in ha_perfschema::rnd_end on CHECKSUM TABLE

Details

    Description

      Non-important issue at all, just logging so it can be added to filters. Extremely sporadic. Expect run times of 1h+ even with 200 threads repeating the SQL. Only one of the CHECKSUM TABLE statements is required, they can all fail with the given assertion.

      SET @@max_statement_time=0.0001;
      		 
      CHECKSUM TABLE performance_schema.rwlock_instances;
      		 
      CHECKSUM TABLE performance_schema.table_io_waits_summary_by_table;
      		 
      CHECKSUM TABLE performance_schema.users;
      		 
      CHECKSUM TABLE performance_schema.cond_instances;

      Leads to:

      11.4.0 57618265ad914824ce78108729829df13c75e224

      		 
      mariadbd: /test/bb-11.4-MDEV-7850_opt/storage/perfschema/ha_perfschema.cc:341: virtual int ha_perfschema::rnd_end(): Assertion `m_table' failed.

      Regrettably no full stack is available:

      11.4.0 57618265ad914824ce78108729829df13c75e224 (Optimized)

      		 
      Core was generated by `/test/MDEV-7850_MD161223-mariadb-11.4.0-linux-x86_64-opt/bin/mariadbd --no-defa'.
      		 
      Program terminated with signal SIGABRT, Aborted.
      		 
      #0  0x00001529ba6969fc in __pthread_mutex_lock_full (mutex=0x3541d3)
      		 
          at ./nptl/pthread_mutex_lock.c:577
      		 
      [Current thread is 1 (LWP 3490259)]
      		 
      (gdb) bt
      		 
      #0  0x00001529ba6969fc in __pthread_mutex_lock_full (mutex=0x3541d3) at ./nptl/pthread_mutex_lock.c:577
      		 
      #1  0x0000000000000000 in ?? ()

      However an earlier run captured the following frames:

      11.4.0 57618265ad914824ce78108729829df13c75e224 (Optimized)

      		 
      #1 ha_perfschema::rnd_end
      		 
      #2 ha_perfschema::rnd_end
      		 
      #3 handler::ha_rnd_end
      		 
      #4 mysql_checksum_table

      Attachments

        Issue Links

          relates to

          New Feature - A new feature of the product, which has yet to be developed. MDEV-34860 Implement MAX_EXECUTION_TIME hint

          • Critical - Crashes, loss of data, severe memory leak.
          • Stalled

          Activity

            Ascending order - Click to sort in descending order
            svoj Sergey Vojtovich added a comment -

            Note that this issue is not PFS specific.

            svoj Sergey Vojtovich added a comment - Note that this issue is not PFS specific.
            Roel Roel Van de Paar added a comment - - edited

            MDEV-34860 (or any other related feature pushed to bb-11.8-MDEV-35504-opt-hints) Seems to make this issue more pronounced (occur more often), as well as creating a stack where mysql_checksum_table is the third frame. This time I was able to capture the full stack:

            bb-11.8-MDEV-35504-opt-hints CS 11.8.0 6d92aa1cf0a042e506189b6739e03a8b45e20ee7 (Optimized)

            		 
            mariadbd: /test/bb-11.8-MDEV-35504-opt-hints_opt/storage/perfschema/ha_perfschema.cc:339: virtual int ha_perfschema::rnd_end(): Assertion `m_table' failed.

            bb-11.8-MDEV-35504-opt-hints CS 11.8.0 6d92aa1cf0a042e506189b6739e03a8b45e20ee7 (Optimized)

            		 
            Core was generated by `/test/MDEV-35504_MD100125-mariadb-11.8.0-linux-x86_64-opt/bin/mariadbd --no-def'.
            		 
            Program terminated with signal SIGABRT, Aborted.
            		 
            #0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>)at ./nptl/pthread_kill.c:44
            		 
             
            		 
            [Current thread is 1 (LWP 3589503)]
            		 
            (gdb) bt
            		 
            #0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>)at ./nptl/pthread_kill.c:44
            		 
            #1  __pthread_kill_internal (signo=6, threadid=<optimized out>)at ./nptl/pthread_kill.c:78
            		 
            #2  __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6)at ./nptl/pthread_kill.c:89
            		 
            #3  0x0000145b7584526e in __GI_raise (sig=sig@entry=6)at ../sysdeps/posix/raise.c:26
            		 
            #4  0x0000145b758288ff in __GI_abort () at ./stdlib/abort.c:79
            		 
            #5  0x0000145b7582881b in __assert_fail_base (fmt=0x145b759d01e8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x561229f176a2 "m_table", file=file@entry=0x561229ecb9c2 "/test/bb-11.8-MDEV-35504-opt-hints_opt/storage/perfschema/ha_perfschema.cc", line=line@entry=339, function=function@entry=0x561229ecba5b "virtual int ha_perfschema::rnd_end()") at ./assert/assert.c:94
            		 
            #6  0x0000145b7583b507 in __assert_fail (assertion=0x561229f176a2 "m_table", file=0x561229ecb9c2 "/test/bb-11.8-MDEV-35504-opt-hints_opt/storage/perfschema/ha_perfschema.cc", line=339, function=0x561229ecba5b "virtual int ha_perfschema::rnd_end()")at ./assert/assert.c:103
            		 
            #7  0x0000561229991e0e in ha_perfschema::rnd_end (this=0x145abcbe57a0)at /test/bb-11.8-MDEV-35504-opt-hints_opt/storage/perfschema/ha_perfschema.cc:339
            		 
            #8  0x00005612295a324e in handler::ha_rnd_end (this=0x36afd2)at /test/bb-11.8-MDEV-35504-opt-hints_opt/sql/handler.h:3562
            		 
            #9  mysql_checksum_table (thd=thd@entry=0x145b14000c68, tables=tables@entry=0x145b140183d0, check_opt=check_opt@entry=0x145b14006330)at /test/bb-11.8-MDEV-35504-opt-hints_opt/sql/sql_table.cc:12983
            		 
            #10 0x00005612294e3b44 in mysql_execute_command (thd=thd@entry=0x145b14000c68, is_called_from_prepared_stmt=false)at /test/bb-11.8-MDEV-35504-opt-hints_opt/sql/sql_parse.cc:4404
            		 
            #11 0x00005612294dd321 in mysql_parse (thd=thd@entry=0x145b14000c68, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x145b7427c420)at /test/bb-11.8-MDEV-35504-opt-hints_opt/sql/sql_parse.cc:7901
            		 
            #12 0x00005612294db7b0 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x145b14000c68, packet=packet@entry=0x145abcd12b79 "", packet_length=packet_length@entry=66, blocking=true)at /test/bb-11.8-MDEV-35504-opt-hints_opt/sql/sql_parse.cc:1903
            		 
            #13 0x00005612294dd731 in do_command (thd=thd@entry=0x145b14000c68, blocking=true)at /test/bb-11.8-MDEV-35504-opt-hints_opt/sql/sql_parse.cc:1416
            		 
            #14 0x000056122960868d in do_handle_one_connection (connect=<optimized out>, connect@entry=0x56122c02b308, put_in_cache=true)at /test/bb-11.8-MDEV-35504-opt-hints_opt/sql/sql_connect.cc:1438
            		 
            #15 0x000056122960844f in handle_one_connection (arg=arg@entry=0x56122c02b308)at /test/bb-11.8-MDEV-35504-opt-hints_opt/sql/sql_connect.cc:1350
            		 
            #16 0x00005612299942d9 in pfs_spawn_thread (arg=0x56122bfd33d8)at /test/bb-11.8-MDEV-35504-opt-hints_opt/storage/perfschema/pfs.cc:2198
            		 
            #17 0x0000145b7589ca94 in start_thread (arg=<optimized out>)at ./nptl/pthread_create.c:447
            		 
            #18 0x0000145b75929c3c in clone3 ()at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

            The semi-reduced (and again sporadic) testcase was in this case:

            INSERT INTO t1  VALUES (CONVERT(_ucs2 0x0645062C06440633 USING utf8));
            		 
            UNLOCK TABLES;
            		 
            SET MAX_STATEMENT_TIME=0.000001;
            		 
            XA RECOVER FORMAT='SQL';
            		 
            INSERT DELAYED INTO t1  VALUES(33);
            		 
            drop table mt1,mt2;
            		 
            SELECT 1;
            		 
            CHECKSUM TABLE performance_schema.table_io_waits_summary_by_table; ;
            		 
            ; ;
            		 
            SELECT 1;
            		 
            CHECKSUM TABLE performance_schema.table_io_waits_summary_by_table;#
            		 
            SELECT SLEEP(2);

            Which clearly is the same bug/issue.

            Roel Roel Van de Paar added a comment - - edited MDEV-34860 (or any other related feature pushed to bb-11.8- MDEV-35504 -opt-hints ) Seems to make this issue more pronounced (occur more often), as well as creating a stack where mysql_checksum_table is the third frame. This time I was able to capture the full stack: bb-11.8-MDEV-35504-opt-hints CS 11.8.0 6d92aa1cf0a042e506189b6739e03a8b45e20ee7 (Optimized) mariadbd: /test/bb-11.8-MDEV-35504-opt-hints_opt/storage/perfschema/ha_perfschema.cc:339: virtual int ha_perfschema::rnd_end(): Assertion `m_table' failed. bb-11.8-MDEV-35504-opt-hints CS 11.8.0 6d92aa1cf0a042e506189b6739e03a8b45e20ee7 (Optimized) Core was generated by `/test/MDEV-35504_MD100125-mariadb-11.8.0-linux-x86_64-opt/bin/mariadbd --no-def'. Program terminated with signal SIGABRT, Aborted. #0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>)at ./nptl/pthread_kill.c:44   [Current thread is 1 (LWP 3589503)] (gdb) bt #0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>)at ./nptl/pthread_kill.c:44 #1 __pthread_kill_internal (signo=6, threadid=<optimized out>)at ./nptl/pthread_kill.c:78 #2 __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6)at ./nptl/pthread_kill.c:89 #3 0x0000145b7584526e in __GI_raise (sig=sig@entry=6)at ../sysdeps/posix/raise.c:26 #4 0x0000145b758288ff in __GI_abort () at ./stdlib/abort.c:79 #5 0x0000145b7582881b in __assert_fail_base (fmt=0x145b759d01e8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x561229f176a2 "m_table", file=file@entry=0x561229ecb9c2 "/test/bb-11.8-MDEV-35504-opt-hints_opt/storage/perfschema/ha_perfschema.cc", line=line@entry=339, function=function@entry=0x561229ecba5b "virtual int ha_perfschema::rnd_end()") at ./assert/assert.c:94 #6 0x0000145b7583b507 in __assert_fail (assertion=0x561229f176a2 "m_table", file=0x561229ecb9c2 "/test/bb-11.8-MDEV-35504-opt-hints_opt/storage/perfschema/ha_perfschema.cc", line=339, function=0x561229ecba5b "virtual int ha_perfschema::rnd_end()")at ./assert/assert.c:103 #7 0x0000561229991e0e in ha_perfschema::rnd_end (this=0x145abcbe57a0)at /test/bb-11.8-MDEV-35504-opt-hints_opt/storage/perfschema/ha_perfschema.cc:339 #8 0x00005612295a324e in handler::ha_rnd_end (this=0x36afd2)at /test/bb-11.8-MDEV-35504-opt-hints_opt/sql/handler.h:3562 #9 mysql_checksum_table (thd=thd@entry=0x145b14000c68, tables=tables@entry=0x145b140183d0, check_opt=check_opt@entry=0x145b14006330)at /test/bb-11.8-MDEV-35504-opt-hints_opt/sql/sql_table.cc:12983 #10 0x00005612294e3b44 in mysql_execute_command (thd=thd@entry=0x145b14000c68, is_called_from_prepared_stmt=false)at /test/bb-11.8-MDEV-35504-opt-hints_opt/sql/sql_parse.cc:4404 #11 0x00005612294dd321 in mysql_parse (thd=thd@entry=0x145b14000c68, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x145b7427c420)at /test/bb-11.8-MDEV-35504-opt-hints_opt/sql/sql_parse.cc:7901 #12 0x00005612294db7b0 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x145b14000c68, packet=packet@entry=0x145abcd12b79 "", packet_length=packet_length@entry=66, blocking=true)at /test/bb-11.8-MDEV-35504-opt-hints_opt/sql/sql_parse.cc:1903 #13 0x00005612294dd731 in do_command (thd=thd@entry=0x145b14000c68, blocking=true)at /test/bb-11.8-MDEV-35504-opt-hints_opt/sql/sql_parse.cc:1416 #14 0x000056122960868d in do_handle_one_connection (connect=<optimized out>, connect@entry=0x56122c02b308, put_in_cache=true)at /test/bb-11.8-MDEV-35504-opt-hints_opt/sql/sql_connect.cc:1438 #15 0x000056122960844f in handle_one_connection (arg=arg@entry=0x56122c02b308)at /test/bb-11.8-MDEV-35504-opt-hints_opt/sql/sql_connect.cc:1350 #16 0x00005612299942d9 in pfs_spawn_thread (arg=0x56122bfd33d8)at /test/bb-11.8-MDEV-35504-opt-hints_opt/storage/perfschema/pfs.cc:2198 #17 0x0000145b7589ca94 in start_thread (arg=<optimized out>)at ./nptl/pthread_create.c:447 #18 0x0000145b75929c3c in clone3 ()at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78 The semi-reduced (and again sporadic) testcase was in this case: INSERT INTO t1 VALUES ( CONVERT (_ucs2 0x0645062C06440633 USING utf8)); UNLOCK TABLES; SET MAX_STATEMENT_TIME=0.000001; XA RECOVER FORMAT= 'SQL' ; INSERT DELAYED INTO t1 VALUES (33); drop table mt1,mt2; SELECT 1; CHECKSUM TABLE performance_schema.table_io_waits_summary_by_table; ; ; ; SELECT 1; CHECKSUM TABLE performance_schema.table_io_waits_summary_by_table;# SELECT SLEEP(2); Which clearly is the same bug/issue.
            serg Sergei Golubchik added a comment -

            ok to push

            serg Sergei Golubchik added a comment - ok to push

            People

              svoj Sergey Vojtovich
              Roel Roel Van de Paar
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.