[MDEV-23701] Assertion `open_tables == __null' failed in translog_flush_buffers *and* Assertion `log_descriptor.bc.buffer->prev_last_lsn == ((LSN)0)' failed. Created: 2020-09-09  Updated: 2023-11-28

Status: Confirmed
Project: MariaDB Server
Component/s: Authentication and Privilege System, Storage Engine - Aria
Affects Version/s: 10.1, 10.2, 10.3, 10.4, 10.5, 10.6, 10.7, 10.8, 10.9, 10.10, 10.11
Fix Version/s: 10.4, 10.5, 10.6

Type: Bug Priority: Critical
Reporter: Roel Van de Paar Assignee: Oleksandr Byelkin
Resolution: Unresolved Votes: 0
Labels: affects-tests, hang, regression, regression-10.4, security, thread_hang

Issue Links:
Relates
relates to MDEV-17225 Assertion `log_descriptor.bc.buffer->... Open
relates to MDEV-22883 Assertion `!parse_error || lex.sphead... Stalled
relates to MDEV-25533 Assertion `!((system_charset_info)->c... Open

 Description   

Bug is sporadic.

SET @save_aria_checkpoint_log_activity=@@ARIA_CHECKPOINT_LOG_ACTIVITY;
SET @save_aria_group_commmit=@@ARIA_GROUP_COMMIT;
SET @save_aria_checkpoint_interval=@@ARIA_CHECKPOINT_INTERVAL;
SET @save_aria_group_commit_interval=@@ARIA_GROUP_COMMIT_INTERVAL;
 
SET GLOBAL ARIA_CHECKPOINT_LOG_ACTIVITY=1;
SET GLOBAL ARIA_GROUP_COMMIT=HARD;
SET GLOBAL ARIA_CHECKPOINT_INTERVAL=1;
# Wait 4 seconds for other commits to join in group commit
SET GLOBAL ARIA_GROUP_COMMIT_INTERVAL=400000000;
CREATE TABLE t1 (a int) engine=aria transactional=1;
INSERT INTO t1 values(1);
DROP TABLE t1;
 
SET GLOBAL ARIA_CHECKPOINT_LOG_ACTIVITY=@save_aria_checkpoint_log_activity;
SET GLOBAL ARIA_GROUP_COMMIT=@save_aria_group_commmit;
SET GLOBAL ARIA_CHECKPOINT_INTERVAL=@save_aria_checkpoint_interval;
SET GLOBAL ARIA_GROUP_COMMIT_INTERVAL=@save_aria_group_commit_interval;

Leads to:

6  0x00007f1b0ba6db1a in __assert_fail_base (fmt=0x7f1b0bbc0f58 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x187f7e0 "log_descriptor.bc.buffer->prev_last_lsn == ((LSN)0)", file=file@entry=0x1879ec8 "/my/maria-10.3/storage/maria/ma_loghandler.c", line=line@entry=8006, function=function@entry=0x1881170 <__PRETTY_FUNCTION__.17860> "translog_flush_buffers") at assert.c:92
#7  0x00007f1b0ba6db92 in __GI___assert_fail (assertion=0x187f7e0 "log_descriptor.bc.buffer->prev_last_lsn == ((LSN)0)", file=0x1879ec8 "/my/maria-10.3/storage/maria/ma_loghandler.c", line=8006, function=0x1881170 <__PRETTY_FUNCTION__.17860> "translog_flush_buffers") at assert.c:101
#8  0x0000000001247097 in translog_flush_buffers (lsn=0x7f1b05c7f308, sent_to_disk=0x7f1b05c7f338, flush_horizon=0x7f1b05c7f330) at /my/maria-10.3/storage/maria/ma_loghandler.c:8005
#9  0x0000000001247862 in translog_flush (lsn=4294976120) at /my/maria-10.3/storage/maria/ma_loghandler.c:8174
#10 0x000000000126329b in ma_commit (trn=0x7f1af40af6f0) at /my/maria-10.3/storage/maria/ma_commit.c:66
#11 0x00000000012293e4 in ha_maria::implicit_commit (thd=0x7f1af4000d40, new_trn=false) at /my/maria-10.3/storage/maria/ha_maria.cc:2857
#12 0x00000000007b4ae6 in mysql_execute_command (thd=0x7f1af4000d40) at /my/maria-10.3/sql/sql_parse.cc:6103
#13 0x00000000007b9938 in mysql_parse (thd=0x7f1af4000d40, rawbuf=0x7f1af4012a68 "INSERT INTO t1 values(1)", length=24, parser_state=0x7f1b05c80570, is_com_multi=false, is_next_command=false) at /my/maria-10.3/sql/sql_parse.cc:7811
#14 0x00000000007a68ad in dispatch_command (command=COM_QUERY, thd=0x7f1af4000d40, packet=0x7f1af4008ec1 "INSERT INTO t1 values(1)", packet_length=24, is_com_multi=false, is_next_command=false) at /my/maria-10.3/sql/sql_parse.cc:1851
#15 0x00000000007a51ea in do_command (thd=0x7f1af4000d40) at /my/maria-10.3/sql/



 Comments   
Comment by Roel Van de Paar [ 2020-09-09 ]

Different assertion and stack on 10.4:

10.4.15 eae968f62d285de97ed607c87bc131cd863d5d03 (Debug)

mysqld: /test/10.4_dbg/storage/maria/ma_loghandler.c:8007: translog_flush_buffers: Assertion `log_descriptor.bc.buffer->prev_last_lsn == ((LSN)0)' failed.

10.4.15 eae968f62d285de97ed607c87bc131cd863d5d03 (Debug)

Core was generated by `/test/MD110820-mariadb-10.4.15-linux-x86_64-dbg/bin/mysqld --no-defaults --core'.
Program terminated with signal SIGABRT, Aborted.
#0  __pthread_kill (threadid=<optimized out>, signo=signo@entry=6)
    at ../sysdeps/unix/sysv/linux/pthread_kill.c:57
[Current thread is 1 (Thread 0x14906fb19700 (LWP 3478863))]
(gdb) bt
#0  __pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:57
#1  0x000055e7890f28a6 in my_write_core (sig=sig@entry=6) at /test/10.4_dbg/mysys/stacktrace.c:482
#2  0x000055e78886ecdc in handle_fatal_signal (sig=6) at /test/10.4_dbg/sql/signal_handler.cc:343
#3  <signal handler called>
#4  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#5  0x000014906ddb08b1 in __GI_abort () at abort.c:79
#6  0x000014906dda042a in __assert_fail_base (fmt=0x14906df27a38 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x55e7894bac68 "log_descriptor.bc.buffer->prev_last_lsn == ((LSN)0)", file=file@entry=0x55e7894b7738 "/test/10.4_dbg/storage/maria/ma_loghandler.c", line=line@entry=8007, function=function@entry=0x55e7894bc690 <__PRETTY_FUNCTION__.20386> "translog_flush_buffers") at assert.c:92
#7  0x000014906dda04a2 in __GI___assert_fail (assertion=assertion@entry=0x55e7894bac68 "log_descriptor.bc.buffer->prev_last_lsn == ((LSN)0)", file=file@entry=0x55e7894b7738 "/test/10.4_dbg/storage/maria/ma_loghandler.c", line=line@entry=8007, function=function@entry=0x55e7894bc690 <__PRETTY_FUNCTION__.20386> "translog_flush_buffers") at assert.c:101
#8  0x000055e788f0aaf9 in translog_flush_buffers (lsn=lsn@entry=0x14906fb15ea8, sent_to_disk=sent_to_disk@entry=0x14906fb15eb0, flush_horizon=flush_horizon@entry=0x14906fb15eb8) at /test/10.4_dbg/storage/maria/ma_loghandler.c:8006
#9  0x000055e788f0afdc in translog_flush (lsn=<optimized out>) at /test/10.4_dbg/storage/maria/ma_loghandler.c:8175
#10 0x000055e788f29e7b in ma_commit (trn=trn@entry=0x14906c9601f0) at /test/10.4_dbg/storage/maria/ma_commit.c:66
#11 0x000055e788ef7436 in ha_maria::implicit_commit (thd=0x149047815070, new_trn=<optimized out>) at /test/10.4_dbg/storage/maria/ha_maria.cc:2946
#12 0x000055e788871e94 in ha_maria_implicit_commit (thd=thd@entry=0x149047815070, new_trn=new_trn@entry=false) at /test/10.4_dbg/sql/handler.cc:146
#13 0x000055e7885c44e7 in mysql_execute_command (thd=thd@entry=0x149047815070) at /test/10.4_dbg/sql/sql_parse.cc:6159
#14 0x000055e7885c6090 in mysql_parse (thd=thd@entry=0x149047815070, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x14906fb18460, is_com_multi=is_com_multi@entry=false, is_next_command=is_next_command@entry=false) at /test/10.4_dbg/sql/sql_parse.cc:7896
#15 0x000055e7885c8920 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x149047815070, packet=packet@entry=0x149047857071 "GRANT SELECT ON *.* TO root@localhost", packet_length=packet_length@entry=37, is_com_multi=is_com_multi@entry=false, is_next_command=is_next_command@entry=false) at /test/10.4_dbg/sql/sql_parse.cc:1834
#16 0x000055e7885cc35b in do_command (thd=0x149047815070) at /test/10.4_dbg/sql/sql_parse.cc:1352
#17 0x000055e7886f88b6 in do_handle_one_connection (connect=connect@entry=0x14906c835790) at /test/10.4_dbg/sql/sql_connect.cc:1412
#18 0x000055e7886f89d6 in handle_one_connection (arg=0x14906c835790) at /test/10.4_dbg/sql/sql_connect.cc:1316
#19 0x000014906ed176db in start_thread (arg=0x14906fb19700) at pthread_create.c:463
#20 0x000014906de91a3f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Comment by Roel Van de Paar [ 2020-09-26 ]

New stack observed with:

SET GLOBAL aria_checkpoint_log_activity=1;
SET GLOBAL aria_group_commit="HARD";
SET GLOBAL aria_group_commit_interval=100000000;
GRANT SELECT ON *.* to root@localhost;

10.5.6 2bac9782aa81ad386beaf00eb126ccf2d15031a1 (Debug)

mysqld: /test/10.5_dbg/storage/maria/ma_loghandler.c:8008: translog_flush_buffers: Assertion `log_descriptor.bc.buffer->prev_last_lsn == ((LSN)0)' failed.

10.5.6 2bac9782aa81ad386beaf00eb126ccf2d15031a1 (Debug)

Core was generated by `/test/MD150920-mariadb-10.5.6-linux-x86_64-dbg/bin/mysqld --no-defaults --core-'.
Program terminated with signal SIGABRT, Aborted.
#0  __pthread_kill (threadid=<optimized out>, signo=signo@entry=6)
    at ../sysdeps/unix/sysv/linux/pthread_kill.c:57
[Current thread is 1 (Thread 0x154566883700 (LWP 179462))]
(gdb) bt
#0  __pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:57
#1  0x000056446540985a in my_write_core (sig=sig@entry=6) at /test/10.5_dbg/mysys/stacktrace.c:424
#2  0x0000564464bd03a9 in handle_fatal_signal (sig=6) at /test/10.5_dbg/sql/signal_handler.cc:330
#3  <signal handler called>
#4  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#5  0x0000154564b198b1 in __GI_abort () at abort.c:79
#6  0x0000154564b0942a in __assert_fail_base (fmt=0x154564c90a38 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x5644657932b0 "log_descriptor.bc.buffer->prev_last_lsn == ((LSN)0)", file=file@entry=0x56446578fd80 "/test/10.5_dbg/storage/maria/ma_loghandler.c", line=line@entry=8008, function=function@entry=0x564465794cd0 <__PRETTY_FUNCTION__.20899> "translog_flush_buffers") at assert.c:92
#7  0x0000154564b094a2 in __GI___assert_fail (assertion=assertion@entry=0x5644657932b0 "log_descriptor.bc.buffer->prev_last_lsn == ((LSN)0)", file=file@entry=0x56446578fd80 "/test/10.5_dbg/storage/maria/ma_loghandler.c", line=line@entry=8008, function=function@entry=0x564465794cd0 <__PRETTY_FUNCTION__.20899> "translog_flush_buffers") at assert.c:101
#8  0x0000564464e3d84a in translog_flush_buffers (lsn=lsn@entry=0x154566880ea8, sent_to_disk=sent_to_disk@entry=0x154566880eb0, flush_horizon=flush_horizon@entry=0x154566880eb8) at /test/10.5_dbg/storage/maria/ma_loghandler.c:8007
#9  0x0000564464e3dd2d in translog_flush (lsn=<optimized out>) at /test/10.5_dbg/storage/maria/ma_loghandler.c:8176
#10 0x0000564464e5d857 in ma_commit (trn=trn@entry=0x1545634c2108) at /test/10.5_dbg/storage/maria/ma_commit.c:66
#11 0x0000564464e24c69 in maria_commit (hton=<optimized out>, thd=0x154542415088, all=<optimized out>) at /test/10.5_dbg/storage/maria/ha_maria.cc:3460
#12 0x0000564464bd5e46 in commit_one_phase_2 (thd=thd@entry=0x154542415088, all=all@entry=false, trans=trans@entry=0x154542418808, is_real_trans=is_real_trans@entry=true, rw_trans=rw_trans@entry=true) at /test/10.5_dbg/sql/handler.cc:1929
#13 0x0000564464bd6155 in ha_commit_one_phase (thd=thd@entry=0x154542415088, all=all@entry=false, rw_trans=rw_trans@entry=true) at /test/10.5_dbg/sql/handler.cc:1881
#14 0x0000564464be9ac7 in ha_commit_trans (thd=thd@entry=0x154542415088, all=all@entry=false) at /test/10.5_dbg/sql/handler.cc:1686
#15 0x0000564464a8af95 in trans_commit_stmt (thd=thd@entry=0x154542415088) at /test/10.5_dbg/sql/transaction.cc:472
#16 0x000056446492335b in mysql_execute_command (thd=thd@entry=0x154542415088) at /test/10.5_dbg/sql/sql_parse.cc:6012
#17 0x000056446492a324 in mysql_parse (thd=thd@entry=0x154542415088, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x154566882350, is_com_multi=is_com_multi@entry=false, is_next_command=is_next_command@entry=false) at /test/10.5_dbg/sql/sql_parse.cc:7994
#18 0x0000564464916d54 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x154542415088, packet=packet@entry=0x154542467089 "GRANT SELECT ON *.* to root@localhost", packet_length=packet_length@entry=37, is_com_multi=is_com_multi@entry=false, is_next_command=is_next_command@entry=false) at /test/10.5_dbg/sql/sql_parse.cc:1867
#19 0x000056446491553e in do_command (thd=0x154542415088) at /test/10.5_dbg/sql/sql_parse.cc:1348
#20 0x0000564464a74893 in do_handle_one_connection (connect=<optimized out>, connect@entry=0x1545458d4808, put_in_cache=put_in_cache@entry=true) at /test/10.5_dbg/sql/sql_connect.cc:1410
#21 0x0000564464a74fb7 in handle_one_connection (arg=arg@entry=0x1545458d4808) at /test/10.5_dbg/sql/sql_connect.cc:1312
#22 0x0000564464edf5c2 in pfs_spawn_thread (arg=0x154563446508) at /test/10.5_dbg/storage/perfschema/pfs.cc:2201
#23 0x00001545657fc6db in start_thread (arg=0x154566883700) at pthread_create.c:463
#24 0x0000154564bfaa3f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Comment by Roel Van de Paar [ 2020-11-06 ]

New stack observed with:

SET GLOBAL aria_group_commit_interval=CAST(-1 AS UNSIGNED INT);
SET GLOBAL aria_checkpoint_log_activity=0;
SET GLOBAL aria_group_commit="HARD";
DELETE FROM mysql.user WHERE USER LIKE "%";

Leads to:

10.6.0 5d4599f9750140f92cfdbbe4d292ae1b8dd456f8 (Debug)

mysqld: /test/10.6_dbg/storage/maria/ma_loghandler.c:8007: translog_flush_buffers: Assertion `log_descriptor.bc.buffer->prev_last_lsn == ((LSN)0)' failed.

10.6.0 5d4599f9750140f92cfdbbe4d292ae1b8dd456f8 (Debug)

Core was generated by `/test/MD211020-mariadb-10.6.0-linux-x86_64-dbg/bin/mysqld --no-defaults --core-'.
Program terminated with signal SIGABRT, Aborted.
#0  __pthread_kill (threadid=<optimized out>, signo=signo@entry=6)
    at ../sysdeps/unix/sysv/linux/pthread_kill.c:56
[Current thread is 1 (Thread 0x14c1e0dd5700 (LWP 2254752))]
(gdb) bt
#0  __pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:56
#1  0x000055c987d22021 in my_write_core (sig=sig@entry=6) at /test/10.6_dbg/mysys/stacktrace.c:424
#2  0x000055c987469321 in handle_fatal_signal (sig=6) at /test/10.6_dbg/sql/signal_handler.cc:330
#3  <signal handler called>
#4  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#5  0x000014c1ea23e859 in __GI_abort () at abort.c:79
#6  0x000014c1ea23e729 in __assert_fail_base (fmt=0x14c1ea3d4588 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x55c988075d70 "log_descriptor.bc.buffer->prev_last_lsn == ((LSN)0)", file=0x55c988072840 "/test/10.6_dbg/storage/maria/ma_loghandler.c", line=8007, function=<optimized out>) at assert.c:92
#7  0x000014c1ea24ff36 in __GI___assert_fail (assertion=assertion@entry=0x55c988075d70 "log_descriptor.bc.buffer->prev_last_lsn == ((LSN)0)", file=file@entry=0x55c988072840 "/test/10.6_dbg/storage/maria/ma_loghandler.c", line=line@entry=8007, function=function@entry=0x55c988077770 <__PRETTY_FUNCTION__.21295> "translog_flush_buffers") at assert.c:101
#8  0x000055c98770c02e in translog_flush_buffers (lsn=lsn@entry=0x14c1e0dd2e28, sent_to_disk=sent_to_disk@entry=0x14c1e0dd2e30, flush_horizon=flush_horizon@entry=0x14c1e0dd2e38) at /test/10.6_dbg/storage/maria/ma_loghandler.c:8007
#9  0x000055c98770c4fe in translog_flush (lsn=<optimized out>) at /test/10.6_dbg/storage/maria/ma_loghandler.c:8176
#10 0x000055c9877578ca in _ma_bitmap_mark_file_changed (flush_translog=1 '\001', share=0x55c98a425128) at /test/10.6_dbg/storage/maria/ma_bitmap.c:384
#11 _ma_change_bitmap_page (info=info@entry=0x55c98a42cbc8, bitmap=bitmap@entry=0x55c98a425c80, page=page@entry=0) at /test/10.6_dbg/storage/maria/ma_bitmap.c:1135
#12 0x000055c98775922b in set_page_bits (info=info@entry=0x55c98a42cbc8, bitmap=bitmap@entry=0x55c98a425c80, page=page@entry=1, fill_pattern=1) at /test/10.6_dbg/storage/maria/ma_bitmap.c:2310
#13 0x000055c98775c49e in _ma_bitmap_set (info=info@entry=0x55c98a42cbc8, page=page@entry=1, head=head@entry=1 '\001', empty_space=7850) at /test/10.6_dbg/storage/maria/ma_bitmap.c:2900
#14 0x000055c98774a225 in delete_head_or_tail (info=info@entry=0x55c98a42cbc8, page=page@entry=1, record_number=record_number@entry=0, head=head@entry=1 '\001', from_update=from_update@entry=0 '\000') at /test/10.6_dbg/storage/maria/ma_blockrec.c:4287
#15 0x000055c98774f51b in _ma_delete_block_record (info=0x55c98a42cbc8, record=0x55c98a421df0 "localhost", ' ' <repeats 171 times>, "mariadb.sys         "...) at /test/10.6_dbg/storage/maria/ma_blockrec.c:4342
#16 0x000055c9877635f2 in maria_delete (info=0x55c98a42cbc8, record=0x55c98a421df0 "localhost", ' ' <repeats 171 times>, "mariadb.sys         "...) at /test/10.6_dbg/storage/maria/ma_delete.c:109
#17 0x000055c9876f0e55 in ha_maria::delete_row (this=<optimized out>, buf=<optimized out>) at /test/10.6_dbg/storage/maria/ha_maria.cc:2369
#18 0x000055c98747e2a2 in handler::ha_delete_row (this=0x55c98a422250, buf=0x55c98a421df0 "localhost", ' ' <repeats 171 times>, "mariadb.sys         "...) at /test/10.6_dbg/sql/handler.cc:7253
#19 0x000055c987663fbf in TABLE::delete_row (this=0x55c98a41ec48) at /test/10.6_dbg/sql/sql_delete.cc:277
#20 0x000055c987662170 in mysql_delete (thd=thd@entry=0x14c198000db8, table_list=0x14c198012780, conds=<optimized out>, order_list=order_list@entry=0x14c1980059e0, limit=18446744073709551615, options=<optimized out>, result=<optimized out>) at /test/10.6_dbg/sql/sql_delete.cc:804
#21 0x000055c9871a8fb6 in mysql_execute_command (thd=thd@entry=0x14c198000db8) at /test/10.6_dbg/sql/sql_limit.h:67
#22 0x000055c987192fd2 in mysql_parse (thd=thd@entry=0x14c198000db8, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x14c1e0dd43d0) at /test/10.6_dbg/sql/sql_parse.cc:7833
#23 0x000055c9871a10c7 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x14c198000db8, packet=packet@entry=0x14c198008ce9 "DELETE FROM mysql.user WHERE USER LIKE \"%\"", packet_length=packet_length@entry=42) at /test/10.6_dbg/sql/sql_class.h:1253
#24 0x000055c9871a43d2 in do_command (thd=0x14c198000db8) at /test/10.6_dbg/sql/sql_parse.cc:1343
#25 0x000055c9872fe994 in do_handle_one_connection (connect=<optimized out>, connect@entry=0x55c98a4788c8, put_in_cache=put_in_cache@entry=true) at /test/10.6_dbg/sql/sql_connect.cc:1410
#26 0x000055c9872ff09b in handle_one_connection (arg=arg@entry=0x55c98a4788c8) at /test/10.6_dbg/sql/sql_connect.cc:1312
#27 0x000055c9877b2abb in pfs_spawn_thread (arg=0x55c98a3a0418) at /test/10.6_dbg/storage/perfschema/pfs.cc:2201
#28 0x000014c1ea74c609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#29 0x000014c1ea33b293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Bug confirmed present in:
MariaDB: 10.4.16 (dbg), 10.5.7 (dbg), 10.6.0 (dbg)

Bug confirmed not present in:
MariaDB: 10.1.48 (dbg), 10.1.48 (opt), 10.2.35 (dbg), 10.2.35 (opt), 10.3.26 (dbg), 10.3.26 (opt), 10.4.16 (opt), 10.5.7 (opt), 10.6.0 (opt)
MySQL: 5.5.62 (dbg), 5.5.62 (opt), 5.6.50 (dbg), 5.6.50 (opt), 5.7.32 (dbg), 5.7.32 (opt), 8.0.22 (dbg), 8.0.22 (opt)

Comment by Roel Van de Paar [ 2020-11-06 ]

The last testcase can also produce thread hangs on release builds. The CLI will hang on the last command. Using another thread, we can see:

10.5.7 e3fc9c1db04bdc5566bcdafcc3d5159199089c00 (Optimized)

10.5.7>SHOW PROCESSLIST;
+----+------+-----------+------+---------+------+----------+--------------------------------------------+----------+
| Id | User | Host      | db   | Command | Time | State    | Info                                       | Progress |
+----+------+-----------+------+---------+------+----------+--------------------------------------------+----------+
|  4 | root | localhost | test | Query   |  213 | Updating | DELETE FROM mysql.user WHERE USER LIKE "%" |    0.000 |
|  5 | root | localhost | test | Query   |    0 | starting | SHOW PROCESSLIST                           |    0.000 |
+----+------+-----------+------+---------+------+----------+--------------------------------------------+----------+
2 rows in set (0.000 sec)

10.3 (optimized) will return to the command prompt instead, and one can re-enter the CLI (no FLUSH PRIVILEGES executed yet), though mysqladmin shutdown hangs during shutdown. Same for 10.2 and 10.1.

This is not the case if only the last DELETE command is executed, so this issue also affects, in another way 10.1 to 10.3.

Leaving the regression tag as the earlier described testcases affect only later versions in a specific way, as described and as per versions listed for each testcase.

Comment by Roel Van de Paar [ 2020-11-06 ]

While the cause of this issue may be Aria related, the effect thereof affects more then just Aria.

Comment by Roel Van de Paar [ 2021-04-22 ]

I just realized (working on another bug) that the priority of this bug is higher than already discussed above, as the crashing stack is generic and produced by other bugs, leading to other bugs being filtered out incorrectly without being handled. A fix would be appreciated.

Comment by Roel Van de Paar [ 2021-04-27 ]

This issue was also seen during a run of the testcase in MDEV-25533.

Comment by Roel Van de Paar [ 2021-11-13 ]

SET GLOBAL aria_group_commit_INTERVAL=100000000;
SET GLOBAL aria_group_commit=HARD;
SET GLOBAL aria_checkpoint_log_activity=1;
CREATE PROCEDURE p() SQL SECURITY INVOKER SELECT 1 QUERY;

Leads to:

10.8.0 bc57ff7cf79504a198d4752b4748340a4afd886c (Debug)

mysqld: /test/10.8_dbg/sql/sql_class.cc:1544: void THD::cleanup(): Assertion `open_tables == __null' failed.

10.8.0 bc57ff7cf79504a198d4752b4748340a4afd886c (Debug)

Core was generated by `/test/MD121121-mariadb-10.8.0-linux-x86_64-dbg/bin/mysqld --no-defaults --core-'.
Program terminated with signal SIGABRT, Aborted.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
[Current thread is 1 (Thread 0x148958c05700 (LWP 3735910))]
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x000014895b7e4859 in __GI_abort () at abort.c:79
#2  0x000014895b7e4729 in __assert_fail_base (fmt=0x14895b97a588 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x5637517744f8 "log_descriptor.bc.buffer->prev_last_lsn == ((LSN)0)", file=0x563751770f00 "/test/10.8_dbg/storage/maria/ma_loghandler.c", line=8006, function=<optimized out>) at assert.c:92
#3  0x000014895b7f5f36 in __GI___assert_fail (assertion=assertion@entry=0x5637517744f8 "log_descriptor.bc.buffer->prev_last_lsn == ((LSN)0)", file=file@entry=0x563751770f00 "/test/10.8_dbg/storage/maria/ma_loghandler.c", line=line@entry=8006, function=function@entry=0x563751775f70 <__PRETTY_FUNCTION__.23059> "translog_flush_buffers") at assert.c:101
#4  0x0000563750e04de6 in translog_flush_buffers (lsn=lsn@entry=0x148958bfef98, sent_to_disk=sent_to_disk@entry=0x148958bfefa0, flush_horizon=flush_horizon@entry=0x148958bfefa8) at /test/10.8_dbg/storage/maria/ma_loghandler.c:8006
#5  0x0000563750e05295 in translog_flush (lsn=<optimized out>) at /test/10.8_dbg/storage/maria/ma_loghandler.c:8175
#6  0x0000563750e4e361 in flush_log_for_bitmap (args=<optimized out>) at /test/10.8_dbg/storage/maria/ma_bitmap.c:3118
#7  0x0000563750e0d89a in pagecache_fwrite (pagecache=pagecache@entry=0x5637527cc4e0 <maria_pagecache_var>, filedesc=0x1489599c9730, buffer=0x148945d6d000 "$\271\361\376?\376\377\355\377\277\377\337\367\375m\263\267\375\277\377\017", pageno=0, type=PAGECACHE_PLAIN_PAGE, flags=36) at /test/10.8_dbg/storage/maria/ma_pagecache.c:675
#8  0x0000563750e13280 in flush_cached_blocks (pagecache=pagecache@entry=0x5637527cc4e0 <maria_pagecache_var>, file=file@entry=0x148914031488, cache=cache@entry=0x148958bff230, end=end@entry=0x148958bff240, type=type@entry=FLUSH_KEEP, first_errno=first_errno@entry=0x148958bff1e4) at /test/10.8_dbg/storage/maria/ma_pagecache.c:4819
#9  0x0000563750e13dbf in flush_pagecache_blocks_int (pagecache=pagecache@entry=0x5637527cc4e0 <maria_pagecache_var>, file=file@entry=0x148914031488, type=type@entry=FLUSH_KEEP, filter=filter@entry=0x0, filter_arg=filter_arg@entry=0x0) at /test/10.8_dbg/storage/maria/ma_pagecache.c:5115
#10 0x0000563750e1b27c in flush_pagecache_blocks_with_filter (pagecache=0x5637527cc4e0 <maria_pagecache_var>, file=file@entry=0x148914031488, type=type@entry=FLUSH_KEEP, filter=filter@entry=0x0, filter_arg=filter_arg@entry=0x0) at /test/10.8_dbg/storage/maria/ma_pagecache.c:5230
#11 0x0000563750e2e816 in _ma_flush_table_files (info=info@entry=0x148914030ff8, flush_data_or_index=flush_data_or_index@entry=3, flush_type_for_data=flush_type_for_data@entry=FLUSH_KEEP, flush_type_for_index=flush_type_for_index@entry=FLUSH_KEEP) at /test/10.8_dbg/storage/maria/ma_extra.c:656
#12 0x0000563750e2fa44 in maria_extra (info=0x148914030ff8, function=function@entry=HA_EXTRA_FLUSH, extra_arg=extra_arg@entry=0x0) at /test/10.8_dbg/storage/maria/ma_extra.c:427
#13 0x0000563750deb763 in ha_maria::extra (this=0x148914025820, operation=HA_EXTRA_FLUSH) at /test/10.8_dbg/storage/maria/ha_maria.cc:2712
#14 0x0000563750d6a87a in Sp_handler::sp_create_routine (this=0x563751e425b0 <sp_handler_procedure>, thd=thd@entry=0x148914000db8, sp=0x14891401c670) at /test/10.8_dbg/sql/sp.cc:1459
#15 0x00005637508b0302 in mysql_create_routine (thd=thd@entry=0x148914000db8, lex=lex@entry=0x1489140050e8) at /test/10.8_dbg/sql/sql_parse.cc:3088
#16 0x00005637508b6595 in mysql_execute_command (thd=thd@entry=0x148914000db8, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false) at /test/10.8_dbg/sql/sql_parse.cc:5706
#17 0x000056375089dcad in mysql_parse (thd=thd@entry=0x148914000db8, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x148958c04400) at /test/10.8_dbg/sql/sql_parse.cc:8028
#18 0x00005637508ac949 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x148914000db8, packet=packet@entry=0x14891400b879 "CREATE PROCEDURE p() SQL SECURITY INVOKER SELECT 1 QUERY", packet_length=packet_length@entry=56, blocking=blocking@entry=true) at /test/10.8_dbg/sql/sql_class.h:1360
#19 0x00005637508afd83 in do_command (thd=0x148914000db8, blocking=blocking@entry=true) at /test/10.8_dbg/sql/sql_parse.cc:1402
#20 0x0000563750a29e2a in do_handle_one_connection (connect=<optimized out>, connect@entry=0x563752ff12e8, put_in_cache=put_in_cache@entry=true) at /test/10.8_dbg/sql/sql_connect.cc:1418
#21 0x0000563750a2a42f in handle_one_connection (arg=arg@entry=0x563752ff12e8) at /test/10.8_dbg/sql/sql_connect.cc:1312
#22 0x0000563750eaa4ce in pfs_spawn_thread (arg=0x563752f04a18) at /test/10.8_dbg/storage/perfschema/pfs.cc:2201
#23 0x000014895bcf3609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#24 0x000014895b8e1293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

10.6.6 3480c3f95b27c8f3e6f4ecbe6262e5b80109c0d4 (Debug)

mysqld: /test/10.6_dbg/storage/maria/ma_loghandler.c:8007: translog_flush_buffers: Assertion `log_descriptor.bc.buffer->prev_last_lsn == ((LSN)0)' failed.

10.6.6 3480c3f95b27c8f3e6f4ecbe6262e5b80109c0d4 (Debug)

Core was generated by `/test/MD121121-mariadb-10.6.6-linux-x86_64-dbg/bin/mysqld --no-defaults --core-'.
Program terminated with signal SIGABRT, Aborted.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
[Current thread is 1 (Thread 0x145e546cd700 (LWP 3800400))]
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x0000145e572ac859 in __GI_abort () at abort.c:79
#2  0x0000145e572ac729 in __assert_fail_base (fmt=0x145e57442588 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x557d124488d8 "log_descriptor.bc.buffer->prev_last_lsn == ((LSN)0)", file=0x557d124452e0 "/test/10.6_dbg/storage/maria/ma_loghandler.c", line=8007, function=<optimized out>) at assert.c:92
#3  0x0000145e572bdf36 in __GI___assert_fail (assertion=assertion@entry=0x557d124488d8 "log_descriptor.bc.buffer->prev_last_lsn == ((LSN)0)", file=file@entry=0x557d124452e0 "/test/10.6_dbg/storage/maria/ma_loghandler.c", line=line@entry=8007, function=function@entry=0x557d1244a330 <__PRETTY_FUNCTION__.21845> "translog_flush_buffers") at assert.c:101
#4  0x0000557d11af09c6 in translog_flush_buffers (lsn=lsn@entry=0x145e546c6f98, sent_to_disk=sent_to_disk@entry=0x145e546c6fa0, flush_horizon=flush_horizon@entry=0x145e546c6fa8) at /test/10.6_dbg/storage/maria/ma_loghandler.c:8007
#5  0x0000557d11af0e75 in translog_flush (lsn=<optimized out>) at /test/10.6_dbg/storage/maria/ma_loghandler.c:8176
#6  0x0000557d11b39f49 in flush_log_for_bitmap (args=<optimized out>) at /test/10.6_dbg/storage/maria/ma_bitmap.c:3117
#7  0x0000557d11af9490 in pagecache_fwrite (pagecache=pagecache@entry=0x557d134922c0 <maria_pagecache_var>, filedesc=0x145e554916c0, buffer=0x145e41d6b000 "\343\266Ѷ-", pageno=0, type=PAGECACHE_PLAIN_PAGE, flags=36) at /test/10.6_dbg/storage/maria/ma_pagecache.c:675
#8  0x0000557d11afee76 in flush_cached_blocks (pagecache=pagecache@entry=0x557d134922c0 <maria_pagecache_var>, file=file@entry=0x145e10032a98, cache=cache@entry=0x145e546c7230, end=end@entry=0x145e546c7240, type=type@entry=FLUSH_KEEP, first_errno=first_errno@entry=0x145e546c71e4) at /test/10.6_dbg/storage/maria/ma_pagecache.c:4819
#9  0x0000557d11aff9b5 in flush_pagecache_blocks_int (pagecache=pagecache@entry=0x557d134922c0 <maria_pagecache_var>, file=file@entry=0x145e10032a98, type=type@entry=FLUSH_KEEP, filter=filter@entry=0x0, filter_arg=filter_arg@entry=0x0) at /test/10.6_dbg/storage/maria/ma_pagecache.c:5115
#10 0x0000557d11b06e72 in flush_pagecache_blocks_with_filter (pagecache=0x557d134922c0 <maria_pagecache_var>, file=file@entry=0x145e10032a98, type=type@entry=FLUSH_KEEP, filter=filter@entry=0x0, filter_arg=filter_arg@entry=0x0) at /test/10.6_dbg/storage/maria/ma_pagecache.c:5230
#11 0x0000557d11b1a402 in _ma_flush_table_files (info=info@entry=0x145e10032628, flush_data_or_index=flush_data_or_index@entry=3, flush_type_for_data=flush_type_for_data@entry=FLUSH_KEEP, flush_type_for_index=flush_type_for_index@entry=FLUSH_KEEP) at /test/10.6_dbg/storage/maria/ma_extra.c:656
#12 0x0000557d11b1b630 in maria_extra (info=0x145e10032628, function=function@entry=HA_EXTRA_FLUSH, extra_arg=extra_arg@entry=0x0) at /test/10.6_dbg/storage/maria/ma_extra.c:427
#13 0x0000557d11ad73cd in ha_maria::extra (this=0x145e10026960, operation=HA_EXTRA_FLUSH) at /test/10.6_dbg/storage/maria/ha_maria.cc:2712
#14 0x0000557d11a3ecc8 in Sp_handler::sp_create_routine (this=0x557d12b0a250 <sp_handler_procedure>, thd=thd@entry=0x145e10000db8, sp=0x145e1001d890) at /test/10.6_dbg/sql/sp.cc:1459
#15 0x0000557d1157f355 in mysql_create_routine (thd=thd@entry=0x145e10000db8, lex=lex@entry=0x145e10005058) at /test/10.6_dbg/sql/sql_parse.cc:3090
#16 0x0000557d115855e6 in mysql_execute_command (thd=thd@entry=0x145e10000db8, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false) at /test/10.6_dbg/sql/sql_parse.cc:5708
#17 0x0000557d1156ccd0 in mysql_parse (thd=thd@entry=0x145e10000db8, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x145e546cc400) at /test/10.6_dbg/sql/sql_parse.cc:8030
#18 0x0000557d1157b995 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x145e10000db8, packet=packet@entry=0x145e1000b769 "CREATE PROCEDURE p() SQL SECURITY INVOKER SELECT 1 QUERY", packet_length=packet_length@entry=56, blocking=blocking@entry=true) at /test/10.6_dbg/sql/sql_class.h:1358
#19 0x0000557d1157edd6 in do_command (thd=0x145e10000db8, blocking=blocking@entry=true) at /test/10.6_dbg/sql/sql_parse.cc:1404
#20 0x0000557d116e4634 in do_handle_one_connection (connect=<optimized out>, connect@entry=0x557d15049938, put_in_cache=put_in_cache@entry=true) at /test/10.6_dbg/sql/sql_connect.cc:1418
#21 0x0000557d116e4c39 in handle_one_connection (arg=arg@entry=0x557d15049938) at /test/10.6_dbg/sql/sql_connect.cc:1312
#22 0x0000557d11b960ac in pfs_spawn_thread (arg=0x557d14f56b08) at /test/10.6_dbg/storage/perfschema/pfs.cc:2201
#23 0x0000145e577bb609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#24 0x0000145e573a9293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Bug confirmed present in:
MariaDB: 10.4.23 (dbg), 10.5.14 (dbg), 10.6.6 (dbg), 10.7.2 (dbg), 10.8.0 (dbg)

Bug (or feature/syntax) confirmed not present in:
MariaDB: 10.2.42 (dbg), 10.2.42 (opt), 10.3.33 (dbg), 10.3.33 (opt), 10.4.23 (opt), 10.5.14 (opt), 10.6.6 (opt), 10.7.2 (opt), 10.8.0 (opt)
MySQL: 5.5.62 (dbg), 5.5.62 (opt), 5.6.51 (dbg), 5.6.51 (opt), 5.7.36 (dbg), 5.7.36 (opt), 8.0.27 (dbg), 8.0.27 (opt)

Comment by Roel Van de Paar [ 2021-12-30 ]

This issue was also seen during a run of the testcase in MDEV-27388.

Comment by Roel Van de Paar [ 2022-03-10 ]

The original testcase was also observed to sporadically generate a thread hang.

10.7.4 3d88f9f34c1223c59272d5086dcd2b4b2cfc167b (Debug)

10.7.4-dbg>show processlist;
+----+------+-----------+------+---------+------+----------+--------------------------+----------+
| Id | User | Host      | db   | Command | Time | State    | Info                     | Progress |
+----+------+-----------+------+---------+------+----------+--------------------------+----------+
|  4 | root | localhost | test | Query   |  113 | Commit   | INSERT INTO t1 values(1) |    0.000 |
|  5 | root | localhost | test | Query   |    0 | starting | show processlist         |    0.000 |
+----+------+-----------+------+---------+------+----------+--------------------------+----------+
2 rows in set (0.000 sec)

Comment by Roel Van de Paar [ 2022-03-10 ]

Another stack with this testcase:

SET GLOBAL ARIA_CHECKPOINT_LOG_ACTIVITY=1;
SET GLOBAL ARIA_GROUP_COMMIT=HARD;
SET GLOBAL ARIA_CHECKPOINT_INTERVAL=1;
SET GLOBAL ARIA_GROUP_COMMIT_INTERVAL=400000000;
CREATE TABLE t1 (a INT) ENGINE=Aria TRANSACTIONAL=1;
INSERT INTO t2 VALUES (1);
DROP TABLE t1;

Leads to:

10.9.0 4a2a9c02cd6611ef36bbb735c2b483dbc83580d4 (Debug)

mysqld: /test/10.9_dbg/storage/maria/ma_loghandler.c:8006: translog_flush_buffers: Assertion `log_descriptor.bc.buffer->prev_last_lsn == ((LSN)0)' failed.

10.9.0 4a2a9c02cd6611ef36bbb735c2b483dbc83580d4 (Debug)

                             
Core was generated by `/test/MD260222-mariadb-10.9.0-linux-x86_64-dbg/bin/mysqld --no-defaults --core-'.                                                                                                  
Program terminated with signal SIGABRT, Aborted.                                                     
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50                            
[Current thread is 1 (Thread 0x148cf80c1700 (LWP 2284428))]       
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x0000148d0dd54859 in __GI_abort () at abort.c:79
#2  0x0000148d0dd54729 in __assert_fail_base (fmt=0x148d0deea588 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x55993b5cbf98 "log_descriptor.bc.buffer->prev_last_lsn == ((LSN)0)", file=0x55993b5c89a0 "/test/10.9_dbg/storage/maria/ma_loghandler.c", line=8006, function=<optimized out>) at assert.c:92
#3  0x0000148d0dd66006 in __GI___assert_fail (assertion=assertion@entry=0x55993b5cbf98 "log_descriptor.bc.buffer->prev_last_lsn == ((LSN)0)", file=file@entry=0x55993b5c89a0 "/test/10.9_dbg/storage/maria/ma_loghandler.c", line=line@entry=8006, function=function@entry=0x55993b5cda10 <__PRETTY_FUNCTION__.23066> "translog_flush_buffers") at assert.c:101
#4  0x000055993ac5dacd in translog_flush_buffers (lsn=lsn@entry=0x148cf80be8d8, sent_to_disk=sent_to_disk@entry=0x148cf80be8e0, flush_horizon=flush_horizon@entry=0x148cf80be8e8) at /test/10.9_dbg/storage/maria/ma_loghandler.c:8006
#5  0x000055993ac5df7c in translog_flush (lsn=<optimized out>) at /test/10.9_dbg/storage/maria/ma_loghandler.c:8175
#6  0x000055993acb8826 in maria_delete_table (name=<optimized out>, name@entry=0x148cf80bf910 "./test/t1") at /test/10.9_dbg/storage/maria/ma_delete_table.c:63
#7  0x000055993ac44443 in ha_maria::delete_table (this=<optimized out>, name=0x148cf80bf910 "./test/t1") at /test/10.9_dbg/storage/maria/ha_maria.cc:2799
#8  0x000055993aa0b35b in hton_drop_table (hton=<optimized out>, path=<optimized out>) at /test/10.9_dbg/sql/handler.cc:576
#9  0x000055993aa14cd8 in ha_delete_table (thd=thd@entry=0x148cc0000db8, hton=hton@entry=0x55993cdc8408, path=path@entry=0x148cf80bf910 "./test/t1", db=db@entry=0x148cf80bf6e0, alias=alias@entry=0x148cf80bf6f0, generate_warning=generate_warning@entry=true) at /test/10.9_dbg/sql/handler.cc:3097
#10 0x000055993a7d68d5 in mysql_rm_table_no_locks (thd=thd@entry=0x148cc0000db8, tables=tables@entry=0x148cc0013e90, current_db=current_db@entry=0x148cc0000e50, ddl_log_state=0x148cf80bf710, ddl_log_state@entry=0x0, if_exists=if_exists@entry=false, drop_temporary=drop_temporary@entry=false, drop_view=false, drop_sequence=false, dont_log_query=false, dont_free_locks=false) at /test/10.9_dbg/sql/sql_table.cc:1611
#11 0x000055993a7d89c3 in mysql_rm_table (thd=thd@entry=0x148cc0000db8, tables=tables@entry=0x148cc0013e90, if_exists=<optimized out>, drop_temporary=<optimized out>, drop_sequence=<optimized out>, dont_log_query=dont_log_query@entry=false) at /test/10.9_dbg/sql/sql_table.cc:1186
#12 0x000055993a701bf4 in mysql_execute_command (thd=thd@entry=0x148cc0000db8, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false) at /test/10.9_dbg/sql/structs.h:568
#13 0x000055993a6eb343 in mysql_parse (thd=thd@entry=0x148cc0000db8, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x148cf80c0400) at /test/10.9_dbg/sql/sql_parse.cc:8027
#14 0x000055993a6f9fdf in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x148cc0000db8, packet=packet@entry=0x148cc000b889 "DROP TABLE t1", packet_length=packet_length@entry=13, blocking=blocking@entry=true) at /test/10.9_dbg/sql/sql_class.h:1362
#15 0x000055993a6fd426 in do_command (thd=0x148cc0000db8, blocking=blocking@entry=true) at /test/10.9_dbg/sql/sql_parse.cc:1402
#16 0x000055993a878036 in do_handle_one_connection (connect=<optimized out>, connect@entry=0x55993d457978, put_in_cache=put_in_cache@entry=true) at /test/10.9_dbg/sql/sql_connect.cc:1418
#17 0x000055993a87863b in handle_one_connection (arg=arg@entry=0x55993d457978) at /test/10.9_dbg/sql/sql_connect.cc:1312
#18 0x000055993acfed23 in pfs_spawn_thread (arg=0x55993d39a3f8) at /test/10.9_dbg/storage/perfschema/pfs.cc:2201
#19 0x0000148d0e265609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#20 0x0000148d0de51163 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

And hang:

10.9.0 4a2a9c02cd6611ef36bbb735c2b483dbc83580d4 (Debug)

10.9.0-dbg>show processlist;
+----+------+-----------+------+---------+------+----------------+------------------+----------+
| Id | User | Host      | db   | Command | Time | State          | Info             | Progress |
+----+------+-----------+------+---------+------+----------------+------------------+----------+
|  4 | root | localhost | test | Query   |   87 | closing tables | DROP TABLE t1    |    0.000 |
|  5 | root | localhost | test | Query   |    0 | starting       | show processlist |    0.000 |
+----+------+-----------+------+---------+------+----------------+------------------+----------+
2 rows in set (0.000 sec)

Comment by Oleksandr Byelkin [ 2022-06-23 ]

how possible it could work:

SET GLOBAL ARIA_CHECKPOINT_LOG_ACTIVITY=1;
SET GLOBAL ARIA_GROUP_COMMIT=HARD;
SET GLOBAL ARIA_CHECKPOINT_INTERVAL=1;
SET GLOBAL ARIA_GROUP_COMMIT_INTERVAL=400000000;
CREATE TABLE t1 (a INT) ENGINE=Aria TRANSACTIONAL=1;
INSERT INTO t2 VALUES (1);
main.test                                [ fail ]
        Test ended at 2022-06-23 12:46:50
 
CURRENT_TEST: main.test
mysqltest: At line 6: query 'INSERT INTO t2 VALUES (1)' failed: 1146: Table 'test.t2' doesn't exist

Comment by Oleksandr Byelkin [ 2022-06-23 ]

If I fix the test case it just waits ARIA_GROUP_COMMIT_INTERVAL

Comment by Roel Van de Paar [ 2022-06-25 ]

It reproduces at the CLI:

10.10.0 081a284712bb661349e2e3802077b12211cede3e (Debug)

10.10.0-dbg>SET GLOBAL ARIA_CHECKPOINT_LOG_ACTIVITY=1;
Query OK, 0 rows affected (0.000 sec)
 
10.10.0-dbg>SET GLOBAL ARIA_GROUP_COMMIT=HARD;
Query OK, 0 rows affected (0.000 sec)
 
10.10.0-dbg>SET GLOBAL ARIA_CHECKPOINT_INTERVAL=1;
Query OK, 0 rows affected (0.004 sec)
 
10.10.0-dbg>SET GLOBAL ARIA_GROUP_COMMIT_INTERVAL=400000000;
Query OK, 0 rows affected (0.000 sec)
 
10.10.0-dbg>CREATE TABLE t1 (a INT) ENGINE=Aria TRANSACTIONAL=1;
Query OK, 0 rows affected (0.023 sec)
 
10.10.0-dbg>INSERT INTO t2 VALUES (1);
ERROR 1146 (42S02): Table 'test.t2' doesn't exist
10.10.0-dbg>DROP TABLE t1;
ERROR 2013 (HY000): Lost connection to server during query

Comment by Roel Van de Paar [ 2022-08-24 ]

SET GLOBAL aria_group_commit_INTERVAL=1000000000;
SET GLOBAL aria_group_commit=HARD;
# Now attempt `mysqladmin shutdown`; it will hang (tested on 10.11 debug + optimized)

Comment by Roel Van de Paar [ 2022-09-10 ]

Oddly, I saw the following stack:

10.11.0 bc563f1a4b0b38de3b41fd0f0d3d8b7f1aacbd8b (Debug)

mysqld: /test/10.11_dbg/sql/sql_class.cc:1533: void THD::cleanup(): Assertion `open_tables == __null' failed.

10.11.0 bc563f1a4b0b38de3b41fd0f0d3d8b7f1aacbd8b (Debug)

Core was generated by `/test/MD190822-mariadb-10.11.0-linux-x86_64-dbg/bin/mysqld --no-defaults --core'.
Program terminated with signal SIGABRT, Aborted.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
[Current thread is 1 (Thread 0x15175fec2940 (LWP 1589060))]
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x000015176009b859 in __GI_abort () at abort.c:79
#2  0x000015176009b729 in __assert_fail_base (fmt=0x151760231588 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x557065fb93de "open_tables == __null", file=0x557065fb7090 "/test/10.11_dbg/sql/sql_class.cc", line=1533, function=<optimized out>) at assert.c:92
#3  0x00001517600acfd6 in __GI___assert_fail (assertion=assertion@entry=0x557065fb93de "open_tables == __null", file=file@entry=0x557065fb7090 "/test/10.11_dbg/sql/sql_class.cc", line=line@entry=1533, function=function@entry=0x557065fb920f "void THD::cleanup()") at assert.c:101
#4  0x000055706536d08c in THD::cleanup (this=this@entry=0x5570677d9418) at /test/10.11_dbg/sql/sql_class.cc:1533
#5  0x000055706536e143 in THD::free_connection (this=this@entry=0x5570677d9418) at /test/10.11_dbg/sql/sql_class.cc:1597
#6  0x000055706537621d in THD::~THD (this=0x5570677d9418, __in_chrg=<optimized out>) at /test/10.11_dbg/sql/sql_class.cc:1676
#7  0x00005570653766c5 in THD::~THD (this=0x5570677d9418, __in_chrg=<optimized out>) at /test/10.11_dbg/sql/sql_class.cc:1646
#8  0x000055706532b1ed in grant_init () at /test/10.11_dbg/sql/sql_acl.cc:7824
#9  0x00005570652ab25f in mysqld_main (argc=<optimized out>, argv=<optimized out>) at /test/10.11_dbg/sql/mysqld.cc:5780
#10 0x000055706529eb76 in main (argc=<optimized out>, argv=<optimized out>) at /test/10.11_dbg/sql/main.cc:34

One time on 10.11 debug while executing this testcase from MDEV-28514:

CREATE TABLE t (f BIT,UNIQUE KEY(f) USING HASH) ENGINE=MyISAM;
REPAIR TABLE t;

Reruns simply resulted in the stack of MDEV-28514. The data dir was 100% clean/fresh when this happened.
This would seem to indicate the the issue can be seen semi-randomly at any time.
I would also not notice this in testing as the issue/stack is generally filtered out.

Comment by Roel Van de Paar [ 2022-10-15 ]

Executing

SET SESSION storage_engine=Aria;
SET GLOBAL aria_group_commit=1;
SET GLOBAL aria_group_commit_interval=1000000000;
CREATE TABLE t (c INT);

At the client, then exiting from the client, and running mysqladmin shutdown will hang the server with new client access blocked.

Comment by Roel Van de Paar [ 2022-10-15 ]

Executing

SET GLOBAL aria_group_commit=1;
INSTALL PLUGIN Spider SONAME 'ha_spider.so';
SET GLOBAL aria_group_commit_INTERVAL=100000000;
CREATE TABLE t (c INT,INDEX (c)) ENGINE=Spider COMMENT='TABLE "st"' PARTITION BY LIST COLUMNS (c) (PARTITION p DEFAULT COMMENT='srv "d"' ENGINE=Spider);

Immediately locks up the client, with this rather strange looking full processlist (note row 12):

10.11.0 6ebdd3013a18b01dbecec76b870810329eb76586 (Debug)

10.11.0-dbg>SHOW FULL PROCESSLIST \G
*************************** 1. row ***************************
      Id: 4
    User: root
    Host: localhost
      db: test
 Command: Query
    Time: 271
   State: creating table
    Info: CREATE TABLE t (c INT,INDEX (c)) ENGINE=Spider COMMENT='TABLE "st"' PARTITION BY LIST COLUMNS (c) (PARTITION p DEFAULT COMMENT='srv "d"' ENGINE=Spider)
Progress: 0.000
*************************** 2. row ***************************
      Id: 5
    User: system user
    Host: 
      db: NULL
 Command: Daemon
    Time: NULL
   State: Spider table background cardinality action handler
    Info: NULL
Progress: 0.000
*************************** 3. row ***************************
      Id: 6
    User: system user
    Host: 
      db: NULL
 Command: Daemon
    Time: NULL
   State: Spider table background cardinality action handler
    Info: NULL
Progress: 0.000
*************************** 4. row ***************************
      Id: 7
    User: system user
    Host: 
      db: NULL
 Command: Daemon
    Time: NULL
   State: Spider table background cardinality action handler
    Info: NULL
Progress: 0.000
*************************** 5. row ***************************
      Id: 8
    User: system user
    Host: 
      db: NULL
 Command: Daemon
    Time: NULL
   State: Spider table background statistics action handler
    Info: NULL
Progress: 0.000
*************************** 6. row ***************************
      Id: 9
    User: system user
    Host: 
      db: NULL
 Command: Daemon
    Time: NULL
   State: Spider table background statistics action handler
    Info: NULL
Progress: 0.000
*************************** 7. row ***************************
      Id: 10
    User: system user
    Host: 
      db: NULL
 Command: Daemon
    Time: NULL
   State: Spider table background statistics action handler
    Info: NULL
Progress: 0.000
*************************** 8. row ***************************
      Id: 11
    User: system user
    Host: 
      db: NULL
 Command: Daemon
    Time: NULL
   State: Spider table background cardinality action handler
    Info: NULL
Progress: 0.000
*************************** 9. row ***************************
      Id: 17
    User: system user
    Host: 
      db: NULL
 Command: Daemon
    Time: NULL
   State: Spider table background statistics action handler
    Info: NULL
Progress: 0.000
*************************** 10. row ***************************
      Id: 12
    User: system user
    Host: 
      db: NULL
 Command: Daemon
    Time: NULL
   State: Spider table background statistics action handler
    Info: NULL
Progress: 0.000
*************************** 11. row ***************************
      Id: 18
    User: system user
    Host: 
      db: NULL
 Command: Daemon
    Time: NULL
   State: Spider table background statistics action handler
    Info: NULL
Progress: 0.000
*************************** 12. row ***************************
      Id: 20
    User: system user
    Host: 
      db: NULL
 Command: Query
    Time: 171
   State: Waiting on cond
    Info: create procedure mysql.spider_fix_system_tables()begin  select 'MariaDB' into @server_name;  select substring_index(version(), '.', 1)    into @server_major_version;  select substring_index(substring_index(version(), '.', 2), '.', -1)    into @server_minor_version;  call mysql.spider_fix_one_table('spider_tables', 'server',   'alter table mysql.spider_tables    add server char(64) default null,    add scheme char(64) default null,    add host char(64) default null,    add port char(5) default null,    add socket char(64) default null,    add username char(64) default null,    add password char(64) default null,    add tgt_db_name char(64) default null,    add tgt_table_name char(64) default null');  select COLUMN_TYPE INTO @col_type from INFORMATION_SCHEMA.COLUMNS    where TABLE_SCHEMA = 'mysql'      AND TABLE_NAME = 'spider_xa'      AND COLUMN_NAME = 'data';  if @col_type != 'binary(128)' then    alter table mysql.spider_xa      modify data binary(128) not null default '';  end if;  select COLUMN_TYPE INTO @col_type from INFORMATION_SCHEMA.COLUMNS    where TABLE_SCHEMA = 'mysql'      AND TABLE_NAME = 'spider_xa_member'      AND COLUMN_NAME = 'data';  if @col_type != 'binary(128)' then    alter table mysql.spider_xa_member      modify data binary(128) not null default '';  end if;  call mysql.spider_fix_one_table('spider_tables', 'link_id',   'alter table mysql.spider_tables    add column link_id int not null default 0 after table_name,    drop primary key,    add primary key (db_name, table_name, link_id)');  call mysql.spider_fix_one_table('spider_tables', 'link_status',   'alter table mysql.spider_tables    add column link_status tinyint not null default 1');  call mysql.spider_fix_one_table('spider_xa_member', 'ssl_ca',   'alter table mysql.spider_xa_member    add column ssl_ca char(64) default null after password,    add column ssl_capath char(64) default null after ssl_ca,    add column ssl_cert char(64) default null after ssl_capath,    add column ssl_cipher char(64) default null after ssl_cert,    add column ssl_key char(64) default null after ssl_cipher,    add column ssl_verify_server_cert tinyint not null default 0      after ssl_key,    add column default_file char(64) default null      after ssl_verify_server_cert,    add column default_group char(64) default null after default_file');  call mysql.spider_fix_one_table('spider_tables', 'ssl_ca',   'alter table mysql.spider_tables    add column ssl_ca char(64) default null after password,    add column ssl_capath char(64) default null after ssl_ca,    add column ssl_cert char(64) default null after ssl_capath,    add column ssl_cipher char(64) default null after ssl_cert,    add column ssl_key char(64) default null after ssl_cipher,    add column ssl_verify_server_cert tinyint not null default 0      after ssl_key,    add column default_file char(64) default null      after ssl_verify_server_cert,    add column default_group char(64) default null after default_file');  call mysql.spider_fix_one_table('spider_link_mon_servers', 'ssl_ca',   'alter table mysql.spider_link_mon_servers    add column ssl_ca char(64) default null after password,    add column ssl_capath char(64) default null after ssl_ca,    add column ssl_cert char(64) default null after ssl_capath,    add column ssl_cipher char(64) default null after ssl_cert,    add column ssl_key char(64) default null after ssl_cipher,    add column ssl_verify_server_cert tinyint not null default 0      after ssl_key,    add column default_file char(64) default null      after ssl_verify_server_cert,    add column default_group char(64) default null after default_file');  select COLUMN_TYPE INTO @col_type from INFORMATION_SCHEMA.COLUMNS    where TABLE_SCHEMA = 'mysql'      AND TABLE_NAME = 'spider_link_mon_servers'      AND COLUMN_NAME = 'sid';  if @col_type != 'int(10) unsigned' then    alter table mysql.spider_link_mon_servers    modify sid int unsigned not null default 0;  end if;  select COLUMN_TYPE INTO @col_type from INFORMATION_SCHEMA.COLUMNS    where TABLE_SCHEMA = 'mysql'      AND TABLE_NAME = 'spider_xa_member'      AND COLUMN_NAME = 'socket';  if @col_type = 'char(64)' then    alter table mysql.spider_xa_member      drop primary key,      add index idx1 (data, format_id, gtrid_length, host),      modify socket text not null,      modify ssl_ca text,      modify ssl_capath text,      modify ssl_cert text,      modify ssl_key text,      modify default_file text;  end if;  select COLUMN_TYPE INTO @col_type from INFORMATION_SCHEMA.COLUMNS    where TABLE_SCHEMA = 'mysql'      AND TABLE_NAME = 'spider_tables'      AND COLUMN_NAME = 'socket';  if @col_type = 'char(64)' then    alter table mysql.spider_tables      modify socket text,      modify ssl_ca text,      modify ssl_capath text,      modify ssl_cert text,      modify ssl_key text,      modify default_file text;  end if;  select COLUMN_TYPE INTO @col_type from INFORMATION_SCHEMA.COLUMNS    where TABLE_SCHEMA = 'mysql'      AND TABLE_NAME = 'spider_link_mon_servers'      AND COLUMN_NAME = 'socket';  if @col_type = 'char(64)' then    alter table mysql.spider_link_mon_servers      modify socket text,      modify ssl_ca text,      modify ssl_capath text,      modify ssl_cert text,      modify ssl_key text,      modify default_file text;  end if;  call mysql.spider_fix_one_table('spider_tables',   'monitoring_binlog_pos_at_failing',   'alter table mysql.spider_tables    add monitoring_binlog_pos_at_failing tinyint not null default 0      after ssl_verify_server_cert');  call mysql.spider_fix_one_table('spider_tables', 'block_status',   'alter table mysql.spider_tables    add column block_status tinyint not null default 0      after link_status');  call mysql.spider_fix_one_table('spider_tables', 'static_link_id',   'alter table mysql.spider_tables    add column static_link_id char(64) default null after block_status,    add unique index uidx1 (db_name, table_name, static_link_id)');  select COLUMN_TYPE INTO @col_type from INFORMATION_SCHEMA.COLUMNS    where TABLE_SCHEMA = 'mysql'      AND TABLE_NAME = 'spider_link_mon_servers'      AND COLUMN_NAME = 'link_id';  if @col_type != 'char(64)' then    alter table mysql.spider_link_mon_servers    modify link_id char(64) not null default '';  end if;  select COLUMN_TYPE INTO @col_type from INFORMATION_SCHEMA.COLUMNS    where TABLE_SCHEMA = 'mysql'      AND TABLE_NAME = 'spider_link_failed_log'      AND COLUMN_NAME = 'link_id';  if @col_type != 'char(64)' then    alter table mysql.spider_link_failed_log    modify link_id char(64) not null default '';  end if;  select COLUMN_TYPE INTO @col_type from INFORMATION_SCHEMA.COLUMNS    where TABLE_SCHEMA = 'mysql'      AND TABLE_NAME = 'spider_tables'      AND COLUMN_NAME = 'table_name';  if @col_type != 'char(199)' then    alter table mysql.spider_tables    modify table_name char(199) not null default '';  end if;  select COLUMN_TYPE INTO @col_type from INFORMATION_SCHEMA.COLUMNS    where TABLE_SCHEMA = 'mysql'      AND TABLE_NAME = 'spider_link_mon_servers'      AND COLUMN_NAME = 'table_name';  if @col_type != 'char(199)' then    alter table mysql.spider_link_mon_servers    modify table_name char(199) not null default '';  end if;  select COLUMN_TYPE INTO @col_type from INFORMATION_SCHEMA.COLUMNS    where TABLE_SCHEMA = 'mysql'      AND TABLE_NAME = 'spider_link_failed_log'      AND COLUMN_NAME = 'table_name';  if @col_type != 'char(199)' then    alter table mysql.spider_link_failed_log    modify table_name char(199) not null default '';  end if;  select COLUMN_TYPE INTO @col_type from INFORMATION_SCHEMA.COLUMNS    where TABLE_SCHEMA = 'mysql'      AND TABLE_NAME = 'spider_table_position_for_recovery'      AND COLUMN_NAME = 'table_name';  if @col_type != 'char(199)' then    alter table mysql.spider_table_position_for_recovery    modify table_name char(199) not null default '';  end if;  select COLUMN_TYPE INTO @col_type from INFORMATION_SCHEMA.COLUMNS    where TABLE_SCHEMA = 'mysql'      AND TABLE_NAME = 'spider_table_sts'      AND COLUMN_NAME = 'table_name';  if @col_type != 'char(199)' then    alter table mysql.spider_table_sts    modify table_name char(199) not null default '';  end if;  select COLUMN_TYPE INTO @col_type from INFORMATION_SCHEMA.COLUMNS    where TABLE_SCHEMA = 'mysql'      AND TABLE_NAME = 'spider_table_crd'      AND COLUMN_NAME = 'table_name';  if @col_type != 'char(199)' then    alter table mysql.spider_table_crd    modify table_name char(199) not null default '';  end if;  call mysql.spider_fix_one_table('spider_table_sts', 'checksum',   'alter table mysql.spider_table_sts    add column checksum bigint unsigned default null after update_time');  if @server_name = 'MariaDB' and    (      @server_major_version > 10 or      (        @server_major_version = 10 and        @server_minor_version >= 4      )    )  then    select ENGINE INTO @engine_name from INFORMATION_SCHEMA.TABLES      where TABLE_SCHEMA = 'mysql'        AND TABLE_NAME = 'spider_link_failed_log';    if @engine_name != 'Aria' then      alter table mysql.spider_link_failed_log        engine=Aria transactional=1;    end if;    select ENGINE INTO @engine_name from INFORMATION_SCHEMA.TABLES      where TABLE_SCHEMA = 'mysql'        AND TABLE_NAME = 'spider_link_mon_servers';    if @engine_name != 'Aria' then      alter table mysql.spider_link_mon_servers        engine=Aria transactional=1;    end if;    select ENGINE INTO @engine_name from INFORMATION_SCHEMA.TABLES      where TABLE_SCHEMA = 'mysql'        AND TABLE_NAME = 'spider_table_crd';    if @engine_name != 'Aria' then      alter table mysql.spider_table_crd        engine=Aria transactional=1;    end if;    select ENGINE INTO @engine_name from INFORMATION_SCHEMA.TABLES      where TABLE_SCHEMA = 'mysql'        AND TABLE_NAME = 'spider_table_position_for_recovery';    if @engine_name != 'Aria' then      alter table mysql.spider_table_position_for_recovery        engine=Aria transactional=1;    end if;    select ENGINE INTO @engine_name from INFORMATION_SCHEMA.TABLES      where TABLE_SCHEMA = 'mysql'        AND TABLE_NAME = 'spider_table_sts';    if @engine_name != 'Aria' then      alter table mysql.spider_table_sts        engine=Aria transactional=1;    end if;    select ENGINE INTO @engine_name from INFORMATION_SCHEMA.TABLES      where TABLE_SCHEMA = 'mysql'        AND TABLE_NAME = 'spider_tables';    if @engine_name != 'Aria' then      alter table mysql.spider_tables        engine=Aria transactional=1;    end if;    select ENGINE INTO @engine_name from INFORMATION_SCHEMA.TABLES      where TABLE_SCHEMA = 'mysql'        AND TABLE_NAME = 'spider_xa';    if @engine_name != 'Aria' then      alter table mysql.spider_xa        engine=Aria transactional=1;    end if;    select ENGINE INTO @engine_name from INFORMATION_SCHEMA.TABLES      where TABLE_SCHEMA = 'mysql'        AND TABLE_NAME = 'spider_xa_failed_log';    if @engine_name != 'Aria' then      alter table mysql.spider_xa_failed_log        engine=Aria transactional=1;    end if;    select ENGINE INTO @engine_name from INFORMATION_SCHEMA.TABLES      where TABLE_SCHEMA = 'mysql'        AND TABLE_NAME = 'spider_xa_member';    if @engine_name != 'Aria' then      alter table mysql.spider_xa_member        engine=Aria transactional=1;    end if;  end if;  if @server_name = 'MariaDB' and    (      @server_major_version > 10 or      (        @server_major_version = 10 and        @server_minor_version >= 999      )    )  then    /* table for ddl pushdown */    create table if not exists mysql.spider_rewrite_tables(      table_id bigint unsigned not null auto_increment,      db_name char(64) not null default '',      table_name char(64) not null default '',      primary key (table_id),      unique uk1(db_name, table_name)    ) engine=Aria transactional=1 default charset=utf8 collate=utf8_bin;    create table if not exists mysql.spider_rewrite_table_tables(      table_id bigint unsigned not null,      partition_id bigint unsigned not null auto_increment,      partition_method varchar(18) default '',      partition_expression varchar(64) default '',      subpartition_method varchar(12) default '',      subpartition_expression varchar(64) default '',      connection_str text not null default '',      comment_str text not null default '',      primary key (table_id, partition_id),      unique uk1(table_id, partition_method, partition_expression,        subpartition_method, subpartition_expression)    ) engine=Aria transactional=1 default charset=utf8 collate=utf8_bin;    create table if not exists mysql.spider_rewrite_table_partitions(      table_id bigint unsigned not null,      partition_id bigint unsigned not null,      partition_ordinal_position bigint unsigned not null auto_increment,      partition_name varchar(64) not null default '',      partition_description varchar(64) not null default '',      connection_str text not null default '',      comment_str text not null default '',      primary key (table_id, partition_id, partition_ordinal_position),      unique key uk1 (table_id, partition_id, partition_name)    ) engine=Aria transactional=1 default charset=utf8 collate=utf8_bin;    create table if not exists mysql.spider_rewrite_table_subpartitions(      table_id bigint unsigned not null,      partition_id bigint unsigned not null,      partition_ordinal_position bigint unsigned not null,      subpartition_ordinal_position bigint unsigned not null        auto_increment,      subpartition_name varchar(64) not null default '',      subpartition_description varchar(64) not null default '',      connection_str text not null default '',      comment_str text not null default '',      primary key (table_id, partition_id, partition_ordinal_position,        subpartition_ordinal_position),      unique key uk1 (table_id, partition_id, partition_ordinal_position,        subpartition_name)    ) engine=Aria transactional=1 default charset=utf8 collate=utf8_bin;    create table if not exists mysql.spider_rewritten_tables(      db_name char(64) not null,      table_name char(64) not null,      table_id bigint unsigned not null,      partition_id bigint unsigned not null,      primary key (db_name, table_name, table_id, partition_id)    ) engine=Aria transactional=1 default charset=utf8 collate=utf8_bin;  end if;  call mysql.spider_fix_one_table('spider_link_mon_servers', 'dsn',   'alter table mysql.spider_link_mon_servers    add column dsn char(64) default null after default_group');  call mysql.spider_fix_one_table('spider_tables', 'dsn',   'alter table mysql.spider_tables    add column dsn char(64) default null after default_group');  call mysql.spider_fix_one_table('spider_xa_failed_log', 'dsn',   'alter table mysql.spider_xa_failed_log    add column dsn char(64) default null after default_group');  call mysql.spider_fix_one_table('spider_xa_member', 'dsn',   'alter table mysql.spider_xa_member    add column dsn char(64) default null after default_group');  call mysql.spider_fix_one_table('spider_link_mon_servers', 'filedsn',   'alter table mysql.spider_link_mon_servers    add column filedsn text default null after dsn');  call mysql.spider_fix_one_table('spider_tables', 'filedsn',   'alter table mysql.spider_tables    add column filedsn text default null after dsn');  call mysql.spider_fix_one_table('spider_xa_failed_log', 'filedsn',   'alter table mysql.spider_xa_failed_log    add column filedsn text default null after dsn');  call mysql.spider_fix_one_table('spider_xa_member', 'filedsn',   'alter table mysql.spider_xa_member    add column filedsn text default null after dsn');  call mysql.spider_fix_one_table('spider_link_mon_servers', 'driver',   'alter table mysql.spider_link_mon_servers    add column driver char(64) default null after filedsn');  call mysql.spider_fix_one_table('spider_tables', 'driver',   'alter table mysql.spider_tables    add column driver char(64) default null after filedsn');  call mysql.spider_fix_one_table('spider_xa_failed_log', 'driver',   'alter table mysql.spider_xa_failed_log    add column driver char(64) default null after filedsn');  call mysql.spider_fix_one_table('spider_xa_member', 'driver',   'alter table mysql.spider_xa_member    add column driver char(64) default null after filedsn');end
Progress: 0.000
*************************** 13. row ***************************
      Id: 22
    User: system user
    Host: 
      db: NULL
 Command: Daemon
    Time: NULL
   State: Spider table background cardinality action handler
    Info: NULL
Progress: 0.000
*************************** 14. row ***************************
      Id: 14
    User: system user
    Host: 
      db: NULL
 Command: Daemon
    Time: NULL
   State: Spider table background statistics action handler
    Info: NULL
Progress: 0.000
*************************** 15. row ***************************
      Id: 13
    User: system user
    Host: 
      db: NULL
 Command: Daemon
    Time: NULL
   State: Spider table background cardinality action handler
    Info: NULL
Progress: 0.000
*************************** 16. row ***************************
      Id: 16
    User: system user
    Host: 
      db: NULL
 Command: Daemon
    Time: NULL
   State: Spider table background cardinality action handler
    Info: NULL
Progress: 0.000
*************************** 17. row ***************************
      Id: 15
    User: system user
    Host: 
      db: NULL
 Command: Daemon
    Time: NULL
   State: Spider table background cardinality action handler
    Info: NULL
Progress: 0.000
*************************** 18. row ***************************
      Id: 21
    User: system user
    Host: 
      db: NULL
 Command: Daemon
    Time: NULL
   State: Spider table background statistics action handler
    Info: NULL
Progress: 0.000
*************************** 19. row ***************************
      Id: 19
    User: system user
    Host: 
      db: NULL
 Command: Daemon
    Time: NULL
   State: Spider table background cardinality action handler
    Info: NULL
Progress: 0.000
*************************** 20. row ***************************
      Id: 23
    User: system user
    Host: 
      db: NULL
 Command: Daemon
    Time: NULL
   State: Spider table background statistics action handler
    Info: NULL
Progress: 0.000
*************************** 21. row ***************************
      Id: 24
    User: system user
    Host: 
      db: NULL
 Command: Daemon
    Time: NULL
   State: Spider table background cardinality action handler
    Info: NULL
Progress: 0.000
*************************** 22. row ***************************
      Id: 25
    User: root
    Host: localhost
      db: test
 Command: Query
    Time: 0
   State: starting
    Info: SHOW FULL PROCESSLIST
Progress: 0.000
22 rows in set (0.001 sec)

Attempting CTRL+c leaves the client hanging, after displaying

^CCtrl-C -- query killed. Continuing normally.

With this new processlist:

10.11.0 6ebdd3013a18b01dbecec76b870810329eb76586 (Debug)

10.11.0-dbg>SHOW FULL PROCESSLIST;
+----+-------------+-----------+-------+---------+------+----------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------+----------+
| Id | User        | Host      | db    | Command | Time | State                                              | Info                                                                                                                                                    | Progress |
+----+-------------+-----------+-------+---------+------+----------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------+----------+
|  4 | root        | localhost | test  | Killed  |  518 | creating table                                     | CREATE TABLE t (c INT,INDEX (c)) ENGINE=Spider COMMENT='TABLE "st"' PARTITION BY LIST COLUMNS (c) (PARTITION p DEFAULT COMMENT='srv "d"' ENGINE=Spider) |    0.000 |
|  5 | system user |           | NULL  | Daemon  | NULL | Spider table background cardinality action handler | NULL                                                                                                                                                    |    0.000 |
|  6 | system user |           | NULL  | Daemon  | NULL | Spider table background cardinality action handler | NULL                                                                                                                                                    |    0.000 |
|  7 | system user |           | NULL  | Daemon  | NULL | Spider table background cardinality action handler | NULL                                                                                                                                                    |    0.000 |
|  8 | system user |           | NULL  | Daemon  | NULL | Spider table background statistics action handler  | NULL                                                                                                                                                    |    0.000 |
|  9 | system user |           | NULL  | Daemon  | NULL | Spider table background statistics action handler  | NULL                                                                                                                                                    |    0.000 |
| 10 | system user |           | NULL  | Daemon  | NULL | Spider table background statistics action handler  | NULL                                                                                                                                                    |    0.000 |
| 11 | system user |           | NULL  | Daemon  | NULL | Spider table background cardinality action handler | NULL                                                                                                                                                    |    0.000 |
| 17 | system user |           | NULL  | Daemon  | NULL | Spider table background statistics action handler  | NULL                                                                                                                                                    |    0.000 |
| 12 | system user |           | NULL  | Daemon  | NULL | Spider table background statistics action handler  | NULL                                                                                                                                                    |    0.000 |
| 18 | system user |           | NULL  | Daemon  | NULL | Spider table background statistics action handler  | NULL                                                                                                                                                    |    0.000 |
| 20 | system user |           | mysql | Query   |  218 | After create                                       | alter table mysql.spider_link_mon_servers        engine=Aria transactional=1                                                                            |    0.000 |
| 22 | system user |           | NULL  | Daemon  | NULL | Spider table background cardinality action handler | NULL                                                                                                                                                    |    0.000 |
| 14 | system user |           | NULL  | Daemon  | NULL | Spider table background statistics action handler  | NULL                                                                                                                                                    |    0.000 |
| 13 | system user |           | NULL  | Daemon  | NULL | Spider table background cardinality action handler | NULL                                                                                                                                                    |    0.000 |
| 16 | system user |           | NULL  | Daemon  | NULL | Spider table background cardinality action handler | NULL                                                                                                                                                    |    0.000 |
| 15 | system user |           | NULL  | Daemon  | NULL | Spider table background cardinality action handler | NULL                                                                                                                                                    |    0.000 |
| 21 | system user |           | NULL  | Daemon  | NULL | Spider table background statistics action handler  | NULL                                                                                                                                                    |    0.000 |
| 19 | system user |           | NULL  | Daemon  | NULL | Spider table background cardinality action handler | NULL                                                                                                                                                    |    0.000 |
| 23 | system user |           | NULL  | Daemon  | NULL | Spider table background statistics action handler  | NULL                                                                                                                                                    |    0.000 |
| 24 | system user |           | NULL  | Daemon  | NULL | Spider table background cardinality action handler | NULL                                                                                                                                                    |    0.000 |
| 27 | root        | localhost | test  | Query   |    0 | starting                                           | SHOW FULL PROCESSLIST                                                                                                                                   |    0.000 |
+----+-------------+-----------+-------+---------+------+----------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------+----------+
22 rows in set (0.001 sec)

Another CTRL+c brings us back to the CLI however there are still Spider threads doing work.
nayuta-yanagisawa This does not look directly related to spider, but FYI.

Generated at Thu Feb 08 09:24:23 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.