main.innodb_ext_key fatal assertion

$ mysql-test/mtr --repeat=30 --parallel=4 main.innodb_ext_key 

Logging: /home/dan/repos/mariadb-server-10.6/mysql-test/mariadb-test-run.pl  --repeat=30 --parallel=4 main.innodb_ext_key

vardir: /home/dan/repos/build-mariadb-server-10.6/mysql-test/var

Checking leftover processes...

Removing old var directory...

Creating var directory '/home/dan/repos/build-mariadb-server-10.6/mysql-test/var'...

Checking supported features...

MariaDB Version 10.6.5-MariaDB

 - SSL connections supported

 - binaries built with wsrep patch

Collecting tests...

Installing system database...

==============================================================================

TEST                                  WORKER RESULT   TIME (ms) or COMMENT

--------------------------------------------------------------------------

worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019

worker[4] Using MTR_BUILD_THREAD 301, with reserved ports 16020..16039

worker[2] Using MTR_BUILD_THREAD 302, with reserved ports 16040..16059

worker[3] Using MTR_BUILD_THREAD 303, with reserved ports 16060..16079

main.innodb_ext_key 'innodb,off,unoptimized' w4 [ pass ]   2918

main.innodb_ext_key 'covering,innodb,off' w3 [ pass ]   2958

main.innodb_ext_key 'innodb,on,unoptimized' w2 [ pass ]   2918

main.innodb_ext_key 'covering,innodb,on' w1 [ pass ]   2972

main.innodb_ext_key 'covering,innodb,off' w3 [ 2 pass ]   2707

main.innodb_ext_key 'innodb,on,unoptimized' w2 [ 2 pass ]   2722

main.innodb_ext_key 'covering,innodb,on' w1 [ 2 pass ]   2616

main.innodb_ext_key 'covering,innodb,off' w3 [ 3 pass ]   2654

main.innodb_ext_key 'covering,innodb,on' w1 [ 3 pass ]   2653

main.innodb_ext_key 'innodb,on,unoptimized' w2 [ 3 pass ]   2733

main.innodb_ext_key 'covering,innodb,off' w3 [ 4 pass ]   2400

main.innodb_ext_key 'covering,innodb,on' w1 [ 4 pass ]   2426

main.innodb_ext_key 'innodb,on,unoptimized' w2 [ 4 pass ]   2507

main.innodb_ext_key 'covering,innodb,off' w3 [ 5 pass ]   2419

main.innodb_ext_key 'covering,innodb,on' w1 [ 5 pass ]   2371

main.innodb_ext_key 'covering,innodb,off' w3 [ 6 pass ]   5280

main.innodb_ext_key 'covering,innodb,on' w1 [ 6 pass ]   5267

main.innodb_ext_key 'covering,innodb,off' w3 [ 7 pass ]   2155

main.innodb_ext_key 'covering,innodb,on' w1 [ 7 pass ]   2143

main.innodb_ext_key 'covering,innodb,on' w1 [ 8 pass ]   2792

main.innodb_ext_key 'covering,innodb,off' w3 [ 8 pass ]   2825

main.innodb_ext_key 'covering,innodb,on' w1 [ 9 pass ]   2069

main.innodb_ext_key 'covering,innodb,off' w3 [ 9 pass ]   2031

main.innodb_ext_key 'covering,innodb,on' w1 [ 10 pass ]   1834

main.innodb_ext_key 'covering,innodb,off' w3 [ 10 pass ]   1848

main.innodb_ext_key 'covering,innodb,on' w1 [ 11 pass ]   3932

main.innodb_ext_key 'covering,innodb,off' w3 [ 11 pass ]   3929

main.innodb_ext_key 'covering,innodb,on' w1 [ 12 pass ]   2203

main.innodb_ext_key 'covering,innodb,off' w3 [ 12 pass ]   2303

main.innodb_ext_key 'covering,innodb,on' w1 [ 13 pass ]   6001

main.innodb_ext_key 'covering,innodb,off' w3 [ 13 pass ]   5990

main.innodb_ext_key 'covering,innodb,on' w1 [ 14 pass ]   1789

main.innodb_ext_key 'covering,innodb,off' w3 [ 14 pass ]   1782

main.innodb_ext_key 'covering,innodb,on' w1 [ 15 pass ]   1881

main.innodb_ext_key 'covering,innodb,off' w3 [ 15 pass ]   1896

main.innodb_ext_key 'covering,innodb,on' w1 [ 16 pass ]   2187

main.innodb_ext_key 'covering,innodb,off' w3 [ 16 pass ]   2261

main.innodb_ext_key 'covering,innodb,on' w1 [ 17 pass ]   2011

main.innodb_ext_key 'covering,innodb,off' w3 [ 17 pass ]   2048

main.innodb_ext_key 'covering,innodb,on' w1 [ 18 pass ]   1838

main.innodb_ext_key 'covering,innodb,off' w3 [ 18 pass ]   1862

main.innodb_ext_key 'covering,innodb,on' w1 [ 19 pass ]   1850

main.innodb_ext_key 'covering,innodb,off' w3 [ 19 pass ]   1761

main.innodb_ext_key 'covering,innodb,on' w1 [ 20 pass ]   1855

main.innodb_ext_key 'covering,innodb,off' w3 [ 20 pass ]   1907

main.innodb_ext_key 'covering,innodb,on' w1 [ 21 pass ]   2080

main.innodb_ext_key 'covering,innodb,off' w3 [ 21 pass ]   2134

main.innodb_ext_key 'covering,innodb,on' w1 [ 22 pass ]   1986

main.innodb_ext_key 'covering,innodb,off' w3 [ 22 pass ]   2019

main.innodb_ext_key 'covering,innodb,on' w1 [ 23 pass ]   1969

main.innodb_ext_key 'covering,innodb,off' w3 [ 23 pass ]   1855

main.innodb_ext_key 'covering,innodb,on' w1 [ 24 pass ]   1899

main.innodb_ext_key 'covering,innodb,off' w3 [ 24 pass ]   1803

main.innodb_ext_key 'covering,innodb,on' w1 [ 25 pass ]   2050

main.innodb_ext_key 'covering,innodb,off' w3 [ 25 pass ]   2034

main.innodb_ext_key 'covering,innodb,on' w1 [ 26 pass ]   2068

main.innodb_ext_key 'covering,innodb,off' w3 [ 26 pass ]   2081

main.innodb_ext_key 'covering,innodb,on' w1 [ 27 pass ]   2056

main.innodb_ext_key 'covering,innodb,off' w3 [ 27 pass ]   2210

main.innodb_ext_key 'covering,innodb,on' w1 [ 28 pass ]   1868

main.innodb_ext_key 'covering,innodb,off' w3 [ 28 pass ]   1855

main.innodb_ext_key 'covering,innodb,on' w1 [ 29 pass ]   1852

main.innodb_ext_key 'covering,innodb,off' w3 [ 29 pass ]   1786

main.innodb_ext_key 'covering,innodb,off' w3 [ 30 pass ]   2099

main.innodb_ext_key 'covering,innodb,on' w1 [ 30 pass ]   2205

main.innodb_ext_key 'innodb,off,unoptimized' w4 [ 2 fail ]

        Test ended at 2021-09-07 22:28:31

CURRENT_TEST: main.innodb_ext_key

mysqltest: At line 588: query 'CREATE TABLE t1 (

pk INT,

f1 VARCHAR(3),

f2 VARCHAR(1024),

PRIMARY KEY (pk),

KEY(f2)

) ENGINE=InnoDB CHARSET utf8 ROW_FORMAT= DYNAMIC' failed: <Unknown> (2013): Lost connection to server during query

The result from queries just before the failure was:

< snip >

id	select_type	table	type	possible_keys	key	key_len	ref	rows	Extra

1	SIMPLE	t1	ref	index_date_updated	index_date_updated	5	const	#	Using index condition

# This used to work from the start:

explain 

select * from t2 force index(index_date_updated) 

where index_date_updated= 10 and index_id < 800;

id	select_type	table	type	possible_keys	key	key_len	ref	rows	Extra

1	SIMPLE	t2	ref	index_date_updated	index_date_updated	5	const	#	Using index condition

drop table t0,t1,t2;

#

# MDEV-11196: Error:Run-Time Check Failure #2 - Stack around the variable 'key_buff'

# was corrupted, server crashes in opt_sum_query

SET @save_optimizer_use_condition_selectivity=@@optimizer_use_condition_selectivity, @@optimizer_use_condition_selectivity=4;

CREATE TABLE t1 (

pk INT,

f1 VARCHAR(3),

f2 VARCHAR(1024),

PRIMARY KEY (pk),

KEY(f2)

) ENGINE=InnoDB CHARSET utf8 ROW_FORMAT= DYNAMIC;

More results from queries before failure can be found in /home/dan/repos/build-mariadb-server-10.6/mysql-test/var/4/log/innodb_ext_key,off.log

Server [mysqld.1 - pid: 1635380, winpid: 1635380, exit: 256] failed during test run

Server log from this test:

----------SERVER LOG START-----------

2021-09-07 22:28:29 0 [ERROR] [FATAL] InnoDB: innodb_fatal_semaphore_wait_threshold was exceeded for dict_sys.latch. Please refer to https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/

210907 22:28:29 [ERROR] mysqld got signal 6 ;

This could be because you hit a bug. It is also possible that this binary

or one of the libraries it was linked against is corrupt, improperly built,

or misconfigured. This error can also be caused by malfunctioning hardware.

To report this bug, see https://mariadb.com/kb/en/reporting-bugs

We will try our best to scrape up some info that will hopefully help

diagnose the problem, but since we have already crashed, 

something is definitely wrong and this may fail.

Server version: 10.6.5-MariaDB-log

Thread 1 (Thread 0x7fa054ff9640 (LWP 1635530)):

#0  __pthread_kill (threadid=<optimized out>, signo=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:54

        pd = <optimized out>

        tid = 1635530

        pid = 1635383

        val = 0

#1  0x00000000008f7bc6 in handle_fatal_signal (sig=6) at /home/dan/repos/mariadb-server-10.6/sql/signal_handler.cc:345

        tm = {tm_sec = 29, tm_min = 28, tm_hour = 22, tm_mday = 7, tm_mon = 8, tm_year = 121, tm_wday = 2, tm_yday = 249, tm_isdst = 0, tm_gmtoff = 36000, tm_zone = 0x2b87e50 "AEST"}

        print_invalid_query_pointer = <optimized out>

        curr_time = 1631017709

        thd = 0x0

#2  <signal handler called>

No locals.

#3  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:49

        set = {__val = {552967, 2147516417, 140326178332088, 46234680, 140326597527936, 13394987, 17938038, 729, 1946169716, 140326597527816, 9223512778638098432, 46234680, 1, 1, 31952248, 46263432}}

        pid = <optimized out>

        tid = <optimized out>

        ret = <optimized out>

#4  0x00007fa08f4128a4 in __GI_abort () at abort.c:79

        save_stage = 1

        act = {__sigaction_handler = {sa_handler = 0x7fa054ff8a08, sa_sigaction = 0x7fa054ff8a08}, sa_mask = {__val = {140327578090128, 140326379457952, 0, 14, 158, 17937887, 158, 140326597528128, 11548080394786522880, 0, 18446744073709551272, 0, 140327478049406, 140327478049407, 47667776, 140326597528088}}, sa_flags = -1891137592, sa_restorer = 0x7fa054ff8910}

        sigs = {__val = {32, 23245968, 140327478049406, 140327478049407, 140326597528000, 17918744, 140326597527808, 10287617, 140326597528048, 10283534, 109, 140326111055120, 158, 0, 140326111055120, 0}}

#5  0x0000000000c8557f in ib::fatal::~fatal (this=0x7fa054ff8a60) at /home/dan/repos/mariadb-server-10.6/storage/innobase/ut/ut0ut.cc:519

No locals.

#6  0x0000000000c5d720 in srv_monitor_task () at /home/dan/repos/mariadb-server-10.6/storage/innobase/srv/srv0srv.cc:1293

        waited = <optimized out>

        start = <optimized out>

        old_lsn = 8539879

        new_lsn = <optimized out>

        now = <optimized out>

        threshold = <optimized out>

#7  0x0000000000d4310c in tpool::thread_pool_generic::timer_generic::run (this=0x2ea5970) at /home/dan/repos/mariadb-server-10.6/tpool/tpool_generic.cc:313

        expected = false

#8  0x0000000000d439b7 in tpool::task::execute (this=0x2ea59b0) at /home/dan/repos/mariadb-server-10.6/tpool/task.cc:52

No locals.

#9  0x0000000000d41596 in tpool::thread_pool_generic::worker_main (this=0x2d663a0, thread_var=0x2d75a40) at /home/dan/repos/mariadb-server-10.6/tpool/tpool_generic.cc:550

        task = 0x2

#10 0x00007fa08f6afc84 in execute_native_thread_routine () from /lib64/libstdc++.so.6

No symbol table info available.

#11 0x00007fa08f807299 in start_thread (arg=0x7fa054ff9640) at pthread_create.c:481

        ret = <optimized out>

        pd = 0x7fa054ff9640

        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140326597531200, -1804064771434488301, 140327478049406, 140327478049407, 0, 140326597531200, 1852674176714399251, 1853147301372616211}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}

        not_first_call = 0

#12 0x00007fa08f4ec353 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

No locals.

(gdb)  p  dict_sys.latch

$1 = {pfs_psi = 0x7fa08b2eccc0, lock = {writer = {lock = {<std::__atomic_base<unsigned int>> = {_M_i = 2147483650}, <No data fields>}}, readers = {<std::__atomic_base<unsigned int>> = {_M_i = 2147483648}, <No data fields>}}}