[MDEV-25065] binlog_encryption.encrypted_master failed in bb with a crash - Jira

XML

Word

Printable

Details

Type: Bug
Status: Closed (View Workflow)
Priority: Major
Resolution: Duplicate
Affects Version/s: 10.2(EOL), 10.3(EOL), 10.4(EOL), 10.5, 10.6, 10.7(EOL)
Fix Version/s: 10.2.41, 10.3.32, 10.4.22, 10.5.13, 10.6.5, 10.7.0
Component/s: Storage Engine - InnoDB, Tests
Labels:
None

Description

http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest/builds/28270/steps/mtr_nm/logs/stdio

10.3 4473d174308eeb14757fc98
binlog_encryption.encrypted_master 'ctr,innodb,row' w3 [ fail ]
Test ended at 2021-02-25 06:06:51

CURRENT_TEST: binlog_encryption.encrypted_master
mysqltest: In included file "./include/sync_slave_sql_with_io.inc":
included from /mnt/buildbot/build/mariadb-10.3.29/mysql-test/suite/binlog_encryption/encrypted_master.test at line 160:
At line 43: query 'let $_sync_slave_sql_with_io_errno= `SELECT MASTER_POS_WAIT('$_master_log_file', $_master_log_pos, $_slave_timeout)`' failed: 2013: Lost connection to MySQL server during query

The result from queries just before the failure was:
< snip >
DROP DATABASE database_name_to_encrypt;
DROP USER new_user_name_to_encrypt;
DROP SERVER server_name_to_encrypt;
#############################
# Final checks for the master
#############################
NOT FOUND /_to_encrypt./ in master-bin.0
NOT FOUND /COMMIT./ in master-bin.0
NOT FOUND /TIMESTAMP./ in master-bin.0
include/save_master_pos.inc
#############################
# Final checks for the slave
#############################
connection server_2;
include/sync_io_with_master.inc
FOUND 1 /_to_encrypt./ in slave-relay-bin.0
FOUND 1 /COMMIT./ in slave-relay-bin.0
FOUND 1 /TIMESTAMP./ in slave-relay-bin.0
include/start_slave.inc
include/sync_slave_sql_with_io.inc

More results from queries before failure can be found in /mnt/buildbot/build/mariadb-10.3.29/mysql-test/var/3/log/encrypted_master.log


Server [mysqld.2 - pid: 4152, winpid: 4152, exit: 256] failed during test run
Server log from this test:
----------SERVER LOG START-----------
2021-02-25 6:06:41 0 [Note] /mnt/buildbot/build/mariadb-10.3.29/sql/mysqld (mysqld 10.3.29-MariaDB-log) starting as process 4154 ...
2021-02-25 6:06:41 0 [Warning] Could not increase number of max_open_files to more than 1024 (request: 32186)
2021-02-25 6:06:41 0 [Warning] Changed limits: max_open_files: 1024 max_connections: 151 (was 151) table_cache: 421 (was 2000)
2021-02-25 6:06:41 0 [Note] InnoDB: Using Linux native AIO
2021-02-25 6:06:41 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2021-02-25 6:06:41 0 [Note] InnoDB: Uses event mutexes
2021-02-25 6:06:41 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2021-02-25 6:06:41 0 [Note] InnoDB: Number of pools: 1
2021-02-25 6:06:41 0 [Note] InnoDB: Using generic crc32 instructions
2021-02-25 6:06:41 0 [Note] InnoDB: Initializing buffer pool, total size = 8M, instances = 1, chunk size = 8M
2021-02-25 6:06:41 0 [Note] InnoDB: Completed initialization of buffer pool
2021-02-25 6:06:41 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2021-02-25 6:06:41 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
2021-02-25 6:06:41 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2021-02-25 6:06:41 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2021-02-25 6:06:41 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2021-02-25 6:06:41 0 [Note] InnoDB: Waiting for purge to start
2021-02-25 6:06:41 0 [Note] InnoDB: 10.3.29 started; log sequence number 1635798; transaction id 33
2021-02-25 6:06:41 0 [Note] InnoDB: Loading buffer pool(s) from /dev/shm/var_auto_fiNA/3/mysqld.2/data/ib_buffer_pool
2021-02-25 6:06:41 0 [Note] Plugin 'SEQUENCE' is disabled.
2021-02-25 6:06:41 0 [Note] Plugin 'INNODB_SYS_DATAFILES' is disabled.
2021-02-25 6:06:41 0 [Note] Plugin 'INNODB_SYS_TABLESTATS' is disabled.
2021-02-25 6:06:41 0 [Note] Plugin 'INNODB_TABLESPACES_SCRUBBING' is disabled.
2021-02-25 6:06:41 0 [Note] Plugin 'INNODB_CMP' is disabled.
2021-02-25 6:06:41 0 [Note] Plugin 'INNODB_FT_DELETED' is disabled.
2021-02-25 6:06:41 0 [Note] Plugin 'INNODB_CMP_RESET' is disabled.
2021-02-25 6:06:41 0 [Note] Plugin 'INNODB_TABLESPACES_ENCRYPTION' is disabled.
2021-02-25 6:06:41 0 [Note] InnoDB: Buffer pool(s) load completed at 210225 6:06:41
2021-02-25 6:06:41 0 [Note] Plugin 'INNODB_CMPMEM_RESET' is disabled.
2021-02-25 6:06:41 0 [Note] Plugin 'FEEDBACK' is disabled.
2021-02-25 6:06:41 0 [Note] Plugin 'INNODB_FT_INDEX_TABLE' is disabled.
2021-02-25 6:06:41 0 [Note] Plugin 'INNODB_CMP_PER_INDEX_RESET' is disabled.
2021-02-25 6:06:41 0 [Note] Plugin 'user_variables' is disabled.
2021-02-25 6:06:41 0 [Note] Plugin 'INNODB_FT_INDEX_CACHE' is disabled.
2021-02-25 6:06:41 0 [Note] Plugin 'INNODB_FT_BEING_DELETED' is disabled.
2021-02-25 6:06:41 0 [Note] Plugin 'INNODB_FT_DEFAULT_STOPWORD' is disabled.
2021-02-25 6:06:41 0 [Note] Plugin 'INNODB_FT_CONFIG' is disabled.
2021-02-25 6:06:41 0 [Note] Plugin 'INNODB_SYS_TABLESPACES' is disabled.
2021-02-25 6:06:41 0 [Note] Plugin 'INNODB_MUTEXES' is disabled.
2021-02-25 6:06:41 0 [Note] Plugin 'INNODB_SYS_SEMAPHORE_WAITS' is disabled.
2021-02-25 6:06:41 0 [Warning] /mnt/buildbot/build/mariadb-10.3.29/sql/mysqld: unknown variable 'loose-feedback-debug-startup-interval=20'
2021-02-25 6:06:41 0 [Warning] /mnt/buildbot/build/mariadb-10.3.29/sql/mysqld: unknown variable 'loose-feedback-debug-first-interval=60'
2021-02-25 6:06:41 0 [Warning] /mnt/buildbot/build/mariadb-10.3.29/sql/mysqld: unknown variable 'loose-feedback-debug-interval=60'
2021-02-25 6:06:41 0 [Warning] /mnt/buildbot/build/mariadb-10.3.29/sql/mysqld: unknown option '--loose-pam-debug'
2021-02-25 6:06:41 0 [Warning] /mnt/buildbot/build/mariadb-10.3.29/sql/mysqld: unknown variable 'loose-file-key-management-encryption-algorithm=aes_ctr'
2021-02-25 6:06:41 0 [Warning] /mnt/buildbot/build/mariadb-10.3.29/sql/mysqld: unknown variable 'loose-debug-sync-timeout=300'
2021-02-25 6:06:41 0 [Note] Server socket created on IP: '127.0.0.1'.
2021-02-25 6:06:41 0 [Note] Reading of all Master_info entries succeeded
2021-02-25 6:06:41 0 [Note] Added new Master_info '' to hash table
2021-02-25 6:06:41 0 [Note] /mnt/buildbot/build/mariadb-10.3.29/sql/mysqld: ready for connections.
Version: '10.3.29-MariaDB-log' socket: '/mnt/buildbot/build/mariadb-10.3.29/mysql-test/var/tmp/3/mysqld.2.sock' port: 16041 MariaDB Server
2021-02-25 6:06:42 10 [Note] Deleted Master_info file '/mnt/buildbot/build/mariadb-10.3.29/mysql-test/var/3/mysqld.2/data/master.info'.
2021-02-25 6:06:42 10 [Note] Deleted Master_info file '/mnt/buildbot/build/mariadb-10.3.29/mysql-test/var/3/mysqld.2/data/relay-log.info'.
2021-02-25 6:06:42 10 [Note] Master connection name: '' Master_info_file: 'master.info' Relay_info_file: 'relay-log.info'
2021-02-25 6:06:42 10 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='', master_port='3306', master_log_file='', master_log_pos='4'. New state master_host='127.0.0.1', master_port='16040', master_log_file='master-bin.000001', master_log_pos='4'.
2021-02-25 6:06:42 12 [Note] Slave I/O thread: Start asynchronous replication to master 'root@127.0.0.1:16040' in log 'master-bin.000001' at position 4
2021-02-25 6:06:42 13 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 4, relay log './slave-relay-bin.000001' position: 4
2021-02-25 6:06:42 12 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16040',replication started in log 'master-bin.000001' at position 4
2021-02-25 6:06:42 13 [Note] Error reading relay log event: slave SQL thread was killed
2021-02-25 6:06:42 13 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 369
2021-02-25 6:06:42 13 [Note] master was 127.0.0.1:16040
2021-02-25 6:06:47 14 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 369, relay log './slave-relay-bin.000002' position: 629
210225 6:06:49 [ERROR] mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.

To report this bug, see https://mariadb.com/kb/en/reporting-bugs

We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.

Server version: 10.3.29-MariaDB-log
key_buffer_size=1048576
read_buffer_size=131072
max_used_connections=2
max_threads=153
thread_count=10
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 63200 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x7f0ed00009a8
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7f0eeaffcc08 thread_stack 0x49000
mysys/stacktrace.c:175(my_print_stacktrace)[0x556caddcc6fe]
sql/signal_handler.cc:209(handle_fatal_signal)[0x556cad8792e7]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7f0f10262390]
bits/list.tcc:73(std::__cxx11::_List_base<dict_v_idx_t, ut_allocator<dict_v_idx_t, true> >::_M_clear())[0x556cadca57b8]
btr/btr0sea.cc:1269(btr_search_drop_page_hash_index(buf_block_t*))[0x556cadc426cf]
buf/buf0lru.cc:1615(buf_LRU_free_page(buf_page_t*, bool))[0x556cadc744ca]
buf/buf0lru.cc:700(buf_LRU_free_from_common_LRU_list)[0x556cadc763f8]
buf/buf0lru.cc:969(buf_LRU_get_free_block(buf_pool_t*))[0x556cadc76ed5]
include/buf0buf.ic:369(buf_page_set_state)[0x556cadc4ed71]
btr/btr0sea.cc:198(btr_search_check_free_space_in_heap(dict_index_t const*))[0x556cadc3f27e]
btr/btr0sea.cc:1562(btr_search_info_update_slow(btr_search_t, btr_cur_t))[0x556cadc44df0]
include/btr0sea.ic:84(btr_search_info_update)[0x556cadc37be6]
include/btr0pcur.ic:448(btr_pcur_open_low(dict_index_t, unsigned long, dtuple_t const, page_cur_mode_t, unsigned long, btr_pcur_t, char const, unsigned int, unsigned long, mtr_t*) [clone .constprop.32])[0x556cadba15f2]
row/row0row.cc:1045(row_search_on_row_ref(btr_pcur_t, unsigned long, dict_table_t const, dtuple_t const, mtr_t))[0x556cadba18c5]
row/row0purge.cc:81(row_purge_reposition_pcur)[0x556cadb9b902]
include/dyn0buf.h:183(mtr_buf_t::erase())[0x556cadb9e4c2]
row/row0purge.cc:1260(row_purge)[0x556cadb9ed21]
que/que0que.cc:1038(que_thr_step)[0x556cadb5942a]
include/sync0types.h:1121(my_atomic_addlint)[0x556cadbca9dc]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba)[0x7f0f102586ba]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f0f0f90382d]

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x0): (null)
Connection ID (thread ID): 4
Status: NOT_KILLED

Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on

The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
information that should help you find out what is causing the crash.

We think the query pointer is invalid, but we will try to print it anyway.
Query:

Writing a core file...
Working directory at /dev/shm/var_auto_fiNA/3/mysqld.2/data
Resource Limits:
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size unlimited unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 23715 23715 processes
Max open files 1024 1024 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 23715 23715 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
Core pattern: \|/usr/share/apport/apport %p %s %c %P

----------SERVER LOG END-------------


- found 'core' (0/0)

Trying 'dbx' to get a backtrace

Trying 'gdb' to get a backtrace from coredump /mnt/buildbot/build/mariadb-10.3.29/mysql-test/var/3/log/binlog_encryption.encrypted_master-ctr,innodb,row/mysqld.2/data/core
Core generated by '/mnt/buildbot/build/mariadb-10.3.29/sql/mysqld'
Output from gdb follows. The first stack trace is from the failing thread.
The following stack traces are from all threads (so the failing one is
duplicated).
--------------------------
[New LWP 4189]
[New LWP 4186]
[New LWP 4160]
[New LWP 4163]
[New LWP 4164]
[New LWP 4197]
[New LWP 4187]
[New LWP 4443]
[New LWP 4188]
[New LWP 4162]
[New LWP 4196]
[New LWP 4161]
[New LWP 4154]
[New LWP 4220]
[New LWP 4159]
[New LWP 4206]
[New LWP 4158]
[New LWP 4205]
[New LWP 4226]
[New LWP 4198]
[New LWP 4234]
[New LWP 4157]
[New LWP 4178]
[New LWP 4177]
[New LWP 4176]
[New LWP 4175]
[New LWP 4174]
[New LWP 4165]
[New LWP 4185]
[New LWP 4195]
[New LWP 4203]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/mnt/buildbot/build/mariadb-10.3.29/sql/mysqld --defaults-group-suffix=.2 --def'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007f0f1025f59f in pthread_kill () from /lib/x86_64-linux-gnu/libpthread.so.0
[Current thread is 1 (Thread 0x7f0eeaffd700 (LWP 4189))]
#0 0x00007f0f1025f59f in pthread_kill () from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x0000556caddcc4f7 in my_write_core (sig=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/mysys/stacktrace.c:386
#2 0x0000556cad87936e in handle_fatal_signal (sig=11) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/signal_handler.cc:343
#3 <signal handler called>
#4 std::__cxx11::_List_base<dict_v_idx_t, ut_allocator<dict_v_idx_t, true> >::_M_clear (this=0x7f0ea406e5c0) at /usr/include/c++/5/bits/list.tcc:73
#5 std::__cxx11::_List_base<dict_v_idx_t, ut_allocator<dict_v_idx_t, true> >::~_List_base (this=0x7f0ea406e5c0, __in_chrg=<optimized out>) at /usr/include/c++/5/bits/stl_list.h:446
#6 std::__cxx11::list<dict_v_idx_t, ut_allocator<dict_v_idx_t, true> >::~list (this=0x7f0ea406e5c0, __in_chrg=<optimized out>) at /usr/include/c++/5/bits/stl_list.h:507
#7 dict_mem_table_free (table=0x7f0ea406e248) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/dict/dict0mem.cc:236
#8 0x0000556cad5b9127 in btr_search_lazy_free (index=index@entry=0x7f0ea40729c8) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/btr/btr0sea.cc:320
#9 0x0000556cadc426cf in btr_search_drop_page_hash_index (block=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/btr/btr0sea.cc:1265
#10 0x0000556cadc744ca in buf_LRU_free_page (bpage=bpage@entry=0x7f0f037f8730, zip=zip@entry=true) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/buf/buf0lru.cc:1612
#11 0x0000556cadc763f8 in buf_LRU_free_from_common_LRU_list (scan_all=<optimized out>, buf_pool=0x7f0f037f8860) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/buf/buf0lru.cc:695
#12 buf_LRU_scan_and_free_block (buf_pool=buf_pool@entry=0x556cafb35810, scan_all=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/buf/buf0lru.cc:737
#13 0x0000556cadc76ed5 in buf_LRU_get_free_block (buf_pool=0x556cafb35810) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/buf/buf0lru.cc:967
#14 0x0000556cadc4ed71 in buf_block_alloc (buf_pool=<optimized out>, buf_pool@entry=0x0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/buf/buf0buf.cc:753
#15 0x0000556cadc3f27e in btr_search_check_free_space_in_heap (index=0x7f0ea40998a8) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/btr/btr0sea.cc:198
#16 0x0000556cadc44df0 in btr_search_info_update_slow (info=0x7f0ea4099bd0, cursor=cursor@entry=0x556cafc24440) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/btr/btr0sea.cc:1559
#17 0x0000556cadc37be6 in btr_search_info_update (cursor=0x556cafc24440, index=0x7f0ea40998a8) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/include/btr0sea.ic:84
#18 btr_cur_search_to_nth_level_func (index=index@entry=0x7f0ea40998a8, level=level@entry=0, tuple=tuple@entry=0x7f0ed00085f8, mode=mode@entry=PAGE_CUR_LE, latch_mode=latch_mode@entry=2, cursor=cursor@entry=0x556cafc24440, ahi_latch=0x0, file=0x556cadfcfbc8 "/home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0row.cc", line=1045, mtr=0x7f0eeaffc500, autoinc=0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/btr/btr0cur.cc:2356
#19 0x0000556cadba15f2 in btr_pcur_open_low (index=index@entry=0x7f0ea40998a8, tuple=tuple@entry=0x7f0ed00085f8, latch_mode=latch_mode@entry=2, cursor=cursor@entry=0x556cafc24440, line=line@entry=1045, mtr=mtr@entry=0x7f0eeaffc500, autoinc=0, file=0x556cadfcfbc8 "/home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0row.cc", mode=PAGE_CUR_LE, level=0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/include/btr0pcur.ic:446
#20 0x0000556cadba18c5 in row_search_on_row_ref (pcur=pcur@entry=0x556cafc24440, mode=mode@entry=2, table=<optimized out>, ref=0x7f0ed00085f8, mtr=mtr@entry=0x7f0eeaffc500) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0row.cc:1045
#21 0x0000556cadb9b902 in row_purge_reposition_pcur (mtr=0x7f0eeaffc500, node=0x556cafc24398, mode=2) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0purge.cc:78
#22 row_purge_reset_trx_id (node=node@entry=0x556cafc24398, mtr=mtr@entry=0x7f0eeaffc500) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0purge.cc:795
#23 0x0000556cadb9e4c2 in row_purge_record_func (node=node@entry=0x556cafc24398, undo_rec=undo_rec@entry=0x7f0edc09be18 "", thr=thr@entry=0x556cafc241e0, updated_extern=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0purge.cc:1208
#24 0x0000556cadb9ed21 in row_purge (thr=0x556cafc241e0, undo_rec=0x7f0edc09be18 "", node=0x556cafc24398) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0purge.cc:1257
#25 row_purge_step (thr=0x556cafc241e0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0purge.cc:1316
#26 0x0000556cadb5942a in que_thr_step (thr=0x1e) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/que/que0que.cc:1038
#27 que_run_threads_low (thr=0x1e) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/que/que0que.cc:1100
#28 que_run_threads (thr=thr@entry=0x556cafc241e0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/que/que0que.cc:1140
#29 0x0000556cadbca9dc in srv_task_execute () at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/srv/srv0srv.cc:2488
#30 srv_worker_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/srv/srv0srv.cc:2536
#31 0x00007f0f102586ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#32 0x00007f0f0f90382d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 31 (Thread 0x7f0f08471700 (LWP 4203)):
#0 0x00007f0f1025e360 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x0000556cad94d6b9 in inline_mysql_cond_wait (that=0x556cae7b2608 <mysql_bin_log+3208>, mutex=0x556cae7b25d8 <mysql_bin_log+3160>, src_file=0x556cadf8b938 "/home/buildbot/buildbot/build/mariadb-10.3.29/sql/log.cc", src_line=9951) at /home/buildbot/buildbot/build/mariadb-10.3.29/include/mysql/psi/mysql_thread.h:1174
#2 binlog_background_thread (arg=arg@entry=0x0) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/log.cc:9951
#3 0x0000556cada0f3f1 in pfs_spawn_thread (arg=0x556caf953c78) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/perfschema/pfs.cc:1869
#4 0x00007f0f102586ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007f0f0f90382d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 30 (Thread 0x7f0eea7fc700 (LWP 4195)):
#0 0x00007f0f1025e360 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x0000556cadb38f16 in os_event::wait (this=0x556cafa50ab0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:158
#2 os_event::wait_low (reset_sig_count=1, this=0x556cafa50ab0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:325
#3 os_event_wait_low (event=0x556cafa50ab0, reset_sig_count=reset_sig_count@entry=0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:502
#4 0x0000556cadc61da7 in buf_dump_thread () at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/buf/buf0dump.cc:834
#5 0x00007f0f102586ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#6 0x00007f0f0f90382d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 29 (Thread 0x7f0efb7fe700 (LWP 4185)):
#0 0x00007f0f10261c1d in nanosleep () from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x0000556cadb39180 in os_thread_sleep (tm=tm@entry=1000000) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0thread.cc:231
#2 0x0000556cadbc97b2 in srv_master_sleep () at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/srv/srv0srv.cc:2354
#3 srv_master_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/srv/srv0srv.cc:2394
#4 0x00007f0f102586ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007f0f0f90382d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 28 (Thread 0x7f0efbfff700 (LWP 4165)):
#0 0x00007f0f1025e360 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x0000556cadb38f16 in os_event::wait (this=0x556cafb5ace0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:158
#2 os_event::wait_low (reset_sig_count=1, this=0x556cafb5ace0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:325
#3 os_event_wait_low (event=0x556cafb5ace0, reset_sig_count=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:502
#4 0x0000556cadbd372c in sync_array_wait_event (arr=arr@entry=0x556cafa27f90, cell=@0x7f0efbffe5b8: 0x556cafa28090) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/sync/sync0arr.cc:471
#5 0x0000556cadbd6e72 in rw_lock_sx_lock_func (lock=lock@entry=0x7f0f037f9be8, pass=pass@entry=2, file_name=file_name@entry=0x556cadfdf470 "/home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/buf/buf0flu.cc", line=line@entry=1214) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/sync/sync0rw.cc:844
#6 0x0000556cadc675e1 in pfs_rw_lock_sx_lock_func (pass=2, file_name=0x556cadfdf470 "/home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/buf/buf0flu.cc", line=1214, lock=0x7f0f037f9be8) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/include/sync0rw.ic:695
#7 buf_flush_page (buf_pool=0x556cafb35810, bpage=0x7f0f037f9b58, flush_type=BUF_FLUSH_LIST, sync=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/buf/buf0flu.cc:1214
#8 0x0000556cadc67fcb in buf_flush_try_neighbors (page_id=..., page_id@entry=..., flush_type=flush_type@entry=BUF_FLUSH_LIST, n_flushed=n_flushed@entry=260, n_to_flush=n_to_flush@entry=1041) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/buf/buf0flu.cc:1460
#9 0x0000556cadc68a97 in buf_flush_page_and_try_neighbors (flush_type=BUF_FLUSH_LIST, count=<synthetic pointer>, n_to_flush=<optimized out>, bpage=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/buf/buf0flu.cc:1533
#10 buf_do_flush_list_batch (buf_pool=buf_pool@entry=0x556cafb35810, min_n=min_n@entry=1041, lsn_limit=lsn_limit@entry=18446744073709551615) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/buf/buf0flu.cc:1791
#11 0x0000556cadc6b218 in buf_flush_batch (n=0x7f0efbffe990, lsn_limit=18446744073709551615, min_n=1041, flush_type=BUF_FLUSH_LIST, buf_pool=0x556cafb35810) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/buf/buf0flu.cc:1861
#12 buf_flush_do_batch (type=BUF_FLUSH_LIST, n=0x7f0efbffe990, lsn_limit=18446744073709551615, min_n=1041, buf_pool=0x556cafb35810) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/buf/buf0flu.cc:2019
#13 pc_flush_slot () at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/buf/buf0flu.cc:2818
#14 0x0000556cadc6ce83 in buf_flush_page_cleaner_coordinator () at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/buf/buf0flu.cc:3231
#15 0x00007f0f102586ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#16 0x00007f0f0f90382d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 27 (Thread 0x7f0efaffd700 (LWP 4174)):
#0 0x00007f0f1025e709 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x0000556cadb388c2 in os_event::timed_wait (this=this@entry=0x556cafbeb0a0, abstime=abstime@entry=0x7f0efaffcbf0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:275
#2 0x0000556cadb38b11 in os_event::wait_time_low (this=this@entry=0x556cafbeb0a0, time_in_usec=time_in_usec@entry=1000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:385
#3 0x0000556cadb38eca in os_event_wait_time_low (event=event@entry=0x556cafbeb0a0, time_in_usec=time_in_usec@entry=1000000, reset_sig_count=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:485
#4 0x0000556cadb0e349 in lock_wait_timeout_thread () at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/lock/lock0wait.cc:522
#5 0x00007f0f102586ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#6 0x00007f0f0f90382d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 26 (Thread 0x7f0efa7fc700 (LWP 4175)):
#0 0x00007f0f1025e709 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x0000556cadb388c2 in os_event::timed_wait (this=this@entry=0x556cafa509d0, abstime=abstime@entry=0x7f0efa7fbb60) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:275
#2 0x0000556cadb38b11 in os_event::wait_time_low (this=0x556cafa509d0, time_in_usec=time_in_usec@entry=1000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:385
#3 0x0000556cadb38eca in os_event_wait_time_low (event=<optimized out>, time_in_usec=time_in_usec@entry=1000000, reset_sig_count=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:485
#4 0x0000556cadbc866b in srv_error_monitor_thread () at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/srv/srv0srv.cc:1864
#5 0x00007f0f102586ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#6 0x00007f0f0f90382d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 25 (Thread 0x7f0ef9ffb700 (LWP 4176)):
#0 0x00007f0f1025e709 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x0000556cadb388c2 in os_event::timed_wait (this=this@entry=0x556cafa50a40, abstime=abstime@entry=0x7f0ef9ffac90) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:275
#2 0x0000556cadb38b11 in os_event::wait_time_low (this=0x556cafa50a40, time_in_usec=time_in_usec@entry=5000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:385
#3 0x0000556cadb38eca in os_event_wait_time_low (event=<optimized out>, time_in_usec=time_in_usec@entry=5000000, reset_sig_count=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:485
#4 0x0000556cadbc7fa2 in srv_monitor_thread () at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/srv/srv0srv.cc:1700
#5 0x00007f0f102586ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#6 0x00007f0f0f90382d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 24 (Thread 0x7f0ef97fa700 (LWP 4177)):
#0 0x00007f0f1025e709 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x0000556cadb388c2 in os_event::timed_wait (this=this@entry=0x556cafc1c4f0, abstime=abstime@entry=0x7f0ef97f9be0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:275
#2 0x0000556cadb38b11 in os_event::wait_time_low (this=0x556cafc1c4f0, time_in_usec=time_in_usec@entry=10000000, reset_sig_count=1, reset_sig_count@entry=0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:385
#3 0x0000556cadb38eca in os_event_wait_time_low (event=<optimized out>, time_in_usec=time_in_usec@entry=10000000, reset_sig_count=reset_sig_count@entry=0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:485
#4 0x0000556cadcb3dc0 in dict_stats_thread () at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/dict/dict0stats_bg.cc:476
#5 0x00007f0f102586ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#6 0x00007f0f0f90382d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 23 (Thread 0x7f0ef8ff9700 (LWP 4178)):
#0 0x00007f0f1025e709 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x0000556cadb388c2 in os_event::timed_wait (this=this@entry=0x556cafbb1f90, abstime=abstime@entry=0x7f0ef8ff8a30) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:275
#2 0x0000556cadb38b11 in os_event::wait_time_low (this=0x556cafbb1f90, time_in_usec=time_in_usec@entry=5000000, reset_sig_count=reset_sig_count@entry=1) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:385
#3 0x0000556cadb38eca in os_event_wait_time_low (event=<optimized out>, time_in_usec=time_in_usec@entry=5000000, reset_sig_count=reset_sig_count@entry=1) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:485
#4 0x0000556cadc1827a in ib_wqueue_timedwait (wq=wq@entry=0x556cafc33e00, wait_in_usecs=wait_in_usecs@entry=5000000) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/ut/ut0wqueue.cc:156
#5 0x0000556cadcfa67a in fts_optimize_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/fts/fts0opt.cc:2830
#6 0x00007f0f102586ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#7 0x00007f0f0f90382d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 22 (Thread 0x7f0f0a5ee700 (LWP 4157)):
#0 0x00007f0f1025e709 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x0000556caddd0e4d in inline_mysql_cond_timedwait (that=0x556caf017e00 <COND_timer>, mutex=0x556caf017e40 <LOCK_timer>, src_file=0x556cadff5a18 "/home/buildbot/buildbot/build/mariadb-10.3.29/mysys/thr_timer.c", src_line=292, abstime=0x7f0f0a5edce0) at /home/buildbot/buildbot/build/mariadb-10.3.29/include/mysql/psi/mysql_thread.h:1211
#2 timer_handler (arg=arg@entry=0x0) at /home/buildbot/buildbot/build/mariadb-10.3.29/mysys/thr_timer.c:292
#3 0x0000556cada0f3f1 in pfs_spawn_thread (arg=0x556caf946f88) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/perfschema/pfs.cc:1869
#4 0x00007f0f102586ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007f0f0f90382d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 21 (Thread 0x7f0f082ff700 (LWP 4234)):
#0 0x00007f0f0f8f7b5d in poll () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x0000556cade2defa in poll (__timeout=120000, __nfds=1, __fds=0x7f0f082fe670) at /usr/include/x86_64-linux-gnu/bits/poll2.h:46
#2 vio_io_wait (vio=vio@entry=0x7f0eb000b858, event=VIO_IO_EVENT_READ, timeout=120000) at /home/buildbot/buildbot/build/mariadb-10.3.29/vio/viosocket.c:962
#3 0x0000556cade2dff8 in vio_socket_io_wait (vio=0x7f0eb000b858, event=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/vio/viosocket.c:119
#4 0x0000556cade2e0ed in vio_read (vio=vio@entry=0x7f0eb000b858, buf=0x7f0eb000ba58 "'", size=size@entry=16384) at /home/buildbot/buildbot/build/mariadb-10.3.29/vio/viosocket.c:196
#5 0x0000556cade2e205 in vio_read_buff (vio=0x7f0eb000b858, buf=0x7f0eb000fa68 "", size=4) at /home/buildbot/buildbot/build/mariadb-10.3.29/vio/viosocket.c:244
#6 0x0000556cad5e0eee in my_real_read (net=0x7f0eb000b178, complen=complen@entry=0x7f0f082fe938, header=header@entry=0 '\000') at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/net_serv.cc:975
#7 0x0000556cad5e21a0 in my_net_read_packet_reallen (net=net@entry=0x7f0eb000b178, read_from_server=read_from_server@entry=0 '\000', reallen=reallen@entry=0x7f0f082feaa8) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/net_serv.cc:1256
#8 0x0000556cad84e307 in cli_safe_read_reallen (mysql=mysql@entry=0x7f0eb000b178, reallen=reallen@entry=0x7f0f082feaa8) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql-common/client.c:594
#9 0x0000556cad5ff2e2 in read_event (network_read_len=0x7f0f082feaa8, suppress_warnings=0x7f0f082fea8a, mi=0x556cafce0890, mysql=0x7f0eb000b178) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/slave.cc:3561
#10 handle_slave_io (arg=arg@entry=0x556cafce0890) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/slave.cc:4708
#11 0x0000556cada0f3f1 in pfs_spawn_thread (arg=0x7f0eb8082148) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/perfschema/pfs.cc:1869
#12 0x00007f0f102586ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#13 0x00007f0f0f90382d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 20 (Thread 0x7f0ee8ff9700 (LWP 4198)):
#0 0x00007f0f1025e360 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x0000556cadabab2b in inline_mysql_cond_wait (src_file=0x556cadfa58c0 "/home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/handler/ha_innodb.cc", src_line=325, mutex=0x7f0ee8ff8cc0, that=0x7f0ee8ff8cf0) at /home/buildbot/buildbot/build/mariadb-10.3.29/include/mysql/psi/mysql_thread.h:1185
#2 thd_destructor_proxy () at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/handler/ha_innodb.cc:325
#3 0x0000556cada0f3f1 in pfs_spawn_thread (arg=0x556cafbc7e98) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/perfschema/pfs.cc:1869
#4 0x00007f0f102586ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007f0f0f90382d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 19 (Thread 0x7f0f08349700 (LWP 4226)):
#0 0x00007f0f0f8f7b5d in poll () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x0000556cade2defa in poll (__timeout=28800000, __nfds=1, __fds=0x7f0f08348970) at /usr/include/x86_64-linux-gnu/bits/poll2.h:46
#2 vio_io_wait (vio=vio@entry=0x556cafc712f8, event=VIO_IO_EVENT_READ, timeout=28800000) at /home/buildbot/buildbot/build/mariadb-10.3.29/vio/viosocket.c:962
#3 0x0000556cade2dff8 in vio_socket_io_wait (vio=0x556cafc712f8, event=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/vio/viosocket.c:119
#4 0x0000556cade2e0ed in vio_read (vio=0x556cafc712f8, buf=0x7f0eac007158 "\001", size=4) at /home/buildbot/buildbot/build/mariadb-10.3.29/vio/viosocket.c:196
#5 0x0000556cad5e0eee in my_real_read (net=0x7f0eac000c08, complen=complen@entry=0x7f0f08348c08, header=header@entry=1 '\001') at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/net_serv.cc:975
#6 0x0000556cad5e21a0 in my_net_read_packet_reallen (net=net@entry=0x7f0eac000c08, read_from_server=read_from_server@entry=1 '\001', reallen=reallen@entry=0x7f0f08348c58) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/net_serv.cc:1256
#7 0x0000556cad5e229d in my_net_read_packet (net=net@entry=0x7f0eac000c08, read_from_server=read_from_server@entry=1 '\001') at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/net_serv.cc:1240
#8 0x0000556cad69c093 in do_command (thd=0x7f0eac0009a8) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/sql_parse.cc:1274
#9 0x0000556cad7701e2 in do_handle_one_connection (connect=connect@entry=0x556cafc71098) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/sql_connect.cc:1403
#10 0x0000556cad77033d in handle_one_connection (arg=arg@entry=0x556cafc71098) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/sql_connect.cc:1308
#11 0x0000556cada0f3f1 in pfs_spawn_thread (arg=0x556cafc71648) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/perfschema/pfs.cc:1869
#12 0x00007f0f102586ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#13 0x00007f0f0f90382d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 18 (Thread 0x7f0f08427700 (LWP 4205)):
#0 0x00007f0f10262156 in sigwait () from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x0000556cad5d491b in signal_hand (arg=arg@entry=0x0) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/mysqld.cc:3590
#2 0x0000556cada0f3f1 in pfs_spawn_thread (arg=0x556cafc6ddd8) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/perfschema/pfs.cc:1869
#3 0x00007f0f102586ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#4 0x00007f0f0f90382d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 17 (Thread 0x7f0f09282700 (LWP 4158)):
#0 0x00007f0f1025e709 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x0000556cada5551c in inline_mysql_cond_timedwait (src_file=0x556cadfa1b18 "/home/buildbot/buildbot/build/mariadb-10.3.29/storage/maria/ma_servicethread.c", src_line=116, abstime=0x7f0f09281c60, mutex=0x556caefbab40 <LOCK_checkpoint>, that=0x556caefbab00 <COND_checkpoint>) at /home/buildbot/buildbot/build/mariadb-10.3.29/include/mysql/psi/mysql_thread.h:1211
#2 my_service_thread_sleep (control=control@entry=0x556cae6fab00 <checkpoint_control>, sleep_time=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/maria/ma_servicethread.c:115
#3 0x0000556cada4cd67 in ma_checkpoint_background (arg=arg@entry=0x1e) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/maria/ma_checkpoint.c:707
#4 0x0000556cada0f3f1 in pfs_spawn_thread (arg=0x556caf958c78) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/perfschema/pfs.cc:1869
#5 0x00007f0f102586ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#6 0x00007f0f0f90382d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 16 (Thread 0x7f0f083dd700 (LWP 4206)):
#0 0x00007f0f1025e360 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x0000556cad688fa9 in inline_mysql_cond_wait (that=0x556cae78abc0 <COND_manager>, mutex=0x556cae78ac00 <LOCK_manager>, src_file=0x556cade67b88 "/home/buildbot/buildbot/build/mariadb-10.3.29/sql/sql_manager.cc", src_line=102) at /home/buildbot/buildbot/build/mariadb-10.3.29/include/mysql/psi/mysql_thread.h:1174
#2 handle_manager (arg=arg@entry=0x0) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/sql_manager.cc:102
#3 0x0000556cada0f3f1 in pfs_spawn_thread (arg=0x556cafcd6ec8) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/perfschema/pfs.cc:1869
#4 0x00007f0f102586ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007f0f0f90382d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 15 (Thread 0x7f0f037d3700 (LWP 4159)):
#0 0x00007f0f10f5764a in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
#1 0x0000556cadb31818 in LinuxAIOHandler::collect (this=this@entry=0x7f0f037d2af0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0file.cc:1932
#2 0x0000556cadb31db2 in LinuxAIOHandler::poll (this=this@entry=0x7f0f037d2af0, m1=m1@entry=0x7f0f037d2b90, m2=m2@entry=0x7f0f037d2b98, request=request@entry=0x7f0f037d2ba0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0file.cc:2093
#3 0x0000556cadb36ba9 in os_aio_linux_handler (request=0x7f0f037d2ba0, m2=0x7f0f037d2b98, m1=0x7f0f037d2b90, global_segment=0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0file.cc:2147
#4 os_aio_handler (segment=segment@entry=0, m1=m1@entry=0x7f0f037d2b90, m2=m2@entry=0x7f0f037d2b98, request=request@entry=0x7f0f037d2ba0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0file.cc:5728
#5 0x0000556cadcbeb85 in fil_aio_wait (segment=segment@entry=0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/fil/fil0fil.cc:4430
#6 0x0000556cadbcd0f8 in io_handler_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/srv/srv0start.cc:326
#7 0x00007f0f102586ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#8 0x00007f0f0f90382d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 14 (Thread 0x7f0f08393700 (LWP 4220)):
#0 0x00007f0f1025e709 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x0000556cad762a28 in inline_mysql_cond_timedwait (src_file=0x556cade77870 "/home/buildbot/buildbot/build/mariadb-10.3.29/sql/rpl_rli.cc", src_line=945, abstime=0x7f0f0838ffc0, mutex=0x556cafce3c30, that=0x556cafce3d00) at /home/buildbot/buildbot/build/mariadb-10.3.29/include/mysql/psi/mysql_thread.h:1211
#2 Relay_log_info::wait_for_pos (this=this@entry=0x556cafce2258, thd=thd@entry=0x7f0eb80009a8, log_name=log_name@entry=0x7f0eb800f638, log_pos=log_pos@entry=1307177, timeout=timeout@entry=300) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/rpl_rli.cc:945
#3 0x0000556cad8d463c in Item_master_pos_wait::val_int (this=0x7f0eb800f8a8) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/item_func.cc:3760
#4 0x0000556cad7cf749 in Type_handler::Item_send_longlong (this=<optimized out>, item=0x7f0eb800f8a8, protocol=0x7f0eb8000ed8, buf=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/sql_type.cc:5450
#5 0x0000556cad5e7621 in Protocol::send_result_set_row (this=this@entry=0x7f0eb8000ed8, row_items=row_items@entry=0x7f0eb8004f60) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/protocol.cc:999
#6 0x0000556cad64e4e3 in select_send::send_data (this=0x7f0eb800fad8, items=...) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/sql_class.cc:2945
#7 0x0000556cad6e82ba in JOIN::exec_inner (this=this@entry=0x7f0eb800fb00) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/sql_select.cc:4000
#8 0x0000556cad6e8663 in JOIN::exec (this=this@entry=0x7f0eb800fb00) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/sql_select.cc:3919
#9 0x0000556cad6e87af in mysql_select (thd=thd@entry=0x7f0eb80009a8, tables=0x0, wild_num=0, fields=..., conds=<optimized out>, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147748608, result=0x7f0eb800fad8, unit=0x7f0eb80046b0, select_lex=0x7f0eb8004e38) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/sql_select.cc:4324
#10 0x0000556cad6e90a8 in handle_select (thd=thd@entry=0x7f0eb80009a8, lex=lex@entry=0x7f0eb80045f0, result=result@entry=0x7f0eb800fad8, setup_tables_done_option=setup_tables_done_option@entry=0) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/sql_select.cc:382
#11 0x0000556cad5a5d49 in execute_sqlcom_select (thd=thd@entry=0x7f0eb80009a8, all_tables=0x0) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/sql_parse.cc:6317
#12 0x0000556cad696ad3 in mysql_execute_command (thd=thd@entry=0x7f0eb80009a8) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/sql_parse.cc:3848
#13 0x0000556cad698c6a in mysql_parse (thd=thd@entry=0x7f0eb80009a8, rawbuf=<optimized out>, length=57, parser_state=parser_state@entry=0x7f0f08392470, is_com_multi=is_com_multi@entry=false, is_next_command=is_next_command@entry=false) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/sql_parse.cc:7841
#14 0x0000556cad69a7cc in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x7f0eb80009a8, packet=packet@entry=0x7f0eb8007159 "", packet_length=packet_length@entry=57, is_com_multi=is_com_multi@entry=false, is_next_command=is_next_command@entry=false) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/sql_parse.cc:1853
#15 0x0000556cad69c121 in do_command (thd=0x7f0eb80009a8) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/sql_parse.cc:1399
#16 0x0000556cad7701e2 in do_handle_one_connection (connect=connect@entry=0x556cafc71098) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/sql_connect.cc:1403
#17 0x0000556cad77033d in handle_one_connection (arg=arg@entry=0x556cafc71098) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/sql_connect.cc:1308
#18 0x0000556cada0f3f1 in pfs_spawn_thread (arg=0x556cafc712f8) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/perfschema/pfs.cc:1869
#19 0x00007f0f102586ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#20 0x00007f0f0f90382d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 13 (Thread 0x7f0f114f68c0 (LWP 4154)):
#0 0x00007f0f0f8f7b5d in poll () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x0000556cad5de50f in poll (__timeout=-1, __nfds=2, __fds=0x7ffd41cf6470) at /usr/include/x86_64-linux-gnu/bits/poll2.h:41
#2 handle_connections_sockets () at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/mysqld.cc:6815
#3 0x0000556cad5dfed7 in mysqld_main (argc=152, argv=0x556caf820f68) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/mysqld.cc:6286
#4 0x00007f0f0f81d830 in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6
#5 0x0000556cad5d1b99 in _start ()

Thread 12 (Thread 0x7f0f027d1700 (LWP 4161)):
#0 0x00007f0f10f5764a in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
#1 0x0000556cadb31818 in LinuxAIOHandler::collect (this=this@entry=0x7f0f027d0af0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0file.cc:1932
#2 0x0000556cadb31db2 in LinuxAIOHandler::poll (this=this@entry=0x7f0f027d0af0, m1=m1@entry=0x7f0f027d0b90, m2=m2@entry=0x7f0f027d0b98, request=request@entry=0x7f0f027d0ba0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0file.cc:2093
#3 0x0000556cadb36ba9 in os_aio_linux_handler (request=0x7f0f027d0ba0, m2=0x7f0f027d0b98, m1=0x7f0f027d0b90, global_segment=0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0file.cc:2147
#4 os_aio_handler (segment=segment@entry=2, m1=m1@entry=0x7f0f027d0b90, m2=m2@entry=0x7f0f027d0b98, request=request@entry=0x7f0f027d0ba0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0file.cc:5728
#5 0x0000556cadcbeb85 in fil_aio_wait (segment=segment@entry=2) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/fil/fil0fil.cc:4430
#6 0x0000556cadbcd0f8 in io_handler_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/srv/srv0start.cc:326
#7 0x00007f0f102586ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#8 0x00007f0f0f90382d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 11 (Thread 0x7f0ee9ffb700 (LWP 4196)):
#0 0x00007f0f10261c1d in nanosleep () from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x0000556cadb39180 in os_thread_sleep (tm=tm@entry=1000000) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0thread.cc:231
#2 0x0000556cadc4c467 in btr_defragment_thread () at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/btr/btr0defragment.cc:715
#3 0x00007f0f102586ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#4 0x00007f0f0f90382d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 10 (Thread 0x7f0f01fd0700 (LWP 4162)):
#0 0x00007f0f10f5764a in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
#1 0x0000556cadb31818 in LinuxAIOHandler::collect (this=this@entry=0x7f0f01fcfaf0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0file.cc:1932
#2 0x0000556cadb31db2 in LinuxAIOHandler::poll (this=this@entry=0x7f0f01fcfaf0, m1=m1@entry=0x7f0f01fcfb90, m2=m2@entry=0x7f0f01fcfb98, request=request@entry=0x7f0f01fcfba0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0file.cc:2093
#3 0x0000556cadb36ba9 in os_aio_linux_handler (request=0x7f0f01fcfba0, m2=0x7f0f01fcfb98, m1=0x7f0f01fcfb90, global_segment=1) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0file.cc:2147
#4 os_aio_handler (segment=segment@entry=3, m1=m1@entry=0x7f0f01fcfb90, m2=m2@entry=0x7f0f01fcfb98, request=request@entry=0x7f0f01fcfba0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0file.cc:5728
#5 0x0000556cadcbeb85 in fil_aio_wait (segment=segment@entry=3) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/fil/fil0fil.cc:4430
#6 0x0000556cadbcd0f8 in io_handler_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/srv/srv0start.cc:326
#7 0x00007f0f102586ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#8 0x00007f0f0f90382d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 9 (Thread 0x7f0eeb7fe700 (LWP 4188)):
#0 0x00007f0f1025e360 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x0000556cadb38f16 in os_event::wait (this=0x556cafb7ef10) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:158
#2 os_event::wait_low (reset_sig_count=1, this=0x556cafb7ef10) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:325
#3 os_event_wait_low (event=0x556cafb7ef10, reset_sig_count=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:502
#4 0x0000556cadbd372c in sync_array_wait_event (arr=arr@entry=0x556cafa27f90, cell=@0x7f0eeb7fc258: 0x556cafa28050) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/sync/sync0arr.cc:471
#5 0x0000556cadbd677b in rw_lock_x_lock_func (lock=lock@entry=0x556cafb7eeb0, pass=pass@entry=0, file_name=file_name@entry=0x556cadfdce78 "/home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/btr/btr0sea.cc", line=line@entry=203) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/sync/sync0rw.cc:739
#6 0x0000556cadc3f31c in pfs_rw_lock_x_lock_func (pass=0, line=203, file_name=0x556cadfdce78 "/home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/btr/btr0sea.cc", lock=0x556cafb7eeb0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/include/sync0rw.ic:561
#7 btr_search_check_free_space_in_heap (index=0x7f0ea4096dd8) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/btr/btr0sea.cc:203
#8 0x0000556cadc44df0 in btr_search_info_update_slow (info=0x7f0ea4097c40, cursor=cursor@entry=0x556cafc2df88) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/btr/btr0sea.cc:1559
#9 0x0000556cadc37be6 in btr_search_info_update (cursor=0x556cafc2df88, index=0x7f0ea4096dd8) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/include/btr0sea.ic:84
#10 btr_cur_search_to_nth_level_func (index=index@entry=0x7f0ea4096dd8, level=level@entry=0, tuple=tuple@entry=0x7f0ecc008058, mode=mode@entry=PAGE_CUR_LE, latch_mode=latch_mode@entry=2, cursor=cursor@entry=0x556cafc2df88, ahi_latch=0x0, file=0x556cadfcfbc8 "/home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0row.cc", line=1045, mtr=0x7f0eeb7fd500, autoinc=0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/btr/btr0cur.cc:2356
#11 0x0000556cadba15f2 in btr_pcur_open_low (index=index@entry=0x7f0ea4096dd8, tuple=tuple@entry=0x7f0ecc008058, latch_mode=latch_mode@entry=2, cursor=cursor@entry=0x556cafc2df88, line=line@entry=1045, mtr=mtr@entry=0x7f0eeb7fd500, autoinc=0, file=0x556cadfcfbc8 "/home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0row.cc", mode=PAGE_CUR_LE, level=0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/include/btr0pcur.ic:446
#12 0x0000556cadba18c5 in row_search_on_row_ref (pcur=pcur@entry=0x556cafc2df88, mode=mode@entry=2, table=<optimized out>, ref=0x7f0ecc008058, mtr=mtr@entry=0x7f0eeb7fd500) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0row.cc:1045
#13 0x0000556cadb9b902 in row_purge_reposition_pcur (mtr=0x7f0eeb7fd500, node=0x556cafc2dee0, mode=2) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0purge.cc:78
#14 row_purge_reset_trx_id (node=node@entry=0x556cafc2dee0, mtr=mtr@entry=0x7f0eeb7fd500) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0purge.cc:795
#15 0x0000556cadb9e4c2 in row_purge_record_func (node=node@entry=0x556cafc2dee0, undo_rec=undo_rec@entry=0x7f0edc09ffd8 "", thr=thr@entry=0x556cafc2de28, updated_extern=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0purge.cc:1208
#16 0x0000556cadb9ed21 in row_purge (thr=0x556cafc2de28, undo_rec=0x7f0edc09ffd8 "", node=0x556cafc2dee0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0purge.cc:1257
#17 row_purge_step (thr=0x556cafc2de28) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0purge.cc:1316
#18 0x0000556cadb5942a in que_thr_step (thr=0x1e) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/que/que0que.cc:1038
#19 que_run_threads_low (thr=0x1e) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/que/que0que.cc:1100
#20 que_run_threads (thr=thr@entry=0x556cafc2de28) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/que/que0que.cc:1140
#21 0x0000556cadbca9dc in srv_task_execute () at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/srv/srv0srv.cc:2488
#22 srv_worker_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/srv/srv0srv.cc:2536
#23 0x00007f0f102586ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#24 0x00007f0f0f90382d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 8 (Thread 0x7f0f0806c700 (LWP 4443)):
#0 0x00007f0f1025e360 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x0000556cadb38f16 in os_event::wait (this=0x556cafb7ef10) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:158
#2 os_event::wait_low (reset_sig_count=1, this=0x556cafb7ef10) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:325
#3 os_event_wait_low (event=0x556cafb7ef10, reset_sig_count=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:502
#4 0x0000556cadbd372c in sync_array_wait_event (arr=arr@entry=0x556cafa27f90, cell=@0x7f0f08069e68: 0x556cafa28110) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/sync/sync0arr.cc:471
#5 0x0000556cadbd677b in rw_lock_x_lock_func (lock=lock@entry=0x556cafb7eeb0, pass=pass@entry=0, file_name=file_name@entry=0x556cadfdc4a0 "/home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/btr/btr0cur.cc", line=line@entry=4158) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/sync/sync0rw.cc:739
#6 0x0000556cadc2c275 in pfs_rw_lock_x_lock_func (lock=lock@entry=0x556cafb7eeb0, file_name=file_name@entry=0x556cadfdc4a0 "/home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/btr/btr0cur.cc", line=line@entry=4158, pass=0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/include/sync0rw.ic:561
#7 0x0000556cadc30f0f in btr_cur_update_in_place (flags=flags@entry=2, cursor=cursor@entry=0x7f0ea40629c0, offsets=0x7f0f0806a1f0, update=update@entry=0x7f0ea407d688, cmpl_info=cmpl_info@entry=1, thr=thr@entry=0x7f0ea40c1798, trx_id=373, mtr=0x7f0f0806acb0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/btr/btr0cur.cc:4158
#8 0x0000556cadc31284 in btr_cur_optimistic_update (flags=flags@entry=2, cursor=cursor@entry=0x7f0ea40629c0, offsets=offsets@entry=0x7f0f0806a0f8, heap=heap@entry=0x7f0f0806a1b0, update=0x7f0ea407d688, cmpl_info=1, thr=0x7f0ea40c1798, trx_id=373, mtr=0x7f0f0806acb0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/btr/btr0cur.cc:4342
#9 0x0000556cadbb4851 in row_upd_clust_rec (flags=flags@entry=0, node=node@entry=0x7f0ea407d568, index=index@entry=0x7f0ea4096dd8, offsets=offsets@entry=0x7f0f0806a1f0, offsets_heap=offsets_heap@entry=0x7f0f0806a1b0, thr=thr@entry=0x7f0ea40c1798, mtr=0x7f0f0806acb0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0upd.cc:2895
#10 0x0000556cadbb8f1b in row_upd_clust_step (node=node@entry=0x7f0ea407d568, thr=thr@entry=0x7f0ea40c1798) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0upd.cc:3228
#11 0x0000556cadbbb5c4 in row_upd (thr=0x7f0ea40c1798, node=0x7f0ea407d568) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0upd.cc:3331
#12 row_upd_step (thr=thr@entry=0x7f0ea40c1798) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0upd.cc:3475
#13 0x0000556cadb8cf74 in row_update_for_mysql (prebuilt=0x7f0ea407ca18) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0mysql.cc:1847
#14 0x0000556cadacd2be in ha_innobase::update_row (this=0x7f0ea403f4d0, old_row=0x7f0ea40963b8 "\370e", new_row=0x7f0ea4096398 "\370e") at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/handler/ha_innodb.cc:8815
#15 0x0000556cad884332 in handler::ha_update_row (this=0x7f0ea403f4d0, old_data=old_data@entry=0x7f0ea40963b8 "\370e", new_data=new_data@entry=0x7f0ea4096398 "\370e") at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/handler.cc:6508
#16 0x0000556cadd9aa8c in ha_partition::update_row (this=0x7f0ea4070a30, old_data=0x7f0ea40963b8 "\370e", new_data=0x7f0ea4096398 "\370e") at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/ha_partition.cc:4430
#17 0x0000556cad8842b5 in handler::ha_update_row (this=0x7f0ea4070a30, old_data=0x7f0ea40963b8 "\370e", new_data=0x7f0ea4096398 "\370e") at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/handler.cc:6508
#18 0x0000556cad96aa3c in Update_rows_log_event::do_exec_row (this=0x7f0ea4096a08, rgi=0x7f0ea40008c0) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/log_event.cc:14715
#19 0x0000556cad95d9ef in Rows_log_event::do_apply_event (this=0x7f0ea4096a08, rgi=0x7f0ea40008c0) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/log_event.cc:11615
#20 0x0000556cad5f25d2 in Log_event::apply_event (rgi=0x7f0ea40008c0, this=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/log_event.h:1484
#21 apply_event_and_update_pos_apply (ev=ev@entry=0x7f0ea4096a08, thd=thd@entry=0x7f0ea4000fb8, rgi=rgi@entry=0x7f0ea40008c0, reason=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/slave.cc:3786
#22 0x0000556cad5f9936 in apply_event_and_update_pos (ev=ev@entry=0x7f0ea4096a08, thd=thd@entry=0x7f0ea4000fb8, rgi=rgi@entry=0x7f0ea40008c0) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/slave.cc:3954
#23 0x0000556cad5fb297 in exec_relay_log_event (serial_rgi=0x7f0ea40008c0, rli=0x556cafce2258, thd=0x7f0ea4000fb8) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/slave.cc:4250
#24 handle_slave_sql (arg=arg@entry=0x556cafce0890) at /home/buildbot/buildbot/build/mariadb-10.3.29/sql/slave.cc:5426
#25 0x0000556cada0f3f1 in pfs_spawn_thread (arg=0x7f0eb8082148) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/perfschema/pfs.cc:1869
#26 0x00007f0f102586ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#27 0x00007f0f0f90382d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 7 (Thread 0x7f0ee37fe700 (LWP 4187)):
#0 0x00007f0f1025e360 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x0000556cadb38f16 in os_event::wait (this=0x556cafb58ac0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:158
#2 os_event::wait_low (reset_sig_count=1, this=0x556cafb58ac0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:325
#3 os_event_wait_low (event=0x556cafb58ac0, reset_sig_count=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/os/os0event.cc:502
#4 0x0000556cadbd372c in sync_array_wait_event (arr=arr@entry=0x556cafa27f90, cell=@0x7f0ee37fc018: 0x556cafa280d0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/sync/sync0arr.cc:471
#5 0x0000556cadbd677b in rw_lock_x_lock_func (lock=<optimized out>, pass=pass@entry=0, file_name=file_name@entry=0x556cadfcfbc8 "/home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0row.cc", line=line@entry=1045) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/sync/sync0rw.cc:739
#6 0x0000556cadc4d67c in pfs_rw_lock_x_lock_func (lock=lock@entry=0x7f0f037f78f8, file_name=file_name@entry=0x556cadfcfbc8 "/home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0row.cc", line=line@entry=1045, pass=0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/include/sync0rw.ic:567
#7 0x0000556cadc4df55 in buf_page_mtr_lock (block=block@entry=0x7f0f037f7868, rw_latch=rw_latch@entry=2, mtr=mtr@entry=0x7f0ee37fd500, file=file@entry=0x556cadfcfbc8 "/home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0row.cc", line=line@entry=1045) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/buf/buf0buf.cc:4181
#8 0x0000556cadc53b40 in buf_page_get_low (page_id=..., page_size=..., rw_latch=rw_latch@entry=2, guess=<optimized out>, mode=10, file=0x556cadfcfbc8 "/home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0row.cc", line=1045, mtr=0x7f0ee37fd500, err=0x7f0ee37fc5bc) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/buf/buf0buf.cc:4854
#9 0x0000556cadc53c82 in buf_page_get_gen (page_id=..., page_id@entry=..., page_size=..., rw_latch=rw_latch@entry=2, guess=guess@entry=0x0, mode=mode@entry=10, file=file@entry=0x556cadfcfbc8 "/home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0row.cc", line=1045, mtr=0x7f0ee37fd500, err=0x7f0ee37fc5bc) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/buf/buf0buf.cc:4902
#10 0x0000556cadc35b5f in btr_cur_search_to_nth_level_func (index=index@entry=0x7f0ea4096dd8, level=level@entry=0, tuple=tuple@entry=0x7f0edc0a60e0, mode=mode@entry=PAGE_CUR_LE, latch_mode=latch_mode@entry=2, cursor=cursor@entry=0x556cafc246b0, ahi_latch=0x0, file=0x556cadfcfbc8 "/home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0row.cc", line=1045, mtr=0x7f0ee37fd500, autoinc=0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/btr/btr0cur.cc:1490
#11 0x0000556cadba15f2 in btr_pcur_open_low (index=index@entry=0x7f0ea4096dd8, tuple=tuple@entry=0x7f0edc0a60e0, latch_mode=latch_mode@entry=2, cursor=cursor@entry=0x556cafc246b0, line=line@entry=1045, mtr=mtr@entry=0x7f0ee37fd500, autoinc=0, file=0x556cadfcfbc8 "/home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0row.cc", mode=PAGE_CUR_LE, level=0) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/include/btr0pcur.ic:446
#12 0x0000556cadba18c5 in row_search_on_row_ref (pcur=pcur@entry=0x556cafc246b0, mode=mode@entry=2, table=<optimized out>, ref=0x7f0edc0a60e0, mtr=mtr@entry=0x7f0ee37fd500) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0row.cc:1045
#13 0x0000556cadb9b902 in row_purge_reposition_pcur (mtr=0x7f0ee37fd500, node=0x556cafc24608, mode=2) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0purge.cc:78
#14 row_purge_reset_trx_id (node=node@entry=0x556cafc24608, mtr=mtr@entry=0x7f0ee37fd500) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0purge.cc:795
#15 0x0000556cadb9e4c2 in row_purge_record_func (node=node@entry=0x556cafc24608, undo_rec=undo_rec@entry=0x7f0edc0a4668 "", thr=thr@entry=0x556cafc24550, updated_extern=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0purge.cc:1208
#16 0x0000556cadb9ed21 in row_purge (thr=0x556cafc24550, undo_rec=0x7f0edc0a4668 "", node=0x556cafc24608) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0purge.cc:1257
#17 row_purge_step (thr=0x556cafc24550) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/row/row0purge.cc:1316
#18 0x0000556cadb5942a in que_thr_step (thr=0x1e) at /home/buildbot/buildbot/build/mariadb-10.3.29/storage/innobase/que/que0que.cc:1038
#19 que_run_threads_low (thr=0x1e) at /home/buildbot/buildbo

Attachments

Issue Links

is duplicated by

MDEV-26383 SUMMARY: AddressSanitizer: heap-use-after-free storage/innobase/btr/btr0sea.cc:209 in btr_search_lazy_free

Closed

relates to

MDEV-23905 Failed test: binlog_encryption.encrypted_master, crash

Open

MDEV-25465 innodb_zip.create_options ' fails in bb, assertion: list.count > 0 in ut0lst.h line 334

Closed

Activity

People

Assignee:: Thirunarayanan Balathandayuthapani

Reporter:: Alice Sherepa

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 2021-03-05 12:52

Updated:: 2022-01-29 18:17

Resolved:: 2022-01-29 18:16

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.