Details
Description
http://buildbot.askmonty.org/buildbot/builders/kvm-rpm-centos74-amd64-debug/builds/3098
http://buildbot.askmonty.org/buildbot/builders/kvm-bintar-centos6-amd64-debug/builds/2351 (earlier occurrence with a decent stack trace)
bb-10.2-release f8b5e147 |
innodb_fts.sync_ddl 'innodb' w4 [ fail ]
|
Test ended at 2019-12-03 18:04:32
|
|
CURRENT_TEST: innodb_fts.sync_ddl
|
mysqltest: At line 148: query 'SET GLOBAL debug_dbug = @save_debug' failed: 2013: Lost connection to MySQL server during query
|
|
The result from queries just before the failure was:
|
< snip >
|
DROP INDEX idx1,
|
ALGORITHM=INPLACE;
|
DROP TABLE t1;
|
SET GLOBAL debug_dbug = @save_debug;
|
CREATE TABLE t1 (
|
value VARCHAR(1024)
|
) ENGINE=InnoDB;
|
CREATE FULLTEXT INDEX idx1 ON t1(value);
|
Warnings:
|
Warning 124 InnoDB rebuilding table to add column FTS_DOC_ID
|
SET GLOBAL debug_dbug = '+d,fts_instrument_sync_request,fts_instrument_msg_sync_sleep';
|
INSERT INTO t1 (value) VALUES
|
('example, a column that uses the latin1 character'),
|
('collation of latin1_bin to make it case sensitive')
|
;
|
ALTER TABLE t1
|
DROP INDEX idx1,
|
ALGORITHM=COPY;
|
DROP TABLE t1;
|
SET GLOBAL debug_dbug = @save_debug;
|
|
More results from queries before failure can be found in /usr/local/mariadb-10.2.30-linux-x86_64/mysql-test/var/4/log/sync_ddl.log
|
|
|
Server [mysqld.1 - pid: 15783, winpid: 15783, exit: 256] failed during test run
|
Server log from this test:
|
----------SERVER LOG START-----------
|
?func: THD::enter_stage: /home/buildbot/buildbot/build/storage/innobase/handler/ha_innodb.cc:1707
|
191203 18:04:31 [ERROR] mysqld got signal 11 ;
|
This could be because you hit a bug. It is also possible that this binary
|
or one of the libraries it was linked against is corrupt, improperly built,
|
or misconfigured. This error can also be caused by malfunctioning hardware.
|
|
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
|
|
We will try our best to scrape up some info that will hopefully help
|
diagnose the problem, but since we have already crashed,
|
something is definitely wrong and this may fail.
|
|
Server version: 10.2.30-MariaDB-debug-log
|
key_buffer_size=1048576
|
read_buffer_size=131072
|
max_used_connections=3
|
max_threads=153
|
thread_count=9
|
It is possible that mysqld could use up to
|
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 63104 K bytes of memory
|
Hope that's ok; if not, decrease some variables in the equation.
|
|
Thread pointer: 0x7f46ec000af0
|
Attempting backtrace. You can use the following information to find out
|
where mysqld died. If you see no messages after this, something went
|
terribly wrong...
|
stack_bottom = 0x7f46ff5fdd98 thread_stack 0x49000
|
/usr/local/mariadb-10.2.30-linux-x86_64/bin/mysqld(my_print_stacktrace+0x2b)[0x7f471d620679]
|
mysys/stacktrace.c:267(my_print_stacktrace)[0x7f471cfa8b3d]
|
sigaction.c:0(__restore_rt)[0x7f471c4634a0]
|
fnmatch.c:0(internal_fnmatch)[0x7f471b3a6133]
|
/lib64/libc.so.6(fnmatch+0x70)[0x7f471b3aa770]
|
/usr/local/mariadb-10.2.30-linux-x86_64/bin/mysqld(+0xda5ac2)[0x7f471d637ac2]
|
dbug/dbug.c:1540(InList)[0x7f471d638494]
|
dbug/dbug.c:1764(_db_keyword_)[0x7f471d6388d3]
|
dbug/dbug.c:1264(_db_pargs_)[0x7f471d24f434]
|
que/que0que.cc:980(que_thr_step)[0x7f471d32712b]
|
include/my_atomic.h:274(my_atomic_add64)[0x7f471d2fc2e6]
|
pthread_create.c:0(start_thread)[0x7f471c45b7f1]
|
/lib64/libc.so.6(clone+0x6d)[0x7f471b3dbccd]
|
|
Trying to get some variables.
|
Some pointers may be invalid and cause the dump to abort.
|
Query (0x0):
|
Connection ID (thread ID): 4
|
Status: NOT_KILLED
|
|
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on
|
|
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
|
information that should help you find out what is causing the crash.
|
Writing a core file...
|
Working directory at /dev/shm/var_auto_O8r_/4/mysqld.1/data
|
Resource Limits:
|
Limit Soft Limit Hard Limit Units
|
Max cpu time unlimited unlimited seconds
|
Max file size unlimited unlimited bytes
|
Max data size unlimited unlimited bytes
|
Max stack size 10485760 unlimited bytes
|
Max core file size unlimited unlimited bytes
|
Max resident set unlimited unlimited bytes
|
Max processes 1024 unlimited processes
|
Max open files 1024 1024 files
|
Max locked memory 65536 65536 bytes
|
Max address space unlimited unlimited bytes
|
Max file locks unlimited unlimited locks
|
Max pending signals 46692 46692 signals
|
Max msgqueue size 819200 819200 bytes
|
Max nice priority 0 0
|
Max realtime priority 0 0
|
Max realtime timeout unlimited unlimited us
|
Core pattern: core
|
|
----------SERVER LOG END-------------
|
#3 <signal handler called>
|
#4 0x00007f471b3a6133 in internal_fnmatch () from /lib64/libc.so.6
|
#5 0x00007f471b3aa770 in fnmatch@@GLIBC_2.2.5 () from /lib64/libc.so.6
|
#6 0x00007f471d637ac2 in InList (linkp=0x8f8f8f8f8f8f8f8f, cp=0x7f471d84196a "ib_que", exact_match=0) at /home/buildbot/buildbot/build/dbug/dbug.c:1540
|
#7 0x00007f471d638494 in _db_keyword_ (cs=0x7f46ec000a30, keyword=0x7f471d84196a "ib_que", strict=0) at /home/buildbot/buildbot/build/dbug/dbug.c:1764
|
#8 0x00007f471d6388d3 in _db_pargs_ (_line_=982, keyword=0x7f471d84196a "ib_que") at /home/buildbot/buildbot/build/dbug/dbug.c:1264
|
#9 0x00007f471d24f434 in que_thr_step (thr=0x7f471f671a08) at /home/buildbot/buildbot/build/storage/innobase/que/que0que.cc:980
|
#10 que_run_threads_low (thr=0x7f471f671a08) at /home/buildbot/buildbot/build/storage/innobase/que/que0que.cc:1102
|
#11 que_run_threads (thr=0x7f471f671a08) at /home/buildbot/buildbot/build/storage/innobase/que/que0que.cc:1142
|
#12 0x00007f471d32712b in trx_purge (n_purge_threads=4, batch_size=300, truncate=false, slot=0x7f471e06a428) at /home/buildbot/buildbot/build/storage/innobase/trx/trx0purge.cc:1739
|
#13 0x00007f471d2fc2e6 in srv_do_purge (arg=Unhandled dwarf expression opcode 0xf3
|
) at /home/buildbot/buildbot/build/storage/innobase/srv/srv0srv.cc:2735
|
#14 srv_purge_coordinator_thread (arg=Unhandled dwarf expression opcode 0xf3
|
) at /home/buildbot/buildbot/build/storage/innobase/srv/srv0srv.cc:2887
|
#15 0x00007f471c45b7f1 in start_thread () from /lib64/libpthread.so.0
|
#16 0x00007f471b3dbccd in clone () from /lib64/libc.so.6
|
innodb_fts.concurrent_insert: http://buildbot.askmonty.org/buildbot/builders/kvm-rpm-centos74-amd64-debug/builds/3102
One more occurrence. Now with ASAN http://buildbot.askmonty.org/buildbot/builders/kvm-asan/builds/2518/steps/mtr_nm/logs/stdio
10.2 3cc0e0befb3d6012a7ed9649a2220bf6cc6c614f
=================================================================
==25904==ERROR: AddressSanitizer: heap-use-after-free on address 0x6060000c5b48 at pc 0x000001e8518d bp 0x7f6a28e941c0 sp 0x7f6a28e941b0
READ of size 1 at 0x6060000c5b48 thread T34
#0 0x1e8518c in InList /home/buildbot/buildbot/build/mariadb-10.2.30/dbug/dbug.c:1545
#1 0x1e85e80 in _db_keyword_ /home/buildbot/buildbot/build/mariadb-10.2.30/dbug/dbug.c:1764
#2 0x1e83a80 in _db_pargs_ /home/buildbot/buildbot/build/mariadb-10.2.30/dbug/dbug.c:1264
#3 0x1e28b60 in my_write /home/buildbot/buildbot/build/mariadb-10.2.30/mysys/my_write.c:29
#4 0x1dcd97c in inline_mysql_file_write /home/buildbot/buildbot/build/mariadb-10.2.30/include/mysql/psi/mysql_file.h:1131
#5 0x1dd5bb7 in _my_b_cache_write /home/buildbot/buildbot/build/mariadb-10.2.30/mysys/mf_iocache.c:1749
#6 0x1dd6daa in my_b_flush_io_cache /home/buildbot/buildbot/build/mariadb-10.2.30/mysys/mf_iocache.c:1949
#7 0xb1df00 in flush_master_info(Master_info*, bool, bool) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/rpl_mi.cc:813
#8 0x67cd25 in handle_slave_io /home/buildbot/buildbot/build/mariadb-10.2.30/sql/slave.cc:4580
#9 0x130b4c1 in pfs_spawn_thread /home/buildbot/buildbot/build/mariadb-10.2.30/storage/perfschema/pfs.cc:1862
#10 0x7f6a40a396b9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76b9)
#11 0x7f6a3fece82c in clone (/lib/x86_64-linux-gnu/libc.so.6+0x10682c)
0x6060000c5b48 is located 8 bytes inside of 55-byte region [0x6060000c5b40,0x6060000c5b77)
freed by thread T33 here:
#0 0x7f6a422172ca in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x982ca)
#1 0x1e86101 in FreeList /home/buildbot/buildbot/build/mariadb-10.2.30/dbug/dbug.c:1830
#2 0x1e85480 in FreeState /home/buildbot/buildbot/build/mariadb-10.2.30/dbug/dbug.c:1620
#3 0x1e7d204 in DbugParse /home/buildbot/buildbot/build/mariadb-10.2.30/dbug/dbug.c:489
#4 0x1e7f9a0 in _db_set_init_ /home/buildbot/buildbot/build/mariadb-10.2.30/dbug/dbug.c:898
#5 0xb8cdae in Sys_var_dbug::global_update(THD*, set_var*) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/sys_vars.ic:923
#6 0x65a2f5 in sys_var::update(THD*, set_var*) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/set_var.cc:208
#7 0x65e725 in set_var::update(THD*) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/set_var.cc:837
#8 0x65dca2 in sql_set_variables(THD*, List<set_var_base>*, bool) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/set_var.cc:738
#9 0x810f38 in mysql_execute_command(THD*) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/sql_parse.cc:4591
#10 0x824cfa in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/sql_parse.cc:7740
#11 0x801361 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/sql_parse.cc:1831
#12 0x7fe457 in do_command(THD*) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/sql_parse.cc:1385
#13 0xb37622 in do_handle_one_connection(CONNECT*) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/sql_connect.cc:1336
#14 0xb37006 in handle_one_connection /home/buildbot/buildbot/build/mariadb-10.2.30/sql/sql_connect.cc:1241
#15 0x130b4c1 in pfs_spawn_thread /home/buildbot/buildbot/build/mariadb-10.2.30/storage/perfschema/pfs.cc:1862
#16 0x7f6a40a396b9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76b9)
previously allocated by thread T33 here:
#0 0x7f6a42217602 in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x98602)
#1 0x1e872cf in DbugMalloc /home/buildbot/buildbot/build/mariadb-10.2.30/dbug/dbug.c:2065
#2 0x1e84dea in ListAddDel /home/buildbot/buildbot/build/mariadb-10.2.30/dbug/dbug.c:1461
#3 0x1e7de96 in DbugParse /home/buildbot/buildbot/build/mariadb-10.2.30/dbug/dbug.c:563
#4 0x1e7f9a0 in _db_set_init_ /home/buildbot/buildbot/build/mariadb-10.2.30/dbug/dbug.c:898
#5 0xb8cdae in Sys_var_dbug::global_update(THD*, set_var*) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/sys_vars.ic:923
#6 0x65a2f5 in sys_var::update(THD*, set_var*) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/set_var.cc:208
#7 0x65e725 in set_var::update(THD*) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/set_var.cc:837
#8 0x65dca2 in sql_set_variables(THD*, List<set_var_base>*, bool) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/set_var.cc:738
#9 0x810f38 in mysql_execute_command(THD*) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/sql_parse.cc:4591
#10 0x824cfa in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/sql_parse.cc:7740
#11 0x801361 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/sql_parse.cc:1831
#12 0x7fe457 in do_command(THD*) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/sql_parse.cc:1385
#13 0xb37622 in do_handle_one_connection(CONNECT*) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/sql_connect.cc:1336
#14 0xb37006 in handle_one_connection /home/buildbot/buildbot/build/mariadb-10.2.30/sql/sql_connect.cc:1241
#15 0x130b4c1 in pfs_spawn_thread /home/buildbot/buildbot/build/mariadb-10.2.30/storage/perfschema/pfs.cc:1862
#16 0x7f6a40a396b9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76b9)
Thread T34 created by T33 here:
#0 0x7f6a421b5253 in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x36253)
#1 0x130b8ae in spawn_thread_v1 /home/buildbot/buildbot/build/mariadb-10.2.30/storage/perfschema/pfs.cc:1912
#2 0x66467a in inline_mysql_thread_create /home/buildbot/buildbot/build/mariadb-10.2.30/include/mysql/psi/mysql_thread.h:1239
#3 0x669467 in start_slave_thread(unsigned int, void* (*)(void*), st_mysql_mutex*, st_mysql_mutex*, st_mysql_cond*, unsigned int volatile*, unsigned long volatile*, Master_info*) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/slave.cc:934
#4 0x669da6 in start_slave_threads(THD*, bool, bool, Master_info*, char const*, char const*, int) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/slave.cc:1056
#5 0x88d374 in start_slave(THD*, Master_info*, bool) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/sql_repl.cc:3146
#6 0x80bf5b in mysql_execute_command(THD*) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/sql_parse.cc:3846
#7 0x824cfa in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/sql_parse.cc:7740
#8 0x801361 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/sql_parse.cc:1831
#9 0x7fe457 in do_command(THD*) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/sql_parse.cc:1385
#10 0xb37622 in do_handle_one_connection(CONNECT*) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/sql_connect.cc:1336
#11 0xb37006 in handle_one_connection /home/buildbot/buildbot/build/mariadb-10.2.30/sql/sql_connect.cc:1241
#12 0x130b4c1 in pfs_spawn_thread /home/buildbot/buildbot/build/mariadb-10.2.30/storage/perfschema/pfs.cc:1862
#13 0x7f6a40a396b9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76b9)
Thread T33 created by T0 here:
#0 0x7f6a421b5253 in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x36253)
#1 0x130b8ae in spawn_thread_v1 /home/buildbot/buildbot/build/mariadb-10.2.30/storage/perfschema/pfs.cc:1912
#2 0x6016d6 in inline_mysql_thread_create /home/buildbot/buildbot/build/mariadb-10.2.30/include/mysql/psi/mysql_thread.h:1239
#3 0x616455 in create_thread_to_handle_connection(CONNECT*) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/mysqld.cc:6506
#4 0x616b79 in create_new_thread /home/buildbot/buildbot/build/mariadb-10.2.30/sql/mysqld.cc:6576
#5 0x617bd9 in handle_connections_sockets() /home/buildbot/buildbot/build/mariadb-10.2.30/sql/mysqld.cc:6851
#6 0x615933 in mysqld_main(int, char**) /home/buildbot/buildbot/build/mariadb-10.2.30/sql/mysqld.cc:6125
#7 0x5ffa15 in main /home/buildbot/buildbot/build/mariadb-10.2.30/sql/main.cc:25
#8 0x7f6a3fde882f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
SUMMARY: AddressSanitizer: heap-use-after-free /home/buildbot/buildbot/build/mariadb-10.2.30/dbug/dbug.c:1545 InList
Shadow bytes around the buggy address:
0x0c0c80010b10: fa fa fa fa fd fd fd fd fd fd fd fa fa fa fa fa
0x0c0c80010b20: fd fd fd fd fd fd fd fa fa fa fa fa fd fd fd fd
0x0c0c80010b30: fd fd fd fa fa fa fa fa fd fd fd fd fd fd fd fd
0x0c0c80010b40: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa
0x0c0c80010b50: fd fd fd fd fd fd fd fd fa fa fa fa 00 00 00 00
=>0x0c0c80010b60: 00 00 00 00 fa fa fa fa fd[fd]fd fd fd fd fd fa
0x0c0c80010b70: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa
0x0c0c80010b80: fd fd fd fd fd fd fd fd fa fa fa fa fd fd fd fd
0x0c0c80010b90: fd fd fd fd fa fa fa fa fd fd fd fd fd fd fd fd
0x0c0c80010ba0: fa fa fa fa 00 00 00 00 00 00 00 00 fa fa fa fa
0x0c0c80010bb0: fd fd fd fd fd fd fd fd fa fa fa fa fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Heap right redzone: fb
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack partial redzone: f4
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe