[MDEV-16544] signal 11 while importing sphinx table Created: 2018-06-21  Updated: 2018-08-14  Resolved: 2018-08-14

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - SphinxSE
Affects Version/s: 10.1, 10.3
Fix Version/s: 10.1.36, 10.2.18, 10.3.10

Type: Bug Priority: Major
Reporter: Petr Hanousek Assignee: Vladislav Vaintroub
Resolution: Fixed Votes: 0
Labels: None
Environment:

Debian 9.4, MariaDB 10.3.7

Issue Links:
Duplicate
is duplicated by MDEV-16819 Server crash on ALTER/CREATE table (S... Closed

 Description   

Hello, I encounter error 11 while importing some 15gb sql dump. Error log says:

Version: '10.3.7-MariaDB-1:10.3.7+maria~stretch'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distributi
 
on
 
180621 11:50:59 [ERROR] mysqld got signal 11 ;
 
This could be because you hit a bug. It is also possible that this binary
 
or one of the libraries it was linked against is corrupt, improperly built,
 
or misconfigured. This error can also be caused by malfunctioning hardware.
 
 
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
 
 
We will try our best to scrape up some info that will hopefully help
 
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
 
 
 
Server version: 10.3.7-MariaDB-1:10.3.7+maria~stretch
 
key_buffer_size=16777216
 
read_buffer_size=131072
 
max_used_connections=1
 
max_threads=153
 
thread_count=7
 
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 352701 K  bytes of memory
 
Hope that's ok; if not, decrease some variables in the equation.
 
 
 
Thread pointer: 0x7f89300009a8
 
Attempting backtrace. You can use the following information to find out
 
where mysqld died. If you see no messages after this, something went
 
terribly wrong...
 
stack_bottom = 0x7f89640fdcf8 thread_stack 0x30000
 
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x5647676adc5e]
 
/usr/sbin/mysqld(handle_fatal_signal+0x41f)[0x56476715eabf]
 
/lib/x86_64-linux-gnu/libpthread.so.0(+0x110c0)[0x7f896a6570c0]
 
/usr/lib/mysql/plugin/ha_sphinx.so(_ZN9ha_sphinx6createEPKcP5TABLEP14HA_CREATE_INFO+0x167)[0x7f8964657167]
 
/usr/sbin/mysqld(_ZN7handler9ha_createEPKcP5TABLEP14HA_CREATE_INFO+0x38)[0x564767166cb8]
 
/usr/sbin/mysqld(_Z15ha_create_tableP3THDPKcS2_S2_P14HA_CREATE_INFOP34st_mysql_const_unsigned_lex_string+0x1e1)[0x564767167511]
 
/usr/sbin/mysqld(_Z16rea_create_tableP3THDP34st_mysql_const_unsigned_lex_stringPKcS4_S4_P14HA_CREATE_INFOP7handlerb+0x9c)[0x564767
 
04366c]
 
/usr/sbin/mysqld(+0x5fe490)[0x56476700f490]
 
/usr/sbin/mysqld(_Z26mysql_create_table_no_lockP3THDPK25st_mysql_const_lex_stringS3_P22Table_specification_stP10Alter_infoPbiP10TA
 
BLE_LIST+0xfb)[0x56476700fa4b]
 
/usr/sbin/mysqld(_Z18mysql_create_tableP3THDP10TABLE_LISTP22Table_specification_stP10Alter_info+0xeb)[0x56476700fc0b]
 
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x84f9)[0x564766f8e769]
 
/usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_statebb+0x21a)[0x564766f8ee5a]
 
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjbb+0x1c81)[0x564766f917f1]
 
/usr/sbin/mysqld(_Z10do_commandP3THD+0x131)[0x564766f924a1]
 
/usr/sbin/mysqld(_Z24do_handle_one_connectionP7CONNECT+0x25a)[0x56476705f81a]
 
/usr/sbin/mysqld(handle_one_connection+0x3d)[0x56476705f99d]
 
nptl/pthread_create.c:333(start_thread)[0x7f896a64d494]
 
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7f8968ba7acf]
 
 
 
Trying to get some variables.
 
Some pointers may be invalid and cause the dump to abort.
 
Query (0x7f8930011610): CREATE TABLE `AttachmentsIndex` (   `id` bigint(20) NOT NULL,   `weight` int(11) NOT NULL,   `query` varch
 
ar(3072) NOT NULL,   KEY `query` (`query`(1024)) ) ENGINE=SPHINX DEFAULT CHARSET=utf8 CONNECTION='sphinx://127.0.0.1:3312/rt'
 
Connection ID (thread ID): 9
 
Status: NOT_KILLED
 
 
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on
 
 
 
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
 
information that should help you find out what is causing the crash.
 
2018-06-21 11:51:05 0 [Note] Using unique option prefix 'myisam-recover' is error-prone and can break in the future. Please use the full name 'myisam-recover-options' instead.
 
2018-06-21 11:51:05 0 [Warning] Plugin 'SPHINX' is of maturity level gamma while the server is stable
 
2018-06-21 11:51:05 0 [Note] InnoDB: Using Linux native AIO
 
2018-06-21 11:51:05 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
 
2018-06-21 11:51:05 0 [Note] InnoDB: Uses event mutexes
 
2018-06-21 11:51:05 0 [Note] InnoDB: Compressed tables use zlib 1.2.8
 
2018-06-21 11:51:05 0 [Note] InnoDB: Number of pools: 1
 
2018-06-21 11:51:05 0 [Note] InnoDB: Using SSE2 crc32 instructions
 
2018-06-21 11:51:05 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
 
2018-06-21 11:51:05 0 [Note] InnoDB: Completed initialization of buffer pool
 
2018-06-21 11:51:05 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
 
2018-06-21 11:51:05 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=103958220381
 
2018-06-21 11:51:06 0 [Note] InnoDB: Recovered page [page id: space=109, page number=658791] from the doublewrite buffer.
 
.....
 
Trying to get some variables.
 
Some pointers may be invalid and cause the dump to abort.
 
Query (0x7f8930011610): CREATE TABLE `AttachmentsIndex` (   `id` bigint(20) NOT NULL,   `weight` int(11) NOT NULL,   `query` varch
 
ar(3072) NOT NULL,   KEY `query` (`query`(1024)) ) ENGINE=SPHINX DEFAULT CHARSET=utf8 CONNECTION='sphinx://127.0.0.1:3312/rt'
 
Connection ID (thread ID): 9
 
Status: NOT_KILLED
 
 
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on
 
 
 
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
 
information that should help you find out what is causing the crash.
 
2018-06-21 11:51:05 0 [Note] Using unique option prefix 'myisam-recover' is error-prone and can break in the future. Please use the full name 'myisam-recover-options' instead.
 
2018-06-21 11:51:05 0 [Warning] Plugin 'SPHINX' is of maturity level gamma while the server is stable
 
2018-06-21 11:51:05 0 [Note] InnoDB: Using Linux native AIO
 
2018-06-21 11:51:05 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
 
2018-06-21 11:51:05 0 [Note] InnoDB: Uses event mutexes
 
2018-06-21 11:51:05 0 [Note] InnoDB: Compressed tables use zlib 1.2.8
 
2018-06-21 11:51:05 0 [Note] InnoDB: Number of pools: 1
 
2018-06-21 11:51:05 0 [Note] InnoDB: Using SSE2 crc32 instructions
 
2018-06-21 11:51:05 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
 
2018-06-21 11:51:05 0 [Note] InnoDB: Completed initialization of buffer pool
 
2018-06-21 11:51:05 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
 
2018-06-21 11:51:05 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=103958220381
 
2018-06-21 11:51:06 0 [Note] InnoDB: Recovered page [page id: space=109, page number=658791] from the doublewrite buffer.

Should I provide more information or is this OK for some debug?
Thank you, Petr

 Comments   
Comment by Elena Stepanova [ 2018-07-18 ]

Thanks for the report.

MTR version

 
INSTALL SONAME 'ha_sphinx';
 
eval CREATE TABLE t (id BIGINT UNSIGNED NOT NULL, weight INT NOT NULL, query VARCHAR(3072) NOT NULL, INDEX(query)) ENGINE=SPHINX CONNECTION='sphinx://127.0.0.1:$MASTER_MYPORT/rt';

Note: for non-MTR version, remove eval and replace $MASTER_MYPORT.

10.3 141a5b2484

 
#3  <signal handler called>
 
#4  0x00007efd1d8012b7 in ha_sphinx::create (this=0x7efd00145788, name=0x7efd1da5e1d0 "./test/t", table_arg=0x7efd1da5b8a0) at /data/src/10.3/storage/sphinx/ha_sphinx.cc:3444
 
#5  0x00005559accbeb3c in handler::ha_create (this=0x7efd00145788, name=0x7efd1da5e1d0 "./test/t", form=0x7efd1da5b8a0, info_arg=0x7efd1da5e7b0) at /data/src/10.3/sql/handler.cc:4554
 
#6  0x00005559accbfbb4 in ha_create_table (thd=0x7efd00000b00, path=0x7efd1da5e1d0 "./test/t", db=0x7efd00015548 "test", table_name=0x7efd00014ec8 "t", create_info=0x7efd1da5e7b0, frm=0x7efd1da5e1c0) at /data/src/10.3/sql/handler.cc:4927
 
#7  0x00005559acaf5486 in rea_create_table (thd=0x7efd00000b00, frm=0x7efd1da5e1c0, path=0x7efd1da5e1d0 "./test/t", db=0x7efd00015548 "test", table_name=0x7efd00014ec8 "t", create_info=0x7efd1da5e7b0, file=0x7efd00015fb0, no_ha_create_table=false) at /data/src/10.3/sql/unireg.cc:514
 
#8  0x00005559aca980ac in create_table_impl (thd=0x7efd00000b00, orig_db=0x7efd00014f18, orig_table_name=0x7efd00014f28, db=0x7efd00014f18, table_name=0x7efd00014f28, path=0x7efd1da5e1d0 "./test/t", options=..., create_info=0x7efd1da5e7b0, alter_info=0x7efd1da5e6f0, create_table_mode=0, is_trans=0x7efd1da5e42e, key_info=0x7efd1da5e1b0, key_count=0x7efd1da5e1a4, frm=0x7efd1da5e1c0) at /data/src/10.3/sql/sql_table.cc:4972
 
#9  0x00005559aca986c5 in mysql_create_table_no_lock (thd=0x7efd00000b00, db=0x7efd00014f18, table_name=0x7efd00014f28, create_info=0x7efd1da5e7b0, alter_info=0x7efd1da5e6f0, is_trans=0x7efd1da5e42e, create_table_mode=0, table_list=0x7efd00014f00) at /data/src/10.3/sql/sql_table.cc:5093
 
#10 0x00005559aca98a64 in mysql_create_table (thd=0x7efd00000b00, create_table=0x7efd00014f00, create_info=0x7efd1da5e7b0, alter_info=0x7efd1da5e6f0) at /data/src/10.3/sql/sql_table.cc:5186
 
#11 0x00005559ac9c80a0 in mysql_execute_command (thd=0x7efd00000b00) at /data/src/10.3/sql/sql_parse.cc:4275
 
#12 0x00005559ac9d42eb in mysql_parse (thd=0x7efd00000b00, rawbuf=0x7efd00014d08 "CREATE TABLE t (id BIGINT UNSIGNED NOT NULL, weight INT NOT NULL, query VARCHAR(3072) NOT NULL, INDEX(query)) ENGINE=SPHINX CONNECTION='sphinx://127.0.0.1:16000/rt'", length=164, parser_state=0x7efd1da5f5f0, is_com_multi=false, is_next_command=false) at /data/src/10.3/sql/sql_parse.cc:8073
 
#13 0x00005559ac9c164a in dispatch_command (command=COM_QUERY, thd=0x7efd00000b00, packet=0x7efd0000b211 "CREATE TABLE t (id BIGINT UNSIGNED NOT NULL, weight INT NOT NULL, query VARCHAR(3072) NOT NULL, INDEX(query)) ENGINE=SPHINX CONNECTION='sphinx://127.0.0.1:16000/rt'", packet_length=164, is_com_multi=false, is_next_command=false) at /data/src/10.3/sql/sql_parse.cc:1847
 
#14 0x00005559ac9c006e in do_command (thd=0x7efd00000b00) at /data/src/10.3/sql/sql_parse.cc:1392
 
#15 0x00005559acb263a5 in do_handle_one_connection (connect=0x5559af6c7c10) at /data/src/10.3/sql/sql_connect.cc:1402
 
#16 0x00005559acb26129 in handle_one_connection (arg=0x5559af6c7c10) at /data/src/10.3/sql/sql_connect.cc:1308
 
#17 0x00005559acfb7b27 in pfs_spawn_thread (arg=0x5559af6eb000) at /data/src/10.3/storage/perfschema/pfs.cc:1862
 
#18 0x00007efd255d9494 in start_thread (arg=0x7efd1da60700) at pthread_create.c:333
 
#19 0x00007efd239bf93f in clone () from /lib/x86_64-linux-gnu/libc.so.6

Comment by Elena Stepanova [ 2018-08-04 ]

In 10.3, the crash started happening after this merge:

commit 2c1067166d7e8a9541578220b408f1e553e23916
 
Merge: 2cf3e2e 61b2618
 
Author: Marko Mäkelä <marko.makela@mariadb.com>
 
Date:   Wed Oct 4 08:24:06 2017 +0300
 
 
 
    Merge bb-10.2-ext into 10.3

And in bb-10.2-ext apparently after this merge

commit 8ae8cd63485eb063de0b70ea6f3acf7102a61fef
 
Merge: 6857cb5 387bdf0
 
Author: Alexander Barkov <bar@mariadb.org>
 
Date:   Mon Oct 2 22:35:13 2017 +0400
 
 
 
    Merge remote-tracking branch 'origin/10.2' into bb-10.2-ext

.

but it doesn't crash on 10.2.

Generated at Thu Feb 08 08:29:42 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.