[MDEV-13908] 10.1.27 Segfaulting Created: 2017-09-26  Updated: 2017-09-29  Resolved: 2017-09-27

Status: Closed
Project: MariaDB Server
Component/s: Galera
Affects Version/s: 10.1.27
Fix Version/s: 10.1.28

Type: Bug Priority: Blocker
Reporter: Logan V Assignee: Andrii Nikitin (Inactive)
Resolution: Fixed Votes: 3
Labels: None
Environment:

Ubuntu Xenial

Attachments: File cluster.cnf     File debian.cnf     File mariadb.cnf     File my.cnf     Text File temp.txt     File tokudb.cnf    
Issue Links:
Duplicate
duplicates MDEV-13910 after update not starting Closed
is duplicated by MDEV-13787 Crash in persistent stats wsrep_on (t... Closed

 Description   

CI gate jobs that were passing this morning with 10.1.26 are now failing as my mirrors pulled in 10.1.27 apt packages.

Log is attached or available at http://osa-ci.objects-us-dfw-1.cloud.lstn.net/454450/8/626/logs/openstack/aio1_galera_container-9fad60d2/mysql_logs/galera_server_error.log.txt (navigate up in the URI path to view the rest of the contents of /var/log in this container such as apt/* for package versions). ie. http://osa-ci.objects-us-dfw-1.cloud.lstn.net/454450/8/626/logs/openstack/aio1_galera_container-9fad60d2/apt/history.log.txt

 Comments   
Comment by Elena Stepanova [ 2017-09-26 ]

Could you please paste the output of 

SHOW CREATE TABLE user;
 
SHOW INDEX IN user;

and attach your cnf file(s)?

Thanks.

Comment by FlorentCoppint [ 2017-09-26 ]

Same here, all upgraded 10.1.27 nodes are crashing after few seconds/minutes. 10.1.26 nodes are fine.

Comment by Antti-Jussi Korjonen [ 2017-09-26 ]

Hit the same issue on CentOS 6.9. Had to downgrade server to 10.1.26.

Comment by Logan V [ 2017-09-26 ] 

MariaDB [keystone]> SHOW CREATE TABLE user;
 
+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 
| Table | Create Table                                                                                                                                                                                                                                                                                                                   |
 
+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 
| user  | CREATE TABLE `user` (
 
  `id` varchar(64) NOT NULL,
 
  `name` varchar(255) NOT NULL,
 
  `extra` text,
 
  `password` varchar(128) DEFAULT NULL,
 
  `enabled` tinyint(1) DEFAULT NULL,
 
  `domain_id` varchar(64) NOT NULL,
 
  `default_project_id` varchar(64) DEFAULT NULL,
 
  PRIMARY KEY (`id`)
 
) ENGINE=InnoDB DEFAULT CHARSET=utf8 |
 
+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 
1 row in set (0.00 sec)


MariaDB [keystone]> SHOW INDEX IN user;
 
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
 
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
 
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
 
| user  |          0 | PRIMARY  |            1 | id          | A         |           0 |     NULL | NULL   |      | BTREE      |         |               |
 
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
 
1 row in set (0.00 sec)

Comment by Florian Strankowski [ 2017-09-26 ]

We've encountered the same? problem after updating to .27 today:

??Sep 26 13:22:59 MariaDB-PROD-002 mysqld: 170926 13:22:59 [ERROR] mysqld got signal 11 ;
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: This could be because you hit a bug. It is also possible that this binary
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: or one of the libraries it was linked against is corrupt, improperly built,
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: or misconfigured. This error can also be caused by malfunctioning hardware.
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: To report this bug, see https://mariadb.com/kb/en/reporting-bugs
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: We will try our best to scrape up some info that will hopefully help
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: diagnose the problem, but since we have already crashed,
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: something is definitely wrong and this may fail.
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: Server version: 10.1.27-MariaDB
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: key_buffer_size=134217728
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: read_buffer_size=131072
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: max_used_connections=0
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: max_threads=1002
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: thread_count=3
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: It is possible that mysqld could use up to
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 2332069 K bytes of memory
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: Hope that's ok; if not, decrease some variables in the equation.
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: Thread pointer: 0x0
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: Attempting backtrace. You can use the following information to find out
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: where mysqld died. If you see no messages after this, something went
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: terribly wrong...
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: stack_bottom = 0x0 thread_stack 0x48400
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x55de18f65a2e]
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: /usr/sbin/mysqld(handle_fatal_signal+0x305)[0x55de18a88d85]
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: /lib64/libpthread.so.0(+0xf5e0)[0x7f5ed2cfa5e0]
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: /usr/sbin/mysqld(wsrep_on+0x18)[0x55de18a24a08]
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: /usr/sbin/mysqld(+0x8df081)[0x55de18da8081]
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: mysys/stacktrace.c:268(my_print_stacktrace)[0x55de18da8db7]
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: sql/wsrep_mysqld.cc:2298(wsrep_on)[0x55de18d5f910]
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: que/que0que.cc:1277(que_eval_sql(pars_info_t*, char const*, unsigned long, trx_t*))[0x55de18d600e2]
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: dict/dict0stats.cc:332(dict_stats_exec_sql(pars_info_t*, char const*, trx_t*))[0x55de18e72f00]
Sep 26 13:22:59 MariaDB-PROD-002 mysqld: dict/dict0stats.cc:2573(dict_stats_save(dict_table_t*, unsigned long const*) [clone .part.72])[0x55de18e79c82]
Sep 26 13:23:00 MariaDB-PROD-002 mysqld: dict/dict0stats.cc:3340(dict_stats_update(dict_table_t*, dict_stats_upd_option_t))[0x55de18e7ad38]
Sep 26 13:23:00 MariaDB-PROD-002 mysqld: dict/dict0stats_bg.cc:464(dict_stats_thread)[0x55de18e7d143]
Sep 26 13:23:00 MariaDB-PROD-002 mysqld: /lib64/libpthread.so.0(+0x7e25)[0x7f5ed2cf2e25]
Sep 26 13:23:00 MariaDB-PROD-002 mysqld: /lib64/libc.so.6(clone+0x6d)[0x7f5ed109634d]
Sep 26 13:23:00 MariaDB-PROD-002 mysqld: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
Sep 26 13:23:00 MariaDB-PROD-002 mysqld: information that should help you find out what is causing the crash.
Sep 26 13:23:00 MariaDB-PROD-002 kernel: mysqld[7542]: segfault at 950 ip 000055de18a24a08 sp 00007f5e927f97d0 error 4 in mysqld[55de184c9000+f91000]
Sep 26 13:23:00 MariaDB-PROD-002 systemd: mariadb.service: main process exited, code=killed, status=11/SEGV
Sep 26 13:23:00 MariaDB-PROD-002 systemd: Unit mariadb.service entered failed state.
Sep 26 13:23:00 MariaDB-PROD-002 systemd: mariadb.service failed.
Sep 26 13:23:00 MariaDB-PROD-002 systemd: Stopped MariaDB database server.??

Comment by Andrii Nikitin (Inactive) [ 2017-09-26 ]

Whoever sees crash in 10.1.27 in dict/dict0stats.cc with wsrep_on on top of stack - please put innodb_stats_persistent=0 into [mysqld] section of configuration file and problem should gone after restart.

Comment by Florian Strankowski [ 2017-09-26 ]

This is most likely still a showstopper because disabling persitent stats can lead to a massive performance drop in querying over indexes without long-term statistics behind.

Comment by Andrii Nikitin (Inactive) [ 2017-09-26 ]

fstrankowski massive performance drop is possible only in very limited cases when dynamic statistics is inaccurate for some reasons (e.g. odd data distribution in index tree). I strongly believe that for absolute majority of cases dynamic stats will work smoothly.

Comment by Logan V [ 2017-09-26 ]

No difference with innodb_stats_persistent = 0

MariaDB [(none)]> show variables LIKE 'innodb_stats_%';
 
+--------------------------------------+-------------+
 
| Variable_name                        | Value       |
 
+--------------------------------------+-------------+
 
| innodb_stats_auto_recalc             | ON          |
 
| innodb_stats_include_delete_marked   | OFF         |
 
| innodb_stats_method                  | nulls_equal |
 
| innodb_stats_modified_counter        | 0           |
 
| innodb_stats_on_metadata             | OFF         |
 
| innodb_stats_persistent              | OFF         |
 
| innodb_stats_persistent_sample_pages | 20          |
 
| innodb_stats_sample_pages            | 8           |
 
| innodb_stats_traditional             | ON          |
 
| innodb_stats_transient_sample_pages  | 8           |
 
+--------------------------------------+-------------+
 
10 rows in set (0.00 sec)
 
 
 
MariaDB [(none)]> drop database keystone;
 
ERROR 2013 (HY000): Lost connection to MySQL server during query


2017-09-26 14:21:49 140356420069632 [Note] WSREP: Read nil XID from storage engines, skipping position init
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so'
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: wsrep_load(): Galera 25.3.20(r3703) by Codership Oy <info@codership.com> loaded successfully.
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: CRC-32C: using hardware acceleration.
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1, safe_to_bootsrap: 1
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: Passing config to GCS: base_dir = /var/lib/mysql/; base_host = 172.29.239.156; base_port = 4567; cert.log_conflicts = no; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.recover = no; gcache.size = 32M; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.checksum = false; pc.
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: GCache history reset: old(c48b6936-a2c5-11e7-a985-dec6ab3a99a7:0) -> new(00000000-0000-0000-0000-000000000000:-1)
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: wsrep_sst_grab()
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: Start replication
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: protonet asio version 0
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: Using CRC-32C for message checksums.
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: backend: asio
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: gcomm thread scheduling priority set to other:0
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: restore pc from disk successfully
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: GMCast version 0
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: (f66de576, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: (f66de576, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: EVS version 0
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: gcomm: connecting to group 'openstack_galera_cluster', peer ''
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: start_prim is enabled, turn off pc_recovery
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: Node f66de576 state prim
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: view(view_id(PRIM,f66de576,3) memb {
 
	f66de576,0
 
} joined {
 
} left {
 
} partitioned {
 
})
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: save pc into disk
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: clear restored view
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: gcomm: connected
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: Opened channel 'openstack_galera_cluster'
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: Waiting for SST to complete.
 
2017-09-26 14:21:49 140356238698240 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 1
 
2017-09-26 14:21:49 140356238698240 [Note] WSREP: Starting new group from scratch: 08ca3c9b-a2c6-11e7-ba35-e788efc1760b
 
2017-09-26 14:21:49 140356238698240 [Note] WSREP: STATE_EXCHANGE: sent state UUID: 08ca4c49-a2c6-11e7-b660-5f4dfe345a11
 
2017-09-26 14:21:49 140356238698240 [Note] WSREP: STATE EXCHANGE: sent state msg: 08ca4c49-a2c6-11e7-b660-5f4dfe345a11
 
2017-09-26 14:21:49 140356238698240 [Note] WSREP: STATE EXCHANGE: got state msg: 08ca4c49-a2c6-11e7-b660-5f4dfe345a11 from 0 (aio1_galera_container-d810e90f)
 
2017-09-26 14:21:49 140356238698240 [Note] WSREP: Quorum results:
 
	version    = 4,
 
	component  = PRIMARY,
 
	conf_id    = 0,
 
	members    = 1/1 (joined/total),
 
	act_id     = 0,
 
	last_appl. = -1,
 
	protocols  = 0/7/3 (gcs/repl/appl),
 
	group UUID = 08ca3c9b-a2c6-11e7-ba35-e788efc1760b
 
2017-09-26 14:21:49 140356238698240 [Note] WSREP: Flow-control interval: [16, 16]
 
2017-09-26 14:21:49 140356238698240 [Note] WSREP: Restored state OPEN -> JOINED (0)
 
2017-09-26 14:21:49 140356238698240 [Note] WSREP: Member 0.0 (aio1_galera_container-d810e90f) synced with group.
 
2017-09-26 14:21:49 140356238698240 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 0)
 
2017-09-26 14:21:49 140356419754752 [Note] WSREP: New cluster view: global state: 08ca3c9b-a2c6-11e7-ba35-e788efc1760b:0, view# 1: Primary, number of nodes: 1, my index: 0, protocol version 3
 
2017-09-26 14:21:49 140356420069632 [Note] WSREP: SST complete, seqno: 0
 
2017-09-26 14:21:49 140356420069632 [Note] InnoDB: Using mutexes to ref count buffer pool pages
 
2017-09-26 14:21:49 140356420069632 [Note] InnoDB: The InnoDB memory heap is disabled
 
2017-09-26 14:21:49 140356420069632 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
 
2017-09-26 14:21:49 140356420069632 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
 
2017-09-26 14:21:49 140356420069632 [Note] InnoDB: Compressed tables use zlib 1.2.8
 
2017-09-26 14:21:49 140356420069632 [Note] InnoDB: Using Linux native AIO
 
2017-09-26 14:21:49 140356420069632 [Note] InnoDB: Using SSE crc32 instructions
 
2017-09-26 14:21:49 140356420069632 [Note] InnoDB: Initializing buffer pool, size = 256.0M
 
2017-09-26 14:21:49 140356420069632 [Note] InnoDB: Completed initialization of buffer pool
 
2017-09-26 14:21:49 140356420069632 [Note] InnoDB: Highest supported file format is Barracuda.
 
2017-09-26 14:21:49 140356420069632 [Note] InnoDB: 128 rollback segment(s) are active.
 
2017-09-26 14:21:49 140356420069632 [Note] InnoDB: Waiting for purge to start
 
2017-09-26 14:21:49 140356420069632 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.36-82.2 started; log sequence number 1783460
 
2017-09-26 14:21:49 140355357898496 [Note] InnoDB: Dumping buffer pool(s) not yet started
 
2017-09-26 14:21:49 140356420069632 [Note] Plugin 'FEEDBACK' is disabled.
 
2017-09-26 14:21:49 140356420069632 [Note] Server socket created on IP: '::'.
 
2017-09-26 14:21:49 140356419754752 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
 
2017-09-26 14:21:49 140356420069632 [Note] /usr/sbin/mysqld: ready for connections.
 
Version: '10.1.27-MariaDB-1~xenial'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
 
2017-09-26 14:21:49 140356419754752 [Note] WSREP: REPL Protocols: 7 (3, 2)
 
2017-09-26 14:21:49 140356419754752 [Note] WSREP: Assign initial position for certification: 0, protocol version: 3
 
2017-09-26 14:21:49 140356296652544 [Note] WSREP: Service thread queue flushed.
 
2017-09-26 14:21:49 140356419754752 [Note] WSREP: GCache history reset: old(00000000-0000-0000-0000-000000000000:0) -> new(08ca3c9b-a2c6-11e7-ba35-e788efc1760b:0)
 
2017-09-26 14:21:49 140356419754752 [Note] WSREP: Synchronized with group, ready for connections
 
2017-09-26 14:21:49 140356419754752 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
 
2017-09-26 14:21:55 140356340321024 [Warning] IP address '172.29.236.100' could not be resolved: Name or service not known
 
170926 14:22:19 [ERROR] mysqld got signal 11 ;
 
This could be because you hit a bug. It is also possible that this binary
 
or one of the libraries it was linked against is corrupt, improperly built,
 
or misconfigured. This error can also be caused by malfunctioning hardware.
 
 
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
 
 
We will try our best to scrape up some info that will hopefully help
 
diagnose the problem, but since we have already crashed,
 
something is definitely wrong and this may fail.
 
 
 
Server version: 10.1.27-MariaDB-1~xenial
 
key_buffer_size=134217728
 
read_buffer_size=131072
 
max_used_connections=2
 
max_threads=802
 
thread_count=10
 
It is possible that mysqld could use up to
 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1892744 K  bytes of memory
 
Hope that's ok; if not, decrease some variables in the equation.
 
 
 
Thread pointer: 0x7fa70d74b008
 
Attempting backtrace. You can use the following information to find out
 
where mysqld died. If you see no messages after this, something went
 
terribly wrong...
 
stack_bottom = 0x7fa741d320b8 thread_stack 0x48400
 
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x55db5a6be4ae]
 
/usr/sbin/mysqld(handle_fatal_signal+0x2f5)[0x55db5a207025]
 
/lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7fa744f99390]
 
/usr/sbin/mysqld(wsrep_on+0x18)[0x55db5a1a0ea8]
 
/usr/sbin/mysqld(+0x8f2336)[0x55db5a559336]
 
/usr/sbin/mysqld(+0x8f329f)[0x55db5a55a29f]
 
/usr/sbin/mysqld(+0x8aa088)[0x55db5a511088]
 
/usr/sbin/mysqld(+0x8aa822)[0x55db5a511822]
 
/usr/sbin/mysqld(+0x9b603f)[0x55db5a61d03f]
 
/usr/sbin/mysqld(+0x8d3cba)[0x55db5a53acba]
 
/usr/sbin/mysqld(+0x817e67)[0x55db5a47ee67]
 
/usr/sbin/mysqld(_Z15ha_delete_tableP3THDP10handlertonPKcS4_S4_b+0x158)[0x55db5a20f6c8]
 
/usr/sbin/mysqld(_Z23mysql_rm_table_no_locksP3THDP10TABLE_LISTbbbbb+0x9f7)[0x55db5a0fbf57]
 
/usr/sbin/mysqld(+0x3e4e36)[0x55db5a04be36]
 
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x5be8)[0x55db5a0763f8]
 
/usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x311)[0x55db5a079d21]
 
/usr/sbin/mysqld(+0x413675)[0x55db5a07a675]
 
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x1ea5)[0x55db5a07cd75]
 
/usr/sbin/mysqld(_Z10do_commandP3THD+0x139)[0x55db5a07dc49]
 
/usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x182)[0x55db5a14c442]
 
/usr/sbin/mysqld(handle_one_connection+0x40)[0x55db5a14c5e0]
 
/lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba)[0x7fa744f8f6ba]
 
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fa74463a3dd]
 
 
 
Trying to get some variables.
 
Some pointers may be invalid and cause the dump to abort.
 
Query (0x7fa702c21020): drop database keystone
 
Connection ID (thread ID): 12
 
Status: NOT_KILLED
 
 
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=off
 
 
 
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
 
information that should help you find out what is causing the crash.

Comment by Andrii Nikitin (Inactive) [ 2017-09-26 ]

it really looks that drop table crashes only if table was created with innodb_stats_persistent=1 . InnoDB still tries to clear persistent stats for that table if it has any, despite innodb_stats_persistent=0 .

So, to work around the problem:
together with disabling persistent stats tables innodb*stats must be truncated:

set global innodb_stats_persistent=0;
 
truncate table mysql.innodb_index_stats;
 
truncate table mysql.innodb_table_stats;

Comment by Rudy Z [ 2017-09-27 ]

The amount of people this bug affected and the number of crashed servers is alarming, especially since Galera is used to operate clusters for fault-tolerance and load-balancing, and this bug could bring down a whole cluster, especially if small. Can there be more tests before a global release happens? I have seen major sites, including several I operate, going down altogether yesterday!

Comment by Florian Strankowski [ 2017-09-27 ]

I agree with Rudy at this point. Its alarming that such a bug made it into the stable tree. Either QA failed at this point or is non-existent. It seems that 10.1.27 got revoked as of today and is not longer announced on official repos, way to late to be honest. Please do not ship such obvious faulty packages into stable tree - this is what the dev-tree is made for, appreciated.

Comment by Antti-Jussi Korjonen [ 2017-09-29 ]

Anybody updated to 10.1.28 yet?

Comment by Andrii Nikitin (Inactive) [ 2017-09-29 ]

10.1.28 got MDEV-13950 , where mysqld_safe (if you use it) may fail to start galera node properly and requires small patch to mysqld_safe .

Generated at Thu Feb 08 08:09:21 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.