[MDEV-16348] the suddenly crashing in MariaDB10.1.29 Created: 2018-05-31  Updated: 2018-09-11  Resolved: 2018-09-11

Status: Closed
Project: MariaDB Server
Component/s: Platform RedHat, Storage Engine - Spider
Affects Version/s: 10.1.29
Fix Version/s: N/A

Type: Bug Priority: Critical
Reporter: wangting Assignee: Jacob Mathew (Inactive)
Resolution: Incomplete Votes: 0
Labels: need_feedback
Environment:

CentOS7


Attachments: File error_mysqld.log-20180513    
Issue Links:
Relates
relates to MDEV-12900 spider tests failed in buildbot with ... Closed
relates to MDEV-15936 Crashes with varying stack trace on W... Closed

 Description   

Install mariaDB10.1.29 in two database server (both centos7), one is master , another is slave.

There are about 170 tables, 5 of them are spider engine (spider to the third database server),
others are innoDB engine.

Suddenly, some crashing occured , master was broken down . But auto repaired by mariadb soon, and now it's normally.

Some info output in the error log :

@180512 12:15:01 [ERROR] mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
 
Server version: 10.1.29-MariaDB
key_buffer_size=402653184
read_buffer_size=2097152
max_used_connections=4
max_threads=153
thread_count=2
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1023042 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x7fa34576a008
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7fa3427fe190 thread_stack 0x48400
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x7fa385e7965e]
/usr/sbin/mysqld(handle_fatal_signal+0x305)[0x7fa38599d015]
/lib64/libpthread.so.0(+0xf5e0)[0x7fa384fab5e0]
/usr/sbin/mysqld(_ZN17Query_cache_block11headers_lenEv+0x0)[0x7fa3857d5ce0]
/usr/sbin/mysqld(_ZN17Query_cache_block4dataEv+0x11)[0x7fa3857d5d01]
/usr/sbin/mysqld(_ZN11Query_cache6insertEP3THDP15Query_cache_tlsPKcmj+0x5c)[0x7fa3857d96ec]
/usr/sbin/mysqld(net_real_write+0x45)[0x7fa385782e25]
/usr/sbin/mysqld(net_flush+0x23)[0x7fa385783163]
/usr/sbin/mysqld(net_write_command+0x3fb)[0x7fa3857838cb]
/usr/sbin/mysqld(cli_advanced_command+0xe2)[0x7fa38597caa2]
/usr/sbin/mysqld(mysql_send_query+0x31)[0x7fa38597a181]
/usr/sbin/mysqld(mysql_real_query+0x11)[0x7fa38597a1a1]
/usr/lib64/mysql/plugin/ha_spider.so(_ZN15spider_db_mysql10exec_queryEPKcji+0x2c7)[0x7fa37d5c8f07]
/usr/lib64/mysql/plugin/ha_spider.so(_ZN20spider_mysql_handler17show_table_statusEiij+0x802)[0x7fa37d5c80c2]
/usr/lib64/mysql/plugin/ha_spider.so(_Z14spider_get_stsP15st_spider_shareilP9ha_spiderdiiij+0xae)[0x7fa37d58fdfe]
/usr/lib64/mysql/plugin/ha_spider.so(_Z20spider_bg_sts_actionPv+0x5dc)[0x7fa37d57d9bc]
/lib64/libpthread.so.0(+0x7e25)[0x7fa384fa3e25]
/lib64/libc.so.6(clone+0x6d)[0x7fa38334934d]
 
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x0): 
Connection ID (thread ID): 21538
Status: NOT_KILLED
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=off
 
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
2018-05-12 12:15:07 140436573571328 [Note] InnoDB: innodb_empty_free_list_algorithm has been changed to legacy because of small buffer pool size. In order to use backoff, increase buffer pool at least up to 20MB.
 
2018-05-12 12:15:07 140436573571328 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2018-05-12 12:15:07 140436573571328 [Note] InnoDB: The InnoDB memory heap is disabled
2018-05-12 12:15:07 140436573571328 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2018-05-12 12:15:07 140436573571328 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2018-05-12 12:15:07 140436573571328 [Note] InnoDB: Compressed tables use zlib 1.2.7
2018-05-12 12:15:07 140436573571328 [Note] InnoDB: Using Linux native AIO
2018-05-12 12:15:07 140436573571328 [Note] InnoDB: Using generic crc32 instructions
2018-05-12 12:15:07 140436573571328 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2018-05-12 12:15:07 140436573571328 [Note] InnoDB: Completed initialization of buffer pool
2018-05-12 12:15:07 140436573571328 [Note] InnoDB: Highest supported file format is Barracuda.
2018-05-12 12:15:07 140436573571328 [Note] InnoDB: The log sequence number 141678940 in ibdata file do not match the log sequence number 141687910 in the ib_logfiles!
2018-05-12 12:15:07 140436573571328 [Note] InnoDB: Restoring possible half-written data pages from the doublewrite buffer...
InnoDB: Last MySQL binlog file position 0 46478766, file name ./mysql-bin.000013
2018-05-12 12:15:08 140436573571328 [Note] InnoDB: 128 rollback segment(s) are active.
2018-05-12 12:15:08 140436573571328 [Note] InnoDB: Waiting for purge to start
2018-05-12 12:15:08 140436573571328 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.36-82.2 started; log sequence number 141687910
2018-05-12 12:15:08 140436573571328 [Note] Plugin 'FEEDBACK' is disabled.
2018-05-12 12:15:08 140436573571328 [Note] Recovering after a crash using mysql-bin
2018-05-12 12:15:08 140435527788288 [Note] InnoDB: Dumping buffer pool(s) not yet started
2018-05-12 12:15:08 140436573571328 [Note] Starting crash recovery...
2018-05-12 12:15:08 140436573571328 [Note] Crash recovery finished.
2018-05-12 12:15:08 140436573571328 [Note] Server socket created on IP: '::'.
2018-05-12 12:15:08 140436573571328 [Warning] Neither --relay-log nor --relay-log-index were used; so replication may break when this MySQL server acts as a slave and has his hostname changed!! Please use '--log-basename=#' or '--relay-log=ryukyu-malldb1-relay-bin' to avoid this problem.
2018-05-12 12:15:08 140436573571328 [Note] /usr/sbin/mysqld: ready for connections.
Version: '10.1.29-MariaDB'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MariaDB Server
2018-05-12 12:45:02 140436572232448 [ERROR] mysqld: Table './mysql/spider_tables' is marked as crashed and should be repaired
2018-05-12 12:45:02 140436572232448 [Warning] Checking table:   './mysql/spider_tables'

About the reason of crashing , please help me find out.



 Comments   
Comment by Elena Stepanova [ 2018-05-31 ]

Looks similar to MDEV-15936.

Comment by Jacob Mathew (Inactive) [ 2018-06-08 ]

It is possible that this problem is a duplicate of MDEV-12900, which I fixed in MariaDB Server 10.1.34. I recommend that the customer install MariaDB Server 10.1.34 when it is released.

Comment by Jacob Mathew (Inactive) [ 2018-06-18 ]

MariaDB Server 10.1.34 is now available.

Comment by Jacob Mathew (Inactive) [ 2018-07-24 ]

Waiting for the customer to confirm after installing 10.1.34 or a later release of MariaDB Server.

Comment by Chris Calender (Inactive) [ 2018-07-31 ]

ting Please install 10.1.34, 10.2.16, 10.3.7, or a later release of MariaDB Server that includes the fix for bug MDEV-12900. Then confirm whether or not the crashes continue to occur.

Generated at Thu Feb 08 08:28:14 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.