Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Duplicate
-
10.2.23
-
CentOS Linux release 7.6.1810 (Core)
Linux db05.local 3.10.0-957.1.3.el7.x86_64 #1 SMP Thu Nov 29 14:49:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linu
Description
Several databases with common size 3.2 TB
Master - 10.1.37-MariaDB (binlog_format=ROW)
Slave1 - 10.1.37-MariaDB
Slave2 - 10.2.23-MariaDB (same crashing problem was with 10.3.13-MariaDB then we revert back to 10.1.37 and restore replication and upgrade to 10.2.23)
Slave2 after update from 10.1.37-MariaDB to 10.2.23-MariaDB and running mysql_upgrade
staring without any problem.
replication also started ok and synced with master.
Then after several hours slave suddenly catching signal 11 and crashing.
Auto-recovery wasn't success (server doing recovery) and controld stops server by timeout.
After restarting with --skip_replication server working ok.
Any selects from db are ok. Seems DB is working fine.
After running 'start slave;' replication is resumed and workign fine just not so long.
After random period 30sec-30min from start replication problem is coming back. Server falling down.
Below is a partial output from /var/log/messages:
BEGIN
mysqld: 190412 10:25:05 [ERROR] mysqld got signal 11 ;
mysqld: This could be because you hit a bug. It is also possible that this binary
mysqld: or one of the libraries it was linked against is corrupt, improperly built,
mysqld: or misconfigured. This error can also be caused by malfunctioning hardware.
mysqld: To report this bug, see https://mariadb.com/kb/en/reporting-bugs
mysqld: We will try our best to scrape up some info that will hopefully help
mysqld: diagnose the problem, but since we have already crashed,
mysqld: something is definitely wrong and this may fail.
mysqld: Server version: 10.2.23-MariaDB-log
mysqld: key_buffer_size=268435456
mysqld: read_buffer_size=131072
mysqld: max_used_connections=1
mysqld: max_threads=102
mysqld: thread_count=10
mysqld: It is possible that mysqld could use up to
mysqld: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 486249 K bytes of memory
mysqld: Hope that's ok; if not, decrease some variables in the equation.
mysqld: Thread pointer: 0x7ef808001298
mysqld: Attempting backtrace. You can use the following information to find out
mysqld: where mysqld died. If you see no messages after this, something went
mysqld: terribly wrong...
mysqld: stack_bottom = 0x7ef8a0168180 thread_stack 0x49000
mysqld: /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x559e1f63ecae]
mysqld: /usr/sbin/mysqld(handle_fatal_signal+0x355)[0x559e1f0c5f35]
mysqld: /lib64/libpthread.so.0(+0xf5d0)[0x7f097b10c5d0]
mysqld: /usr/sbin/mysqld(ZN16Item_func_or_sum4walkEM4ItemFbPvEbS1+0x60)[0x559e1eef1a60]
mysqld: /usr/sbin/mysqld(_Z21fix_session_vcol_exprP3THDP19Virtual_column_info+0x3c)[0x559e1efd36dc]
mysqld: /usr/sbin/mysqld(_Z11lock_tablesP3THDP10TABLE_LISTjj+0x3ff)[0x559e1eeeae6f]
mysqld: /usr/sbin/mysqld(_Z20open_and_lock_tablesP3THDRK14DDL_options_stP10TABLE_LISTbjP19Prelocking_strategy+0x8a)[0x559e1eeec2ea]
mysqld: /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEP14rpl_group_info+0xa3c)[0x559e1f1b8e2c]
mysqld: /usr/sbin/mysqld(+0x46648b)[0x559e1eeb448b]
mysqld: /usr/sbin/mysqld(handle_slave_sql+0x1833)[0x559e1eebceb3]
mysqld: /lib64/libpthread.so.0(+0x7dd5)[0x7f097b104dd5]
mysqld: /lib64/libc.so.6(clone+0x6d)[0x7f09794a6ead]
mysqld: Trying to get some variables.
mysqld: Some pointers may be invalid and cause the dump to abort.
mysqld: Query (0x0):
mysqld: Connection ID (thread ID): 11
mysqld: Status: NOT_KILLED
kernel: mysqld[41013]: segfault at 2d9 ip 0000559e1eef1a60 sp 00007ef8a01675e0 error 4 in mysqld[559e1ea4e000+1125000]
mysqld: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on
mysqld: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
mysqld: information that should help you find out what is causing the crash.
systemd: mariadb.service: main process exited, code=killed, status=11/SEGV
systemd: Unit mariadb.service entered failed state.
systemd: mariadb.service failed.
systemd: mariadb.service holdoff time over, scheduling restart.
systemd: Stopped MariaDB 10.2.23 database server.
END
Thank you very much
Attachments
Issue Links
- is duplicated by
-
MDEV-16048 crashing with virtual columns
- Closed