Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Incomplete
-
10.6.16
-
None
-
Red Hat Enterprise Linux Server release 7.9 (Maipo)
16 CPU
150 GB
Description
Hi,
I can't get my head around the cause for a crash we've had recently.
It only happened once, and I couldn't replicate the issue. Could you help me understand the backtrace ?
I have to mention that we upgraded from 10.1.38 to 10.6.16 recently (late January). We never had such a crash before.
240216 10:55:38 [ERROR] mysqld got signal 11 ;
|
Sorry, we probably made a mistake, and this is a bug.
|
|
|
Your assistance in bug reporting will enable us to fix this for the next release.
|
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
|
|
|
We will try our best to scrape up some info that will hopefully help
|
diagnose the problem, but since we have already crashed,
|
something is definitely wrong and this may fail.
|
|
|
Server version: 10.6.16-MariaDB-log source revision: b83c379420a8846ae4b28768d3c81fa354cca056
|
key_buffer_size=134217728
|
read_buffer_size=131072
|
max_used_connections=268
|
max_threads=4002
|
thread_count=233
|
It is possible that mysqld could use up to
|
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 4846808 K bytes of memory
|
Hope that's ok; if not, decrease some variables in the equation.
|
|
|
Thread pointer: 0x7f258c4a9588
|
Attempting backtrace. You can use the following information to find out
|
where mysqld died. If you see no messages after this, something went
|
terribly wrong...
|
stack_bottom = 0x7f27217bac40 thread_stack 0x49000
|
Can't start addr2line
|
/usr/sbin/mariadbd(my_print_stacktrace+0x2e)[0x55605f839eae]
|
/usr/sbin/mariadbd(handle_fatal_signal+0x2f7)[0x55605f28eac7]
|
/lib64/libpthread.so.0(+0xf630)[0x7f401d447630]
|
/usr/sbin/mariadbd(_ZN13st_join_table17save_explain_dataEP20Explain_table_accessybPS_+0x30b)[0x55605f0a926b]
|
/usr/sbin/mariadbd(_ZN4JOIN24save_explain_data_internEP13Explain_querybbbPKc+0xd6f)[0x55605f0aaf1f]
|
/usr/sbin/mariadbd(_ZN4JOIN17save_explain_dataEP13Explain_querybbbb+0x94)[0x55605f0ab084]
|
/usr/sbin/mariadbd(_ZN4JOIN13build_explainEv+0x7c)[0x55605f0ab2dc]
|
/usr/sbin/mariadbd(_ZN4JOIN8optimizeEv+0x71)[0x55605f0c2751]
|
/usr/sbin/mariadbd(_Z12mysql_selectP3THDP10TABLE_LISTR4ListI4ItemEPS4_jP8st_orderS9_S7_S9_yP13select_resultP18st_select_lex_unitP13st_select_lex+0xb8)[0x55605f0c2868]
|
/usr/sbin/mariadbd(_Z13handle_selectP3THDP3LEXP13select_resultm+0x1e4)[0x55605f0c3174]
|
/usr/sbin/mariadbd(+0x67bd11)[0x55605ef12d11]
|
/usr/sbin/mariadbd(_Z21mysql_execute_commandP3THDb+0x34a3)[0x55605f0610c3]
|
/usr/sbin/mariadbd(_Z11mysql_parseP3THDPcjP12Parser_state+0x20b)[0x55605f06346b]
|
/usr/sbin/mariadbd(_Z16dispatch_command19enum_server_commandP3THDPcjb+0x14a0)[0x55605f0658a0]
|
/usr/sbin/mariadbd(_Z10do_commandP3THDb+0x123)[0x55605f066dc3]
|
/usr/sbin/mariadbd(_Z24do_handle_one_connectionP7CONNECTb+0x187)[0x55605f164cd7]
|
/usr/sbin/mariadbd(handle_one_connection+0x34)[0x55605f164f74]
|
/usr/sbin/mariadbd(+0xc466ac)[0x55605f4dd6ac]
|
/lib64/libpthread.so.0(+0x7ea5)[0x7f401d43fea5]
|
/lib64/libc.so.6(clone+0x6d)[0x7f401c95ab0d]
|
|
|
Trying to get some variables.
|
Some pointers may be invalid and cause the dump to abort.
|
Query (0x7f258c851880): SELECT sys_ui_ng_action0.`sys_overrides` FROM (sys_ui_ng_action sys_ui_ng_action0 INNER JOIN sys_metadata sys_metadata0 ON sys_ui_ng_action0.`sys_id` = sys_metadata0.`sys_id` ) WHERE sys_ui_ng_action0.`table` IN ('sys_user' , 'global') AND sys_ui_ng_action0.`active` = 1 AND sys_ui_ng_action0.`list_button` = 1 AND (sys_ui_ng_action0.`show_for` = 'any' OR sys_ui_ng_action0.`show_for` = 'update') AND sys_ui_ng_action0.`sys_overrides` IS NOT NULL AND sys_ui_ng_action0.`sys_domain` != 'global' AND (sys_ui_ng_action0.`sys_domain_path` = '/' OR sys_ui_ng_action0.`sys_domain_path` = '!!!/!!!/!!!/!!#/!!!/!!#/!!!/!!$/!!)/' OR sys_ui_ng_action0.`sys_domain_path` = '!!!/!!!/!!!/!!#/!!!/!!#/!!!/!!$/' OR sys_ui_ng_action0.`sys_domain_path` = '!!!/!!!/!!!/!!#/!!!/!!#/!!!/' OR sys_ui_ng_action0.`sys_domain_path` = '!!!/!!!/!!!/!!#/!!!/!!#/' OR sys_ui_ng_action0.`sys_domain_path` = '!!!/!!!/!!!/!!#/!!!/' OR sys_ui_ng_action0.`sys_domain_path` = '!!!/!!!/!!!/!!#/' OR sys_ui_ng_action0.`sys_domain_path` = '!!!/!!!/!!!/' OR sys_ui_ng_action0.`sys_domain_path` = '!!!/!!!/' OR sys_ui_ng_action0.`sys_domain_path` = '!!!/' OR sys_ui_ng_action0.`sys_domain_path` = '/') /* serv1, gs:F6CC5A87FD108A50DB4614677409CE67, tx:a60e560bfd108a50db4614677409ce93, hash:-1250232669 */
|
|
|
Connection ID (thread ID): 42129
|
Status: KILL_QUERY
|
|
|
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off,hash_join_cardinality=off,cset_narrowing=off
|
|
|
The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mariadbd/ contains
|
information that should help you find out what is causing the crash.
|
Writing a core file...
|
Working directory at /basedata00/INSTANCE_NAME/data
|
Resource Limits:
|
Limit Soft Limit Hard Limit Units
|
Max cpu time unlimited unlimited seconds
|
Max file size unlimited unlimited bytes
|
Max data size unlimited unlimited bytes
|
Max stack size 8388608 unlimited bytes
|
Max core file size unlimited unlimited bytes
|
Max resident set unlimited unlimited bytes
|
Max processes 611711 611711 processes
|
Max open files 1048576 1048576 files
|
Max locked memory 65536 65536 bytes
|
Max address space unlimited unlimited bytes
|
Max file locks unlimited unlimited locks
|
Max pending signals 611711 611711 signals
|
Max msgqueue size 819200 819200 bytes
|
Max nice priority 0 0
|
Max realtime priority 0 0
|
Max realtime timeout unlimited unlimited us
|
Core pattern: core
|
|
|
Kernel version: Linux version 3.10.0-1160.81.1.el7.x86_64 (mockbuild@x86-vm-38.build.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC) ) #1 SMP Thu Nov 24 12:21:22 UTC 2022
|
Demangled relevant stack:
/usr/sbin/mariadbd(st_join_table::save_explain_data(Explain_table_access*, unsigned long long, bool, st_join_table*)+0x30b)[0x55605f0a926b]
|
/usr/sbin/mariadbd(JOIN::save_explain_data_intern(Explain_query*, bool, bool, bool, char const*)+0xd6f)[0x55605f0aaf1f]
|
/usr/sbin/mariadbd(JOIN::save_explain_data(Explain_query*, bool, bool, bool, bool)+0x94)[0x55605f0ab084]
|
/usr/sbin/mariadbd(JOIN::build_explain()+0x7c)[0x55605f0ab2dc]
|
/usr/sbin/mariadbd(JOIN::optimize()+0x71)[0x55605f0c2751]
|
/usr/sbin/mariadbd(mysql_select(THD*, TABLE_LIST*, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*)+0xb8)[0x55605f0c2868]
|
/usr/sbin/mariadbd(handle_select(THD*, LEX*, select_result*, unsigned long)+0x1e4)[0x55605f0c3174]
|
/usr/sbin/mariadbd(+0x67bd11)[0x55605ef12d11]
|
/usr/sbin/mariadbd(mysql_execute_command(THD*, bool)+0x34a3)[0x55605f0610c3]
|
Here is the cnf file :
#
|
# These groups are read by MariaDB server.
|
# Use it for options that only the server (but not clients) should see |
#
|
# See the examples of server my.cnf files in /usr/share/mysql/
|
#
|
|
|
# this is read by the standalone daemon and embedded servers |
[server]
|
|
|
# this is only for the mysqld standalone daemon |
[mysqld]
|
|
|
server-id = 1 |
log_bin = /base/INSTANCE_NAME/arc/mysql-bin
|
skip_name_resolve = 1 |
skip-slave-start
|
tmpdir = /base/INSTANCE_NAME/data
|
log_slave_updates = 1 |
max_connect_errors = 20000 # 100 default |
binlog_format = MIXED # Mixed default |
|
|
basedir = /usr
|
|
|
port = 3306 |
socket = /base/INSTANCE_NAME/conf/mysql_INSTANCE_NAME.sock
|
|
|
datadir = /base/INSTANCE_NAME/data
|
innodb_tmpdir = /base/INSTANCE_NAME/data
|
|
|
general_log = off
|
general_log_file = /base/INSTANCE_NAME/log/mariadb_INSTANCE_NAME_gal.log
|
log-error = /base/INSTANCE_NAME/log/mariadb_INSTANCE_NAME.log
|
|
|
secure_auth = ON
|
|
|
key_buffer_size = 128M
|
max_allowed_packet = 32M
|
tmp_table_size = 64M
|
max_heap_table_size = 64M
|
sort_buffer_size = 1M
|
myisam_sort_buffer_size = 256K
|
|
|
net_write_timeout = 900 |
net_read_timeout = 900 |
|
|
sync_binlog = 0 |
|
|
# Slow query conf
|
slow_query_log = ON
|
long_query_time = 10 |
slow_query_log_file = /base/INSTANCE_NAME/log/slow_queries_INSTANCE_NAME.log
|
|
|
|
expire_logs_days = 3 |
#log-slave-updates = 1 |
#read-only = 0 |
#binlog_cache_size = 1M
|
slave_compressed_protocol = 1 |
#log-bin-trust-function-creators = 1 |
|
|
default_storage_engine = innodb
|
character_set_server = utf8
|
|
|
query_cache_size = 0 |
query_cache_type = 0 |
innodb_buffer_pool_size = 96G
|
innodb_io_capacity = 65535 |
innodb_open_files = 8000 |
open_files_limit = 37535 |
table_open_cache = 18000 |
table_definition_cache = 12000 |
|
|
max_connections = 4000 |
innodb_lock_wait_timeout = 50 |
innodb_sync_spin_loops = 20 |
innodb_log_file_size = 512M
|
innodb_read_io_threads = 32 |
innodb_write_io_threads = 32 |
innodb_autoinc_lock_mode = 1 |
innodb_purge_threads = 4 |
innodb_purge_batch_size = 300 |
innodb_max_purge_lag = 5000000 |
thread_cache_size = 128 |
|
|
back_log = 2500 |
|
|
innodb_log_buffer_size = 64M
|
innodb_page_size = 16K
|
join_buffer_size = 256K
|
innodb_strict_mode = off
|
|
|
innodb_use_native_aio = 1 |
innodb_stats_persistent = 1 |
innodb_spin_wait_delay = 48 |
|
|
innodb_adaptive_flushing = 1 |
innodb_checksum_algorithm = crc32
|
|
|
#optimizer_switch = index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,engine_condition_pushdown=on,index_condition_pushdown=off,mrr=on,mrr_cost_based=on,block_nested_loop=on,batched_key_access=off,materialization=on,semijoin=on,loosescan=on,firstmatch=on,subquery_materialization_cost_based=on,use_index_extensions=on
|
|
|
innodb_flush_log_at_trx_commit = 1 |
innodb_stats_on_metadata = 0 |
innodb_flush_method = O_DIRECT
|
innodb_file_per_table = on
|
|
|
innodb_data_file_path = ibdata1:10M:autoextend
|
|
|
innodb_monitor_disable = ALL
|
performance_schema = on
|
|
|
local-infile = off
|
|
|
bind-address = 0.0.0.0 |
|
|
symbolic-links = 0 |
lock_wait_timeout = 14400 |
|
|
# Upgrade MariaDB 10.6.16 |
sql_mode = NO_ENGINE_SUBSTITUTION
|
slave_parallel_max_queued = 524288 |
innodb_print_all_deadlocks = on
|
innodb_max_dirty_pages_pct = 75 |
slave_parallel_mode = conservative
|
innodb_adaptive_hash_index = on
|
log_queries_not_using_indexes = 0 |
log_bin_trust_function_creators = 1 |
slave_parallel_threads = 16 |
eq_range_index_dive_limit = 0 |
|
|
#innodb_log_files_in_group = 3 # Removed from 10.6 |
#innodb_support_xa = 1 # Removed from 10.3.0 |
#innodb_thread_concurrency = 0 # Removed from 10.6 |
#innodb_buffer_pool_instances = 12 # Removed from 10.6 |
#innodb_additional_mem_pool_size = 20M # Removed from 10.2 |
#innodb_file_format = Barracuda # Removed from 10.6 |
|
|
|
|
|
|
|
|
#
|
# * Galera-related settings
|
#
|
[galera]
|
# Mandatory settings
|
#wsrep_on = ON
|
#wsrep_provider =
|
#wsrep_cluster_address =
|
#binlog_format = row
|
#default_storage_engine = InnoDB
|
#innodb_autoinc_lock_mode = 2 |
#
|
# Allow server to accept connections on all interfaces.
|
#
|
#bind-address = 0.0.0.0 |
#
|
# Optional setting
|
#wsrep_slave_threads = 1 |
#innodb_flush_log_at_trx_commit = 0 |
|
|
# this is only for embedded server |
[embedded]
|
|
|
# This group is only read by MariaDB servers, not by MySQL.
|
# If you use the same .cnf file for MySQL and MariaDB, |
# you can put MariaDB-only options here
|
[mariadb]
|
|
|
# This group is only read by MariaDB-10.2 servers. |
# If you use the same .cnf file for MariaDB of different versions, |
# use this group for options that older servers don't understand |
[mariadb-10.2] |
#enforce_gtid_consistency=1 |
|
|
#Erreur reverse lookup
|
skip_name_resolve = 1 |
|
|
[mysql]
|
no-auto-rehash
|
Attachments
Issue Links
- relates to
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-