Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Cannot Reproduce
-
10.6.14, 10.5
Description
After crash (probably related to MDEV-24769) eventually MariaDB 10.5.12-8 instance got a different crash directly after startup, as we can see in this log snippet:
...
|
2021-12-09 1:14:49 0 [Note] Loaded 'server_audit2.so' with offset 0x7fb034965000
|
2021-12-09 1:14:49 0 [Note] Loaded 'disks.so' with offset 0x7fb034762000
|
2021-12-09 1:14:49 0 [Note] Loaded 'auth_ed25519.so' with offset 0x7fb034551000
|
2021-12-09 1:14:49 0 [Note] Loaded 'simple_password_check.so' with offset 0x7fb03434e000
|
2021-12-09 1:14:49 0 [Note] Loaded 'ha_spider.so' with offset 0x7fb02fc9e000
|
2021-12-09 1:14:53 0 [Note] mariadbd: Aria engine: starting recovery
|
tables to flush: 9 8 7 6 5 4 3 2 1 0
|
(0.0 seconds);
|
2021-12-09 1:14:53 0 [Note] mariadbd: Aria engine: recovery done
|
2021-12-09 1:14:53 0 [Warning] The parameter innodb_thread_concurrency is deprecated and has no effect.
|
2021-12-09 1:14:53 0 [Warning] InnoDB: innodb_open_files 1000000 should not be greater than the open_files_limit 32768
|
2021-12-09 1:14:53 0 [Note] InnoDB: Uses event mutexes
|
2021-12-09 1:14:53 0 [Note] InnoDB: Compressed tables use zlib 1.2.7
|
2021-12-09 1:14:53 0 [Note] InnoDB: Number of pools: 1
|
2021-12-09 1:14:53 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
|
2021-12-09 1:14:53 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts)
|
2021-12-09 1:14:53 0 [Note] InnoDB: Using Linux native AIO
|
2021-12-09 1:14:53 0 [Note] InnoDB: Initializing buffer pool, total size = 214748364800, chunk size = 134217728
|
2021-12-09 1:15:04 0 [Note] InnoDB: Completed initialization of buffer pool
|
2021-12-09 1:15:04 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=9289381264203,9289408027088
|
2021-12-09 1:15:08 0 [Note] InnoDB: Transaction 1550231304 was in the XA prepared state.
|
2021-12-09 1:15:08 0 [Note] InnoDB: 1 transaction(s) which must be rolled back or cleaned up in total 0 row operations to undo
|
2021-12-09 1:15:08 0 [Note] InnoDB: Trx id counter is 1550231305
|
2021-12-09 1:15:08 0 [Note] InnoDB: Starting final batch to recover 39401 pages from redo log.
|
2021-12-09 1:15:11 0 [Note] InnoDB: Last binlog file '/mariadblog/binlog/db2-bin.000240', position 567549
|
2021-12-09 1:15:11 0 [Note] InnoDB: 128 rollback segments are active.
|
2021-12-09 1:15:11 0 [Note] InnoDB: Starting in background the rollback of recovered transactions
|
2021-12-09 1:15:11 0 [Note] InnoDB: Rollback of non-prepared transactions completed
|
2021-12-09 1:15:11 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
|
2021-12-09 1:15:11 0 [Note] InnoDB: Creating shared tablespace for temporary tables
|
2021-12-09 1:15:11 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
|
2021-12-09 1:15:11 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
|
2021-12-09 1:15:11 0 [Note] InnoDB: 10.5.12 started; log sequence number 9289421791370; transaction id 1550231306
|
2021-12-09 1:15:11 0 [Note] InnoDB: Loading buffer pool(s) from /data/mysql/ib_buffer_pool
|
2021-12-09 1:15:11 0 [Note] Plugin 'FEEDBACK' is disabled.
|
2021-12-09 1:15:11 server_audit: MariaDB Audit Plugin version 2.4.0 STARTED.
|
2021-12-09 1:15:11 0 [Note] Loaded 'sql_errlog.so' with offset 0x7f77a05f6000
|
2021-12-09 1:15:11 0 [ERROR] mariadbd: Plugin 'spider' already installed
|
2021-12-09 1:15:11 0 [ERROR] mariadbd: Plugin 'spider_alloc_mem' already installed
|
2021-12-09 1:15:11 0 [Note] Loaded 'metadata_lock_info.so' with offset 0x7f77a03f3000
|
2021-12-09 1:15:11 0 [Note] Using unique option prefix 'sql_error_log_file' is error-prone and can break in the future. Please use the full name 'sql-error-log-filename' instead.
|
2021-12-09 1:15:11 0 [Note] Recovering after a crash using /mariadblog/binlog/db2-bin
|
2021-12-09 1:15:11 0 [Note] Starting crash recovery...
|
2021-12-09 1:15:11 0 [Note] InnoDB: Starting recovery for XA transactions...
|
2021-12-09 1:15:11 0 [Note] InnoDB: Transaction 1550231304 in prepared state after recovery
|
2021-12-09 1:15:11 0 [Note] InnoDB: Transaction contains changes to 1 rows
|
2021-12-09 1:15:11 0 [Note] InnoDB: 1 transactions in prepared state after recovery
|
2021-12-09 1:15:11 0 [Note] Found 1 prepared transaction(s) in InnoDB
|
2021-12-09 1:15:11 0 [Note] Crash recovery finished.
|
211209 1:15:11 [ERROR] mysqld got signal 11 ;
|
This could be because you hit a bug. It is also possible that this binary
|
or one of the libraries it was linked against is corrupt, improperly built,
|
or misconfigured. This error can also be caused by malfunctioning hardware.
|
 |
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
|
 |
We will try our best to scrape up some info that will hopefully help
|
diagnose the problem, but since we have already crashed,
|
something is definitely wrong and this may fail.
|
 |
Server version: 10.5.12-8-MariaDB-enterprise-log
|
key_buffer_size=8388608
|
read_buffer_size=2097152
|
max_used_connections=0
|
max_threads=65537
|
thread_count=20
|
It is possible that mysqld could use up to
|
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 939623430 K bytes of memory
|
Hope that's ok; if not, decrease some variables in the equation.
|
 |
Thread pointer: 0x0
|
Attempting backtrace. You can use the following information to find out
|
where mysqld died. If you see no messages after this, something went
|
terribly wrong...
|
stack_bottom = 0x0 thread_stack 0x16d000
|
/usr/sbin/mariadbd(my_print_stacktrace+0x2e)[0x557b97dabd0e]
|
/usr/sbin/mariadbd(handle_fatal_signal+0x307)[0x557b9777bcc7]
|
sigaction.c:0(__restore_rt)[0x7fb0388b8630]
|
??:0(spider_sys_open_table(THD*, TABLE_LIST*, start_new_trans**))[0x7fb02fcff5bd]
|
??:0(spider_open_sys_table(THD*, char const*, int, bool, start_new_trans**, bool, int*))[0x7fb02fcff839]
|
??:0(spider_internal_xa_rollback_by_xid(THD*, st_spider_transaction*, xid_t*))[0x7fb02fd0d889]
|
??:0(spider_xa_rollback_by_xid(handlerton*, xid_t*))[0x7fb02fd108da]
|
/usr/sbin/mariadbd(+0x994d36)[0x557b9777dd36]
|
/usr/sbin/mariadbd(_Z24plugin_foreach_with_maskP3THDPFcS0_P13st_plugin_intPvEijS3_+0x148)[0x557b9757de08]
|
/usr/sbin/mariadbd(+0x9940db)[0x557b9777d0db]
|
/usr/sbin/mariadbd(my_hash_iterate+0x50)[0x557b97d89b00]
|
/usr/sbin/mariadbd(_Z19ha_recover_completeP7st_hashPSt4pairIjyE+0x2c)[0x557b9777fbbc]
|
/usr/sbin/mariadbd(_ZN16Recovery_context8completeEP13MYSQL_BIN_LOGR7st_hash+0x8a)[0x557b97875b1a]
|
/usr/sbin/mariadbd(_ZN13MYSQL_BIN_LOG7recoverEP11st_log_infoPKcP11st_io_cacheP28Format_description_log_eventb+0x5c3)[0x557b97876863]
|
/usr/sbin/mariadbd(_ZN13MYSQL_BIN_LOG18do_binlog_recoveryEPKcb+0x28d)[0x557b97876bfd]
|
sql/handler.cc:2479(Vers_type_trx::check_sys_fields(st_mysql_const_lex_string const&, Column_definition const*, Column_definition const*) const)[0x557b9787b77a]
|
sql/sql_plugin.cc:2508(plugin_foreach_with_mask(THD*, char (*)(THD*, st_plugin_int*, void*), int, unsigned int, void*))[0x557b9749e263]
|
sql/handler.cc:2535(Discovered_table_list::add_file(char const*))[0x557b974a3cfb]
|
??:0(__libc_start_main)[0x7fb037cef555]
|
/usr/sbin/mariadbd(+0x6af503)[0x557b97498503]
|
The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
|
information that should help you find out what is causing the crash.
|
Writing a core file...
|
Working directory at /data/mysql
|
Resource Limits:
|
Limit Soft Limit Hard Limit Units
|
Max cpu time unlimited unlimited seconds
|
Max file size unlimited unlimited bytes
|
Max data size unlimited unlimited bytes
|
Max stack size 8388608 unlimited bytes
|
Max core file size unlimited unlimited bytes
|
Max resident set unlimited unlimited bytes
|
Max processes 1006755 1006755 processes
|
Max open files 32768 32768 files
|
Max locked memory 65536 65536 bytes
|
Max address space unlimited unlimited bytes
|
Max file locks unlimited unlimited locks
|
Max pending signals 1006755 1006755 signals
|
Max msgqueue size 819200 819200 bytes
|
Max nice priority 0 0
|
Max realtime priority 0 0
|
Max realtime timeout unlimited unlimited us
|
Core pattern: core
|
...
|
After that the instance continues crashing like that on every startup.
Attachments
Issue Links
- is blocked by
-
-
- In Review
-
- relates to
-
-
- Open
-
-
-
- Open
-
-
-
- In Review
-
- split to
-
-
- Closed
-