Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Fixed
-
10.11.7
-
None
-
Debian GNU/Linux 11 (bullseye)
Dell PowerEdge R750
XFS filesystem
Description
We experienced a server crash, on a production server.
This is the second time it happens on the same box. The previous time was reported in MDEV-33189
(Hardware unchanged:
Dell PowerEdge R750 with 64 cores (Intel(R) Xeon(R) Gold 6326 CPU @ 2.90GHz),
512 GiB RAM
software RAID-1 NVMe)
Linux db01 5.10.0-32-amd64 #1 SMP Debian 5.10.223-1 (2024-08-10) x86_64 GNU/Linux
mariadb-server 1:10.11.7+maria~deb11
[ERROR] InnoDB: Trying to read 16384 bytes at 70368744161280 outside the bounds of the file: ./ibdata1
|
[ERROR] InnoDB: File './ibdata1' is corrupted
|
[ERROR] InnoDB: We detected index corruption in an InnoDB type table
|
: We will try our best to scrape up some info that will hopefully help
|
: diagnose the problem, but since we have already crashed,
|
: something is definitely wrong and this may fail.
|
: Server version: 10.11.7-MariaDB-1:10.11.7+maria~deb11-log source revision: 87e13722a95af5d9378d990caf48cb6874439347
|
: key_buffer_size=134217728
|
: read_buffer_size=131072
|
: max_used_connections=5
|
: max_threads=4002
|
: thread_count=7
|
: It is possible that mysqld could use up to
|
: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 4848246 K bytes of memory
|
: Hope that's ok; if not, decrease some variables in the equation.
|
: Thread pointer: 0x7f5d30001598
|
: Attempting backtrace. You can use the following information to find out
|
: where mysqld died. If you see no messages after this, something went
|
: terribly wrong...
|
: stack_bottom = 0x7f5d6d372660 thread_stack 0x49000
|
: /usr/sbin/mariadbd(my_print_stacktrace+0x2e)[0x562e41d0632e]
|
: /usr/sbin/mariadbd(handle_fatal_signal+0x475)[0x562e417df465]
|
: /lib/x86_64-linux-gnu/libpthread.so.0(+0x13140)[0x7f68ea15a140]
|
: /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x141)[0x7f68e9c92d51]
|
: /lib/x86_64-linux-gnu/libc.so.6(abort+0x123)[0x7f68e9c7c537]
|
: /usr/sbin/mariadbd(+0x6ab7d9)[0x562e414087d9]
|
: /usr/sbin/mariadbd(+0x6a71d9)[0x562e414041d9]
|
: /usr/sbin/mariadbd(+0xe6a9e1)[0x562e41bc79e1]
|
: /usr/sbin/mariadbd(+0x70b580)[0x562e41468580]
|
: /usr/sbin/mariadbd(+0xe68be0)[0x562e41bc5be0]
|
: /usr/sbin/mariadbd(+0xe68c39)[0x562e41bc5c39]
|
: /usr/sbin/mariadbd(+0xe5aead)[0x562e41bb7ead]
|
: /usr/sbin/mariadbd(+0xe59b11)[0x562e41bb6b11]
|
: /usr/sbin/mariadbd(+0xd5293e)[0x562e41aaf93e]
|
: /usr/sbin/mariadbd(_Z17ha_rollback_transP3THDb+0x120)[0x562e417e3550]
|
: /usr/sbin/mariadbd(_Z14trans_rollbackP3THD+0x43)[0x562e416c0473]
|
: /usr/sbin/mariadbd(_ZN14rpl_group_info15cleanup_contextEP3THDb+0x70)[0x562e416a58c0]
|
: /usr/sbin/mariadbd(handle_slave_sql+0xe1b)[0x562e414c26eb]
|
: /usr/sbin/mariadbd(+0xca9cc2)[0x562e41a06cc2]
|
And then:
|
|
2024-11-11 01:18:53 0x7f5d6d2c3700 InnoDB: Assertion failure in file ./storage/innobase/buf/buf0lru.cc line 281
|
mariadbd[3792424]: InnoDB: Failing assertion: !block->page.in_file()
|
mariadbd[3792424]: InnoDB: We intentionally generate a memory trap.
|
mariadbd[3792424]: InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
|
mariadbd[3792424]: InnoDB: If you get repeated assertion failures or crashes, even
|
mariadbd[3792424]: InnoDB: immediately after the mariadbd startup, there may be
|
mariadbd[3792424]: InnoDB: corruption in the InnoDB tablespace. Please refer to
|
mariadbd[3792424]: InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
|
mariadbd[3792424]: InnoDB: about forcing recovery.
|
We have kept the original ibdata1 ibtmp1 and ib_logfile0, as well as the database directory, in case thay are needed for debugging purposes.
Forced recovery flags were not used. We restarted the Server operations by restoring from backup.
Attachments
Issue Links
- duplicates
-
-
- Closed
-
- relates to
-
-
- Closed
-
-
-
- Needs Feedback
-