Details
-
Bug
-
Status: Needs Feedback (View Workflow)
-
Major
-
Resolution: Unresolved
-
10.6.17
-
None
-
CPanel 11.118.0.11, CloudLinux 8, Virtual Machine (KVM), 178G RAM
Description
During the day with no events happening at the same time, innodb1 turns out corrupted.
2024-05-23 12:12:01 8668229 [ERROR] InnoDB: Trying to read 16384 bytes at 70368744161280 outside the bounds of the file: ./ibdata1
|
2024-05-23 12:12:01 8668229 [ERROR] InnoDB: File './ibdata1' is corrupted
|
2024-05-23 12:12:01 0x7f0944bad700 InnoDB: Assertion failure in file /builddir/build/BUILD/cl-MariaDB106-10.6.17/mariadb-10.6.17/storage/innobase/trx/trx0purge.cc line 268
|
InnoDB: Failing assertion: undo_page
|
InnoDB: We intentionally generate a memory trap.
|
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
|
InnoDB: If you get repeated assertion failures or crashes, even
|
InnoDB: immediately after the mariadbd startup, there may be
|
InnoDB: corruption in the InnoDB tablespace. Please refer to
|
InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
|
InnoDB: about forcing recovery.
|
240523 12:12:01 [ERROR] mysqld got signal 6 ;
|
Sorry, we probably made a mistake, and this is a bug.
|
Your assistance in bug reporting will enable us to fix this for the next release.
|
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
|
We will try our best to scrape up some info that will hopefully help
|
diagnose the problem, but since we have already crashed,
|
something is definitely wrong and this may fail.
|
Server version: 10.6.17-MariaDB-cll-lve source revision: 15c75ad083a55e198ae78324f22970694b72f22b
|
key_buffer_size=402653184
|
read_buffer_size=2097152
|
max_used_connections=307
|
max_threads=802
|
thread_count=33
|
It is possible that mysqld could use up to
|
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 3699403 K bytes of memory
|
Hope that's ok; if not, decrease some variables in the equation.
|
Thread pointer: 0x7f078463b9a8
|
Attempting backtrace. You can use the following information to find out
|
where mysqld died. If you see no messages after this, something went
|
terribly wrong...
|
stack_bottom = 0x7f0944bacb98 thread_stack 0x49000
|
/usr/sbin/mariadbd(my_print_stacktrace+0x2e)[0x55ec22da47ae]
|
/usr/sbin/mariadbd(handle_fatal_signal+0x475)[0x55ec2287ed95]
|
/lib64/libpthread.so.0(+0x12cf0)[0x7f0c8d514cf0]
|
2024-05-23 12:12:05 0x7f0944ed1700 InnoDB: Assertion failure in file /builddir/build/BUILD/cl-MariaDB106-10.6.17/mariadb-10.6.17/storage/innobase/buf/buf0lru.cc line 281
|
InnoDB: Failing assertion: !block->page.in_file()
|
InnoDB: We intentionally generate a memory trap.
|
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
|
InnoDB: If you get repeated assertion failures or crashes, even
|
InnoDB: immediately after the mariadbd startup, there may be
|
InnoDB: corruption in the InnoDB tablespace. Please refer to
|
InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
|
InnoDB: about forcing recovery.
|
mariadb.service: Main process exited, code=killed, status=6/ABRT
|
mariadb.service: Failed with result 'signal'.
|
The size of the ibdata1 is about 6931087360 bytes for reference.
This resulted in broken mini-transaction which was later recovered from via innodb_force_recovery=1 (setting the value, starting, stopping, removing the value, starting). The database is running without issues after that.
There is no core dump logged.
What is concerning is that the previous week the database was rsynced and migrated from CloudLinux 7. However no problems where noticed while migrating the database.
I would like to offer more information about the incident, but non is available. If you feel that information about the following mini-transaction problem is interesting I can post that too. I post this issue hoping that someone can shed some light on what our next step would be. Known bug? Need to upgrade?
My.cnf
[mysqld]
|
character_set_server = utf8mb4
|
collation_server = utf8mb4_unicode_ci
|
ft_min_word_len = 3
|
innodb_buffer_pool_size = 12G
|
innodb_file_per_table = 1
|
key_buffer_size = 384M
|
max_allowed_packet = 268435456
|
max_connections = 800
|
max_user_connections = 100
|
max_heap_table_size = 256M
|
myisam_sort_buffer_size = 64M
|
myisam_use_mmap = 1
|
open_files_limit = 80000
|
performance-schema = 1
|
read_buffer_size = 2M
|
read_rnd_buffer_size = 8M
|
sort_buffer_size = 2M
|
table_open_cache = 4000
|
thread_cache_size = 8
|
tmp_table_size = 256M
|
join_buffer_space_limit = 24M
|
join_cache_level = 8
|
join_buffer_size = 24M
|
query_cache_size = 128M
|
query_cache_limit = 8M
|
query_cache_type = 0
|
tmp_disk_table_size = 805306368
|
ignore_db_dirs = lost+found
|
explicit_defaults_for_timestamp = On
|
optimizer_switch = ""
|
sql_mode = ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE
|
log_warnings=4
|
tmpdir = "/var/cache/mysql"
|
ssl_cert = /etc/mysql/ssl/fullchain.pem
|
ssl_key = /etc/mysql/ssl/privkey.pem
|
/var/cache/mysql is an ext4 fs with 10G space.
Attachments
Issue Links
- relates to
-
-
- Closed
-
-
-
- Closed
-
-
-
- Open
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-