Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
10.11.7
-
None
-
Amazon AL2 Linux
Description
MariaDB 10.11.7 set with innodb log file size of 70GB and 516GB of buffer pool memory. When stopped and attempted to restart, it fails to restart as it was unable to apply redo log file records to disk and it throws error as below
[Note] Starting MariaDB 10.11.7-MariaDB-log source revision as process 8497 |
[Note] InnoDB: Compressed tables use zlib 1.3.1 |
[Note] InnoDB: Number of transaction pools: 1 |
[Note] InnoDB: Using crc32 + pclmulqdq instructions
|
[Note] InnoDB: Using Linux native AIO |
[Note] InnoDB: Initializing buffer pool, total size = 0.516TiB, chunk size = 8.000GiB |
[Note] InnoDB: Setting NUMA memory policy to MPOL_INTERLEAVE
|
[Note] InnoDB: Setting NUMA memory policy to MPOL_DEFAULT
|
[Note] InnoDB: Completed initialization of buffer pool
|
[Note] InnoDB: Buffered log writes (block size=512 bytes) |
[Warning] InnoDB: 22531489280 bytes should have been read at 47565664768 from (unknown file), but got only 2147479552. Retrying. |
[Warning] InnoDB: 22531489280 bytes should have been read at 47565664768 from (unknown file), but got only 4294959104. Retrying. |
[Warning] InnoDB: 22531489280 bytes should have been read at 47565664768 from (unknown file), but got only 6442438656. Retrying. |
[Warning] InnoDB: 22531489280 bytes should have been read at 47565664768 from (unknown file), but got only 8589918208. Retrying. |
[Warning] InnoDB: 22531489280 bytes should have been read at 47565664768 from (unknown file), but got only 10737397760. Retrying. |
[Warning] InnoDB: 22531489280 bytes should have been read at 47565664768 from (unknown file), but got only 12884877312. Retrying. |
[Warning] InnoDB: 22531489280 bytes should have been read at 47565664768 from (unknown file), but got only 15032356864. Retrying. |
[Warning] InnoDB: 22531489280 bytes should have been read at 47565664768 from (unknown file), but got only 17179836416. Retrying. |
[Warning] InnoDB: 22531489280 bytes should have been read at 47565664768 from (unknown file), but got only 19327315968. Retrying. |
[Warning] InnoDB: 22531489280 bytes should have been read at 47565664768 from (unknown file), but got only 21474795520. Retrying. |
[Warning] InnoDB: Retry attempts for reading partial data failed. |
[ERROR] InnoDB: Failed to read log at 47565664768: I/O error |
[Note] InnoDB: Read redo log up to LSN=15096222607497 |
[ERROR] InnoDB: Log scan aborted at LSN 15096222607497 |
[ERROR] InnoDB: Plugin initialization aborted with error Generic error
|
[Note] InnoDB: Starting shutdown...
|
[ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed. |
[Note] Plugin 'FEEDBACK' is disabled. |
[Note] Semi-sync replication enabled on the master.
|
[ERROR] Unknown/unsupported storage engine: InnoDB
|
[ERROR] Aborting
|
Any guidance on how it can be resolved.
Context:
Initially we observed performance impact with a smaller redo log file of 4G. There was more load to the background task as Adaptive flushing was active all the time. We increased the log file size after reading through some recommendation provided on blog and MDEV-30501 and we saw better performance results as flushing reduced.
At the same time we wanted to test the trade off of restart time, where redo logs will have to reapply all changes to disk. `innodb_fast_shutdown` is set to `1` so it will not apply the changes during shutdown/crash. It resulted in above error and not allowing mariadb to start.