Details
-
Bug
-
Status: Open (View Workflow)
-
Major
-
Resolution: Unresolved
-
10.11.7
-
None
-
Linux 6.8.8-gentoo, AMD EPYC 7451, 386GB RAM
Description
It was noticed that on receiving SST node logs series of warnings:
2024-05-02 23:44:38 0 [Warning] InnoDB: 2147483648 bytes should have been read at 45590839296 from (unknown file), but got only 2147479552. Retrying. |
2024-05-02 23:44:39 0 [Warning] InnoDB: 2147483648 bytes should have been read at 45590839296 from (unknown file), but got only 2147479552. Retrying. |
2024-05-02 23:44:41 0 [Warning] InnoDB: 2147483648 bytes should have been read at 45590839296 from (unknown file), but got only 2147479552. Retrying. |
2024-05-02 23:44:42 0 [Warning] InnoDB: 2147483648 bytes should have been read at 45590839296 from (unknown file), but got only 2147479552. Retrying. |
2024-05-02 23:44:44 0 [Warning] InnoDB: 2147483648 bytes should have been read at 45590839296 from (unknown file), but got only 2147479552. Retrying. |
[05] 2024-05-04 09:13:04 Streaming ./db210441/trigger_action_stats.ibd |
[10] 2024-05-04 09:13:04 ...done |
[05] 2024-05-04 09:13:04 ...done |
[08] 2024-05-04 09:13:04 Streaming ./db210441/attributes.ibd |
2024-05-04 9:13:05 0 [Warning] InnoDB: 2147483648 bytes should have been read at 17356201984 from (unknown file), but got only 2147479552. Retrying. |
[14] 2024-05-04 09:13:05 Streaming ./db210441/generic_event_v001#P#up_to_2023_10_31.ibd |
[16] 2024-05-04 09:13:05 ...done |
[11] 2024-05-04 09:13:05 ...done |
[08] 2024-05-04 09:13:05 ...done |
The same error appears on joining the cluster after start-up and IST:
2024-04-19 22:10:43 3 [Note] WSREP: SST received |
2024-04-19 22:10:43 3 [Note] WSREP: Server status change joiner -> initializing |
2024-04-19 22:10:43 0 [Note] InnoDB: Compressed tables use zlib 1.3 |
2024-04-19 22:10:43 0 [Note] InnoDB: Number of transaction pools: 1 |
2024-04-19 22:10:43 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions |
2024-04-19 22:10:43 0 [Note] InnoDB: Using liburing |
2024-04-19 22:10:43 0 [Note] InnoDB: Initializing buffer pool, total size = 64.000GiB, chunk size = 1.000GiB |
2024-04-19 22:10:43 0 [Note] InnoDB: Completed initialization of buffer pool |
2024-04-19 22:10:43 0 [Note] InnoDB: Buffered log writes (block size=4096 bytes) |
2024-04-19 22:10:44 0 [Note] WSREP: (e50354a0-929f, 'tcp://[::]:4567') turning message relay requesting off |
2024-04-19 22:10:44 0 [Warning] InnoDB: 2147483648 bytes should have been read at 4592611328 from (unknown file), but got only 2147479552. Retrying. |
2024-04-19 22:10:45 0 [Warning] InnoDB: 2147483648 bytes should have been read at 4592611328 from (unknown file), but got only 2147479552. Retrying. |
2024-04-19 22:10:45 0 [Note] InnoDB: End of log at LSN=158417014560773 |
2024-04-19 22:10:45 0 [Note] InnoDB: 128 rollback segments are active. |
2024-04-19 22:10:52 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ... |
2024-04-19 22:10:52 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB. |
2024-04-19 22:10:52 0 [Note] InnoDB: log sequence number 158417014560773; transaction id 50466490434 |
2024-04-19 22:10:52 0 [Note] Plugin 'FEEDBACK' is disabled. |
2024-04-19 22:10:52 0 [Note] InnoDB: Loading buffer pool(s) from /data/mysql/ib_buffer_pool |
2024-05-05 18:51:55 2 [Note] WSREP: Server status change connected -> joiner |
2024-05-05 18:51:55 2 [Note] WSREP: Server status change joiner -> initializing |
2024-05-05 18:51:55 0 [Note] InnoDB: Compressed tables use zlib 1.3 |
2024-05-05 18:51:55 0 [Note] InnoDB: Number of transaction pools: 1 |
2024-05-05 18:51:55 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions |
2024-05-05 18:51:55 0 [Note] InnoDB: Using liburing |
2024-05-05 18:51:55 0 [Note] InnoDB: Initializing buffer pool, total size = 96.000GiB, chunk size = 1.500GiB |
2024-05-05 18:51:56 0 [Note] InnoDB: Completed initialization of buffer pool |
2024-05-05 18:51:56 0 [Note] InnoDB: File system buffers for log disabled (block size=4096 bytes) |
2024-05-05 18:51:57 0 [Warning] InnoDB: 2147483648 bytes should have been read at 3576475648 from (unknown file), but got only 2147479552. Retrying. |
2024-05-05 18:52:00 0 [Warning] InnoDB: 2147483648 bytes should have been read at 66178359296 from (unknown file), but got only 2147479552. Retrying. |
2024-05-05 18:52:00 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=162945365494496 |
2024-05-05 18:52:11 0 [Note] InnoDB: Parsed redo log up to LSN=162945879327135; to recover: 383181 pages |
2024-05-05 18:52:26 0 [Note] InnoDB: Parsed redo log up to LSN=162947230932576; to recover: 672094 pages |
Notable point is that 2147483648 - 2147479552 = 4096
2G is 2147483648 (so 2147479552+4096).
We have following settings in configuration file:
innodb_log_buffer_size = 2G
|
innodb_log_file_size = 64G
|
As an experiment, we tried to replace the value in log file to:
innodb_log_buffer_size = 2147479552 |
and surprisingly the warning doesn't appear anymore!
While it's not completely clear for us yet whether this triggers any other issues and whether the problem is just for some parameters or global, but from this mentioned case it seems that metric prefixes are misinterpreted by mariadb, at least for innodb parameters, that is already significant issue itself