Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Fixed
-
10.1(EOL)
-
None
Description
We've ran into an unusual case where InnoDB Monitor output is flooding the error logs, and eventually leading to disk space issues.
Apparently these cases where the InnoDB Monitor was turned on was triggered by this kind of error message:
2017-12-19 23:38:26 7f5f729fc700 InnoDB: Warning: difficult to find free blocks in
|
InnoDB: the buffer pool (338 search iterations)!
|
InnoDB: 0 failed attempts to flush a page! Consider
|
InnoDB: increasing the buffer pool size.
|
InnoDB: It is also possible that in your Unix version
|
InnoDB: fsync is very slow, or completely frozen inside
|
InnoDB: the OS kernel. Then upgrading to a newer version
|
InnoDB: of your operating system may help. Look at the
|
InnoDB: number of fsyncs in diagnostic info below.
|
InnoDB: Pending flushes (fsync) log: 0; buffer pool: 0
|
InnoDB: 105756 OS file reads, 62064 OS file writes, 16768 OS fsyncs
|
InnoDB: Starting InnoDB Monitor to print further
|
InnoDB: diagnostics to the standard output.
|
Once this occurs, innodb monitor seems to get switched on permanently here, and there does not seem to be a way to stop it that point.
A restart seems to "fix" it temporarily.
Also, explicitly setting innodb_status_output to OFF seems to prevent it (after a restart).
Unlike with long semaphore waits monitor output is enabled permanently here, not just for 30s like with long semaphore waits?
What exactly should we look for in the generated monitor output to track down the root cause?
Attachments
Issue Links
- duplicates
-
MDEV-12327 InnoDB status monitor gets enabled after server startup
- Closed
-
MDEV-12557 INNODB MONITOR OUTPUT write every minutes old messages to error log
- Closed
- relates to
-
MDEV-11215 Several locks taken to same record inside a transaction.
- Stalled
-
MDEV-16339 Upgrading to 10.1.32 shows innodb_empty_free_list_algorithm=BACKOFF as default when it should be 'LEGACY'
- Closed
-
MDEV-16617 After upgrading from 5.5.52 to 10.1.32 getting [Warning] InnoDB: Difficult to find free blocks in the buffer pool
- Closed