[MDEV-33272] innodb_fatal_semaphore_wait_threshold was exceeded for dict_sys.latch Created: 2024-01-18 Updated: 2024-01-22 |
|
| Status: | Needs Feedback |
| Project: | MariaDB Server |
| Component/s: | Server, Storage Engine - InnoDB |
| Affects Version/s: | 10.11.5 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major |
| Reporter: | ASHWANI SINGH | Assignee: | Marko Mäkelä |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | None | ||
| Environment: |
Linux |
||
| Attachments: |
|
||||||||
| Issue Links: |
|
||||||||
| Description |
|
Hi Team, We are facing issue with MariaDB Server 10.11.5 which is getting crashed automatically with error Couldn't start addr2line. Please find attached mysqld.log & my.cnf |
| Comments |
| Comment by Daniel Black [ 2024-01-18 ] | ||||
|
Hi, Thanks for the bug report. Failing to start addr2line isn't as big a problem (it just may not be installed). You've got something in the previous message "InnoDB: innodb_fatal_semaphore_wait_threshold was exceeded for dict_sys.latch.". To help us try to work out how this occurred to you can you:
Then include its output as an attachment here. | ||||
| Comment by ASHWANI SINGH [ 2024-01-18 ] | ||||
|
Hi Daniel, We tried to run thread apply all bt -frame-arguments all full inside gdb prompt but getting error mentioned below: Thread 71 (LWP 151687): But when we executed thread apply all bt we got some output which am attaching as attachment. | ||||
| Comment by Daniel Black [ 2024-01-22 ] | ||||
|
Thanks for the information and getting past the frame-arguments difference. Its hard to remember sometimes which OS has the gdb late enough to support this. set print frame-arguments all was the version compatible with older versions. Seems the 2G limit applied somewhere (possibly systemd-coredump settings) to the coredump has missed the pages most related to the assertion. Apparently ~20G was expected. This obviously is excluding your 300G innodb buffer pool (mariadb excludes this), part of which might be required to resolve this. If you do notice this before it asserts ([https://mariadb.com/kb/en/innodb-system-variables/#innodb_fatal_semaphore_wait_threshold|innodb_fatal_semaphore_wait_threshold] defaults to 10 minutes), and you have space gcore (from gdb package) can capture the full process memory. If you have the ability to find and raise this limit before the next occurrence this would be appreciated. Advice on where this is set will help us update our documentation. |