[MDEV-30271] Mysqld crash - Semaphore wait has lasted > 600 seconds Created: 2022-12-19 Updated: 2023-02-20 Resolved: 2023-02-20 |
|
| Status: | Closed |
| Project: | MariaDB Server |
| Component/s: | Locking, Platform RedHat |
| Affects Version/s: | 10.4.26 |
| Fix Version/s: | N/A |
| Type: | Bug | Priority: | Major |
| Reporter: | Anton | Assignee: | Unassigned |
| Resolution: | Cannot Reproduce | Votes: | 0 |
| Labels: | None | ||
| Attachments: |
|
| Description |
|
Hi ,
dmesg had next records
Some variables
I found that the same issue was in We rapidly almost reached limit of connections , but strange that this is caused a lot of locks waiting and as result - segfault I've uploaded archive with mysqld log and system variables - |
| Comments |
| Comment by Daniel Black [ 2022-12-20 ] |
|
Hi R, for these kinds of problems is critical to get the full backtrace of all threads. If this hang did generate a core dump, can you extract the all threads with these instructions. |
| Comment by Anton [ 2022-12-20 ] |
|
Hi danblack I can do backtrace of current running service , but it would be useless I think. |
| Comment by Daniel Black [ 2022-12-21 ] |
|
> I can do backtrace of current running service , but it would be useless I think. You're right, it would need to be captured in the 600 seconds between when it occurred and the mariadb assertion that causes it termination. While gcore or the gdb scripting could grab a sample every 400ish seconds it would be a bit wasteful. I don't think core_file is the limiting factor. Is there a system LimitCore in effect (https://access.redhat.com/solutions/649193)? Is the kernel.core_pattern sysctl set? Does /proc/$(pidof mariabd)/limits limit the core? > Can I do something else for investigation of the problem ? Install the debuginfo packages for when it does occur. If you get stuck with a backtrace you can upload the core to ftp. Please note if you're using a RHEL package or a mariadb package. By default the buffer pool isn't included in the dump so there will be none or very little user data in the dump. Mainly prepare for when a core dump occurs is the most useful thing. There just isn't sufficient information exposed elsewhere. |
| Comment by Anton [ 2023-02-20 ] |
|
Hi danblack The problem didn't appear again for a long time |