[MDEV-21418] [Warning] InnoDB: A long semaphore wait Created: 2020-01-03  Updated: 2020-03-12  Resolved: 2020-03-12

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 10.2.22
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Dermot Brereton Assignee: Unassigned
Resolution: Cannot Reproduce Votes: 0
Labels: need_feedback, semaphore
Environment:

Ubuntu 16.04.6 LTS

Attachments: HTML File syslog     File syslog.1    
Issue Links:
Relates
relates to MDEV-13983 Mariadb becomes unresponsive Closed
relates to MDEV-16467 MariaDB crashes because of "long sema... Closed

 Description   

"[Warning] InnoDB: A long semaphore wait" messages logged in /var/log syslog.

MariaDB eventually crashes and restarts;

mysqld[926]: where mysqld died. If you see no messages after this, something went
Jan 3 04:39:17 localhost mysqld[926]: terribly wrong...

 Comments   
Comment by Dermot Brereton [ 2020-01-03 ] 

MariaDB [(none)]> select @@innodb_adaptive_hash_index;
 
+------------------------------+
 
| @@innodb_adaptive_hash_index |
 
+------------------------------+
 
|                            1 |
 
+------------------------------+

Comment by Marko Mäkelä [ 2020-01-03 ]

There are know problems with innodb_adaptive_hash_index; see MDEV-20487 and related bugs.

Dermot.Brereton, if you see hangs with innodb_adaptive_hash_index=0, please run

gdb -ex "set pagination 0" -ex "thread apply all bt" -ex "print dict_operation_lock" -ex "print dict_sys->mutex" --batch -p $(pgrep -x mysqld)

and try to create and preserve a core dump for further analysis.

Comment by Dermot Brereton [ 2020-01-03 ]

I have no evidence from the attached logs that this issue is related to innodb_adaptive_hash_index

Do you recommend setting innodb_adaptive_hash_index = 0 (the current value is 1)?

Also based on the information provided in the attached syslog's is this issue similar to MDEV-13983?

Comment by Marko Mäkelä [ 2020-01-03 ]

Dermot.Brereton, it is hard to analyze hangs solely based on server error log output. It could easily take more than 1 hour of my time, and still the cause could remain unresolved. Stack traces of all threads are a much more convenient and reliable way of analyzing hangs. I cannot say for sure which hang reports are related, because I have not seen the requested gdb output in any of them.

My motivation to disable the adaptive hash index in MariaDB Server 10.5 by default (MDEV-20487) is the bugs that we are aware of, but have not had time to fix yet. In fact, I would have disabled it at compilation time, but we got some user and customer feedback that it can actually help the performance of read-mostly workloads.

Comment by Sergei Golubchik [ 2020-02-03 ]

Your crash could have the same underlying reason as in MDEV-16467 that was fixed in 10.2.25

Try upgrading

Comment by Dermot Brereton [ 2020-02-13 ]

Issue was resolved when the DRBD replication software was removed from the database server and the data file system created as ext4.

Comment by Dermot Brereton [ 2020-02-13 ]

This ticket can now be closed.

Generated at Thu Feb 08 09:06:59 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.