[MDEV-27774] Reduce scalability bottlenecks in mtr_t::commit() Created: 2022-02-08  Updated: 2024-01-22  Resolved: 2022-02-10

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 10.8.1
Fix Version/s: 10.8.3

Type: Bug Priority: Major
Reporter: Marko Mäkelä Assignee: Marko Mäkelä
Resolution: Fixed Votes: 0
Labels: performance

Issue Links:
Problem/Incident
causes MDEV-27868 buf_pool.flush_list is in the wrong o... Closed
causes MDEV-27876 Server building fails on MacOS caused... Closed
causes MDEV-27969 buf_page_cleaner not getting signaled... Closed
causes MDEV-28043 Race condition between mtr_t::commit(... Closed
causes MDEV-32374 log_sys.lsn_lock is a performance hog Closed
Relates
relates to MDEV-27866 Switching log_sys.latch to use spin b... Closed
relates to MDEV-27910 Internal compiler error on CentOS 7 A... Closed
relates to MDEV-28111 Redo log writes are being buffered on... Closed
relates to MDEV-14425 Change the InnoDB redo log format to ... Closed
relates to MDEV-14462 Confusing error message: ib_logfiles ... Closed

 Description   

MDEV-14425 improved write performance less than it was hoped. We are still seeing log_sys.mutex and buf_pool.mutex dominating the wait times.

A prominent bottleneck was identified to be log_sys.mutex between log_sys.append_prepare() and log_close(). To protect the copying of data to log_sys.buf, we only need a shared log_sys.latch, which can replace log_sys.mutex and log_sys.flush_order_mutex. The log_sys.lsn and the position of log_sys.buf needs to be protected by a new log_sys.lsn_lock. Writes of log_sys.buf to the file system will have to acquire exclusive log_sys.latch.

While working on this code, we can also make mtr_t::commit() invoke buf_pool.page_cleaner_wakeup() and update Innodb_buffer_pool_write_requests at most once per call.



 Comments   
Comment by Marko Mäkelä [ 2022-02-10 ]

Thanks to wlad for the idea, and for further optimization suggestions.

Comment by Marko Mäkelä [ 2022-02-10 ]

greenman, please note that with this change, the minimum value of innodb_log_file_size will increase from 1M to 4M. That was done in order to remove a very unlikely startup error condition, as well as a very unlikely error message output from log_sys.append().

Comment by Ian Gilfillan [ 2022-02-11 ]

Thanks, updated

Generated at Thu Feb 08 09:55:29 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.