[MDEV-17380] innodb_flush_neighbors=ON should be ignored on SSD Created: 2018-10-05  Updated: 2023-06-01  Resolved: 2019-04-01

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: None
Fix Version/s: 10.4.4

Type: Bug Priority: Major
Reporter: Vladislav Vaintroub Assignee: Marko Mäkelä
Resolution: Fixed Votes: 0
Labels: defaults, performance

Attachments: Zip Archive benchmark.zip    
Issue Links:
Problem/Incident
causes MDEV-23526 InnoDB leaks memory for some static o... Closed
Relates
relates to MDEV-14533 Provide information_schema tables usi... Closed
relates to MDEV-15528 Avoid writing freed InnoDB pages Closed
relates to MDEV-22343 Remove SYS_TABLESPACES and SYS_DATAFILES Closed
relates to MDEV-23929 innodb_flush_neighbors is not being i... Closed
relates to MDEV-15528 Avoid writing freed InnoDB pages Closed
relates to MDEV-20400 Implement a way to query an InnoDB ta... Confirmed
relates to MDEV-31347 fil_ibd_create() may hijack the file ... Closed

 Description   

innodb_flush_neighbors was an optimization for rotating disk, to have larger IO size. It costs CPU, and it does not bring anything for the SSD. I think OFF is now a good default (and also it has good in the past couple of years)

See for example
https://stackoverflow.com/questions/40732595/is-there-any-configuration-option-for-innodb-that-increase-the-performance-for-s



 Comments   
Comment by Vladislav Vaintroub [ 2018-11-22 ]

Based on test benchmark, this might need some more investigation, on HDD it seems to bring a good improvement.
benchmark.zip contains the config file, and script to run

sysbench 0.4 parameters (update_index)

sysbench --test=oltp --oltp-table-size=50000000 --mysql-host=localhost --mysql-db=sbtest --mysql-user=root --mysql-password= --db-driver=mysql 
--mysql-table-engine=innodb --max-time=300 --oltp-test-mode=complex --oltp-read-only=off --max-requests=0 --num-threads=NUM_THREADS
--oltp-point-selects=0 --oltp-simple-ranges=0 --oltp-sum-ranges=0 --oltp-order-ranges=0 --oltp-distinct-ranges=0 ^
--oltp-index-updates=1 --oltp-delete-inserts=0 --oltp-non-index-updates=0  --oltp-skip-trx=on --oltp-dist-type=uniform  run

relevant config settings

innodb_buffer_pool_size=20G
innodb_log_file_size=6G
innodb_log_buffer_size=100M 
innodb_doublewrite=0
innodb_flush_log_at_trx_commit=2

Results for 24 users, in TPS, for the above run shows about 2x increase in throughput for HDD

HDD

innodb_flush_neighbors 1 user 24 users
0 3853.60 19301.41
1 6551.23 35736.78
Comment by Marko Mäkelä [ 2018-11-23 ]

On Linux since kernel version 2.6.29, the type of the medium can be detected by querying a file like /sys/block/sda/queue/rotational.

We could introduce a flag fil_space_t::is_rotational, and only allow innodb_flush_neighbors=ON to take effect on files that reside on rotational media. The same flag could also be used in MDEV-15528 to choose between punching holes and overwriting data with zero bytes.

Comment by Marko Mäkelä [ 2019-03-29 ]

axel, I hope that you can test bb-10.4-marko on Linux. On HDD, the setting innodb_flush_neighbors (default ON) should continue to have a similar impact that wlad reported above.
On SSD, the setting innodb_flush_neighbors should have no impact, that is, we should never flush nearby dirty pages.

On my Linux system, /sys/block listed only nvme0n1. I additionally tested after plugging in USB devices. Both were reported as /sys/block/sd*/queue/rotational=1 even though only one of them is a hard disk, and the other is a memory card reader. I have earlier tested that a SATA SSD was reported as non-rotational.

Comment by Marko Mäkelä [ 2020-09-01 ]

Quoting the commit message:

For diagnostics, we introduce the column ON_SSD to the table INFORMATION_SCHEMA.INNODB_TABLESPACES_SCRUBBING. The table INNODB_SYS_TABLESPACES might seem more appropriate, but its purpose is to reflect the contents of the InnoDB system table SYS_TABLESPACES, which we would like to remove at some point.

In MDEV-22343 we plan to remove the internal InnoDB tables SYS_TABLESPACES and SYS_DATAFILES.

I just realized that MDEV-15528 removed the view INFORMATION_SCHEMA.INNODB_TABLESPACES_SCRUBBING altogether from 10.5. It would allow us to fix scrubbing (MDEV-8139) by making it part of the normal page flushing mechanism. Hence, no status information table was necessary.

ralf.gebhardt@mariadb.com and serg, perhaps we would like to retain INFORMATION_SCHEMA.INNODB_SYS_TABLESPACES even if we remove the system table in 10.6? We could include a column ON_SSD there already starting with 10.4, to allow smooth upgrades to 10.5 and later versions. Currently we basically have a regression in 10.5 that removed the ON_SSD attribute.

Comment by Marko Mäkelä [ 2020-09-23 ]

I think that we could export the ON_SSD attribute as part of MDEV-20400, in a new view INFORMATION_SCHEMA.INNODB_TABLES or similar that contains one row per table (or partition or subpartition).

A possible alternative might be to revive INFORMATION_SCHEMA.FILES, which was originally implemented for NDB, implemented in MySQL 5.7 for InnoDB’s CREATE TABLESPACE support, and reverted from MariaDB in MDEV-11426.

Comment by Marko Mäkelä [ 2020-09-23 ]

An even more natural alternative could be to extend the MDEV-14533 INFORMATION_SCHEMA.DISKS plugin with an Boolean column that discloses whether the device is rotational (traditional hard disk). After all, that is a property of storage media, not individual files.

Generated at Thu Feb 08 08:36:02 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.