[MDEV-13190] InnoDB write-only performance regression Created: 2017-06-27 Updated: 2021-11-23 |
|
| Status: | Open |
| Project: | MariaDB Server |
| Component/s: | Storage Engine - InnoDB |
| Affects Version/s: | 10.2.2 |
| Fix Version/s: | 10.2 |
| Type: | Bug | Priority: | Major |
| Reporter: | Axel Schwenke | Assignee: | Marko Mäkelä |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | None | ||
| Attachments: |
|
||||||||||||
| Issue Links: |
|
||||||||||||
| Description |
|
Starting with MariaDB 10.2.2 there is a heavy performance regression for write-only workload and low thread counts.
Benchmark: sysbench OLTP with --oltp_simple_ranges=0 --oltp-distinct-ranges=0 --oltp-sum-ranges=0 --oltp-order-ranges=0 --oltp-point-selects=0 --oltp_non_index_updates=9 --oltp_index_updates=9. 32 tables with 10 mio rows total (yields ~2.5GB tablespaces). Datadir residing on RAID-0 on two SSD. my.cnf
This issue popped up during work on |
| Comments |
| Comment by Marko Mäkelä [ 2017-07-05 ] | ||||||||||||||||||||||||||||||||
|
I believe that this is simply caused by the merge of MySQL 5.7.9 into MariaDB 10.2.2. | ||||||||||||||||||||||||||||||||
| Comment by Marko Mäkelä [ 2021-10-25 ] | ||||||||||||||||||||||||||||||||
|
I am assigning this back to me. I wonder how much we have improved since those days. I think that already 10.5.12 or 10.6.4 should perform considerably better, and 10.6.5 includes some more improvements. | ||||||||||||||||||||||||||||||||
| Comment by Marko Mäkelä [ 2021-10-26 ] | ||||||||||||||||||||||||||||||||
|
Today, I observed 130ktps at 16 threads on 10.6 when testing the Linux kernel io_uring hang ( | ||||||||||||||||||||||||||||||||
| Comment by Axel Schwenke [ 2021-11-17 ] | ||||||||||||||||||||||||||||||||
|
Things got no better in recent releases:
| ||||||||||||||||||||||||||||||||
| Comment by Axel Schwenke [ 2021-11-17 ] | ||||||||||||||||||||||||||||||||
|
Attached a summary of all InnoDB-related benchmarks from the regression suite for 10.2.1, 10.2.2 and 10.6.5 - MDEV-13190.pdf | ||||||||||||||||||||||||||||||||
| Comment by Axel Schwenke [ 2021-11-22 ] | ||||||||||||||||||||||||||||||||
|
I have rerun the benchmarks using the current kernel (with mitigations against SPECTRE & co) and have also recompiled everything, so using the same compiler and libraries. I still see an increase in latency and accordingly a decrease in performance at low thread counts for 10.2.2 (vs. 10.2.1). 10.6.5 looks quite good in direct comparison. For read-only it is better du to faster collation algorithm. For writes 10.6 is just better. Since the regression vanishes at higher load, it seems like the extra time is spent waiting in the kernel with the user threads suspended. Attached: sysbench_de.pdf | ||||||||||||||||||||||||||||||||
| Comment by Marko Mäkelä [ 2021-11-22 ] | ||||||||||||||||||||||||||||||||
|
Thank you. Some regression for updates at low thread counts is still present. I hope that applying https://www.brendangregg.com/offcpuanalysis.html in a similar way as in The regression for point selects happens at any thread count. I wonder if using the READ UNCOMMITTED isolation level would make it go away. Possibly MDEV-21423 could address this. | ||||||||||||||||||||||||||||||||
| Comment by Axel Schwenke [ 2021-11-23 ] | ||||||||||||||||||||||||||||||||
|
Tested with transaction-isolation = 'READ-UNCOMMITTED'. For point-selects the performance didn't change at all, for read-only it became actually slower. Attached: sysbench_read_uncommitted.pdf |