[MDEV-23379] Deprecate and ignore options for InnoDB concurrency throttling Created: 2020-08-03  Updated: 2023-05-09  Resolved: 2020-08-04

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 10.5, 10.6
Fix Version/s: 10.5.5

Type: Bug Priority: Minor
Reporter: Rick Pizzi Assignee: Marko Mäkelä
Resolution: Fixed Votes: 4
Labels: performance

Issue Links:
Blocks
blocks MDEV-23397 Remove deprecated InnoDB options in 10.6 Closed
Relates
relates to MDEV-23382 Change DB_ROLL_PTR format to allow mo... Open
relates to MDEV-25883 Galera Cluster hangs while "DELETE FR... Closed
relates to MDEV-11382 AliSQL: [Feature] Issue#9: PROVIDE AD... Closed
relates to MDEV-14462 Confusing error message: ib_logfiles ... Closed
relates to MDEV-23369 False sharing in page_hash_latch::rea... Closed
relates to MDEV-24759 Server hangs when innodb_thread_concu... Closed
relates to MDEV-31220 MariaDB:performance seems to be limit... Closed

 Description   

The variable innodb_thread_concurrency was useful years ago when computing resources were limited, but nowadays setting this value to anything than unlimited (0) makes little sense and actually makes things worse ,because queries have to wait outside InnoDB.

We have seen many customers mistakenly setting this to a small value like 16 or 64 and then complaining the server was slow.

In the latest instantation of the above problem I've seen (just today), the available slots were all taken by long running transactions (left open) and the server was stalled as a result.

So, a dangerous config option and IMHO a good candidate for deprecation/removal.

Thanks
Rick



 Comments   
Comment by Marko Mäkelä [ 2020-08-03 ]

Funny, I was thinking exactly the same yesterday, after fixing MDEV-23369.

With the MDEV-23369 fix in 10.5, we seem to scale rather well to an insane number of concurrent threads. I tested 1000 connections on a server where nproc reports 56. That would be at least 1033 threads on the server, and at least 1000 more threads for the client workload that was running on the same server.

Yes, there is an inherent limit caused by there being only 128 rollback segments (the component in DB_ROLL_PTR is only 7 bits). The maximum write scalability ought to be reached with innodb_undo_tablespaces=128. But, we do not see a dramatic drop of the total throughput when we add more threads.

Hence, the ‘thread throttling’ at the high level by innodb_thread_concurrency or innodb_commit_concurrency does not seem to make much sense, and those parameters should probably be deprecated and ignored starting with the 10.6 release.

Comment by Marko Mäkelä [ 2020-08-03 ]

I would deprecate and ignore and hard-wire to 0 the following parameters:

  • innodb_thread_concurrency
  • innodb_commit_concurrency
  • innodb_replication_delay
  • innodb_concurrency_tickets
  • innodb_thread_sleep_delay
  • innodb_adaptive_max_sleep_delay

Also, the column INFORMATION_SCHEMA.INNODB_TRX.trx_concurrency_tickets would become the constant 0.

Comment by Marko Mäkelä [ 2020-08-04 ]

We will deprecate and ignore these parameters in MariaDB 10.5.5 Server and remove in 10.6, declaring them as MARIADB_REMOVED_OPTION for compatibility with old configuration files.

Generated at Thu Feb 08 09:21:57 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.