[MDEV-30182] Optimize open_tables to take O(N) time Created: 2022-12-09  Updated: 2023-11-05

Status: Open
Project: MariaDB Server
Component/s: None
Fix Version/s: 11.1

Type: Task Priority: Major
Reporter: Nikita Malyavin Assignee: Nikita Malyavin
Resolution: Unresolved Votes: 0
Labels: None

Issue Links:
Relates
relates to MDEV-22180 Planner opens unnecessary tables when... Closed

 Description   

See also MDEV-22180 and MDEV-30103 (comment).

find_fk_prelocked_table traverses all opened tables in connection, MDL_context::find_ticket traverses all mdl tickets which is also not less amount of steps. Both functions are called for each table, taking O(tables^2) time total, during opening tables.

Opening a table for an operation like UPDATE or DELETE for a table with 12k child tables takes about 3.5 seconds on my laptop. Making the two mentioned fuinctions noop takes 0.15 seconds.

What should be done

A list traversal should be changed to a hash lookup when a number of tables to open is big.
Hashing is an expensive operation, so for a smaller number of tables (an exact value is to be determined) it should be left as a list traversal.

We can't predict an exact number of tables to open preliminarily: it grows dynamically, once the tables are opened (and triggers/FK relations are checked), so we need a data structure which will also dynamically adapt.



 Comments   
Comment by Nikita Malyavin [ 2023-03-06 ]

It's now in progress by Sergey Vanislavskiy, a FEFU student. So I don't expect it to be done before July

Comment by Nikita Malyavin [ 2023-10-19 ]

As it was found, another use case is DROP DATABASE, which is very slow for tables over 10K table databases

Generated at Thu Feb 08 10:14:17 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.