[MDEV-16223] Background ADD INDEX Created: 2018-05-20  Updated: 2021-09-21  Resolved: 2020-04-23

Status: Closed
Project: MariaDB Server
Component/s: Data Definition - Alter Table, Storage Engine - InnoDB
Fix Version/s: N/A

Type: Task Priority: Major
Reporter: Marko Mäkelä Assignee: Marko Mäkelä
Resolution: Won't Fix Votes: 4
Labels: None

Issue Links:
Relates
relates to MDEV-16264 Implement a common work queue for Inn... Closed
relates to MDEV-16281 Implement parallel CREATE INDEX, ALTE... Open
relates to MDEV-11655 Transactional data dictionary Open
relates to MDEV-11675 Lag Free Alter On Slave Closed

 Description   

MySQL 5.6 (and MariaDB 10.0) introduced online ADD INDEX, that is, creating a secondary index while the table is being modified. (It also introduced online table rebuild for cases like adding or dropping columns.)

MariaDB could extend this and support background ADD INDEX, that is, submit a request an index to be created, and do not care when it completes. Technically, the index would exist in the data dictionary, but queries would have to avoid using the index until the storage engine confirms that it is usable.

We cannot support ADD UNIQUE INDEX in the background, because the ability to flag duplicate key errors requires the index to exist. If concurrent INSERT or UPDATE opreations would introduce duplicate keys, that will have to result in the failure of the ADD UNIQUE INDEX operation.

Implementing this feature should require a new storage engine handler call for checking whether an index is usable, and some change of the InnoDB persistent data dictionary. Preferrably, I would first eliminate the InnoDB dictionary tables, such as SYS_INDEXES, and store the secondary index root page numbers and some status information within the .ibd file. This would allow crash-safe operation. (In case the server is killed before the index creation completes, on server restart the index creation would have to start from the beginning, possibly after an explicit statement to request it.)



 Comments   
Comment by Marko Mäkelä [ 2018-05-24 ]

It might be best to implement this after MDEV-16281 (parallel ADD INDEX) or MDEV-16264 (pool of background threads).

Comment by Marko Mäkelä [ 2020-04-23 ]

If there is no practical demand for a feature that is by design incompatible with transactions, we should rather not do it at all.

Comment by Rick James [ 2021-08-25 ]

A thought... Can ADD UNIQUE INDEX be performed in 2 steps?
1. ADD INDEX (in background)
2. Convert INDEX to UNIQUE INDEX

Would this speed up ADD UNIQUE enough to be worth doing?

Comment by Sergei Golubchik [ 2021-09-21 ]

The point of UNIQUE index is to maintain uniqueness. If it wasn't done when the index was built, the fastest was to check it is to rebuild the index.

The problem is not building index in the background. This is easy to do, just open a new connection and do your ALTER TABLE there, from other connections' point of view it'll be "in the background". Or do it with CREATE EVENT.

The problem is that it doesn't make a logical sense to add a constraint that starts working in some unspecified point in time in the future.

Generated at Thu Feb 08 08:27:18 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.