[MDEV-7317] Make an index ignorable to the optimizer Created: 2014-12-14  Updated: 2021-08-05  Resolved: 2021-03-04

Status: Closed
Project: MariaDB Server
Component/s: Data Definition - Alter Table, Data Definition - Create Table, Optimizer
Fix Version/s: 10.6.0

Type: Task Priority: Critical
Reporter: James Briggs Assignee: Varun Gupta (Inactive)
Resolution: Fixed Votes: 11
Labels: index, optimizer

Issue Links:
PartOf
includes MDEV-22199 Add VISIBLE attribute for indexes in ... Closed
Problem/Incident
causes MDEV-25075 Ignorable index makes the resulting C... Closed

 Description   

SYNTAX

An index can be ignored in the following way:

1) Adding IGNORED to the key specification in CREATE TABLE
Example

CREATE TABLE t1(a INT, key key1(a) IGNORE);

2) Adding IGNORED to the key specification in CREATE INDEX
Example

CREATE INDEX key1 on t1 (a) IGNORE;

3) Adding IGNORED to the key in the ALTER statement

New syntax: ALTER TABLE table_name ALTER INDEX key_name ignorability

ignorability:

IGNORE_SYM
NOT IGNORE_SYM

Example:

 ALTER TABLE t1 ADD INDEX key1 IGNORE;

Specifications

  • Store the information about the ignorability of the index in the FRM.
  • A primary index cannot be made IGNORABLE.
  • A non-nullable unique key that is promoted to a primary key cannot be made IGNORABLE.
  • CHANGES in the output of SHOW CREATE TABLE and SHOW INDEXES|KEYS to show if the index is IGNORED or not.
  • An ALTER INDEX operation should use the INPLACE algorithm by default.


 Comments   
Comment by James Briggs [ 2014-12-15 ]

Here's the syntax I've implemented in the parser:

\W
use test
drop table if exists t1;
create table t1 (id int, col1 char(3), key idx_id (id) visible) ENGINE=InnoDB DEFAULT CHARSET=latin1;
create index idx_col1 on t1 (col1) visible;
show create table t1;
insert into t1 (id, col1) values (1,'abc'), (2, 'def');
alter table t1 change index idx_col1 invisible;
alter table t1 change index idx_col1 visible;
alter table t1 change index idx_id invisible, change index idx_col1 invisible;
select id, col1 from t1 order by id;
drop table if exists t1;
show warnings;

mysql> \W
Show warnings enabled.
mysql> drop table if exists t1;
Query OK, 0 rows affected, 2 warnings (0.00 sec)
Note (Code 1051): Unknown table 'test.t1'
 
mysql> create table t1 (id int, col1 char(3), key idx_id (id) visible) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Query OK, 0 rows affected (0.20 sec)
 
mysql> create index idx_col1 on t1 (col1) visible;
Query OK, 0 rows affected (0.07 sec)
Records: 0  Duplicates: 0  Warnings: 0
 
mysql> show create table t1;
| Table | Create Table                                                                                                                                                            |
| t1    | CREATE TABLE `t1` (
  `id` int(11) DEFAULT NULL,
  `col1` char(3) DEFAULT NULL,
  KEY `idx_id` (`id`),
  KEY `idx_col1` (`col1`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 | 
1 row in set (0.00 sec)
 
mysql> insert into t1 (id, col1) values (1,'abc'), (2, 'def');
Query OK, 2 rows affected (0.01 sec)
Records: 2  Duplicates: 0  Warnings: 0
 
mysql> alter table t1 change index idx_col1 invisible;
Query OK, 0 rows affected (0.00 sec)
Records: 0  Duplicates: 0  Warnings: 0
 
mysql> alter table t1 change index idx_col1 visible;
Query OK, 0 rows affected (0.00 sec)
Records: 0  Duplicates: 0  Warnings: 0
 
mysql> alter table t1 change index idx_id invisible, change index idx_col1 invisible;
Query OK, 0 rows affected (0.00 sec)
Records: 0  Duplicates: 0  Warnings: 0
 
mysql> select id, col1 from t1 order by id;
+------+------+
| id   | col1 |
+------+------+
|    1 | abc  | 
|    2 | def  | 
+------+------+
2 rows in set (0.00 sec)
 
mysql> drop table if exists t1;
Query OK, 0 rows affected, 1 warning (0.03 sec)

Comment by James Briggs [ 2014-12-16 ]

The Oracle 11g documentation is fuzzy on whether invisible indexes are session or global or both. If anybody has any thoughts on that, please add a comment.

In MySQL, various index properties are scoped like this:

set foreign_key_checks (session or global)
alter table disable keys (MyISAM-only, global)
create temporary table (session)

Comment by luntik2012 [ 2018-10-28 ]

mysql has visible keyword now, so even not still broken mariadb 10.1 is incompatible with mysql

Comment by Colm Smyth [ 2018-12-28 ]

Upvoting this issue as it blocks compatibility with recent versions of MySQL Workbench, and as it is a straightforward feature that has already been prototyped.

Comment by Sergei Golubchik [ 2020-04-28 ]

colm_smyth, you can configure a target mysql version in workbench preferences. Set it to 5.7 to get a compatible dump.

Comment by Sergei Golubchik [ 2020-04-28 ]

varun, we cannot really call them INVISIBLE, because we already use the concept of invisibility for something completely different. Perhaps, "IGNORED"? That exactly explains what they are, and also matches the IGNORE INDEX syntax in the FROM clause.

Comment by Rick James [ 2020-05-04 ]

While I understand that IGNORE is a better keyword than INVISIBLE, I worry that MariaDB's incompatibility with MySQL will be made worse. It means that transferring some datasets between MariaDB and MySQL, in either direction, will lead to errors fixable only by hand-editing of the dump.

Perhaps a partial solution is for loading the file can accept either keyword. Or a commandline switch can allow for ignoring the conflicting keyword.

Note that Workbench raised the problem by adding [unnecessarily] "VISIBLE" when making dumps. "mysqldump" should not include VISIBLE.

Comment by Colm Smyth [ 2020-05-04 ]

There is no logical conflict between the concept of an invisible column (to SELECT) and an invisible (to the optimizer) index. If there is no syntactic issue, it would be better to use the same INVISIBLE keyword to maximise compatibility with MySQL as it is important for MariaDB to be able to simplify migration and adoption for former MySQL users.

Comment by Sergei Golubchik [ 2020-05-05 ]

rjasdfiii, VISIBLE is already silently parsed and ignored after MDEV-22199. Unless one actually tries to load a dump that has "invisible" indexes, there is no compatibility issue here. Perhaps we can do this "partial solution" or may them work only in some kind of "mysql" mode.

colm_smyth, unfortunately, there is. Currently invisible indexes are indexes that only index invisible columns. There can be columns so invisible to the user that even SHOW CREATE TABLE doesn't show then, and indexes that index them are not shown either. Although they're perfectly "visible" to the optimizer and it is allowed to use them to optimize the query.

Comment by Rick James [ 2020-05-05 ]

Thanks, Colm and Sergei. In addition to the invisible index on an invisible column, I can think of two others: The hidden 6-byte PK that InnoDB uses as a last resort. And the extra column added for InnoDB FULLTEXT.

That's 3 cases. Is there a list somewhere of all such odd indexes? Perhaps such a list would be made as a checklist on whether all cases are being handled for this feature request and other situations.

Comment by Varun Gupta (Inactive) [ 2020-05-06 ]

The patch is in the branch 10.5-mdev7317

Comment by Sergei Petrunia [ 2021-02-24 ]

Review input provided:
https://lists.launchpad.net/maria-developers/msg12543.html

Comment by Sergei Petrunia [ 2021-05-11 ]

Note:

From the Server Team Lead Call: The MySQL syntax to use "INVISIBLE" should be supported as an alias

This hasn't been implemented.

Comment by Sergei Petrunia [ 2021-08-05 ]

Hi ralf.gebhardt@mariadb.com, right. We did not add an optimizer switch flag for this.

Generated at Thu Feb 08 07:18:40 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.