[MDEV-21785] sequences used as default by other table not dumped in right order by mysqldump Created: 2020-02-20  Updated: 2021-04-19  Resolved: 2021-01-26

Status: Closed
Project: MariaDB Server
Component/s: Backup, Sequences
Affects Version/s: 10.4.12
Fix Version/s: 10.3.28, 10.4.18, 10.5.9

Type: Bug Priority: Critical
Reporter: Hartmut Holzgraefe Assignee: Oleksandr Byelkin
Resolution: Fixed Votes: 1
Labels: mysqldump


 Description   

mysqldump treads sequences the same as other tables, dumping all of them in alphabetical order. A sequence may provide the default value for a column on another table though. So if the table name alphabetically comes before the sequence name, the tables CREATE statement will be in the dump output before the CREATE for the sequence table, and so the table creation will fail as it refers to a sequence not created yet.

How to reproduce:

  • create sequence and table:

CREATE SEQUENCE b;
CREATE TABLE a (id INT PRIMARY KEY DEFAULT (NEXT VALUE FOR b));

  • dump database with mysqldump, then restore into an empty database
  • restore will fail when trying to create table `a`:

ERROR 1146 (42S02) at line ...: Table 'test.b' doesn't exist



 Comments   
Comment by Elena Stepanova [ 2020-02-20 ]

I can add to that. In such a case, SHOW CREATE TABLE a refers to b by fully qualified name, e.g.:

CREATE TABLE `a` (
  `id` int(11) NOT NULL DEFAULT nextval(`test`.`b`),
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1

So, if the dump is restored into a different database, there will be other problems – either an error if the sequence in the initial schema no longer exists, or the table will end up referencing a wrong sequence.

Something needs to be done (at least about the initially reported problem), but I don't know what should be a fix for either of this – either we should have a variable which temporarily suppresses the reference check, like it happens with foreign keys, or the tricky logic with fake temporary tables, as it happens with views, or recognizing sequences (which might also be needed due to MDEV-21786) and dumping them first, or something else.

Comment by Michael Widenius [ 2020-02-24 ]

Jani Tolonen once made a patch that would fix this by in the beginning of the dump create an inital sequence table and than at the end of the dump add the value to it. This was to ensure that at the end of the dump we will have the latest values of the sequence.

That code was never added to 10.3, but it should have. I think we should add it now as it will make this patch easier.

In any case, to first find the sequences so that we can dump them first, we should change mysqldump to instead of using "mysql_list_tables() to find the tables, instead use performance schema and get both the table name and the engine used.
This would enable us to first dump the sequence definition, then the tables and last the sequence values.
Note that sequence tables should not be part of any LOCK table statement as it's ok that these changes during the dump!

Comment by Hartmut Holzgraefe [ 2020-12-01 ]

"instead use performance schema and get both the table name and the engine used"

that would be INFORMATION_SCHEMA, not PERFORMANCE_SCHEMA, right? (we can't rely on P_S being enabled anyway)

Comment by Oleksandr Byelkin [ 2020-12-10 ]

monty hholzgra so we drop support for version earlier than 5.0.3 (first version with information schema). I hope it is OK?

Comment by Hartmut Holzgraefe [ 2020-12-10 ]

I'd be ok with that, 4.x has loooong been EOL, and whoever still has it running also has old mysqldump binary of the same version ...

Comment by Oleksandr Byelkin [ 2020-12-11 ]

It appeared that we already use Information schema inside, so it is only 5.0.3 and up compatible, so checks of the first information schema version inside looks a bit useless. Maybe check it once and issue warning/error is better solution...

Comment by Oleksandr Byelkin [ 2020-12-11 ]

OK, it was only for views, which was introduced in 5.0 also

Comment by Oleksandr Byelkin [ 2021-01-14 ]

commit 3e01da668d14b29f8b9f3f8dacb16f685d8c825e (HEAD -> bb-10.3-MDEV-21785, origin/bb-10.3-MDEV-21785)
Author: Oleksandr Byelkin <sanja@mariadb.com>
Date:   Wed Jan 13 18:36:48 2021 +0100
 
    MDEV-21785: sequences used as default by other table not dumped in right order by mysqldump
    
    Dump sequences first.
    
    This atch made to keep it small and
    to keep number of queries to the server the same.
    
    Order of tables in a dump can not be changed
    (except sequences first) because mysql_list_tables
    uses SHOW TABLES and I used SHOW FULL TABLES.

Generated at Thu Feb 08 09:09:46 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.