Details
-
Bug
-
Status: Closed (View Workflow)
-
Blocker
-
Resolution: Fixed
-
10.6.1
Description
MDEV-17567 and MDEV-25180 changed something in InnoDB startup. We forgot to adjust Mariabackup accordingly.
Specifically, the command mariabackup --prepare --export is supposed to purge the history of old transactions, and also apply all logs.
Starting with MDEV-25180, InnoDB no longer treats specially any tables whose names start with #sql. The only exception are table names that start with #sql-ib. It is the responsibility of the SQL layer's DDL log recovery to drop any such tables, in case the server was killed (or a backup was made) while a table-rebuilding DDL operation was in progress.
The following patch proves that a regression was introduced. If the function (which would start the InnoDB purge of transaction history) would be invoked during mariabackup --prepare --export, the execution would be aborted due to assertion failure.
diff --git a/storage/innobase/handler/ha_innodb.cc b/storage/innobase/handler/ha_innodb.cc
|
index db5c8524776..73f7a1aa248 100644
|
--- a/storage/innobase/handler/ha_innodb.cc
|
+++ b/storage/innobase/handler/ha_innodb.cc
|
@@ -1954,6 +1954,7 @@ static int innodb_check_version(handlerton *hton, const char *path,
|
static void innodb_ddl_recovery_done(handlerton*)
|
{
|
ut_ad(!ddl_recovery_done);
|
+ ut_a(srv_operation != SRV_OPERATION_RESTORE_EXPORT);
|
ut_d(ddl_recovery_done= true);
|
if (!srv_read_only_mode && srv_operation == SRV_OPERATION_NORMAL &&
|
srv_force_recovery < SRV_FORCE_NO_BACKGROUND) |
We must ensure that ha_signal_ddl_recovery_done() will be invoked. Should it for some reason be unacceptable to invoke ddl_log_execute_recovery() in mariabackup --prepare --export, then we would have to invoke special code that would drop all #sql- tables from InnoDB, before executing ha_signal_ddl_recovery_done(). In any case, we must somehow compensate for the change that was made in MDEV-25666.
Note: Nothing must change in mariabackup outside --prepare --export. We do not want to ruin incremental backups by executing actions that would generate InnoDB redo log records.
Attachments
Issue Links
- causes
-
MDEV-25989 Crash (or hang) during first startup after restoring backup
-
- Closed
-
- includes
-
MDEV-25899 intermediate files operations are not protected by backup locks
-
- Closed
-
- is caused by
-
MDEV-25180 Atomic ALTER TABLE
-
- Closed
-
- relates to
-
MDEV-18184 Do not copy temporary files (prefixed with #sql-) to backup
-
- Closed
-
-
MDEV-24184 InnoDB RENAME TABLE recovery failure if names are reused
-
- Closed
-
-
MDEV-25180 Atomic ALTER TABLE
-
- Closed
-
-
MDEV-25666 Atomic DDL: InnoDB: Operating system error number 2, Could not find a valid tablespace file
-
- Closed
-
Activity
Field | Original Value | New Value |
---|---|---|
Link |
This issue is caused by |
Link |
This issue relates to |
Link |
This issue relates to |
Link |
This issue relates to |
Description |
Specifically, the command {{mariabackup --prepare --export}} is supposed to purge the history of old transactions, and also apply all logs. Starting with The following patch proves that a regression was introduced. If the function (which would start the InnoDB purge of transaction history) would be invoked during {{mariabackup --prepare --export}}, the execution would be aborted due to assertion failure. {code:diff} diff --git a/storage/innobase/handler/ha_innodb.cc b/storage/innobase/handler/ha_innodb.cc index db5c8524776..73f7a1aa248 100644 --- a/storage/innobase/handler/ha_innodb.cc +++ b/storage/innobase/handler/ha_innodb.cc @@ -1954,6 +1954,7 @@ static int innodb_check_version(handlerton *hton, const char *path, static void innodb_ddl_recovery_done(handlerton*) { ut_ad(!ddl_recovery_done); + ut_a(srv_operation != SRV_OPERATION_RESTORE_EXPORT); ut_d(ddl_recovery_done= true); if (!srv_read_only_mode && srv_operation == SRV_OPERATION_NORMAL && srv_force_recovery < SRV_FORCE_NO_BACKGROUND) {code} We must ensure that {{ha_signal_ddl_recovery_done()}} will be invoked. Should it for some reason be unacceptable to invoke {{ddl_log_execute_recovery()}} in {{mariabackup --prepare --export}}, then we would have to invoke special code that would drop all {{#sql-}} tables from InnoDB, before executing {{ha_signal_ddl_recovery_done()}}. In any case, we must somehow compensate for the change that was made in Note: Nothing must change in {{mariabackup}} outside {{--prepare --export}}. We do not want to ruin incremental backups by executing actions that would generate InnoDB redo log records. |
Specifically, the command {{mariabackup --prepare --export}} is supposed to purge the history of old transactions, and also apply all logs. Starting with The following patch proves that a regression was introduced. If the function (which would start the InnoDB purge of transaction history) would be invoked during {{mariabackup --prepare --export}}, the execution would be aborted due to assertion failure. {code:diff} diff --git a/storage/innobase/handler/ha_innodb.cc b/storage/innobase/handler/ha_innodb.cc index db5c8524776..73f7a1aa248 100644 --- a/storage/innobase/handler/ha_innodb.cc +++ b/storage/innobase/handler/ha_innodb.cc @@ -1954,6 +1954,7 @@ static int innodb_check_version(handlerton *hton, const char *path, static void innodb_ddl_recovery_done(handlerton*) { ut_ad(!ddl_recovery_done); + ut_a(srv_operation != SRV_OPERATION_RESTORE_EXPORT); ut_d(ddl_recovery_done= true); if (!srv_read_only_mode && srv_operation == SRV_OPERATION_NORMAL && srv_force_recovery < SRV_FORCE_NO_BACKGROUND) {code} We must ensure that {{ha_signal_ddl_recovery_done()}} will be invoked. Should it for some reason be unacceptable to invoke {{ddl_log_execute_recovery()}} in {{mariabackup --prepare --export}}, then we would have to invoke special code that would drop all {{#sql\-}} tables from InnoDB, before executing {{ha_signal_ddl_recovery_done()}}. In any case, we must somehow compensate for the change that was made in Note: Nothing must change in {{mariabackup}} outside {{--prepare --export}}. We do not want to ruin incremental backups by executing actions that would generate InnoDB redo log records. |
Link | This issue relates to MENT-1213 [ MENT-1213 ] |
Description |
Specifically, the command {{mariabackup --prepare --export}} is supposed to purge the history of old transactions, and also apply all logs. Starting with The following patch proves that a regression was introduced. If the function (which would start the InnoDB purge of transaction history) would be invoked during {{mariabackup --prepare --export}}, the execution would be aborted due to assertion failure. {code:diff} diff --git a/storage/innobase/handler/ha_innodb.cc b/storage/innobase/handler/ha_innodb.cc index db5c8524776..73f7a1aa248 100644 --- a/storage/innobase/handler/ha_innodb.cc +++ b/storage/innobase/handler/ha_innodb.cc @@ -1954,6 +1954,7 @@ static int innodb_check_version(handlerton *hton, const char *path, static void innodb_ddl_recovery_done(handlerton*) { ut_ad(!ddl_recovery_done); + ut_a(srv_operation != SRV_OPERATION_RESTORE_EXPORT); ut_d(ddl_recovery_done= true); if (!srv_read_only_mode && srv_operation == SRV_OPERATION_NORMAL && srv_force_recovery < SRV_FORCE_NO_BACKGROUND) {code} We must ensure that {{ha_signal_ddl_recovery_done()}} will be invoked. Should it for some reason be unacceptable to invoke {{ddl_log_execute_recovery()}} in {{mariabackup --prepare --export}}, then we would have to invoke special code that would drop all {{#sql\-}} tables from InnoDB, before executing {{ha_signal_ddl_recovery_done()}}. In any case, we must somehow compensate for the change that was made in Note: Nothing must change in {{mariabackup}} outside {{--prepare --export}}. We do not want to ruin incremental backups by executing actions that would generate InnoDB redo log records. |
Specifically, the command {{mariabackup --prepare --export}} is supposed to purge the history of old transactions, and also apply all logs. Starting with The following patch proves that a regression was introduced. If the function (which would start the InnoDB purge of transaction history) would be invoked during {{mariabackup --prepare --export}}, the execution would be aborted due to assertion failure. {code:diff} diff --git a/storage/innobase/handler/ha_innodb.cc b/storage/innobase/handler/ha_innodb.cc index db5c8524776..73f7a1aa248 100644 --- a/storage/innobase/handler/ha_innodb.cc +++ b/storage/innobase/handler/ha_innodb.cc @@ -1954,6 +1954,7 @@ static int innodb_check_version(handlerton *hton, const char *path, static void innodb_ddl_recovery_done(handlerton*) { ut_ad(!ddl_recovery_done); + ut_a(srv_operation != SRV_OPERATION_RESTORE_EXPORT); ut_d(ddl_recovery_done= true); if (!srv_read_only_mode && srv_operation == SRV_OPERATION_NORMAL && srv_force_recovery < SRV_FORCE_NO_BACKGROUND) {code} We must ensure that {{ha_signal_ddl_recovery_done()}} will be invoked. Should it for some reason be unacceptable to invoke {{ddl_log_execute_recovery()}} in {{mariabackup --prepare --export}}, then we would have to invoke special code that would drop all {{#sql\-}} tables from InnoDB, before executing {{ha_signal_ddl_recovery_done()}}. In any case, we must somehow compensate for the change that was made in Note: Nothing must change in {{mariabackup}} outside {{--prepare --export}}. We do not want to ruin incremental backups by executing actions that would generate InnoDB redo log records. |
Assignee | Sergei Golubchik [ serg ] | Michael Widenius [ monty ] |
Link |
This issue relates to |
Assignee | Michael Widenius [ monty ] | Sergei Golubchik [ serg ] |
Component/s | Storage Engine - InnoDB [ 10129 ] | |
Assignee | Sergei Golubchik [ serg ] | Vladislav Vaintroub [ wlad ] |
Summary | mariabackup --prepare --export fails to run some actions | Restoring a backup may result in garbage intermediate tables from DDL |
Assignee | Vladislav Vaintroub [ wlad ] | Marko Mäkelä [ marko ] |
Status | Open [ 1 ] | In Progress [ 3 ] |
Assignee | Marko Mäkelä [ marko ] | Vladislav Vaintroub [ wlad ] |
Status | In Progress [ 3 ] | In Review [ 10002 ] |
issue.field.resolutiondate | 2021-06-17 10:47:50.0 | 2021-06-17 10:47:50.346 |
Fix Version/s | 10.6.2 [ 25800 ] | |
Fix Version/s | 10.6 [ 24028 ] | |
Assignee | Vladislav Vaintroub [ wlad ] | Marko Mäkelä [ marko ] |
Resolution | Fixed [ 1 ] | |
Status | In Review [ 10002 ] | Closed [ 6 ] |
Link |
This issue duplicates |
Link |
This issue includes |
Link |
This issue duplicates |
Link |
This issue relates to |
Link |
This issue causes |
Link | This issue includes MENT-1213 [ MENT-1213 ] |
Workflow | MariaDB v3 [ 122448 ] | MariaDB v4 [ 159361 ] |
Link |
This issue relates to |