[MDEV-25866] Upgrade from pre-10.5.10 to 10.5.10 causes CHECK errors on encrypted Aria tables Created: 2021-06-06  Updated: 2021-06-07  Resolved: 2021-06-07

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - Aria, Upgrades
Affects Version/s: 10.5
Fix Version/s: 10.5.11

Type: Bug Priority: Critical
Reporter: Elena Stepanova Assignee: Michael Widenius
Resolution: Fixed Votes: 0
Labels: regression

Issue Links:
Relates
relates to MDEV-17913 Encrypted transactional Aria tables r... Closed

 Description   
  • bootstrap 10.5.9 with aria-encrypt-tables=1, e.g. 

    --aria-encrypt-tables=1 --file-key-management --file-key-management-filename=`pwd`/mysql-test/std_data/keys.txt --plugin-load-add=file_key_management

  • start 10.5.10 on the datadir from 10.5.9;
  • run CHECK TABLE `mysql`.`tables_priv`

    10.5 8426c741

    		 
    MariaDB [(none)]> CHECK TABLE `mysql`.`tables_priv`;
    		 
    +-------------------+-------+----------+----------------------------------------------------------------------------------------------------------------------------------+
    		 
    | Table             | Op    | Msg_type | Msg_text                                                                                                                         |
    		 
    +-------------------+-------+----------+----------------------------------------------------------------------------------------------------------------------------------+
    		 
    | mysql.tables_priv | check | error    | Page at 8192 as wrong LSN (3971177,0x66334873). Current LSN is (1,0x5a5b)                                                        |
    		 
    | mysql.tables_priv | check | error    | Page at 16384 as wrong LSN (11590737,0x19495cff). Current LSN is (1,0x5a5b)                                                      |
    		 
    | mysql.tables_priv | check | error    | Page 1:  Wrong LSN (9127271,0x327b23c6). Current LSN is (1,0x5a5b)                                                               |
    		 
    | mysql.tables_priv | check | error    | Table is probably from another system and must be zerofilled or repaired ('REPAIR TABLE table_name') to be usable on this system |
    		 
    | mysql.tables_priv | check | error    | Corrupt                                                                                                                          |
    		 
    +-------------------+-------+----------+----------------------------------------------------------------------------------------------------------------------------------+
    		 
    5 rows in set (0.012 sec)

The problem was introduced by this commit in 10.5.10:

commit 81258f14323e1d1ad0203bae93bc55a30d47c1b3 (HEAD)
 
Author: Monty <monty@mariadb.org>
 
Date:   Fri Apr 2 22:00:36 2021 +0300
 
 
 
    MDEV-17913 Encrypted transactional Aria tables remain corrupt after crash recovery, automatic repairment does not work
 
 
 
...
 
 
 
    Other things:
 
    - Extended Aria check_table() to check for wrong (= too big) LSN numbers.
 
    - If check_table() failed just because of wrong LSN or TRN numbers,
 
      a following repair table will just do a zerofill which is much faster.
 
    - Limit number of LSN errors in one check table to MAX_LSN_ERROR (10).
 
    - Removed old obsolete test of 'if (error_count & 2)'. Changed error_count
 
      and warning_count from bits to numbers of errors/warnings as this is
 
      more useful.

I don't know whether it is a real problem or the extended check produces false positives, but in any case corruption-like errors can cause troubles for users.

REPAIR makes the errors disappear.

 Comments   
Comment by Michael Widenius [ 2021-06-07 ]

The issue is that 10.5.9 did not store the LSN (log number) for encrypted in the pages but instead a random number.

The effect of this is:

  • Automatic repair after crash on encrypted tables will not work in 10.5.9
  • The new check in 10.5.10 will check the LSN numbers and report errors (as these are wrong)

10.5.11 fixes the LSN number is stored correctly in all pages also for encrypted tables. CHECK TABLE now checks also for this kind of errors, which is what is shown in this Jira entry.
There is no way to fix the issue in code. What should be done is that anyone using encrypted Aria tables should run a repair on all of them after upgrading to 10.5.9:

REPAIR TABLE mysql.column_stats;
REPAIR TABLE mysql.columns_priv;
REPAIR TABLE mysql.db;
REPAIR TABLE mysql.event;
REPAIR TABLE mysql.func;
REPAIR TABLE mysql.global_priv;
REPAIR TABLE mysql.help_category;
REPAIR TABLE mysql.help_keyword;
REPAIR TABLE mysql.help_relation;
REPAIR TABLE mysql.help_topic;
REPAIR TABLE mysql.index_stats;
REPAIR TABLE mysql.plugin;
REPAIR TABLE mysql.proc;
REPAIR TABLE mysql.procs_priv;
REPAIR TABLE mysql.proxies_priv;
REPAIR TABLE mysql.roles_mapping;
REPAIR TABLE mysql.servers;
REPAIR TABLE mysql.table_stats;
REPAIR TABLE mysql.tables_priv;
REPAIR TABLE mysql.time_zone;
REPAIR TABLE mysql.time_zone_leap_second;
REPAIR TABLE mysql.time_zone_name;
REPAIR TABLE mysql.time_zone_transition;

The main effect if the above repairs are not done is that CHECK TABLE will report errors for the not updated tables. Repair of the above tables after a crash will work in most cases but not necessary in all cases until repair is done.

In theory we could fix that mysql_upgrade would notice this and automatically does the above.

Comment by Michael Widenius [ 2021-06-07 ]

Fixed that mysql_upgrade will repair old encrypted Aria tables

Generated at Thu Feb 08 09:40:59 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.