[MDEV-23116] mariabackup doesn't recognize encryption options provided on the command line Created: 2020-07-07  Updated: 2021-08-03

Status: Stalled
Project: MariaDB Server
Component/s: mariabackup
Affects Version/s: 10.4, 10.5
Fix Version/s: 10.4, 10.5

Type: Bug Priority: Major
Reporter: Elena Stepanova Assignee: Vladislav Lesin
Resolution: Unresolved Votes: 3
Labels: regression

Issue Links:
Problem/Incident
is caused by MDEV-18215 mariabackup does not report unknown c... Closed
Relates
relates to MDEV-23696 mariabackup options processing code r... Open

 Description   

Till recently (I assume until MDEV-18215) one could override encryption options specified in the generated cnf file from the command line. It is convenient, for example, when you move the backup between hosts or environments, and paths change. It could look like this (notice file-key-management-filename treatment):

10.4 9d479e25

 
$ /data/bld/10.4-debug/bin/mariabackup --prepare --use-memory=1073741824  --innodb-file-io-threads=1 --target-dir=`pwd`/mdev23115/backup_before_prepare_0 --file-key-management-filename=`pwd`/mysql-test/std_data/keys.txt --plugin-dir=`pwd`/lib/plugin
 
Warning: option 'innodb_file_io_threads': signed value 1 adjusted to 4
 
/data/bld/10.4-debug/bin/mariabackup based on MariaDB server 10.4.14-MariaDB Linux (x86_64)
 
[00] 2020-07-07 21:13:56 cd to /data/bld/10.5-debug-nightly/mdev23115/backup_before_prepare_0/
 
[00] 2020-07-07 21:13:56 Loading encryption plugin from file_key_management=file_key_management
 
[00] 2020-07-07 21:13:56 Loading encryption plugin
 
[00] 2020-07-07 21:13:56 	 Encryption plugin parameter :  '--plugin_load=file_key_management=file_key_management'
 
[00] 2020-07-07 21:13:56 	 Encryption plugin parameter :  '--file_key_management_encryption_algorithm=aes_cbc'
 
[00] 2020-07-07 21:13:56 	 Encryption plugin parameter :  '--file_key_management_filekey='
 
[00] 2020-07-07 21:13:56 	 Encryption plugin parameter :  '--file_key_management_filename=/xxhome/mdbe/data/file_key_management_keys.txt'
 
[00] 2020-07-07 21:13:56 	 Encryption plugin parameter :  '--innodb_encrypt_log=1'
 
[00] 2020-07-07 21:13:56 	 Encryption plugin parameter :  '--prepare'
 
[00] 2020-07-07 21:13:56 	 Encryption plugin parameter :  '--use-memory=1073741824'
 
[00] 2020-07-07 21:13:56 	 Encryption plugin parameter :  '--target-dir=/data/bld/10.5-debug-nightly/mdev23115/backup_before_prepare_0'
 
[00] 2020-07-07 21:13:56 	 Encryption plugin parameter :  '--file-key-management-filename=/data/bld/10.5-debug-nightly/mysql-test/std_data/keys.txt'
 
...
 
[00] 2020-07-07 21:13:58 completed OK!

Now I don't seem to be able to do it anymore.
If I try to put it on the command line as a "regular" mariabackup option, it rejects it:

10.5 ee584137

 
$ /data/bld/10.5-debug-nightly/bin/mariabackup --prepare --use-memory=1073741824  --innodb-file-io-threads=1 --target-dir=`pwd`/mdev23115/backup_before_prepare_0 --file-key-management-filename=`pwd`/mysql-test/std_data/keys.txt --plugin-dir=`pwd`/lib/plugin
 
Warning: option 'innodb_file_io_threads': signed value 1 adjusted to 4
 
Warning: option 'innodb_file_io_threads': signed value 1 adjusted to 4
 
/data/bld/10.5-debug-nightly/bin/mariabackup: unknown variable 'file-key-management-filename=/data/bld/10.5-debug-nightly/mysql-test/std_data/keys.txt'

And if I put it after mysqld-args, it gets ignored and the one from the cnf file is still used:

$ /data/bld/10.5-debug-nightly/bin/mariabackup --prepare --use-memory=1073741824  --innodb-file-io-threads=1 --target-dir=`pwd`/mdev23115/backup_before_prepare_0 --plugin-dir=`pwd`/lib/plugin --mysqld-args --file-key-management-filename=`pwd`/mysql-test/std_data/keys.txt 
Warning: option 'innodb_file_io_threads': signed value 1 adjusted to 4
 
Warning: option 'innodb_file_io_threads': signed value 1 adjusted to 4
 
/data/bld/10.5-debug-nightly/bin/mariabackup based on MariaDB server 10.5.5-MariaDB Linux (x86_64)
 
[00] 2020-07-07 21:22:15 cd to /data/bld/10.5-debug-nightly/mdev23115/backup_before_prepare_0/
 
[00] 2020-07-07 21:22:15 Loading encryption plugin from file_key_management=file_key_management
 
[00] 2020-07-07 21:22:15 Loading encryption plugin
 
[00] 2020-07-07 21:22:15 	 Encryption plugin parameter :  '--plugin_load=file_key_management=file_key_management'
 
[00] 2020-07-07 21:22:15 	 Encryption plugin parameter :  '--file_key_management_encryption_algorithm=aes_cbc'
 
[00] 2020-07-07 21:22:15 	 Encryption plugin parameter :  '--file_key_management_filekey='
 
[00] 2020-07-07 21:22:15 	 Encryption plugin parameter :  '--file_key_management_filename=/xxhome/mdbe/data/file_key_management_keys.txt'
 
[00] 2020-07-07 21:22:15 	 Encryption plugin parameter :  '--innodb_encrypt_log=1'
 
[00] 2020-07-07 21:22:15 	 Encryption plugin parameter :  '--prepare'
 
[00] 2020-07-07 21:22:15 	 Encryption plugin parameter :  '--use-memory=1073741824'
 
[00] 2020-07-07 21:22:15 	 Encryption plugin parameter :  '--target-dir=/data/bld/10.5-debug-nightly/mdev23115/backup_before_prepare_0'
 
/data/bld/10.5-debug-nightly/bin/mariabackup: Can't find file: '/xxhome/mdbe/data/file_key_management_keys.txt' (errno: 2 "No such file or directory")



 Comments   
Comment by Vladislav Lesin [ 2020-07-15 ]

Encryption plugin is initialized from xtrabackup_prepare_func():

▾ encryption_plugin_init                                      
  ▾ encryption_plugin_prepare_init
 
    ▾ xtrabackup_prepare_func                                                   
      ▾ main_low
 
        • main

But the options are parsed before xtrabackup_prepare_func() call, in handle_options(), which is invoked from main().

The fix can be in moving encryption_plugin_prepare_init() call from xtrabackup_prepare_func() to handle_options().

Comment by Sergei Golubchik [ 2021-08-03 ]

It was apparently caused by

commit 9bdf35e90f3
 
Author: Vlad Lesin <vladislav.lesin@mariadb.com>
 
Date:   Mon Jun 8 11:45:56 2020 +0300
 
 
 
    MDEV-18215: mariabackup does not report unknown command line options
 
    MDEV-21298: mariabackup doesn't read from the [mariadbd] and [mariadbd-X.Y]
 
    server option groups from configuration files
 
    MDEV-21301: mariabackup doesn't read [mariadb-backup] option group in
 
    configuration file
 
    
    All three issues require to change the same code, that is why their
 
    fixes are joined in one commit.
 
    
    The fix is in invoking load_defaults_or_exit() and handle_options() for
 
    backup-specific groups separately from client-server groups to let the last
 
    handle_options() call fail on unknown backup-specific options.
 
    
    The order of options procesing is the following:
 
    1) Load server groups and process server options, ignore unknown
 
    options
 
    2) Load client groups and process client options, ignore unknown
 
    options
 
    3) Load backup groups and process client-server options, exit on
 
    unknown option
 
    4) Process --mysqld-args command line options, ignore unknown options
 
...

Generated at Thu Feb 08 09:19:58 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.