[MDEV-19015] mysql_plugin doesn't read all server option groups Created: 2019-03-21  Updated: 2019-04-25  Resolved: 2019-04-25

Status: Closed
Project: MariaDB Server
Component/s: Configuration, Scripts & Clients
Affects Version/s: 10.4.3, 10.1.38, 10.2.22, 10.3.13
Fix Version/s: 10.2.24, 10.1.39, 10.3.15, 10.4.5

Type: Bug Priority: Major
Reporter: Geoff Montee (Inactive) Assignee: Sergei Golubchik
Resolution: Fixed Votes: 0
Labels: beginner-friendly

Issue Links:
Relates
relates to MDEV-19227 mysql_plugin doesn't run bootstrap fr... Closed

 Description   

The mysql_plugin utility doesn't read all server option groups from the configuration file. It only reads the [mysqld] option group. This is because it executes the following:

my_print_defaults mysqld

Instead of the following:

my_print_defaults --mysqld

See the following function:

https://github.com/MariaDB/server/blob/3a7d7e180a871c5b2b2d98812d9d7f800cf33952/client/mysql_plugin.c#L314



 Comments   
Comment by Anel Husakovic [ 2019-04-08 ]

Hi I have done this:

anel@ubuntu:~/workspace/server/client$ ./mysql_plugin -v -f ../extra --plugin_ini=./rock.ini rocksdb ENABLE -P
./mysql_plugin would have been started with the following arguments:
# Found tool 'my_print_defaults' as '../extra/my_print_defaults'.
# Command: ../extra/my_print_defaults --mysqld > /tmp/txtqdKhB7
--datadir=/home/anel/workspace/datadir-10.4/ --basedir=/home/anel/workspace/server --plugin_dir=/home/anel/workspace/server/storage/rocksdb --plugin_ini=./rock.ini --my_print_defaults=../extra 

It is documented that mysql_plugin is using mysqld group., and I curious why would we need this, since only 3 arguments are read (basedir, datadir and plugindir) from config?
Is the motivation to have those parameters used in other groups ?
If this patch get applied we need to change this documentation also.
However I have a problem in applying this plugin on rocksdb and auth_socket(this is my first time to use this plugin), so can you please let me know how to proceed.
I'm working on 10.4. in debug mode.
1. rocksdb
I got the error that messagefile cannot be obtained in bootstrap mode:, although the plugin is loaded!

anel@ubuntu:~/workspace/server/client$ echo -e "ha_rocksdb\nrocksdb">>rock.ini
 
anel@ubuntu:~/workspace/server/client$ ./mysql_plugin -v -f ../extra --plugin_ini=./rock.ini rocksdb ENABLE 
# Found tool 'my_print_defaults' as '../extra/my_print_defaults'.
# Command: ../extra/my_print_defaults --mysqld > /tmp/txtmW8I4K
#    basedir = /home/anel/workspace/server
# plugin_dir = /home/anel/workspace/server/storage/rocksdb
#    datadir = /home/anel/workspace/datadir-10.4/
# plugin_ini = ./rock.ini
# Found tool 'mysqld' as '/home/anel/workspace/server/sql/mysqld'.
# Found plugin 'rocksdb' as '/home/anel/workspace/server/storage/rocksdb/ha_rocksdb.so'
# Enabling rocksdb...
# Query: REPLACE INTO mysql.plugin VALUES ('rocksdb','ha_rocksdb.so');
 
# Command: /home/anel/workspace/server/sql/mysqld --no-defaults --bootstrap --datadir=/home/anel/workspace/datadir-10.4/ --basedir=/home/anel/workspace/server < /tmp/sqlSOMLR9
2019-04-08  9:08:26 0 [Note] /home/anel/workspace/server/sql/mysqld (mysqld 10.4.5-MariaDB-debug) starting as process 76227 ...
2019-04-08  9:08:26 0 [Warning] Could not increase number of max_open_files to more than 1024 (request: 4186)
2019-04-08  9:08:26 0 [Warning] Changed limits: max_open_files: 1024  max_connections: 151 (was 151)  table_cache: 421 (was 2000)
2019-04-08  9:08:26 0 [ERROR] Can't find messagefile '/home/anel/workspace/server/share/errmsg.sys'
2019-04-08  9:08:26 0 [ERROR] Aborting
ERROR: Unexpected result from bootstrap. Error code: 256.

2. auth_socket
Following MDEV-18206 and documentation for authentication plugin
I cannot find auth_socket.so in order to use it in `plugin_dir`

anel@ubuntu:~/workspace/server/plugin/auth_socket$ ls
auth_socket.c  CMakeFiles  cmake_install.cmake  CMakeLists.txt  CTestTestfile.cmake  libauth_socket.a

Comment by Geoff Montee (Inactive) [ 2019-04-08 ]

It is documented that mysql_plugin is using mysqld group., and I curious why would we need this, since only 3 arguments are read (basedir, datadir and plugindir) from config?

We need to fix this because [mysqld] is not the only server option group. basedir, datadir, plugindir can also be ready from server option groups like [server] and [mariadb]. See here:

https://mariadb.com/kb/en/library/configuring-mariadb-with-option-files/#server-option-groups

Is the motivation to have those parameters used in other groups ?

Yes. [mysqld] is not the only server option group. Those parameters can be set in any server option group, so mysql_plugin should read all of them, not just [mysqld].

If this patch get applied we need to change this documentation also.

Yes, obviously we would also need to update the documentation if this gets fixed.

However I have a problem in applying this plugin on rocksdb and auth_socket(this is my first time to use this plugin), so can you please let me know how to proceed.

The unix_socket authentication plugin is statically linked into the server in MariaDB 10.4. If you are testing with MariaDB 10.4, then you cannot test mysql_plugin with the unix_socket authentication plugin.

For your rocksdb issue, the server is complaining that it can't find the error message file at /home/anel/workspace/server/share/errmsg.sys. This looks unrelated to rocksdb and to mysql_plugin. It looks like you are running mysql_plugin from your source tree, and the file might not have been built, or if it was built, then that file isn't where the server expects it should be in a source tree. If that is the case, then you might need to set lc_messages or lc_messages_dir.

https://mariadb.com/kb/en/library/server-system-variables/#lc_messages

https://mariadb.com/kb/en/library/server-system-variables/#lc_messages_dir

Comment by Anel Husakovic [ 2019-04-08 ]

Hi GeoffMontee thanks.
Yes I'm working from source and regarding the `rocksdb` options are ok (instead I couldn't run mysqld)

*.my.cnf*
[mysqld]
basedir=/home/anel/workspace/10.2/
datadir=/home/anel/workspace/datadir-10.2/
lc_messages_dir=/home/anel/workspace/10.2/sql/share
lc-messages=en_us
plugin_dir=/home/anel/workspace/10.2/storage/rocksdb/

Even if `no-defaults` is used the situation is the same

./client/mysql_plugin -n -v --basedir=. --datadir=../datadir-10.2 --plugin_dir=./storage/rocksdb/ --plugin_ini=./client/rock.ini rocksdb ENABLE
#    basedir = ./
# plugin_dir = ./storage/rocksdb/
#    datadir = ../datadir-10.2
# plugin_ini = ./client/rock.ini
# Found tool 'mysqld' as './sql/mysqld'.
# Found plugin 'rocksdb' as './storage/rocksdb/ha_rocksdb.so'
# Enabling rocksdb...
# Query: REPLACE INTO mysql.plugin VALUES ('rocksdb','ha_rocksdb.so');
 
# Command: ./sql/mysqld --no-defaults --bootstrap --datadir=../datadir-10.2 --basedir=./ < /tmp/sqlOMLtua
2019-04-08 12:28:58 139909366516736 [Note] ./sql/mysqld (mysqld 10.2.24-MariaDB-debug) starting as process 67475 ...
2019-04-08 12:28:58 139909366516736 [Warning] Could not increase number of max_open_files to more than 1024 (request: 4186)
2019-04-08 12:28:58 139909366516736 [Warning] Changed limits: max_open_files: 1024  max_connections: 151 (was 151)  table_cache: 421 (was 2000)
2019-04-08 12:28:58 139909366516736 [ERROR] Can't find messagefile '/home/anel/workspace/10.2/share/errmsg.sys'
2019-04-08 12:28:58 139909366516736 [ERROR] Aborting
 
ERROR: Unexpected result from bootstrap. Error code: 256.

Even if we provide `lc_messages_dir` and `lc_messages` it doesn't work

anel@ubuntu:~/workspace/10.2$ ./client/mysql_plugin -n -v --lc_messages_dir=./sql/share --basedir=. --datadir=../datadir-10.2 --plugin_dir=./plugin/auth_socket --plugin_ini=./client/auth-ini.ini  unix_socket ENABLE
./client/mysql_plugin: unknown variable 'lc_messages_dir=./sql/share'

Regarding the `auth_socket` in 10.2 the situation is the same.
This is what I do (again without configuration file)

anel@ubuntu:~/workspace/10.2/client$ echo -e "auth_socket\nunix_socket">>auth-ini.ini
anel@ubuntu:~/workspace/10.2/client$ cat auth-ini.ini 
auth_socket
unix_socket

Invoking:

anel@ubuntu:~/workspace/10.2$ ./client/mysql_plugin -n -v --basedir=. --datadir=../datadir-10.2 --plugin_dir=./plugin/auth_socket --plugin_ini=./client/auth-ini.ini unix_socket ENABLE
#    basedir = ./
# plugin_dir = ./plugin/auth_socket
#    datadir = ../datadir-10.2
# plugin_ini = ./client/auth-ini.ini
# Found tool 'mysqld' as './sql/mysqld'.
# Found plugin 'unix_socket' as './plugin/auth_socket/auth_socket.so'
# Enabling unix_socket...
# Query: REPLACE INTO mysql.plugin VALUES ('unix_socket','auth_socket.so');
 
# Command: ./sql/mysqld --no-defaults --bootstrap --datadir=../datadir-10.2 --basedir=./ < /tmp/sqlAMSKPm
2019-04-08 12:32:23 140214250675200 [Note] ./sql/mysqld (mysqld 10.2.24-MariaDB-debug) starting as process 67492 ...
2019-04-08 12:32:23 140214250675200 [Warning] Could not increase number of max_open_files to more than 1024 (request: 4186)
2019-04-08 12:32:23 140214250675200 [Warning] Changed limits: max_open_files: 1024  max_connections: 151 (was 151)  table_cache: 421 (was 2000)
2019-04-08 12:32:23 140214250675200 [ERROR] Can't find messagefile '/home/anel/workspace/10.2/share/errmsg.sys'
2019-04-08 12:32:23 140214250675200 [ERROR] Aborting
 
ERROR: Unexpected result from bootstrap. Error code: 256.

In 10.4 since there is statically linking what should `--plugin_dir` be?
Thanks

Comment by Geoff Montee (Inactive) [ 2019-04-08 ]

Hi anel,

Even if `no-defaults` is used the situation is the same

Even if we provide `lc_messages_dir` and `lc_messages` it doesn't work

It sounds like the mysql_plugin utility may have some kind of bug that may prevent it from working when run from a source tree. This may be another bug that needs to be debugged.

In 10.4 since there is statically linking what should `--plugin_dir` be?

I'm not sure that I understand the question. --plugin-dir is for shared libraries for dynamically linked plugins. If a plugin is statically linked, then --plugin-dir is irrelevant.

I don't think that the mysql_plugin utility can be used with statitcally linked plugins at all.

Comment by Anel Husakovic [ 2019-04-08 ]

Hi GeoffMontee
maybe I'm doing something wrong while configuring (today I used for first time this plugin ) but this is what I have obtained till now.
Will raise soon PR on 10.2 and point this.
Thanks

Comment by Daniel Black [ 2019-04-09 ]

An installed mysql has the plugin shared libraries in their separate build directories. mysql_install_db only uses the auth_socket so fakes plugin-dir to "$builddir/plugin/auth_socket". Once installed they end up in the same directory so its less of a problem.

Comment by Anel Husakovic [ 2019-04-09 ]

links to

Comment by Geoff Montee (Inactive) [ 2019-04-10 ]

Hi anel,

Your PR is for MariaDB 10.2. A bug should be fixed in the earliest branch that we want the fix in. Don't you think that a bug this simple and easy to fix should be fixed in MariaDB 10.1 too? Or maybe even as far back as MariaDB 5.5?

Comment by Anel Husakovic [ 2019-04-11 ]

Isn't 10.1 getting close to the EOL ?
That was the reason why I used 10.2 from combination of `10.1, 10.2, 10.3, 10.4` you wrote in MDEV.
If needed ok, we can do that on 10.1, just let me know about question above.

Comment by Geoff Montee (Inactive) [ 2019-04-11 ]

Hi anel,

See here:

https://mariadb.org/about/maintenance-policy/

MariaDB 10.1 doesn't reach EOL for another 1.5 years. You might be thinking of MariaDB 10.0. That reached EOL a couple weeks ago.

Comment by Anel Husakovic [ 2019-04-12 ]

Hi GeoffMontee
I updated PR to `10.1`.
Thanks

Generated at Thu Feb 08 08:48:26 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.