A tool that helps to adjust config files after MariaDB is upgraded.
It can list all options that are no longer recognized by the server, suggest replacements, if any.
It could also edit files in place (when requested), removing, commenting out, or adding loose- prefix to such options.
Also it could check for invalid values of enum/set variables.
To parse all files it could use my_print_defaults, the list of supported options could be hard-coded or extracted from mariadbd --help and such.
perl/python or C/C++ ? C or C++ is probably more portable for windows.
For code how to parse options like !include, check mysys/my_defaults.c
Suggested options:
No options: List unknown server options on stdout (those that would prevent the server from starting).
--update
Update the my.cnf file in place (and all files included with !include or !includedir)
--backup
Store modified files as backup (for example my.cnf-backup)
--current-version=... (for example --current-version=mysqld-5.7)
Section to use for unknown options, see --mode
The purpose of --current-version is to enable the tool to create a config file that will work for both MySQL and MariaDB at
the same time. In this case, options that works for both MySQL and MariaDB should use the section [mysqld]. MySQL unique options should be under [mysqld-5.7] and MariaDB specific options
would be under [mariadbd].
--edit=...
Select what to do with unknown options. List of suggested mode options:
remove Remove unoption
comment Add # before option
inline-old-version Add ['current-version'] before the option and [mysqld] afterwards. 'curent-version' is the tag specified with --current-version=...
last-old-version Move all unknown version last in the file with [old-version] before.
--print
Print converted file on stdout. In this case all !include and !includedir files should be included.
We need both inline- and last- as some options may depend on their place in the config file.
The tool should also be prepared for special handling of some specific options.
Note that MariaDB don't have the audit_log plugin (Percona Server).
Instead MariaDB has a server_audit plugin which takes different options.
If the tool notice that 'audit_plugin' is used, it should suggest one to use the server_audit plugin instead.
--plugin-load=QUERY_RESPONSE_TIME_READ=query_response_time.so;QUERY_RESPONSE_TIME_WRITE=query_response_time.so
This is only supported in MariaDB 10.11 and above. If used with an earlier MariaDB version it would be good to get a note that this is supported in 10.11.6 and above.
As a reference, here are some common config options found in Percona server that is not in MariaDB: [mysqld-5.7]
--binlog-ignore-db=information_schema
--innodb_log_files_in_group=2
--innodb_buffer_pool_instances=3
--max_binlog_files=14
--log_slow_rate_type=query
--slow_query_log_always_write_time=0.5
--slow_query_log_use_global_control=all
--plugin-load=audit_log.so
Attachments
Issue Links
relates to
MDEV-26923MariaDB will abort server startup if it finds an invalid parameter, but won't check for other invalid params
Closed
MDEV-33503innodb_log_files_in_group is quietly ignored
while ! mariadbd --no-defaults "${opts[@]}" --help --verbose > /dev/null 2>&1
do
maybebad+=( "${opts[-1]}" )
unset "opts[-1]"
done
while [ ${#maybebad[@]} -gt 0 ]
do
opts+=( "${maybebad[-1]}" )
unset "maybebad[-1]"
if ! mariadbd --no-defaults "${opts[@]}" --help --verbose > /dev/null 2>&1
then
isbad+=( "${opts[-1]}" )
unset "opts[-1]"
fi
done
echo Good options: "${opts[@]}"
echo Bad options: "${isbad[@]}"
Example:
./my_convert.sh
Good options: --table_open_cache=17384 --max_connections=150 --datadir=/home/dan/mariadb-datadir --slow_query_log_file=/home/tom/x.slow
Bad options: --bababa=foo
Daniel Black
added a comment - First concept to show incorrect options:
my_convert.sh
#!/bin/bash
declare -a opts
declare -a maybebad
declare -a isbad
readarray -t opts < <( my_print_defaults --mysqld )
while ! mariadbd --no-defaults "${opts[@]}" --help --verbose > /dev/null 2>&1
do
maybebad+=( "${opts[-1]}" )
unset "opts[-1]"
done
while [ ${#maybebad[@]} -gt 0 ]
do
opts+=( "${maybebad[-1]}" )
unset "maybebad[-1]"
if ! mariadbd --no-defaults "${opts[@]}" --help --verbose > /dev/null 2>&1
then
isbad+=( "${opts[-1]}" )
unset "opts[-1]"
fi
done
echo Good options: "${opts[@]}"
echo Bad options: "${isbad[@]}"
Example:
./my_convert.sh
Good options: --table_open_cache=17384 --max_connections=150 --datadir=/home/dan/mariadb-datadir --slow_query_log_file=/home/tom/x.slow
Bad options: --bababa=foo
It will not work for anyone wanting to create an automatic script that will migratate MySQL 5.7 to MariaDB.
What I would like to see in the final script/tool:
Option to list 'bad options' (like above)
Be able to run it on an existing config files and print the resulting config file.
In this case we should move 'bad' options' in the 'server' or 'mysqld' section to [mysqld-5.7] (user specified) to the end of the script or add a #not-in-mariadb comment before them. The benefit of using [mysqld-5.7] is that the config files would be usable both by MariaDB and MySQL.
Be able to run it on all config file and fix them inplace.
When pointing to a config file for automatic conversion, support the '!include" and "!includedir" options.
When it comes to fixing things inplace, check out the replace tool. It does this and also has optimization to not change files that does not need conversion.
Michael Widenius
added a comment - Great first version.
Some comments:
This will probably not work on windows (bash)
It will not work for anyone wanting to create an automatic script that will migratate MySQL 5.7 to MariaDB.
What I would like to see in the final script/tool:
Option to list 'bad options' (like above)
Be able to run it on an existing config files and print the resulting config file.
In this case we should move 'bad' options' in the 'server' or 'mysqld' section to [mysqld-5.7] (user specified) to the end of the script or add a #not-in-mariadb comment before them. The benefit of using [mysqld-5.7] is that the config files would be usable both by MariaDB and MySQL.
Be able to run it on all config file and fix them inplace.
When pointing to a config file for automatic conversion, support the '!include" and "!includedir" options.
When it comes to fixing things inplace, check out the replace tool. It does this and also has optimization to not change files that does not need conversion.
mysql_upgrade_service.exe (windows upgrade tool) has edited my.ini to remove invalid options for a while now, itself. Without suggestions, it removed known outdated options. It did not support "!include" , but neither our nor MySQL installer use any includes on Windows
Vladislav Vaintroub
added a comment - - edited mysql_upgrade_service.exe (windows upgrade tool) has edited my.ini to remove invalid options for a while now, itself. Without suggestions, it removed known outdated options. It did not support "!include" , but neither our nor MySQL installer use any includes on Windows
As noted in MDEV-33503, replacing some parameters will involve some arithmetics. An example: If innodb_log_files_in_group was specified, then it must be removed and innodb_log_file_size be multiplied by that number. If no innodb_log_file_size had been specified but a default value was being used, then it gets trickier. I think that in that case, it would be easiest to issue a warning. In that way, the tool will not have to remember the default innodb_log_file_size in different server versions.
Marko Mäkelä
added a comment - As noted in MDEV-33503 , replacing some parameters will involve some arithmetics. An example: If innodb_log_files_in_group was specified, then it must be removed and innodb_log_file_size be multiplied by that number. If no innodb_log_file_size had been specified but a default value was being used, then it gets trickier. I think that in that case, it would be easiest to issue a warning. In that way, the tool will not have to remember the default innodb_log_file_size in different server versions.
Another example from MDEV-33503 is that innodb_stats_sample_pages should be renamed to innodb_stats_transient_sample_pages. That we can do independent of server version.
Marko Mäkelä
added a comment - Another example from MDEV-33503 is that innodb_stats_sample_pages should be renamed to innodb_stats_transient_sample_pages . That we can do independent of server version.
People
Unassigned
Sergei Golubchik
Votes:
0Vote for this issue
Watchers:
6Start watching this issue
Dates
Created:
Updated:
Git Integration
Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.
{"report":{"fcp":1112.7000000029802,"ttfb":176.70000000298023,"pageVisibility":"visible","entityId":126311,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":0.5,"journeyId":"c6ed8820-59dc-46a7-a69b-0ffffe7d0fb5","navigationType":0,"readyForUser":1171,"redirectCount":0,"resourceLoadedEnd":1445.9000000059605,"resourceLoadedStart":182.40000000596046,"resourceTiming":[{"duration":469.70000000298023,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2bv2/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":182.40000000596046,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":182.40000000596046,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":652.1000000089407,"responseStart":0,"secureConnectionStart":0},{"duration":469.70000000298023,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2bv2/820016/12ta74/2380add21a9a1006587582385952de73/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":182.70000000298023,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":182.70000000298023,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":652.4000000059605,"responseStart":0,"secureConnectionStart":0},{"duration":478.20000000298023,"initiatorType":"script","name":"https://jira.mariadb.org/s/e9b27a47da5fb0f74a35acd57e9847fb-CDN/lu2bv2/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":182.90000000596046,"connectEnd":182.90000000596046,"connectStart":182.90000000596046,"domainLookupEnd":182.90000000596046,"domainLookupStart":182.90000000596046,"fetchStart":182.90000000596046,"redirectEnd":0,"redirectStart":0,"requestStart":182.90000000596046,"responseEnd":661.1000000089407,"responseStart":661.1000000089407,"secureConnectionStart":182.90000000596046},{"duration":556.6999999880791,"initiatorType":"script","name":"https://jira.mariadb.org/s/c32eb0da7ad9831253f8397e6cc26afd-CDN/lu2bv2/820016/12ta74/2380add21a9a1006587582385952de73/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true","startTime":183.1000000089407,"connectEnd":183.1000000089407,"connectStart":183.1000000089407,"domainLookupEnd":183.1000000089407,"domainLookupStart":183.1000000089407,"fetchStart":183.1000000089407,"redirectEnd":0,"redirectStart":0,"requestStart":183.1000000089407,"responseEnd":739.7999999970198,"responseStart":739.7999999970198,"secureConnectionStart":183.1000000089407},{"duration":559.9000000059605,"initiatorType":"script","name":"https://jira.mariadb.org/s/bc0bcb146314416123c992714ee00ff7-CDN/lu2bv2/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":183.20000000298023,"connectEnd":183.20000000298023,"connectStart":183.20000000298023,"domainLookupEnd":183.20000000298023,"domainLookupStart":183.20000000298023,"fetchStart":183.20000000298023,"redirectEnd":0,"redirectStart":0,"requestStart":183.20000000298023,"responseEnd":743.1000000089407,"responseStart":743.1000000089407,"secureConnectionStart":183.20000000298023},{"duration":559.8999999910593,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bv2/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":183.6000000089407,"connectEnd":183.6000000089407,"connectStart":183.6000000089407,"domainLookupEnd":183.6000000089407,"domainLookupStart":183.6000000089407,"fetchStart":183.6000000089407,"redirectEnd":0,"redirectStart":0,"requestStart":183.6000000089407,"responseEnd":743.5,"responseStart":743.5,"secureConnectionStart":183.6000000089407},{"duration":560.3000000119209,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bv2/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":183.79999999701977,"connectEnd":183.79999999701977,"connectStart":183.79999999701977,"domainLookupEnd":183.79999999701977,"domainLookupStart":183.79999999701977,"fetchStart":183.79999999701977,"redirectEnd":0,"redirectStart":0,"requestStart":183.79999999701977,"responseEnd":744.1000000089407,"responseStart":744.1000000089407,"secureConnectionStart":183.79999999701977},{"duration":618.3999999910593,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2bv2/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":183.90000000596046,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":183.90000000596046,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":802.2999999970198,"responseStart":0,"secureConnectionStart":0},{"duration":560.3999999910593,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":184.1000000089407,"connectEnd":184.1000000089407,"connectStart":184.1000000089407,"domainLookupEnd":184.1000000089407,"domainLookupStart":184.1000000089407,"fetchStart":184.1000000089407,"redirectEnd":0,"redirectStart":0,"requestStart":184.1000000089407,"responseEnd":744.5,"responseStart":744.5,"secureConnectionStart":184.1000000089407},{"duration":618.2000000029802,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2bv2/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":184.20000000298023,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":184.20000000298023,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":802.4000000059605,"responseStart":0,"secureConnectionStart":0},{"duration":561.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/719848dd97ebe0663199f49a3936487a-CDN/lu2bv2/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":184.29999999701977,"connectEnd":184.29999999701977,"connectStart":184.29999999701977,"domainLookupEnd":184.29999999701977,"domainLookupStart":184.29999999701977,"fetchStart":184.29999999701977,"redirectEnd":0,"redirectStart":0,"requestStart":184.29999999701977,"responseEnd":745.7999999970198,"responseStart":745.7000000029802,"secureConnectionStart":184.29999999701977},{"duration":1259.7999999970198,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bv2/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":185.40000000596046,"connectEnd":185.40000000596046,"connectStart":185.40000000596046,"domainLookupEnd":185.40000000596046,"domainLookupStart":185.40000000596046,"fetchStart":185.40000000596046,"redirectEnd":0,"redirectStart":0,"requestStart":185.40000000596046,"responseEnd":1445.2000000029802,"responseStart":1445.2000000029802,"secureConnectionStart":185.40000000596046},{"duration":1260.4000000059605,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bv2/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":185.5,"connectEnd":185.5,"connectStart":185.5,"domainLookupEnd":185.5,"domainLookupStart":185.5,"fetchStart":185.5,"redirectEnd":0,"redirectStart":0,"requestStart":185.5,"responseEnd":1445.9000000059605,"responseStart":1445.9000000059605,"secureConnectionStart":185.5},{"duration":146.29999999701977,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":829.9000000059605,"connectEnd":829.9000000059605,"connectStart":829.9000000059605,"domainLookupEnd":829.9000000059605,"domainLookupStart":829.9000000059605,"fetchStart":829.9000000059605,"redirectEnd":0,"redirectStart":0,"requestStart":829.9000000059605,"responseEnd":976.2000000029802,"responseStart":976.2000000029802,"secureConnectionStart":829.9000000059605},{"duration":458.20000000298023,"initiatorType":"link","name":"https://jira.mariadb.org/s/d5715adaadd168a9002b108b2b039b50-CDN/lu2bv2/820016/12ta74/be4b45e9cec53099498fa61c8b7acba4/_/download/contextbatch/css/jira.project.sidebar,-_super,-project.issue.navigator,-jira.general,-jira.browse.project,-jira.view.issue,-jira.global,-atl.general,-com.atlassian.jira.projects.sidebar.init/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":1069,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":1069,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1527.2000000029802,"responseStart":0,"secureConnectionStart":0},{"duration":418.70000000298023,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bv2/820016/12ta74/e65b778d185daf5aee24936755b43da6/_/download/contextbatch/js/browser-metrics-plugin.contrib,-_super,-project.issue.navigator,-jira.view.issue,-atl.general/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":1069.9000000059605,"connectEnd":1069.9000000059605,"connectStart":1069.9000000059605,"domainLookupEnd":1069.9000000059605,"domainLookupStart":1069.9000000059605,"fetchStart":1069.9000000059605,"redirectEnd":0,"redirectStart":0,"requestStart":1069.9000000059605,"responseEnd":1488.6000000089407,"responseStart":1488.6000000089407,"secureConnectionStart":1069.9000000059605},{"duration":422.59999999403954,"initiatorType":"script","name":"https://jira.mariadb.org/s/53a43b6764f587426c7bb9a150184c00-CDN/lu2bv2/820016/12ta74/be4b45e9cec53099498fa61c8b7acba4/_/download/contextbatch/js/jira.project.sidebar,-_super,-project.issue.navigator,-jira.general,-jira.browse.project,-jira.view.issue,-jira.global,-atl.general,-com.atlassian.jira.projects.sidebar.init/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true","startTime":1070.2000000029802,"connectEnd":1070.2000000029802,"connectStart":1070.2000000029802,"domainLookupEnd":1070.2000000029802,"domainLookupStart":1070.2000000029802,"fetchStart":1070.2000000029802,"redirectEnd":0,"redirectStart":0,"requestStart":1070.2000000029802,"responseEnd":1492.7999999970198,"responseStart":1492.7999999970198,"secureConnectionStart":1070.2000000029802}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":11,"responseStart":177,"responseEnd":179,"domLoading":180,"domInteractive":1485,"domContentLoadedEventStart":1485,"domContentLoadedEventEnd":1527,"domComplete":1821,"loadEventStart":1822,"loadEventEnd":1822,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":1459.2000000029802},{"name":"bigPipe.sidebar-id.end","time":1460},{"name":"bigPipe.activity-panel-pipe-id.start","time":1460.2000000029802},{"name":"bigPipe.activity-panel-pipe-id.end","time":1462.7999999970198},{"name":"activityTabFullyLoaded","time":1540.7000000029802}],"measures":[],"correlationId":"c89ae6f73111ce","effectiveType":"4g","downlink":10,"rtt":0,"serverDuration":111,"dbReadsTimeInMs":9,"dbConnsTimeInMs":17,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}
First concept to show incorrect options:
my_convert.sh
#!/bin/bash
declare -a opts
declare -a maybebad
declare -a isbad
readarray -t opts < <( my_print_defaults --mysqld )
while ! mariadbd --no-defaults "${opts[@]}" --help --verbose > /dev/null 2>&1
do
maybebad+=( "${opts[-1]}" )
unset "opts[-1]"
done
while [ ${#maybebad[@]} -gt 0 ]
do
opts+=( "${maybebad[-1]}" )
unset "maybebad[-1]"
if ! mariadbd --no-defaults "${opts[@]}" --help --verbose > /dev/null 2>&1
then
isbad+=( "${opts[-1]}" )
unset "opts[-1]"
fi
done
echo Good options: "${opts[@]}"
echo Bad options: "${isbad[@]}"
Example:
./my_convert.sh
Good options: --table_open_cache=17384 --max_connections=150 --datadir=/home/dan/mariadb-datadir --slow_query_log_file=/home/tom/x.slow
Bad options: --bababa=foo