• So you can set wsrep_provider_options value to more than 2048 characters or does it truncate it already there ?
• We could provide some I_S.WSREP_PROVIDER_OPTIONS interface where wsrep_provider_options is spitted to variable_name and variable_value pairs so that every option is it's own row.
• I do not know what we can do for show variables
• If wsrep_provider is a plugin we could add these as normal variables (there would be just many of them)

It seems to be possible to implement wsrep_provider plugin to expose wsrep provider options as system variables. This however requires that the provider library is loaded before plugins so that the default options string can be read from the provider and plugin variables can be generated before the plugin in loaded.

I have POC implementation under work, and see that there will be following limitations which this approach will bring:

• Provider library must always be loaded at server startup before plugins are initialized.
• Provider cannot be reloaded while the server is running.
• wsrep_provider and wsrep_provider_options will be read only.
• All the variables must be string variables because there is no type information available from the provider.

Do these limitations sound reasonable from usage point of view?

I saw also the following issues with current Galera:

• There may be some variables on Galera side which may not allow changing the value back to default once changed.
• There are also many variables on Galera side which are no exposed by default unless set in initial provider options.

I believe these can be worked on Galera side.

teemu.ollakka As now wsrep_provider is read only I do not mind that wsrep_provider_options is also read only.

ralf.gebhardt you will get around 80 - 90 options out of this.
However, these options won't show up in non-Galera setups. Also, the feature can be enabled / disabled.

ralf.gebhardt would the above be acceptable?

If plugin-load-add=wsrep_plugin was used wsrep_provider_plugin_enabled as a variable becomes redundant.

Or better, plugin-load-add=galera (with plugin-dir symlink in packaging) as an implementer of the wsrep API is there (replacing wsrep_provider=x and the fake wsrep_provider plugin). Galera can expose variables in a standard MYSQL_SYSVAR way, which it almost does anyway. Then all provider system variables automaticity become galera_gcache_dir, galera_repl.max_ws_size etc.

For compatibility/legacy interface wsrep_provider_options_check / wsrep_provider_options_check break up the option list and pass each option though the plugin check/update mechanism.

With this the number of new options is reduced (eliminated?) and the facility to treat galera system variables as standard unlimited variables becomes available.

sciascid Can you see if suggested changes could be done please.

danblack thanks for the feedback! I'll look into your suggestions

danblack I looked into your suggestions.

If plugin-load-add=wsrep_plugin was used wsrep_provider_plugin_enabled as a variable becomes redundant.

I think this is possible.

Or better, plugin-load-add=galera (with plugin-dir symlink in packaging) as an implementer of the wsrep API is there (replacing wsrep_provider=x and the fake wsrep_provider plugin). Galera can expose variables in a standard MYSQL_SYSVAR way, which it almost does anyway. Then all provider system variables automaticity become galera_gcache_dir, galera_repl.max_ws_size etc.

Are you suggesting we load galera library (i.e. libgalera_ssm.so) through mariadb's plugin interface?
Even if we were to do this, we would still not be able to have a list of pre-declared variables (to expose using the standard MYSQL_SYSVAR way), because the library does not have one. Internally, different modules populate their own variables at runtime.

For compatibility/legacy interface wsrep_provider_options_check / wsrep_provider_options_check break up the option list and pass each option though the plugin check/update mechanism.

Notice that in the current pull request wsrep_provider_options is still supported when the plugin is disabled. And when the plugin is enabled wsrep_provider_options becomes read only. This way a user is either using the old wsrep_provider_options or the new single options. Not both at the same. I think this is a sensible choice, do you agree?

> Are you suggesting we load galera library (i.e. libgalera_ssm.so) through mariadb's plugin interface?

yes

> Even if we were to do this, we would still not be able to have a list of pre-declared variables (to expose using the standard MYSQL_SYSVAR way), because the library does not have one. Internally, different modules populate their own variables at runtime.

It could be made could populate the MYSQL_SYSVAR array during the plugin initialization with a restructure of the internal modules of galera.

> Notice that in the current pull request wsrep_provider_options is still supported when the plugin is disabled.

Why is this useful? How is error handling on value bounds done? No other plugin needs such a feature. A careful look at the initialization order would remove this need.

> And when the plugin is enabled wsrep_provider_options becomes read only. This way a user is either using the old wsrep_provider_options or the new single options. Not both at the same. I think this is a sensible choice, do you agree?

So rather restructuring galera to work like a mariadb/mysql plugin, the wsrep_api gained an entire configuration service, its own re-invention of variable types.

So mariadb is getting a lot of padding (debt would be the unfriend term) to avoid galera doing any change or actually behaving like a mariadb plugin.

>> Notice that in the current pull request wsrep_provider_options is still supported when the plugin is disabled.

> Why is this useful? How is error handling on value bounds done? No other plugin needs such a feature. A careful look at the initialization order would remove this need.

This comment is confusing to me. Which feature are you talking about?
You originally commented about wsrep_provider_options variable and suggested how to ensure compatibility and still support the legacy interface. And my reply was that the current pull request still supports the legacy interface, when the plugin is disabled. I don't see how this is related to initialization order.

> So rather restructuring galera to work like a mariadb/mysql plugin, the wsrep_api gained an entire configuration service, its own re-invention of variable types.
> So mariadb is getting a lot of padding (debt would be the unfriend term) to avoid galera doing any change or actually behaving like a mariadb plugin.

I'm not sure I agree with this. The "configuration service" is literally one function call.
In its current form, the feature is entirely implemented in galera, wsrep-lib, and wsrep_* files. No interference with the rest of mariadb.
And even if Galera was loaded through plugin interface, you'd still need to interact with it. So assuming that you'd still use wsrep-api, you'd need one call to somehow expose the variables of the library.

Overall, I think what this pull request is doing, is very similar to what you propose, minus the loading of galera through mariadb plugin interface.

The pull request has been updated to get rid of variable wsrep_provider_plugin_enabled, as per first suggestion by danblack:
> If plugin-load-add=wsrep_plugin was used wsrep_provider_plugin_enabled as a variable becomes redundant.

Since this is a built-in plugin, it appears that instead of plugin-load-add, one is supposed to use -plugin<name of the plugin>=ON, to enable the plugin.
So in our case --plugin-wsrep-provider=ON, enables the feature.

edit: needed upstream galera 4.x latest to work.

• I still think plugin-load-add=galera would be a good way to start the service
• wsrep_provider_options_pc_linger=PT2S - I thought the "wsrep" was notionally an API, and galera was the specific bit, so now we've got an implementation detail in the "wsrep" namespace and no resemblance of proxying it through "wsrep_provider_options".
• The galera code now registers centrally the config, so why not populate an MYSQL_SYSVAR array in galera's maria_declare_plugin.
• If you can't validate a parameter, maybe its better if the user migrates their config to galera_pc_linger= system variables, and they can be validated on initialization.
• this concept would brutally remove the wsrep_provider_options, which by this and linked bugs was never a good choice anyway.

Elsewhere, it was suggested to me that we could just hard code variables in the plugin, instead of fetching them from Galera and populate them dynamically (as it is done in the current proposal).
There are two problems with hard coding the options in the plugin: 1) This would somehow fix the dependency between Mariadb and Galera. You may end up with a version of Galera that has some option, which is not in the MariaDB plugin (or viceversa). It is my understanding that there is no version dependency right now. 2) Even if we do fix this version dependency somehow (let's say by packaging Galera together with MariaDB), any change to Galera configuration must be reflected into the mariadb plugin. Any change in default values, ranges, addition or removal of variable must be done in two places. That's not good, and could easily turn into a maintenance mess.
I don't see many advantages with the hard coding of Galera variables approach . Other than range checking done in server side.
I could try to extend the current approach so that range checking is done in server side. Let me know if that's of interest.

branch : bb-10.1-MDEV-22570-wsrep-provider-options-plugin
commit : 9cf15ff3335f77031b71ca0590d5b2e9ce337f05
Requires Galera library : 26.4.13 commit e6c1388b

Config required to use new feature :

[mysqld.1]

wsrep-on=ON

wsrep-cluster-address=gcomm://

wsrep-provider=@ENV.WSREP_PROVIDER

binlog-format=ROW

plugin-wsrep-provider=ON

Test plan :

• Make sure that new variables are not visible if

  plugin-wsrep-provider=OFF

  or default
• Make sure that new variables are visible and you can change them if

  plugin-wsrep-provider=ON

• Defaults for parameters should be same

Updated branch : preview-10.11-MDEV-22570-wsrep-provider-options-plugin

ok to push

Updated branch: bb-11.0-galera-split-wsrep-provider-options
Requires Galera library : 26.4.14 from mariadb-4.x branch commit cf9e5e253