I don't quite get why does the title include "Fixing debian...". How MariaDB does in-place upgrades for its data has nothing to do with distribution and packaging of files in distros, let alone anything specific to Debian?

MariaDB should not rely on deb nor rpm pre/post scripts to ensure its data files are in correct format (=upgraded). In the most simple scenario deb/rpm is just a way to download files and unpack them on the system with a couple of integrations to the distro paths and facilities. There is no guarantee for example that the MariaDB Server would be running nor restarted during the deb/rpm pre/post script execution.

The most sane solution is that the mariadbd binary itself checks that its data is on correct format, and if not, upgrades it or does whatever is needed. There should be just one way for it to do it and it should not be specific to any distro type or distro version. If mariadbd can't do it independently but needs a separate script mariadb-upgrade, then injecting a call to mariadb-upgrade in systemd and SysV init scripts is probably the best solution, as that is the way systems typically run mariadbd instead of calling it directly. You seem to be on that path already. If you choose to use PreExec or Exec is just a systemd implementation detail.

faust

On Debian, the mariadb_upgrade process is run at every start/reboot of the mariadb-server.

When does other distros or Windows run mariadb-upgrade if not at start of the server? As stated in MDEV-27606, mariadb-upgrade runs as part of the systemd/init.d scripts and are largely the same across all distros.

In native Fedora they made a custom script to check for the need of running mariadb-upgrade, but it is still run as part of the server startup: https://src.fedoraproject.org/rpms/mariadb/blob/rawhide/f/mariadb-check-upgrade.sh

Ideally the mariadbd would itself keep track of its data directory and upgrade it when needed, and not rely on external bash scripts to do it for it, nor even have the possibility to run new mariadbd version without having the datadir also upgraded.

Discussed plan with faust. Finalizing the details will happen next week, but for record:

1. We will use systemd to automate the process where possible.
   (in the future, we will have a way to --enable-networking, much like we have --skip-networking for similar automation in Docker Files)
2. We will use mariadb-upgrade's --check-if-upgrade-is-needed flag
   1. check-if-upgrade-is-needed only looks at the datadir for mysql_upgrade_info file and returns 1 if there is a need to upgrade, otherwise 0. This is a fast operation, but requires read access on the datadir.
   2. We can run --check-if-upgrade-is-needed as part of PreExec step of SystemD, provided we can save the result for the actual Exec step.
   3. For the Exec step, we will start the server and only run mariadb-upgrade if the PreExec step returned 1.

Remember the main thread starting this this proposal ?

The conclusion was that mariadb-upgrade is only required for major and minor upgrades, but not for patchlevel upgrades.

So, you shouldn't need to run mariadb-upgrade for patchlevel upgrades. At all.

As a concept:

[Unit]

Description=MariaDB Upgrade

Documentation=man:mariadb-upgrade(1)

Documentation=https://mariadb.com/kb/en/library/systemd/

Requires=mariadb.service

[Service]

Type=oneshot

ExecCondition=/usr/bin/mariadb-upgrade --check-if-upgrade-is-needed

ExecStart=/usr/bin/mariadb-upgrade --verbose

PrivateTmp=yes

○ mariadb-upgrade.service - MariaDB Upgrade

     Loaded: loaded (/etc/systemd/system/mariadb-upgrade.service; static)

    Drop-In: /usr/lib/systemd/system/service.d

             └─10-timeout-abort.conf, 50-keep-warm.conf

     Active: inactive (dead)

       Docs: man:mariadb-upgrade(1)

             https://mariadb.com/kb/en/library/systemd/

Dec 11 13:19:00 bark strace[184268]: information_schema

Dec 11 13:19:00 bark strace[184268]: performance_schema

Dec 11 13:19:00 bark strace[184268]: sys

Dec 11 13:19:00 bark strace[184268]: sys.sys_config                                     OK

Dec 11 13:19:00 bark strace[184268]: test

Dec 11 13:19:00 bark strace[184268]: Phase 7/8: uninstalling plugins

Dec 11 13:19:00 bark strace[184268]: Phase 8/8: Running 'FLUSH PRIVILEGES'

Dec 11 13:19:00 bark strace[184268]: OK

Dec 11 13:19:00 bark systemd[1]: mariadb-upgrade.service: Deactivated successfully.

Dec 11 13:19:00 bark systemd[1]: Finished mariadb-upgrade.service - MariaDB Upgrade.

(note the strace here for some reason when strace was placed in the Exec*= it worked when without it was doing a EPERM on the mariadb socket. I'm sure this is solveable if the concept is ok)

To resolve MDEV-27606, SET GLOBAL read_only into the mariadb-upgrade execution and clearing after execution?