[MDEV-30499] mariadb in-server upgrade Created: 2023-01-29  Updated: 2023-07-28

Status: Open
Project: MariaDB Server
Component/s: None
Fix Version/s: 11.1

Type: Task Priority: Major
Reporter: Daniel Black Assignee: Unassigned
Resolution: Unresolved Votes: 3
Labels: None

Issue Links:
Relates
relates to MDEV-27060 mariadb_upgrade runs every startup Closed
relates to MDEV-27068 running mariadb-upgrade on mariadb se... Closed
relates to MDEV-27107 prevent two mariadb-upgrade running i... Closed
relates to MDEV-27279 mariadb_upgrade add --check-if-upgrad... Closed
relates to MDEV-27606 change debian packaging to not run ma... Open
relates to MDEV-27613 Fixing debian to only run the full my... Stalled
relates to MDEV-27636 mariadb_upgrade --check-if-upgrade-is... Open
relates to MDEV-30498 Rename mysql_upgrade state file to ma... Closed

 Description   

The competing requirements around mariadb-upgrade make the current solution undesirable:

  • mariadb-upgrade should be conducted without user load
  • multiple startups of the server takes time and can was a lot of time in innodb recovery/page loading
  • scripted deployments shouldn't bare the responsibility of understanding the complexity
  • service scripts like systemd and containers make mariadb available on sd_notify=READY or the network connection being listened too.

So ideally:

  • compile the upgrade SQL into the server (cold attribute)
  • Use include/mysql/service_sql.h or bootstrap like interface to run the SQL
  • allow skipping by non-default system variable auto_upgrade=1
  • execute before tcp/unix sockets are bound/listening
  • use service_manager_extend_timeout in some way
  • lock state file like mariadb-upgrade (and keep this for compatibility)

 Comments   
Comment by Christian Gonzalez [ 2023-03-07 ]

Is there any update on this issue? I think this can help to improve the upgrade process reliability. Knowing that the server itself will take care of updating the DB tables will make the upgrade process more simpler i.e less prone to errors, specially in automated environments when the upgrade process is automated and it might be harder to catch unexpected errors. Also, if some extra hands are needed I could lock some time to start working on this.

Comment by Otto Kekäläinen [ 2023-03-26 ]

I don't see any drawbacks with the server doing the upgrade automatically.

The server startup speed will be fast if the server check the existence of /var/lib/mysql/mariadb-upgrade-info file on startup and based on its contents decides to attempt automatic upgrade, as such a file check sub-millisecond operation.

In rare cases when a DBA has a use case to not do it, they can use the auto_upgrade=0 config or --skip-auto-upgrade.

The upside however is huge: When the server does the upgrade by itself it will be much more reliable, as it is very hard to end up having none or multiple mariadb-upgrades running in parallel. That whole category of bugs would go away.

This architecture would also greatly simplify the SysV init and systemd startup files as they don't need to check for any upgrades anymore. Most likely also the deb and rpm install/upgrade maintainer scripts could be simplified after this.

Comment by Otto Kekäläinen [ 2023-07-23 ]

If we had this automatic upgrade inside mariadbd, bugs like MDEV-27060 and MDEV-27606 would not occur.

I am definitely in favor of having automatic in-server upgrade on every mariadbd start.

Comment by Andrew Hutchings [ 2023-07-28 ]

I believe monty was not keen on this feature being implemented. I don't remember the details, just that there was a discussion over it between him and danblack.

Comment by Daniel Black [ 2023-07-28 ]

Lucky there's a documented discussion with a user interest and valid points around function and testing that can supersede that.

Generated at Thu Feb 08 10:16:41 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.