Details
-
Bug
-
Status: Open (View Workflow)
-
Major
-
Resolution: Unresolved
-
23.02.24
-
None
-
None
Description
cmapi/prerm.template (the RPM %preun and DEB prerm of MariaDB-columnstore-cmapi) runs unconditionally:
systemctl stop ${SYSTEMD_UNIT_NAME}
|
...
|
systemctl disable ${SYSTEMD_UNIT_NAME}
|
systemctl unmask ${SYSTEMD_ENGINE_UNIT_NAME}
|
It never checks $1, so it also runs on an upgrade or reinstall, not just on erase. On RPM the old package's %preun runs after the new package's %post, so it undoes what postinst.template just did: CMAPI ends up stopped and disabled, and mariadb-columnstore (the umbrella unit) unmasked. Same code on develop-23.02 and stable-23.10.
Nothing looks wrong until the next reboot. Then CMAPI doesn't start and the enabled umbrella unit starts ColumnStore on every node outside CMAPI.
What we saw (23.02 build, Rocky 8, 3-node cluster, shared storage; dnf reinstall of the CMAPI rpm on all nodes, cluster started through CMAPI afterwards and healthy, then the instances were stopped and started the next day)
- mariadb-columnstore-cmapi disabled and inactive on all 3 nodes; mariadb-columnstore enabled.
- Every node started its own ColumnStore stack at boot: DBRM read-only, DMLProc refused to start ("Rollback will be deferred"), controllernode bind failures on two nodes, and a replica running its own controllernode and workernode.
- mcs cluster start hung, since there was no CMAPI to answer it.
- Recovered by stopping the stray mcs-* units, systemctl enable mariadb-columnstore-cmapi, systemctl mask mariadb-columnstore on each node. The shared BRM files were not modified in our case, but with several controllernodes up against shared storage that is luck, not design.
Any customer who upgrades the CMAPI rpm and later reboots is exposed.
Suggested fix
Only do the disable/unmask on a real removal. The same file is used for both package types, so check both forms of the argument:
case "$1" in
|
0|remove|purge) # rpm erase / deb remove
|
systemctl stop ${SYSTEMD_UNIT_NAME}
|
systemctl disable ${SYSTEMD_UNIT_NAME}
|
systemctl unmask ${SYSTEMD_ENGINE_UNIT_NAME}
|
;;
|
esac
|
(The _pycache_ cleanup can stay unconditional.) On upgrade the new %post already restarts CMAPI. We have only seen this on RPM; on DEB the old prerm runs before the new postinst, so the enable/mask should win there, but that's not tested.
Related: MCOL-5848 (engine and CMAPI scriptlets fight over the same unit across packages), MCOL-4938 (where the mask/unmask came from).