Note: I'm not filing FORCE_UPGRADE issues (apart of it not being transactional), but I will log them here for the future reference.

CentOS 5 x86:
upgrade from mysql-server requires FORCE_UPGRADE. With FORCE_UPGRADE it works, new server gets installed, old server gets removed, but the old one is running, not restarted.

On some reason, when we do yum install mysql-server mysql on CentOS 5 x86_64, it installs both mysql x86_64 and mysql i386. It causes further problems on upgrade to MariaDB:

--> Finished Dependency Resolution

mysql-5.0.95-5.el5_9.i386 from installed has depsolving problems

  --> mysql conflicts with MariaDB-server

Error: mysql conflicts with MariaDB-server

If we remove i386 version of mysql, the upgrade goes the same way as on CentOS 5 i386 – it suggests FORCE_UPGRADE instead of just failing with conflicts; with FORCE_UPGRADE it upgrades, but does not get restarted.

Tests are now in the buildbot config.
They are enabled for the special category "rpm test" branch, which is defined as any branch which has "rpm" in its name.
All RPM builders are enabled for this branch, and all others are disabled except for a few which are not that easy to disable.

For other branches the tests are not enabled, because MDEV-9584 isn't pushed yet.
Some tests can be enabled for main trees as soon as MDEV-9584 is pushed, even in its current state. It mostly affects upgrade from MariaDB of a previous version to the current one – apart from a few exceptions, it works okay.

Most other tests will fail on "rpm test" branches due to bugs submitted as subtasks to MDEV-9584. As a rule, such tests have comments in the buildbot config pointing at known bugs which make them fail, although there might be more problems which will show up when existing ones are fixed.

In some cases, when there seems to be an acceptable workaround for the failure, it is implemented and enabled – e.g. manual server restart when the automatic one does not work, or additional package manager options if they are suggested during the installation. When such a workaround is used, a warning is printed to the log and captured in buildbot warnings.

When/if the bugs are fixed and tests start passing on the "rpm test" branch, they can be considered to be enabled for main branches. However, it should not be done for all tests, only for those upgrade paths which we see as most important. While every test is not long (~2-3 min), the amount of them will make builds much longer.

Some tests which are not expected to do anything are disabled for all branches, but they are still in the config just in case (e.g. installation of MySQL followed by upgrade to MariaDB, as opposed to install MariaDB over MySQL). They can also be converted to negative test cases.

Technical info:

The test flow is implemented within def getRpmUpgradeStep, and tests are in fact calls for this method.

Plans:

One thing that is missing in the test flow is creating some structures (tables, views) and checking that they upgrade well. It is omitted for now on purpose, since the goal is to test the packaging change; but when packaging is considered to be finished, the data creation/upgrade should be added.
For a note, some tests might fail because of the missing data (e.g. upgrade from mysql-server on CentOS 6) – it fails because of the InnoDB file size mismatch. It is a known nuisance related to the empty database. When data is big enough, it will go away.

See serg's comments in MDEV-9796, take into account his case 1, and re-check case 2.

More comments/notes from chat:

elenst
re: 9796, your example `sudo zypper -v install mArIaDb-cLiEnT` . I know that it's case-insensitive, but why does it not consider them to be the same package of a different version? Why does it choose both, rather than the one of the highest version (10.1 in this case)? (edited)

serg
they're from different repositories, different packagers, etc

elenst
is it not possible to somehow indicate "replaceability"?

serg
not when you explicitly select a package
if it's pulled as a dependency - the package manager selects the best matching one
we can affect that to a degree
but if a user indicates a wish to install something specific
we cannot affect that, I think
it's all "as far as I understand"
and I've only spent few hours with zypper
I could be wrong

elenst
the other part is just weird, this one "On the other hand, installing our packages over SUSE ones, works without any additional options" – from my previous comment it seems that it didn't work for me before
either i did something differently, or something has changed since then

serg
right, that's why I tried to install over
another option to try for you (zypper) instead of --from, try -r
if you'll have something that --from doesn't solve
I think that --from means "take the package from this repo" while -r means "think this is the only repo, others don't exist for you"
so I think it's kind of a stronger --from

elenst cool, thanks for this .