Reconcile Install & upgrade test differences from old buildbot

The old buildbot has a few upgrade tests that are slightly different from the new one.

Debian based

Install tests

Summary:

• after installation old / new buildbot are performing a different set of tests. Details to come.
• old buildbot waits for mariadb upgrade after spider packages installation also.
• old buildbot is not installing columnstore at all. Looks like a bug to me, because is setting columnstore_package_list
• new buildbot is installing columnstore only for amd and arm. Actually there's a typo for arm and is skipping it on this arch
• old buildbot does the service health check later . Not sure if this is an important aspect
• new buildbot does some final steps which seem redundant to me but will confirm

Check the image below for more details:

Minor upgrade

TODO

Major upgrade

I will start by saying that the notion of previous differs completely in the case of Debian between the old and the new buildbot.
In the new buildbot, previous is exactly like in RHEL, meaning substracting 1 from the major.

Whereas in the old buildbot, previous is:

• for major upgrade 1: mysql from distro repos
• for major upgrade 2: mariadb from distro repos

For Debian:

• upgrade 1 is mariadb to mariadb
• upgrade 2 is also mariadb to mariadb, a slightly different minor, newer but not latest

Does not make a lot of sense to perform upgrade 1.

For Ubuntu:

• upgrade 1 is mysql to mariadb. for mysql 8 (currently in all supported ubuntu) there are a lot of workarounds
• upgrade 2: mariadb to mariadb

We can further analyze whether the upgrade steps differ, even if previous is defined differently.

RHEL based

Install tests

Summary:

• tests are conducted the same (basic + Columnstore)
• new buildbot is checking for wsrep status right before the final server stop. Not documented why.
• old buildbot is disabling the cracklib plugin. Not sure why, need to ask for feedback. In any case, if that is failing then will set reinstall_cracklib_plugin=YES and do nothing with it. Looks redundant to me.

Check the image below for more details:

Minor upgrade

New buildbot is not doing:

• minor upgrade (server)
• minor upgrade (dependencies)
  But as far as I remember this was agreed to be the case. Will re-confirm.

Major upgrade

Both buildbots treat the server case only i.e. upgrading MariaDB-server MariaDB-client

Summary:

• some upgrades are handled automatically in old-bb (MDEV-33459). new-bb removes the old-packages for all versions, thus, not executing this test. Let's revisit the rule and decide for which x -> y we can handle the upgrade automatically.
• i think new-bb it's more accurate at determining the previous major version 10.11 -> 11.0; 11.8 -> 12.0; else {10,11,12}

  .X-1; X > 0 where old-bb only does the x-1 part . Of course, 11.0 it's not a case anymore.

• in old-bb there's a cracklib workaround; given that it only applies to 10.4 -> 10.5 upgrades, I'd say it's no longer applicable

Check the image below for more details: