Details
-
Task
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
None
Description
Time consuming, as 10.000 lines of code need to be moved to New Buildbot
Expectations outlined by @elenst. If they turn out to be wrong, @elenst would like to hear what exactly is different. The terminology is from the old buildbot.
pre-conditions (not related to the scripts themselves):
- make KVM machines work, create a couple of VMs – e.g. one with deb-based linux, one with rpm-based – with a default installation, as clean as possible, no development environment and alike;
- create builders which will launch these machines and copy into them packages built elsewhere, currently in docker;
moving the scripts: - copy-paste from the old buildbot config to a separate file php functions which define installation and upgrade tests, e.g. getRpmInstallStep and such, each of which is basically one gigantic shell step (generic Shell or more likely the step Test which is pretty much the same).
There are probably half a dozen of them, or so; - rename the steps and concepts if necessary (it's one few-line block per function);
- search for properties like %( and see if they match what the new buildbot has, or need to be renamed, or need to be created (some were defined in the old buildbot, you can see how);
- search for php injections like """ + ... + """ and do the same;
- move the edited functions to the new config;
- make the builders created in pre-conditions call the steps;
- take the same approach for remaining one or two steps (I think in Debian factory, the legacy ones) which are not defined as php functions but directly as shell steps in the factory;
debugging: - run, get it fail, fix, repeat.
Additional things:
- Upgrade tests only needed from previous main version to the next (like 10.4 -> 10.5 but not 10.3 -> 10.5)
- In old buildbot for RPM's we try to upgrade from the latest RPM released for the previous MariaDB version to the built one. For Debian we use a random previous package. In new buildbot we should use the latest released package in all cases.
- We also need to do minor upgrade tests from previous released version from mariadb.org, for example MariaDB 10.5.1 -> MariaDB 10.5.2. This doesn't have to be done for a 0 version, like MariaDB 10.6.0
Attachments
Issue Links
- causes
-
MDBF-818 Missing columnstore test fixes in new Buildbot after upgrade from old BB
-
- Closed
-
- includes
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
- is part of
-
-
- Closed
-
-
-
- Closed
-
Activity
To summarize once again which of installation/upgrade tests from the old buildbot are expected to be in the new buildbot from the beginning.
Notes:
Special tricks may be needed to work around existing problems. See old buildbot tests for the hints.
Some tests aren't run by the old buildbot on early development releases.
"deb-based platforms" are all versions of Ubuntu and Debian for which MariaDB provides packages to the public. The list is MariaDB-version-specific and dynamic (new Ubuntu releases happen often).
"rpm-based platforms" are all versions of CentOS, RHEL, and Fedora for which MariaDB provides packages to the public. The list is MariaDB-version-specific and dynamic (new Fedora releases happen often).
"zyp-based platforms" are all versions of openSUSE and SLES for which MariaDB provides packages to the public. They are in fact also RPM-based, but with zypper as a package manager.
Installation tests on all platforms ("install" step in old buildbot)
The test installs all packages, and makes sure the server is functioning. The test is performed on a clean image, without any specific non-default packages pre-installed. Only standard distro repos, Galera repository and the repository pointing at the packages under test (for debs) are configured. Galera repo of the corresponding version is needed in this test because while MariaDB public repositories have the galera library, package sets under test don't. Alternatively the test can download the galera package and install it from the local file.
Major upgrade tests on rpm-based platforms ("major-upgrade-server" step in kvm-rpm* builders in the old buildbot)
The test is performed on a clean image, without any specific non-default packages pre-installed.
Based on the version of the packages under test, the test determines the previous release line, configures the package manager to point at the official MariaDB repo of this release line, and installs selected packages from there. The test starts the server, makes sure it works, creates some data. Then it removes the installed packages and the repo configuration, configures it to point at a new Galera if needed, and installs new packages instead. It checks that there are no old packages left, starts the server, runs mysql_upgrade, makes sure the previously created data is accessible.
The test shouldn't fail if it can't find any previously released version in the repos, as it can happen naturally.
On historical reasons, the test isn't run for zyp-based systems. While adding it to new buildbot, there is no point to make this distinction, the test should run there as well.
Major upgrade tests on deb-based platforms ("upgrade1" and "upgrade2" steps in kvm-deb* builders in the old buildbot)
In the old buildbot, major upgrade tests on deb-based platforms are performed from fixed pre-defined versions of MySQL and MariaDB. The test starts a VM with a pre-installed server, creates some data, configures Galera library, runs new package installation and checks that the server is upgraded and the data is accessible. The tests can be replicated in the new buildbot if it's simpler for the start, but in future they should better be converted to the same approach as other upgrade tests take, installing a previous version dynamically. The pre-defined upgrade can still have a value if on some reason we want to test upgrade from a specific server version rather than the latest one, but it's an extension.
Minor upgrade tests on deb-based, rpm-based and zyp-based platforms ("minor-upgrade-*" steps in the old buildbot)
The test is performed on a clean image, without any specific non-default packages pre-installed.
Based on the version of the packages under test, the test determines the release line, configures the package manager to point at the official MariaDB repo of this release line, and installs selected packages from there. The test starts the server (for RPMs), makes sure it works, creates some data. Then it removes/modifies the repo configuration and installs/upgrades to new packages. It checks that there are no old packages left, starts the server, makes sure the previously created data is accessible.
The test shouldn't fail if it can't find any previously released version in the repos, as it can happen naturally.
There are several types of minor upgrade tests in the old buildbot:
- minor-upgrade-server: Installs and upgrades the minimal possible set of packages to get the server, like suggested in MariaDB KB, e.g. sudo apt-get install mariadb-server.
- minor-upgrade-deps: Installs and upgrades a limited set of packages. The point of the test is to make sure that the new minor version doesn't bring any new dependencies, so before running the upgrade, the test disables all repositories in the package manager configuration, even the default ones. Thus, if a new dependency appears, the upgrade will fail, which is the idea.
- minor-upgrade-all: installs all packages available in MariaDB repository (except for Columnstore, see below) and then upgrades them. The test performs some extra checks, particularly that the list of plugins and engines hasn't changed between the minor versions, and that all binaries are linked with the same libraries as before.
- minor-upgrade-columnstore: installs and upgrades only the server and Columnstore. That's a new hopefully temporary test, added because Columnstore isn't yet mature enough to be a part of "minor-upgrade-all". It should be removed in the future, but then extra logic will need to be added to minor-upgrade-all, with additional Columnstore-specific checks.
An additional category of tests which is currently performed as an extra step after installation on deb-based platforms is SST tests. I don't insist on adding them as they are to the new buildbot. It would be better to communicate with the Galera team and maybe re-implement the tests the way they want.
Things to fix: