MDBF-1237 · dev deployment (ENVIRON=DEV)

Foundry test plan

0 / 0

Validates everything on branch MDBF-1237-rev: access control, force inputs, server sources, ci_only, builder naming, the dispatcher's source archive, per-plugin results, base-image install and test, MTR suite handling, the dispatcher waiting for its package builds, pull request status, the status report, and that nothing outside Foundry broke. Items marked new changed since the first version of this plan. Sections build on each other; run them in order. Ticks are saved in this browser only.

00Before you start

Add your GitHub username to FORCE_USERS in settings.py, then deploy master-web and master-migration. Deploy MDBF-1237-rev at f5470a9 or later newit is MDBF-1237 squashed into six commits on the current dev, plus the status report (see 11), with 11.8 and 12.3 enabled. It includes the deb install fix (older builds fail every deb install), the bintar step-name fix (older bintar builds fail on a database error), bintar MTR log saving, CI-only versions and Foundry's own storage (provision it first, see below). Foundry's settings are in settings.py instead of foundry.yaml, and the Debian 11 security-repo workaround is gone (see 09). Provision Foundry's own storage before deploying, as for the connectors: create /srv/buildbot/foundry on the master host next to connectors (buildmaster:buildmaster), share it with hz-bbw10 and bg-arm-rasmus-5 the way connectors reaches them, as /mnt/autofs/foundry_dev_packages (prod: foundry_packages), writable by the containers' buildbot user; then regenerate docker-compose.yaml and restart nginx. newevery Foundry step's docker run would fail without the worker directory. Move the old files (mv /srv/buildbot/packages/foundry/* /srv/buildbot/foundry/) or their links 404. bb_package_rotation.sh doesn't rotate the new directory, as it doesn't rotate connectors. Check the RHEL entitlement on both workers: ls /etc/pki/entitlement /etc/rhsm on hz-bbw10 and bg-arm-rasmus-5.both directories exist and hold the entitlement files. Check the GitHub token gh_mdbci.access_token can write commit statuses on the repository FOUNDRY_REPO_URL points at: RazvanLiviuVarzaru/foundry while testing, MariaDB/foundry after. newa test POST /repos/<repo>/statuses/<sha> returns 201. 401 is a bad token, 403 or 404 no access. Push the Foundry test branches to the fork: testing (six test plugins, see its TESTING.md), testing-2 (two passing ones) and testing-bintar (one bintar-only plugin whose MTR test fails). new Validate every master: ./validate_master_cfg.sh -e DEV."Config file is good!" for every master. master-web and master-migration passed in the dev master image on the old MDBF-1237 branch (19348c7); not rerun on MDBF-1237-rev.

01Access control

Force as a listed user.the dispatcher build starts. Force as a MariaDB member not in FORCE_USERS.403. Rebuild an existing Foundry run as that same user.allowed. Force another (non-Foundry) builder as a MariaDB member.still allowed, unchanged.

02Force: mirrors, no commit

The main path. Press Force with every field left at its default.

03Force: pinned commit

Short SHA or random text in the Foundry commit field.rejected in the dialog. Full SHA of an older main commit. newthe dispatcher's Git step fetches that SHA; the plan log and the archive's name show it, and every build's foundry_revision is its short form. A well-formed full SHA that isn't in the repository. newthe dispatcher's Git step fails; nothing is archived or triggered.

04Force: server source choices

CI tarball + a valid tarbuildnum for 11.4. newthe plan says 11.4: <CI_URL>/<tarbuildnum>/; "Add server CI repo (worker)" and "(base)" run, adding CI's galera repo too, and the mirror steps are skipped; MariaDB-server installs (no "nothing provides galera-4"); files saved under 11.4-<tarbuildnum>/. CI tarball, no number.dispatcher fails, the log says "Nothing was triggered", no child builds start. Skip this version.dispatcher succeeds, the log says "skipped, no builders triggered". Mirrors, with a tarbuildnum typed in anyway. newthe number is ignored: the plan says mirrors, the builds have no tarbuildnum and save under 11.4-mirror/. Bintar builder on a CI tarball run. newDownload CI bintar runs from <CI_URL>/<tarbuildnum>/amd64-centos-7-bintar and the mirror one is skipped. TideSQL on a CI tarball run (a tarbuildnum whose MariaDB-devel ships the mariadb-plugin CMake package). newtidesql builds and installs; MTR runs tidesdb, tidesdb_rpl and tidesdb_galera. The galera suite passes without hanging, which needs ps in the base image and MTR running as buildbot.

05`ci_only` targets

Temporarily add ci_only=["debian-13-deb-autobake"] to the 11.4 MariaDBVersion in settings.py (on CI, not on the 11.4 mirrors) and redeploy. Revert when done.

06Pull requests

On test pull requests in the repository FOUNDRY_REPO_URL points at: the fork RazvanLiviuVarzaru/foundry while testing (e.g. PR #1), MariaDB/foundry after.

07Several plugins: the test plugins

Force with the tip SHA of testing in the Foundry commit field (git rev-parse origin/testing), or open pull requests from it. It also has tidesql from main; see Known issues.

rpm/deb: build step. partial newWARNINGS: dummy_bintar_mtr fails at build, dummy_bintar at package ("No packages found"); the other four build. rpm/deb: install step. partial newWARNINGS: dummy_uninstallable fails on its missing dependency; dummy_mtr_fail, dummy_pkg and dummy_pkg_mtr install, on deb builders too. rpm/deb: suite step. newplugin_suites is dummy_mtr_fail,dummy_pkg_mtr; the log notes that dummy_pkg has no MTR suite and that dummy_uninstallable isn't installed. rpm/deb: MTR failure. failure newdummy_mtr_fail fails and dummy_pkg_mtr passes; var.tar.gz saved under …/<rev>/logs/<builder>/; the Logs link opens. Bintar builder. newall six test plugins build; MTR runs dummy_pkg_mtr, dummy_bintar_mtr, dummy_uninstallable and dummy_mtr_fail, fails on the last and saves its logs; the log notes dummy_pkg and dummy_bintar have no suite. No suites at all: a PR touching only dummy_pkg and dummy_bintar. new"Install test packages" and "Run plugin MTR suite" are skipped (grey) on every builder. All green: a PR from testing-2 touching only its two plugins. newevery builder succeeds; MTR runs dummy_pkg_mtr; the PR status is success. Bintar MTR failure: a PR from testing-bintar, which only has dummy_bintar_mtr_fail. failure newrpm/deb builders fail at the build step (the plugin fails to configure, so no plugin built). The bintar builder builds it, MTR fails, var.tar.gz is saved under …/<rev>/logs/<builder>/ and its Logs link opens.

08Concurrency and cancelling

Two Force runs in a row.the second dispatcher waits until the first run's package builds finish. Stop the dispatcher mid-run.already-triggered package builds keep running (known limitation). Cancel a package build during "Build plugins". newthe step shows cancelled, with no built_plugins property and no "printed no summary" blame on the plugins. Rebuild a finished package build. newit downloads the same archive and builds the same foundry_revision, even if the branch has moved since. Rebuild a package build from before the archive change (no foundry_source_url property). new"Download Foundry" fails at once: "only the dispatcher can start this build". Cancel one of a PR's package builds. newthe dispatcher doesn't succeed, and neither does the PR status.

09Nothing else broken

Connector builds: one C++ and one ODBC, deb and rpm (they share packages.py and ShellStep).unchanged results. Debian 11 connector build. newpasses, unless Debian's security mirror serves a stale bullseye-security Release file again: the workaround is gone, as Debian 11 leaves buildbot soon. GitHub statuses for other builders.still posted; the only new reporter is the Foundry one. They share the gh_mdbci token.

10More MariaDB versions: 11.8 and 12.3

11.8 and 12.3 are enabled in MARIADB_VERSIONS, so every run builds them too. These check what differs from 11.4.

Force with defaults. newthe plan lists 11.4, 11.8 and 12.3; all three schedulers fire; a builder in several versions runs once per version. Targets not in 11.4 (SLES, openSUSE, Debian 13, Ubuntu 26.04, almalinux-8-bintar). newbuilt for 11.8 and 12.3 only; almalinux-8-bintar downloads the newest GA bintar of each. On SLES and openSUSE, "Install test packages" also installs perl (for perl(Memoize)), and MTR runs. SLES and openSUSE: Run plugin MTR suite. newMTR runs the suites (no Can't locate Memoize.pm). If MTR ever fails before running a test, the step says MTR failed before running any test, left no logs to save and saves no plugins.tar.gz or mariadbd.gz. One build per version on shared builders. newa builder in two versions (e.g. sles-1600-rpm-autobake in 11.8 and 12.3) runs twice, once per mariadb_version, even while its worker is busy; no request is skipped in the dispatcher's trigger step or on the Build Requests page. SLES 15.7, SLES 16 and openSUSE 16: Install test packages. newpasses, and the docker run line has -e SYSTEMD_OFFLINE=1. Its log shows Running in chroot, ignoring command 'is-active' from MariaDB-server's %posttrans, not System has not been booted with systemd and exit 107. Needs a deploy with the change in builders.py. Saved files. new11.8 and 12.3 packages and logs under 11.8-mirror/ and 12.3-mirror/, next to, not over, the 11.4 ones.

11Status report

Every run ends with a status page: each plugin's build, install and MTR result per MariaDB version and platform. Use the runs from the earlier sections.

—Known issues

TideSQL 5.0.0 fails to configure against the 11.4 mirrors: MariaDB-devel 11.4.13 doesn't ship the mariadb-plugin CMake package it asks for. Its earlier 5.0.0 packages were built from CI tarball 73276. Expect tidesql to fail at build on mirror runs.
Nothing prunes Foundry's sources/: one archive (about 40 KB) per dispatcher build, left to the artifacts server's retention.
Nothing prunes runs/ either: a few small files per package build, and the page.
The status page can't link to the buildbot builds: a shell step doesn't know a builder's numeric ID. Open them from the dispatcher's "Trigger Foundry Builders" step.
Rebuilding a package build rewrites its status files, but not the run's status.html.
On buildbot 4.x the Status, Logs and Packages links won't show until the shared ShellCommandWithURL moves from start() to run(); the files are still written.

—Already checked locally

buildbot checkconfig passes for master-migration and master-web in the dev master image; all 27 Foundry builders generate.
Moving the settings from foundry.yaml to settings.py changes nothing generated: all 27 builders' steps (names, commands, environment), the Triggerables and the Force and pull request schedulers are byte-identical before and after.
In settings.py, a misspelled target, a target in both targets and ci_only, a version with neither, an rpm/deb target without base_image and a misspelled field each fail at config load with a message saying what is wrong.
Every rendered Foundry script passes bash -n; the one-pass server bintar unpack and the suite discovery give the same results as before on test tarballs.
Status report: unit tests for every status, including a builder failing before each stage, every builder of a version failing early, and a ci_only builder that never reported.
Status report, end to end: the real rendered build, install, suite and MTR scripts, run with stubbed cmake, apt-get, dpkg and MTR, each wrote its status file and kept its exit code; the dispatcher's generated report command turned them into the right page.
On the 2.7 fork, a Force run with 11.4 from a CI tarball and 12.3 skipped: the dispatch step passes foundry_run to every build and records the two Triggerables it fired and each version's source (11.4=1234 11.8=mirror); with one build reporting, the page shows 17 of 11.4's 18 platforms and all 25 of 11.8's with no status, and 12.3 as not triggered; 11.4 links its CI tarball and 11.8 says mirrors.
Generated steps: image switch, RHEL mounts, x86 image and MTR user as intended.
Reporter filter: only pull request builds are reported.
Dispatch rules: mirror, CI and pull request runs trigger the right scheduler groups for ci_only, and pass on the commit and archive.
Build step verdicts and properties: unit tests, and buildbot's own step harness on the 2.x fork and on 4.3, including a cancel.
Source archive: published, served over HTTP, downloaded and unpacked byte for byte; a wrong checksum or missing file retries, then fails.
Saved packages' sha256sums.txt verifies, and a corrupted package fails it.
Plugin discovery gives the same results as before in 7 force and pull request cases.
Test plugins built with run.cmake in rpm, deb and bintar mode, with the expected results and suites.
Install and suite discovery with the test plugins in plain debian:12, centos:stream9 and opensuse/leap:16.0.
Plain debian:12, ubuntu:24.04, centos:stream9 with 11.4: prepare, repo, install and main.1st MTR pass.
All 27 Foundry builders on master-migration have collapseRequests=False; the other 97 keep the default.
MariaDB 12.3.3 from the mirrors on bci-base:15.7 (systemd 254) and opensuse/leap:16.0 (systemd 257): zypper install MariaDB-server MariaDB-test exits 107 without SYSTEMD_OFFLINE=1, and 0 with it, all packages installed.
SLES 15.7, SLES 16, Leap 16 (12.3.3): MariaDB-test only requires /usr/bin/perl, and perl-base has no Memoize.pm, so MTR couldn't start. With perl(Memoize) in test_packages (perl-Memoize on CentOS/UBI 8-10), main.1st and main.select pass. The rendered SLES 16 steps: MTR not starting saves nothing and says so; a failing test saves only var.tar.gz; passing saves nothing.