Uploaded image for project: 'MariaDB Foundation Development'
  1. MariaDB Foundation Development
  2. MDBF-1265

Cap MTR --parallel per builder and worker

    XMLWordPrintable

Details

    • Task
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • BB V1.17
    • BB V1.18
    • Buildbot
    • None

    Description

      Buildbot's one-size-fits-all rule has been that a builder asking for N jobs (vCPUs) runs MTR with --parallel=2N, and that this is safe on a host where several builds are scheduled. It relies on most of our hosts having about 2 GB of memory per vCPU, and on a test run peaking at about 2 GB per job.

      Some builder configurations, such as --big-test or the sanitizers, are outliers and no longer fit this rule.

      With this patch:

      • Builders keep declaring jobs, which sets the compile parallelism and the job claim on the worker, so they stay fast while compiling.
      • A builder can cap MTR --parallel on specific workers:

        get_config(mtr_max_parallel={"<worker>": N})
        

        • The cap is applied when the build starts, for the worker running it.
        • Workers not listed keep 2 x jobs.
        • A worker name that is not one of the builder's workers fails the config load.

      Attachments

        Activity

          People

            rvarzaru Varzaru Razvan-Liviu
            rvarzaru Varzaru Razvan-Liviu
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0d
                0d
                Logged:
                Time Spent - 5h
                5h