Uploaded image for project: 'MariaDB Foundation Development'
  1. MariaDB Foundation Development
  2. MDBF-522

Add performance regression testing to buildbot

    XMLWordPrintable

Details

    • Task
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      Buildbot should run performance testing to show any performance regressions or improvements before they are found by other means. Preferably on the Intel hardware provided.

      The proposal is this:
      1. We run a nightly job that executes HammerDB single thread and something like 64 threads (see https://github.com/TPC-Council/HammerDB/tree/master/Docker/scripts/python/maria/tprocc)

      2. Weekly or monthly there is a job that runs the same full benchmark as Intel, which is the HammerDB above for 1, 2, 4, 8 threads and then increments of 4 threads until 100 threads.

      Part 1 should notify us if there is a statistically significant regression between days. Part 2 should store reports ready to graph somewhere (email?), this will be useful to show performance improvements over time for blogs and conference talks.

      This should be against a release build of MariaDB instead of a debug build.

      Attachments

        Activity

          People

            Unassigned Unassigned
            TheLinuxJedi Andrew Hutchings
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: