Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-37924

buf_pool.mutex contention under I/O-limited OLTP workload

    XMLWordPrintable

Details

    Description

      Goal

      Reduce buf_pool.mutex contention observed under heavy I/O-limited OLTP workloads (HammerDB TPROC-C UAW), where the bottleneck is page-flushing — both long stalls at checkpoint and free-list starvation when the buffer pool is under pressure.

      Current plan (revised 2026-06-16)

      After PERF-475 results came in, the priority order for this Epic has changed. Flushing is the dominant issue for the RW workload tested here; page_hash contention work moves to the front.

      Status Workstream Sprint Notes
      Primary MDEV-39084 — Split buf_pool.page_hash to fil_space_t::page_hash Q3 2026 Server Maintenance Promoted to top priority. Targets relieving contention around flushing. Targeted release: 11.8. Pulled into the Q3 Dev Sprint.
      Read-heavy track MDEV-33966 — Sysbench performance regression with concurrent workloads Q4 2026 Server Maintenance STALLED. PERF-475 confirms its benefits are concentrated on read-only / read-heavy workloads, so it is no longer the lever for this Epic's RW workload. Patch exists under PERF-475 testing. Scope reduced to Phase 1 (profiling / RCA) until read-heavy benefit is requantified.
      Completed MDEV-38814 — index_lock_upgrades from btr_cur_need_opposite_intention() Q2 2026 Server Development CLOSED. Delivered in

      Under investigation (no MDEV yet)

      Alessandro has pushed several small candidate commits under the PERF-475 umbrella for testing. Rahul Raj is running those through a perf round. Any that prove promising will be opened as new MDEVs and linked here as Epic children.

      Background

      The contention surfaces in HammerDB TPROC-C UAW runs at 64G and 128G buffer pool sizes (see attachments). Flame graphs and off-CPU profiles point at the flushing path rather than the page-hash lookup path for this specific workload — the original assumption (that page_hash sharding alone would carry the win) does not hold; flushing-side wins matter more.

      Definition of done for this Epic

      • MDEV-39084 lands and demonstrates a measurable reduction of buf_pool.mutex contention on the HammerDB TPROC-C UAW reference workload.
      • Any PERF-475-derived MDEVs that get opened are landed or formally deferred with rationale.
      • MDEV-33966 either lands its patch under the read-heavy scope or is decoupled into its own tracking Epic if it grows out of scope here.
      • No remaining "background research" line items — anything we learn from this Epic that isn't fixed gets either a new MDEV or a documented "won't fix" rationale.

      Sprint allocation guidance

      This Epic remains a research / coordination container — it does not consume sprint capacity itself (5d estimate kept). All actual development effort is tracked under the child MDEVs above.

      Attachments

        1. Drop_Percentage.png
          Drop_Percentage.png
          28 kB
        2. Flamegraph_Use_all_Warehouse_128G.zip
          4.76 MB
        3. Flamegraph_Use_all_Warehouse_64G.zip
          4.25 MB
        4. 64G_use_all_warehouse.png
          64G_use_all_warehouse.png
          64 kB
        5. 128G_use_all_warehouse.png
          128G_use_all_warehouse.png
          90 kB
        6. MySQL_MariaDB_Graph.png
          MySQL_MariaDB_Graph.png
          46 kB
        7. buf_pool_mutex.png
          buf_pool_mutex.png
          55 kB
        8. OFFCPU_MDEV_37924.zip
          632 kB

        Issue Links

          Activity

            People

              alessandro.vetere Alessandro Vetere
              adamluciano Adam Luciano
              Votes:
              1 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - 5d Original Estimate - 5d
                  5d
                  Remaining:
                  Remaining Estimate - 0d
                  0d
                  Logged:
                  Time Spent - 7d 7h
                  7d 7h

                  Git Integration

                    Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.