Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-41261

wsrep: rejected oversized writeset leaks memory, eventually OOM-kills mariadbd

    XMLWordPrintable

Details

    • Bug
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • 11.8.9
    • 11.8
    • Galera
    • None
    • Can result in hang or crash
    • Hide
      Rejecting an oversized transaction with `ERROR 1105 (HY000): Maximum writeset size exceeded` does not release the memory that was allocated while building that writeset. Retrying the same oversized transaction repeatedly grows `mariadbd`'s memory until the process is OOM-killed by the kernel - and this happens regardless of how much memory is provisioned, since the growth appears unbounded rather than converging on some fixed size proportional to the transaction.
      Show
      Rejecting an oversized transaction with `ERROR 1105 (HY000): Maximum writeset size exceeded` does not release the memory that was allocated while building that writeset. Retrying the same oversized transaction repeatedly grows `mariadbd`'s memory until the process is OOM-killed by the kernel - and this happens regardless of how much memory is provisioned, since the growth appears unbounded rather than converging on some fixed size proportional to the transaction.

    Description

        1. How it's triggered

      A background job (in our case, a third-party JMAP server's account-deletion task, but the trigger is generic: any sufficiently large single transaction) attempts to `DELETE` several hundred thousand rows belonging to one account in one transaction. The resulting writeset exceeds `wsrep_max_ws_size` and MariaDB correctly rejects it:

      ```
      ERROR 1105 (HY000): Maximum writeset size exceeded
      ```

      The caller retries the same (still oversized) transaction on a backoff schedule. Over a ~70-minute window we saw this rejection recur 4 times across our 3 nodes, and 2 of those retries ended in the node's `mariadbd` being OOM-killed by the kernel.

        1. Observed behavior (evidence)

      Kernel OOM-killer report (`dmesg -T`) from two independent nodes, for two independent occurrences of the same rejected transaction, ~52 minutes apart:

      ```
      Node A: Memory cgroup out of memory: Killed process (mariadbd) total-vm:8602138600kB, anon-rss:16152468kB, file-rss:13940kB
      Node B: Memory cgroup out of memory: Killed process (mariadbd) total-vm:8602111008kB, anon-rss:16130300kB, file-rss:13552kB
      ```

      Note how close these are - anon-rss within 0.14% of each other, file-rss within 2.8%, on two different physical hosts. That's a deterministic memory footprint tied to this specific operation, not noise.

      The `anon-rss`/`file-rss` split is the interesting part. Contrast with a live, healthy `mariadbd` on the same node right now, same workload otherwise:

      ```
      Normal: RssAnon: 11581684 kB (~11.0 GiB) RssFile: 2729380 kB (~2.6 GiB) total ~13.65 GiB
      At OOM kill: anon-rss: ~16.14 GiB file-rss: ~13.6 MiB total ~15.4 GiB (= cgroup limit)
      ```

      In normal operation there's a healthy mix of anonymous and file-backed (page-cache) resident memory. At the moment of the OOM kill, file-backed RSS had been reduced to almost nothing - the kernel's own reclaim was clearly working hard right up to the kill (`pgscan`/`pgsteal` counters in the OOM report show tens of millions of pages scanned/reclaimed), but couldn't free enough because nearly all resident memory was anonymous (private, application-owned, not reclaimable by the kernel on its own). That anonymous allocation grew to just under the container's memory limit and stayed there until killed.

      `total_vm` (~8.6TB) is not unusual by itself - it matches this same node's normal steady-state `VmSize` right now, so that's just baseline glibc/InnoDB address-space reservation, not part of the problem.

        1. What doesn't work

      We've raised the container memory limit for this cluster repeatedly over rouecifically to get past recurring OOM kills (15Gi -> 16Gi -> 17Gi -> back down to 15872Mi as the currently active value) - each time, the same class of OOM eventually recurs once a large-enough transaction/writeset is attempted again, right at whatever the
      current limit is. We've also hit this same pattern from our own application'rge-transaction load in the past, independent of ever hitting the 1105 limitspecifically - raising memory has never durably fixed it. That's what makes us suspect this isn't "just needs more RAM" but a genuine unbounded-growth/non-release issue somewhere in
      writeset construction or certification-failure handling.

        1. Expected behavior

      When a writeset is rejected for exceeding `wsrep_max_ws_size` (or, more genetransaction fails/rolls back), the memory allocated while building it should bereleased back to the process/OS, the same way we assume it is for a transaction that completes successfully. Right now it looks like that release either doesn't happen, or doesn't
      happen completely, on the rejection path.

        1. Notes

      We don't have wsrep/InnoDB-internal instrumentation (heap profiling, valgrinctual incident - what's above is kernel-level cgroup/OOM evidence, not asource-level diagnosis. Happy to provide the full `dmesg` OOM reports, or try to build a minimal, deliberately-oversized-transaction reproduction against a memory-limited container if that would help - let us know what would be most useful.

      Attachments

        Activity

          People

            seppo Seppo Jaakola
            ralfbecker Ralf Becker
            Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.