Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-10814

Feature request: Optionally exclude large buffers from core dumps

Details

    Description

      As the size of core dump files by default mostly equals the process size producing core dumps can become an issue on systems with memory buffers, esp. large innodb_buffer_pool_size.

      There needs to be enough file system space to store such large core dumps, and with multi gigabyte files it also takes a non-trivial amount of time to write these, so delaying process restart quite a bit.

      And then there's also a security aspect to it: as the core dump contains the complete innodb buffer pool it contains a substantial amount, or even all, of the actual user data in the database.

      At the same time actual buffer contents are rarely needed when doing a post mortem analysis (usually we only need stack frames and a few pieces of local data).

      So I'm proposing a server option to exclude certain buffers from core dumps by marking them as DONOTDUMP with the madvise() system call.

      Attachments

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. MDEV-18946 munmap of 1 byte during shutdown is EINVAL

          • Major - Major loss of function.
          • Closed
          relates to

          Task - A task that needs to be done. MDEV-17159 Document what is included in core dumps

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. MDEV-16605 Always include buf_madvise_do_dump in binaries.

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-20684 innodb: use madvise CORE/NOCORE on FreeBSD for selective large allocations

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-21741 [Warning] InnoDB: Failed to set memory to MADV_DONTDUMP: Function not implemented

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Task - A task that needs to be done. MDEV-22186 Please add innodb_buffer_pool_in_core_file option (Facebook/MySQL 8 cherry pick patch)

          • Major - Major loss of function.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            hholzgra Hartmut Holzgraefe created issue -
            elenst Elena Stepanova made changes -
            Field Original Value New Value
            Environment Linux
            Issue Type Bug [ 1 ] Task [ 3 ]
            elenst Elena Stepanova made changes -
            Priority Trivial [ 5 ] Minor [ 4 ]
            danblack Daniel Black made changes -
            Labels patch
            danblack Daniel Black made changes -
            Labels patch contribution patch
            danblack Daniel Black made changes -
            Summary Feature requst: Optionally exclude large buffers from core dumps Feature request: Optionally exclude large buffers from core dumps
            svoj Sergey Vojtovich made changes -
            Fix Version/s 10.1 [ 16100 ]
            Assignee Vladislav Vaintroub [ wlad ]
            Description

            As the size of core dump files by default mostly equals the process size producing core dumps can become an issue on systems with memory buffers, esp. large innodb_buffer_pool_size.

            There needs to be enough file system space to store such large core dumps, and with multi gigabyte files it also takes a non-trivial amount of time to write these, so delaying process restart quite a bit.

            And then there's also a security aspect to it: as the core dump contains the complete innodb buffer pool it contains a substantial amount, or even all, of the actual user data in the database.

            At the same time actual buffer contents are rarely needed when doing a post mortem analysis (usually we only need stack frames and a few pieces of local data).

            So I'm proposing a server option to exclude certain buffers from core dumps by marking them as {{DONOTDUMP}} with the {{madvise()}} system call.
            As the size of core dump files by default mostly equals the process size producing core dumps can become an issue on systems with memory buffers, esp. large innodb_buffer_pool_size.

            There needs to be enough file system space to store such large core dumps, and with multi gigabyte files it also takes a non-trivial amount of time to write these, so delaying process restart quite a bit.

            And then there's also a security aspect to it: as the core dump contains the complete innodb buffer pool it contains a substantial amount, or even all, of the actual user data in the database.

            At the same time actual buffer contents are rarely needed when doing a post mortem analysis (usually we only need stack frames and a few pieces of local data).

            So I'm proposing a server option to exclude certain buffers from core dumps by marking them as {{DONOTDUMP}} with the {{madvise()}} system call.
            Due Date 2017-05-05
            Labels contribution patch contribution foundation patch
            Priority Minor [ 4 ] Major [ 3 ]
            svoj Sergey Vojtovich made changes -
            Assignee Vladislav Vaintroub [ wlad ] Marko Mäkelä [ marko ]
            marko Marko Mäkelä made changes -
            Component/s Storage Engine - InnoDB [ 10129 ]
            Fix Version/s 10.3.5 [ 22905 ]
            Fix Version/s 10.1 [ 16100 ]
            Resolution Fixed [ 1 ]
            Status Open [ 1 ] Closed [ 6 ]
            hholzgra Hartmut Holzgraefe made changes -
            Resolution Fixed [ 1 ]
            Status Closed [ 6 ] Stalled [ 10000 ]
            serg Sergei Golubchik made changes -
            Fix Version/s 10.3 [ 22126 ]
            Fix Version/s 10.3.5 [ 22905 ]
            marko Marko Mäkelä made changes -
            Assignee Marko Mäkelä [ marko ] Oleksandr Byelkin [ sanja ]
            marko Marko Mäkelä made changes -
            Component/s Query Cache [ 10120 ]
            sanja Oleksandr Byelkin made changes -
            Fix Version/s 10.3.7 [ 23005 ]
            Fix Version/s 10.3 [ 22126 ]
            Resolution Fixed [ 1 ]
            Status Stalled [ 10000 ] Closed [ 6 ]
            ratzpo Rasmus Johansson (Inactive) made changes -
            GeoffMontee Geoff Montee (Inactive) made changes -
            danblack Daniel Black made changes -
            anel Anel Husakovic made changes -
            GeoffMontee Geoff Montee (Inactive) made changes -
            alice Alice Sherepa made changes -
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 77108 ] MariaDB v4 [ 132947 ]
            marko Marko Mäkelä made changes -

            People

              sanja Oleksandr Byelkin
              hholzgra Hartmut Holzgraefe
              Votes:
              4 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.