Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-36182

Use libaio when running kernel 5.14

Details

    Description

      The change in MDEV-26674 sets innodb_use_native_aio=OFF for any system running kernel < 5.15.3 due to hangs related to liburing. Since RHEL9 is likely to use kernel 5.14 for quite some time (it goes EoL in 2028), the fix in MDEV-26674 will likely affect quite a lot of people, particularly enterprises. Users of Rocky and Alma Linux will also be affected.

      Could MariaDB be compiled with libaio and allow the use of libaio for kernel 5.14 instead of setting innodb_use_native_aio=OFF which impacts performance?

      There's also the possibility that Red Hat will (perhaps already?) backport the fix to 5.14. If that's the case a more rigorous check for kernel version/platform would be needed in ha_innodb.cc.

      Attachments

        Issue Links

          is caused by

          Bug - A problem which impairs or prevents the functions of the product. MDEV-26674 io_uring related hangs on the Linux kernel

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-35886 MariaDB Server frequently hanging, causing data corruption.

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-36234 mysqld: io_uring_queue_init() failed with errno 1

          • Major - Major loss of function.
          • Open

          Activity

            Ascending order - Click to sort in descending order
            danblack Daniel Black added a comment - - edited

            > Could MariaDB be compiled with libaio and allow the use of libaio for kernel 5.14 instead of setting innodb_use_native_aio=OFF which impacts performance?

            It gets a bit messy to affect build time compilation by the kernel version of what is being built.

            Also a fun situation where running containers on a host kernel that isn't mapping to the container build. This was the reason behind runtime checks of version (despite their imperfection).

            Another invasive solution, that I don't immediately advocate, is to link against uring and libaio at the same time.

            > There's also the possibility that Red Hat will (perhaps already?) backport the fix to 5.14.

            Based on Red Hat kernel processes I think its highly likely the RHEL image has this corrected.

            > If that's the case a more rigorous check for kernel version/platform would be needed in ha_innodb.cc.

            I'm open to this. If you've got a uname -v and a uname -r output from a RHEL9 kernel that would be most useful.

            Or given how old the version checks where in MDEV-26674, 2.5 years ago, just remove the version checks and let the responsible people who pay for kernel stability, or are responsible to update kernels to gain fixes, let them gain the use of the features that are fixed.

            Do you have a preference Ali.maria or marko?

            danblack Daniel Black added a comment - - edited > Could MariaDB be compiled with libaio and allow the use of libaio for kernel 5.14 instead of setting innodb_use_native_aio=OFF which impacts performance? It gets a bit messy to affect build time compilation by the kernel version of what is being built. Also a fun situation where running containers on a host kernel that isn't mapping to the container build. This was the reason behind runtime checks of version (despite their imperfection). Another invasive solution, that I don't immediately advocate, is to link against uring and libaio at the same time. > There's also the possibility that Red Hat will (perhaps already?) backport the fix to 5.14. Based on Red Hat kernel processes I think its highly likely the RHEL image has this corrected. > If that's the case a more rigorous check for kernel version/platform would be needed in ha_innodb.cc. I'm open to this. If you've got a uname -v and a uname -r output from a RHEL9 kernel that would be most useful. Or given how old the version checks where in MDEV-26674 , 2.5 years ago, just remove the version checks and let the responsible people who pay for kernel stability, or are responsible to update kernels to gain fixes, let them gain the use of the features that are fixed. Do you have a preference Ali.maria or marko ?
            marko Marko Mäkelä added a comment -

            We should also keep in mind that there may be regressions in the kernel forks of GNU/Linux distributions, such as MDEV-35886. As far as I understand, this bug was never present in an upstream Linux kernel.

            I was thinking that it might be useful to link to both libaio and liburing and enhance innodb_use_native_aio to an enumeration that would include the explicit values innodb_use_native_aio=libaio and innodb_use_native_aio=uring where available. The libaio interface is much older and much less likely to change.

            It could also be a good idea to remove the run-time check for the 5.x kernels, from 11.4 onwards, if 11.4 is the version that would be included in the GNU/Linux distributions that use a Linux 5.14 based kernel fork.

            marko Marko Mäkelä added a comment - We should also keep in mind that there may be regressions in the kernel forks of GNU/Linux distributions, such as MDEV-35886 . As far as I understand, this bug was never present in an upstream Linux kernel. I was thinking that it might be useful to link to both libaio and liburing and enhance innodb_use_native_aio to an enumeration that would include the explicit values innodb_use_native_aio=libaio and innodb_use_native_aio=uring where available. The libaio interface is much older and much less likely to change. It could also be a good idea to remove the run-time check for the 5.x kernels, from 11.4 onwards, if 11.4 is the version that would be included in the GNU/Linux distributions that use a Linux 5.14 based kernel fork.
            danblack Daniel Black added a comment -

            With UBI container images (starting at 10.6) based on RHEL9 userspace and using RHEL9 MariaDB packages if a version check is removed I would prefer it happen in 10.6.

            I'd rather not rely on seccomp filters blocking uring or aio syscalls as that may disappear at any time.

            danblack Daniel Black added a comment - With UBI container images (starting at 10.6) based on RHEL9 userspace and using RHEL9 MariaDB packages if a version check is removed I would prefer it happen in 10.6. I'd rather not rely on seccomp filters blocking uring or aio syscalls as that may disappear at any time.

            People

              marko Marko Mäkelä
              Ali.maria Alasdair Haswell
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.