Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-29910

mysql_install_db fails with libpam-tmpdir

Details

    • Bug
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • 10.6.10
    • 10.6
    • None
    • OS: Debian GNU/Linux (unstable)
      Package: mariadb-server-10.6 (version 1:10.6.10-1)

    Description

      On Debian GNU/Linux, when the package libpam-tmpdir is installed, mysql_install_db script fails during post install setup. As a result, mariadb daemon fails to start. The following error message is shown:

      rm -rf /var/lib/mysql ; mysql_install_db --rpm --cross-bootstrap --user=mysql --disable-log-bin --skip-test-db
      

      2022-10-28 19:33:00 0 [ERROR] mariadbd: Can't create/write to file '/tmp/user/0/ib2C7oNS' (Errcode: 13 "Permission denied")
      2022-10-28 19:33:00 0 [ERROR] InnoDB: Unable to create temporary file; errno: 13
      2022-10-28 19:33:00 0 [ERROR] mariadbd: Can't create/write to file '/tmp/user/0/ibykVtxz' (Errcode: 13 "Permission denied")
      2022-10-28 19:33:00 0 [ERROR] InnoDB: Unable to create temporary file; errno: 13
      2022-10-28 19:33:00 0 [ERROR] InnoDB: Database creation was aborted with error Generic error. You may need to delete the ibdata1 file before trying to start up again.
      2022-10-28 19:33:00 0 [ERROR] Plugin 'InnoDB' init function returned error.
      2022-10-28 19:33:00 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
      2022-10-28 19:33:00 0 [ERROR] Unknown/unsupported storage engine: InnoDB
      2022-10-28 19:33:00 0 [ERROR] Aborting
       
      Installation of system tables failed!  Examine the logs in
      /var/lib/mysql for more information.
      

      Environment

      On FreedomBox (a pure blend of Debian), several applications that depend on mariadb fail to install when running on Debian testing/unstable. This is due to mariadb not running soon after installation. FreedomBox installs that package libpam-tmpdir by default. If this package is removed, mariadb server is running successfully after install.

      This bug was reproduced on Debian unstable (as of 2022-10-28) with mariadb-server package version 1:10.6.10-1+b1.

      Workarounds

      1. If libpam-tmpdir package is removed, the installation and daemon start succeed.
      2. When the environment variable TMPDIR is set to empty value, the mysql_install_db command succeeds. Example:

        rm -rf /var/lib/mysql ; TMPDIR= mysql_install_db --rpm --cross-bootstrap --user=mysql --disable-log-bin --skip-test-db

      3. When mysql_install_db is not run are root, the problem is not observed. Example:

        rm -rf /var/lib/mysql ; mkdir /var/lib/mysql; chown mysql:mysql /var/lib/mysql/ ; sudo -u mysql mysql_install_db --rpm --cross-bootstrap --user=mysql --disable-log-bin --skip-test-db

      Regression:

      This error does not occur on Debian stable (bullseye) where mariadb package version is 1:10.5.15-0+deb11u1. Hence this is a regression since that version.

      Analysis

      According to pam-tmpdir: "Many programs use $TMPDIR for storing temporary files. Not all of them are good at securing the permissions of those files. libpam-tmpdir sets $TMPDIR and $TMP for PAM sessions and sets the permissions quite tight. This helps system security by having an extra layer of security, making such symlink attacks and other /tmp based attacks harder or impossible".

      Errors like the one being reported are typically seen when directories/files are created by root user in the $TMPDIR and later a non-root user tries to access those files without any further permission changes. libpam-tmpdir tries to ensure that temporary files created by one user are not accidentally accessible to unauthorized users.

      During 10.6.x release cycle a change was introduced that makes this mistake. It creates files as 'root' and then tries to access them as 'mysql' user. The problem can be fixed by:

      1. Copying the files temporarily created by 'root' user to a location accessible to the 'mysql' user and then setting proper ownership, or by
      2. Creating all the temporary files with 'mysql' user to start with.

      Attachments

        Issue Links

          Activity

            Thank you for good bug report sunil.

            After little bit digging this I like to ask some questions if I understood correctly. does I understand correctly that pam-tmpdir does this:

            1. Create dir

              /tmp/users

              with chmod 0711 (rwx-x-x) which is little problematic but if it's always run by root when creating new tmp-dir then it works

            2. After that it create

              /tmp/users/uid

              in this case as we use root it's

              /tmp/users/0

              with chmod 0700 (rwx------)

            3. Then try to write to

              /tmp/users/0

              with user mysql

            4. Fail with error message

            I would see that TMPDIR should be filled with

            mkstemp

            neither than just unset it.

            illuusio Tuukka Pasanen added a comment - Thank you for good bug report sunil . After little bit digging this I like to ask some questions if I understood correctly. does I understand correctly that pam-tmpdir does this: Create dir /tmp/users with chmod 0711 (rwx- x -x) which is little problematic but if it's always run by root when creating new tmp-dir then it works After that it create /tmp/users/uid in this case as we use root it's /tmp/users/0 with chmod 0700 (rwx------) Then try to write to /tmp/users/0 with user mysql Fail with error message I would see that TMPDIR should be filled with mkstemp neither than just unset it.

            1. Create dir

            /tmp/users

            with chmod 0711 (rwx-x-x) which is little problematic but if it's always run by root when creating new tmp-dir then it works

            It looks like /tmp/users is created by the PAM module with superuser privileges. I did the following to confirm:

            root@freedombox:~# rm -rf /tmp/user
            root@freedombox:~# exit

            (SSH into machine as user fbx)
            fbx@freedombox:~$ ls -lad /tmp/user/
            drwx-x-x 3 root root 60 Nov 7 10:22 /tmp/user/
            fbx@freedombox:~$ ls -lad /tmp/user/$UID
            drwx------ 2 fbx root 40 Nov 7 10:22 /tmp/user/10000

            sunil Sunil Mohan Adapa added a comment - 1. Create dir /tmp/users with chmod 0711 (rwx-x-x) which is little problematic but if it's always run by root when creating new tmp-dir then it works It looks like /tmp/users is created by the PAM module with superuser privileges. I did the following to confirm: root@freedombox:~# rm -rf /tmp/user root@freedombox:~# exit (SSH into machine as user fbx) fbx@freedombox:~$ ls -lad /tmp/user/ drwx- x -x 3 root root 60 Nov 7 10:22 /tmp/user/ fbx@freedombox:~$ ls -lad /tmp/user/$UID drwx------ 2 fbx root 40 Nov 7 10:22 /tmp/user/10000
            illuusio Tuukka Pasanen added a comment - - edited

            Thank you for fast reply. Need little bit digging and testing on my side to get this straight. danblack do you have opinion on this?

            illuusio Tuukka Pasanen added a comment - - edited Thank you for fast reply. Need little bit digging and testing on my side to get this straight. danblack do you have opinion on this?
            danblack Daniel Black added a comment -

            Opinion on "I would see that TMPDIR should be filled with mkstemp neither than just unset it.":

            • If you're thinking of something like TMPDIR=$(mktemp -d) in the mysql_install_db then that would create a root:root 700 permission directory that --user=mysql wouldn't be able to write to. chown to user, problematic, path might be not +x, and selinux/apparmor restrictions could apply.
            • Or are you thinking of EPERM errors in the server code to call mkstemp and changing the env TMPDIR (or DEFAULT_TMPDIR) there to the created directory? Messy, but plausible.
            danblack Daniel Black added a comment - Opinion on "I would see that TMPDIR should be filled with mkstemp neither than just unset it.": If you're thinking of something like TMPDIR=$(mktemp -d) in the mysql_install_db then that would create a root:root 700 permission directory that --user=mysql wouldn't be able to write to. chown to user, problematic, path might be not +x, and selinux/apparmor restrictions could apply. Or are you thinking of EPERM errors in the server code to call mkstemp and changing the env TMPDIR (or DEFAULT_TMPDIR ) there to the created directory? Messy, but plausible.

            I don't know the exact code we are talking about here but in order for 'mysql' user to get a proper value for TMPDIR, one could start a proper PAM session. So, instead of doing sometime like 'setuid ; do task' one could do 'su -c task'.

            sunil Sunil Mohan Adapa added a comment - I don't know the exact code we are talking about here but in order for 'mysql' user to get a proper value for TMPDIR, one could start a proper PAM session. So, instead of doing sometime like 'setuid ; do task' one could do 'su -c task'.

            People

              Unassigned Unassigned
              sunil Sunil Mohan Adapa
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.