Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-12310

openat(<directory>, ...O_EXEC) fails on Illumos / Solaris.

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • 10.1.22
    • 5.5.56, 10.1.23, 10.0.31
    • Server
    • None
    • OmniOS r151020 - Illumos derivative.
      Solaris 11.3 (maybe earlier).

    Description

      MariaDB fails to start with error:

      ERROR: 1030  Got error 8 "Exec format error" from storage engine MyISAM
      

      This is due to openat() being called on a directory with the O_EXEC flag set.

      The man page for openat() on these platforms specifies that:

             O_EXEC      Open ordinary file for execute only.
      

      and returns:

             ENOEXEC         The O_EXEC access mode was specified and the file to be
                             opened is not an ordinary file.
      

      Trace shows that this is occurring:

      28152/27:       openat(4294967295, "/data", O_EXEC|O_NOFOLLOW)  Err#8 ENOEXEC
      28152/27:       write(2, " E R R O R :  ", 7)                   = 7
      

      My MariaDB files are under /data/mariadb/ — the parent directory is being opened with O_EXEC hence the failure.

      The last working version that I've tried is 10.1.19

      Attachments

        Issue Links

          Activity

            fiddaman Andy added a comment - - edited

            Tried this quick patch and I can confirm that my server now works. This isn't the proper fix, just a quick test - the semantics of the various operating systems need taking into account.

            --- mariadb-10.1.22.distrib/mysys/mysys_priv.h  Sat Mar 11 19:09:10 2017
            +++ mariadb-10.1.22/mysys/mysys_priv.h  Mon Mar 20 13:27:02 2017
            @@ -108,9 +108,13 @@
             
             void my_error_unregister_all(void);
             
            +#if !defined(O_PATH) && defined(O_SEARCH) /* Illumos */
            +#define O_PATH O_SEARCH
            +#else
             #if !defined(O_PATH) && defined(O_EXEC) /* FreeBSD */
             #define O_PATH O_EXEC
             #endif
            +#endif
             
             #ifdef O_PATH
             #define HAVE_OPEN_PARENT_DIR_NOSYMLINKS
            

            fiddaman Andy added a comment - - edited Tried this quick patch and I can confirm that my server now works. This isn't the proper fix, just a quick test - the semantics of the various operating systems need taking into account. --- mariadb-10.1.22.distrib/mysys/mysys_priv.h Sat Mar 11 19:09:10 2017 +++ mariadb-10.1.22/mysys/mysys_priv.h Mon Mar 20 13:27:02 2017 @@ -108,9 +108,13 @@   void my_error_unregister_all(void);   +#if !defined(O_PATH) && defined(O_SEARCH) /* Illumos */ +#define O_PATH O_SEARCH +#else #if !defined(O_PATH) && defined(O_EXEC) /* FreeBSD */ #define O_PATH O_EXEC #endif +#endif   #ifdef O_PATH #define HAVE_OPEN_PARENT_DIR_NOSYMLINKS
            fiddaman Andy added a comment - - edited

            I've done some reading around on this and it seems that O_PATH is Linux specific whereas O_SEARCH/O_EXEC are defined by POSIX. They are almost equivalent except that when opening a symlink with O_NOFOLLOW:

            • Linux O_PATH|O_NOFOLLOW opens a file descriptor referring to the symlink inode itself.
            • POSIX O_NOFOLLOW with O_SEARCH or O_EXEC forces failure if the pathname refers to a symlink.

            These are the results of running a simple test program using my_open_parent_dir_nosymlinks() on Linux and illumos where the illumos variant is using O_SEARCH:

            Linux:

            % ls -li ~/test
            7871180 lrwxrwxrwx 1 root root    5 Mar 23 10:56 fred -> wilma/
            7871179 drwxr-xr-x 2 root root 4.0K Mar 23 10:56 wilma/
             
            % ./pdfd ~/test/wilma/barney.txt
            RET 3 (inode: 7871179), barney.txt
            % ./pdfd ~/test/fred/barney.txt
            RET 3 (inode: 7871180), barney.txt
            

            illumos (same directory structure):

            % ./pdfd ~/test/wilma/barney.txt
            RET 4 (inode: 70302), barney.txt
            % ./pdfd ~/test/fred/barney.txt
            RET 4927 (inode: -1), NULL
            

            I don't have access to *BSD at present but I would expect the use of O_EXEC|O_NOFOLLOW to follow POSIX and therefore behave as O_SEARCH does on illumos.

            Which behaviour is really wanted here? The POSIX result seems better from my limited understanding of the code where this is used (still reading...)

            [1] https://sourceware.org/ml/libc-alpha/2013-08/msg00016.html
            [2] https://lkml.org/lkml/2013/8/2/856

            fiddaman Andy added a comment - - edited I've done some reading around on this and it seems that O_PATH is Linux specific whereas O_SEARCH/O_EXEC are defined by POSIX. They are almost equivalent except that when opening a symlink with O_NOFOLLOW: Linux O_PATH|O_NOFOLLOW opens a file descriptor referring to the symlink inode itself. POSIX O_NOFOLLOW with O_SEARCH or O_EXEC forces failure if the pathname refers to a symlink. These are the results of running a simple test program using my_open_parent_dir_nosymlinks() on Linux and illumos where the illumos variant is using O_SEARCH: Linux: % ls -li ~/test 7871180 lrwxrwxrwx 1 root root 5 Mar 23 10:56 fred -> wilma/ 7871179 drwxr-xr-x 2 root root 4.0K Mar 23 10:56 wilma/   % ./pdfd ~/test/wilma/barney.txt RET 3 (inode: 7871179), barney.txt % ./pdfd ~/test/fred/barney.txt RET 3 (inode: 7871180), barney.txt illumos (same directory structure): % ./pdfd ~/test/wilma/barney.txt RET 4 (inode: 70302), barney.txt % ./pdfd ~/test/fred/barney.txt RET 4927 (inode: -1), NULL I don't have access to *BSD at present but I would expect the use of O_EXEC|O_NOFOLLOW to follow POSIX and therefore behave as O_SEARCH does on illumos. Which behaviour is really wanted here? The POSIX result seems better from my limited understanding of the code where this is used (still reading...) [1] https://sourceware.org/ml/libc-alpha/2013-08/msg00016.html [2] https://lkml.org/lkml/2013/8/2/856

            We don't care what behavior of these two will be, this code doesn't rely on linux specific O_PATH|O_NOFOLLOW.

            What I need is openat() to fail if there's a symlink involved. On linux it'll open a symlink, and it will fail on the next openat(), because symlink is not a directory, and symlink's fd cannot be used as a first argument for openat(). With POSIX behavior it'll fail right away. Which is just fine too.

            The logic of my_open_parent_dir_nosymlinks() is as follows — the path argument comes from realpath(), there must be no symlinks in it. If a symlink is found, it means someone replaced a path component with a symlink between realpath() and my_open_parent_dir_nosymlinks() calls, clearly an attempt to exploit a TOCTOU race condition.

            serg Sergei Golubchik added a comment - We don't care what behavior of these two will be, this code doesn't rely on linux specific O_PATH|O_NOFOLLOW . What I need is openat() to fail if there's a symlink involved. On linux it'll open a symlink, and it will fail on the next openat() , because symlink is not a directory, and symlink's fd cannot be used as a first argument for openat() . With POSIX behavior it'll fail right away. Which is just fine too. The logic of my_open_parent_dir_nosymlinks() is as follows — the path argument comes from realpath() , there must be no symlinks in it. If a symlink is found, it means someone replaced a path component with a symlink between realpath() and my_open_parent_dir_nosymlinks() calls, clearly an attempt to exploit a TOCTOU race condition.
            fiddaman Andy added a comment -

            Thanks for the explanation. It would seem that you should use O_SEARCH if available, O_EXEC if not and fallback to O_PATH, although the *BSD behaviour falls into 'undefined' as per POSIX.

            fiddaman Andy added a comment - Thanks for the explanation. It would seem that you should use O_SEARCH if available, O_EXEC if not and fallback to O_PATH, although the *BSD behaviour falls into 'undefined' as per POSIX.
            fiddaman Andy added a comment -

            Here's my updated local patch:

            diff -ru mariadb-10.1.22.distrib/mysys/my_symlink.c mariadb-10.1.22/mysys/my_symlink.c
            --- mariadb-10.1.22.distrib/mysys/my_symlink.c  Sat Mar 11 19:09:10 2017
            +++ mariadb-10.1.22/mysys/my_symlink.c  Fri Mar 24 08:31:32 2017
            @@ -244,7 +244,7 @@
                   return pathname + (s - buf);
                 }
             
            -    fd = openat(dfd, s, O_NOFOLLOW | O_PATH);
            +    fd = openat(dfd, s, O_NOFOLLOW | O_SEARCH);
                 if (fd < 0)
                   goto err;
             
            diff -ru mariadb-10.1.22.distrib/mysys/mysys_priv.h mariadb-10.1.22/mysys/mysys_priv.h
            --- mariadb-10.1.22.distrib/mysys/mysys_priv.h  Sat Mar 11 19:09:10 2017
            +++ mariadb-10.1.22/mysys/mysys_priv.h  Fri Mar 24 08:31:18 2017
            @@ -108,11 +108,15 @@
             
             void my_error_unregister_all(void);
             
            -#if !defined(O_PATH) && defined(O_EXEC) /* FreeBSD */
            -#define O_PATH O_EXEC
            +#ifndef O_SEARCH       /* POSIX */
            +#if defined(O_EXEC)    /* FreeBSD */
            +#define O_SEARCH O_EXEC
            +#elif defined(O_PATH)  /* Linux */
            +#define O_SEARCH O_PATH
             #endif
            +#endif /* !defined(O_SEARCH) */
             
            -#ifdef O_PATH
            +#ifdef O_SEARCH
             #define HAVE_OPEN_PARENT_DIR_NOSYMLINKS
             const char *my_open_parent_dir_nosymlinks(const char *pathname, int *pdfd);
             #define NOSYMLINK_FUNCTION_BODY(AT,NOAT)                                \
            

            fiddaman Andy added a comment - Here's my updated local patch: diff -ru mariadb-10.1.22.distrib/mysys/my_symlink.c mariadb-10.1.22/mysys/my_symlink.c --- mariadb-10.1.22.distrib/mysys/my_symlink.c Sat Mar 11 19:09:10 2017 +++ mariadb-10.1.22/mysys/my_symlink.c Fri Mar 24 08:31:32 2017 @@ -244,7 +244,7 @@ return pathname + (s - buf); }   - fd = openat(dfd, s, O_NOFOLLOW | O_PATH); + fd = openat(dfd, s, O_NOFOLLOW | O_SEARCH); if (fd < 0) goto err;   diff -ru mariadb-10.1.22.distrib/mysys/mysys_priv.h mariadb-10.1.22/mysys/mysys_priv.h --- mariadb-10.1.22.distrib/mysys/mysys_priv.h Sat Mar 11 19:09:10 2017 +++ mariadb-10.1.22/mysys/mysys_priv.h Fri Mar 24 08:31:18 2017 @@ -108,11 +108,15 @@   void my_error_unregister_all(void);   -#if !defined(O_PATH) && defined(O_EXEC) /* FreeBSD */ -#define O_PATH O_EXEC +#ifndef O_SEARCH /* POSIX */ +#if defined(O_EXEC) /* FreeBSD */ +#define O_SEARCH O_EXEC +#elif defined(O_PATH) /* Linux */ +#define O_SEARCH O_PATH #endif +#endif /* !defined(O_SEARCH) */   -#ifdef O_PATH +#ifdef O_SEARCH #define HAVE_OPEN_PARENT_DIR_NOSYMLINKS const char *my_open_parent_dir_nosymlinks(const char *pathname, int *pdfd); #define NOSYMLINK_FUNCTION_BODY(AT,NOAT) \

            People

              serg Sergei Golubchik
              fiddaman Andy
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.