Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-6531

MySQL Bug #72809 - InnoDB Linux native aio setup missing barrier after setup

Details

    Description

      The following patch is suggested by Stewart Smith:

      memory barrier at end of setting up innodb aio
      		 
       
      		 
      we need a memory barrier here as otherwise when IO threads start they
      		 
      can get a stale view of the data structure.
      		 
       
      		 
      It turns out that POWER8 is fast enough with large enough caches that
      		 
      this can actually be a problem.
      		 
       
      		 
      You can reproduce this by attempting to do a lot of writes shortly after
      		 
      server startup - you'll hit a really odd assert.
      		 
       
      		 
      Index: mysql-5.6.17/storage/innobase/os/os0file.cc
      		 
      ===================================================================
      		 
      --- mysql-5.6.17.orig/storage/innobase/os/os0file.cc
      		 
      +++ mysql-5.6.17/storage/innobase/os/os0file.cc
      		 
      @@ -3782,7 +3782,7 @@ skip_native_aio:
      		 
                      slot->ret = 0;
      		 
       #endif /* WIN_ASYNC_IO */
      		 
              }
      		 
      -
      		 
      +       __sync_synchronize();
      		 
              return(array);
      		 
       }

      Attachments

        Issue Links

          is part of

          Task - A task that needs to be done. MDEV-6530 Examine and apply Power8 patches suggested by Stewart Smith

          • Major - Major loss of function.
          • Closed
          links to

          Web Link Kernel patch which may be relevant

          Web Link MySQL BUG#72809

          Activity

            Ascending order - Click to sort in descending order
            svoj Sergey Vojtovich added a comment - - edited

            Quoting Stewart:

            Basically, as InnoDB sets up the data structures for aio during startup, it creates a mutex but doesn't take it, thus not issuing any memory barriers.

            It turns out that a POWER8 seems to be fast enough and with enough caches that you can get into runtime inside InnoDB with InnoDB doing IO where one of the InnoDB IO threads will hit an assert due to reading stale data left over from setting up AIO rather than runtime.

            This patch adds __sync_synchronize() to
            innobase_init()/innobase_start_or_create_for_mysql()/os_aio_init()/os_aio_array_create()

            But I/O threads are created later in
            innobase_init()/innobase_start_or_create_for_mysql()

            And pthread_create() is supposed to synchronize memory anyway:
            http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html#tag_04_11

            svoj Sergey Vojtovich added a comment - - edited Quoting Stewart: Basically, as InnoDB sets up the data structures for aio during startup, it creates a mutex but doesn't take it, thus not issuing any memory barriers. It turns out that a POWER8 seems to be fast enough and with enough caches that you can get into runtime inside InnoDB with InnoDB doing IO where one of the InnoDB IO threads will hit an assert due to reading stale data left over from setting up AIO rather than runtime. This patch adds __sync_synchronize() to innobase_init()/innobase_start_or_create_for_mysql()/os_aio_init()/os_aio_array_create() But I/O threads are created later in innobase_init()/innobase_start_or_create_for_mysql() And pthread_create() is supposed to synchronize memory anyway: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html#tag_04_11
            svoj Sergey Vojtovich added a comment -

            Closing as can't repeat for now. We will reopen this issue if we will be able to reproduce anything similar.

            svoj Sergey Vojtovich added a comment - Closing as can't repeat for now. We will reopen this issue if we will be able to reproduce anything similar.
            axel Axel Schwenke added a comment -

            will try to reproduce

            axel Axel Schwenke added a comment - will try to reproduce
            axel Axel Schwenke added a comment -

            could not reproduce
            since this is a race condition this means not much

            axel Axel Schwenke added a comment - could not reproduce since this is a race condition this means not much

            People

              axel Axel Schwenke
              svoj Sergey Vojtovich
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.