Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-34543

Shutdown hangs while freeing the asynchronous i/o slots

Details

    Description

      mariadb service is failing to restart due to mariadb zombie process already locking some files. Stopping mariadb using systemctl is not working.

      It seems to occurs when a service stop occurs before mariadb could finish to load InnoDB buffer pool. Mariadb service is not waiting for loading InnoDB buffer pool to end before starting.

      # systemctl start mariadb
      10:22:22 0 [Note] Starting MariaDB 10.11.7-MariaDB-log source revision 87e13722a95af5d9378d990caf48cb6874439347 as process 12127
      10:22:23 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
      10:22:23 0 [Note] InnoDB: Number of transaction pools: 1
      10:22:23 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
      10:22:23 0 [Note] InnoDB: Using Linux native AIO
      10:22:23 0 [Note] InnoDB: Initializing buffer pool, total size = 3.032GiB, chunk size = 48.516MiB
      10:22:23 0 [Note] InnoDB: Completed initialization of buffer pool
      10:22:23 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes)
      10:22:25 0 [Note] InnoDB: End of log at LSN=202113430606
      10:22:26 0 [Note] InnoDB: 128 rollback segments are active.
      10:22:26 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
      10:22:26 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
      10:22:26 0 [Note] InnoDB: log sequence number 202113430606; transaction id 530959006
      10:22:26 0 [Note] InnoDB: Loading buffer pool(s) from <somePath>/ib_buffer_pool
      10:22:26 0 [Note] Plugin 'FEEDBACK' is disabled.
      10:22:26 0 [Warning] 'innodb-thread-concurrency' was removed. It does nothing now and exists only for compatibility with old my.cnf files.
      10:22:26 0 [Note] Server socket created on IP: '0.0.0.0'.
      10:22:26 0 [Note] Server socket created on IP: '::'.
      10:22:26 0 [Note] /usr/sbin/mariadbd: ready for connections.
      Version: '10.11.7-MariaDB-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MariaDB Server
      

      # systemctl stop mariadb
      10:23:11 0 [Note] /usr/sbin/mariadbd (initiated by: unknown): Normal shutdown
      10:23:20 0 [Note] InnoDB: FTS optimize thread exiting.
      10:23:20 0 [Note] InnoDB: Buffer pool(s) load aborted due to user instigated abort at XXXXXX10:23:20
      10:23:20 0 [Note] InnoDB: Dumping of buffer pool not started as load was incomplete
      10:23:20 0 [Note] InnoDB: Starting shutdown...
      10:23:20 0 [Note] InnoDB: Dumping of buffer pool not started as load was incomplete
      

      # systemctl start mariadb
      10:34:48 0 [Note] Starting MariaDB 10.11.7-MariaDB-log source revision 87e13722a95af5d9378d990caf48cb6874439347 as process 17288
      10:34:48 0 [ERROR] mariadbd: Can't lock aria control file '<somePath>/aria_log_control' for exclusive use, error: 11. Will retry for 30 seconds
      10:35:18 0 [ERROR] mariadbd: Got error 'Could not get an exclusive lock; file is probably in use by another process' when trying to use aria control file '<somePath>/aria_log_control'
      10:35:18 0 [ERROR] Plugin 'Aria' registration as a STORAGE ENGINE failed.
      10:35:18 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
      10:35:18 0 [Note] InnoDB: Number of transaction pools: 1
      10:35:18 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
      10:35:18 0 [Note] InnoDB: Using Linux native AIO
      10:35:18 0 [Note] InnoDB: Initializing buffer pool, total size = 3.032GiB, chunk size = 48.516MiB
      10:35:18 0 [Note] InnoDB: Completed initialization of buffer pool
      10:35:18 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes)
      10:35:18 0 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
      10:35:18 0 [Note] InnoDB: Check that you do not already have another mariadbd process using the same InnoDB data or log files.
      10:35:18 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
      10:35:18 0 [Note] InnoDB: Starting shutdown...
      10:35:18 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
      10:35:18 0 [Note] Plugin 'FEEDBACK' is disabled.
      10:35:18 0 [ERROR] Could not open mysql.plugin table: "Unknown storage engine 'Aria'". Some plugins may be not loaded
      10:35:18 0 [ERROR] Failed to initialize plugins.
      10:35:18 0 [ERROR] Aborting
      15:55:03 0 [Note] Starting MariaDB 10.11.7-MariaDB-log source revision 87e13722a95af5d9378d990caf48cb6874439347 as process 109521
      15:55:04 0 [ERROR] mariadbd: Can't lock aria control file '<somePath>/aria_log_control' for exclusive use, error: 11. Will retry for 30 seconds
      15:55:34 0 [ERROR] mariadbd: Got error 'Could not get an exclusive lock; file is probably in use by another process' when trying to use aria control file '<somePath>/aria_log_control'
      15:55:34 0 [ERROR] Plugin 'Aria' registration as a STORAGE ENGINE failed.
      15:55:34 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
      15:55:34 0 [Note] InnoDB: Number of transaction pools: 1
      15:55:34 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
      15:55:34 0 [Note] InnoDB: Using Linux native AIO
      15:55:34 0 [Note] InnoDB: Initializing buffer pool, total size = 3.032GiB, chunk size = 48.516MiB
      15:55:34 0 [Note] InnoDB: Completed initialization of buffer pool
      15:55:34 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes)
      15:55:34 0 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
      15:55:34 0 [Note] InnoDB: Check that you do not already have another mariadbd process using the same InnoDB data or log files.
      15:55:34 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
      15:55:34 0 [Note] InnoDB: Starting shutdown...
      15:55:34 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
      15:55:34 0 [Note] Plugin 'FEEDBACK' is disabled.
      15:55:34 0 [ERROR] Could not open mysql.plugin table: "Unknown storage engine 'Aria'". Some plugins may be not loaded
      15:55:34 0 [ERROR] Failed to initialize plugins.
      15:55:34 0 [ERROR] Aborting
      

      Attachments

        Activity

          People

            thiru Thirunarayanan Balathandayuthapani
            math.nao MathNao
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.