Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-34543

Shutdown hangs while freeing the asynchronous i/o slots

    XMLWordPrintable

Details

    Description

      mariadb service is failing to restart due to mariadb zombie process already locking some files. Stopping mariadb using systemctl is not working.

      It seems to occurs when a service stop occurs before mariadb could finish to load InnoDB buffer pool. Mariadb service is not waiting for loading InnoDB buffer pool to end before starting.

      # systemctl start mariadb
      10:22:22 0 [Note] Starting MariaDB 10.11.7-MariaDB-log source revision 87e13722a95af5d9378d990caf48cb6874439347 as process 12127
      10:22:23 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
      10:22:23 0 [Note] InnoDB: Number of transaction pools: 1
      10:22:23 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
      10:22:23 0 [Note] InnoDB: Using Linux native AIO
      10:22:23 0 [Note] InnoDB: Initializing buffer pool, total size = 3.032GiB, chunk size = 48.516MiB
      10:22:23 0 [Note] InnoDB: Completed initialization of buffer pool
      10:22:23 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes)
      10:22:25 0 [Note] InnoDB: End of log at LSN=202113430606
      10:22:26 0 [Note] InnoDB: 128 rollback segments are active.
      10:22:26 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
      10:22:26 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
      10:22:26 0 [Note] InnoDB: log sequence number 202113430606; transaction id 530959006
      10:22:26 0 [Note] InnoDB: Loading buffer pool(s) from <somePath>/ib_buffer_pool
      10:22:26 0 [Note] Plugin 'FEEDBACK' is disabled.
      10:22:26 0 [Warning] 'innodb-thread-concurrency' was removed. It does nothing now and exists only for compatibility with old my.cnf files.
      10:22:26 0 [Note] Server socket created on IP: '0.0.0.0'.
      10:22:26 0 [Note] Server socket created on IP: '::'.
      10:22:26 0 [Note] /usr/sbin/mariadbd: ready for connections.
      Version: '10.11.7-MariaDB-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MariaDB Server
      

      # systemctl stop mariadb
      10:23:11 0 [Note] /usr/sbin/mariadbd (initiated by: unknown): Normal shutdown
      10:23:20 0 [Note] InnoDB: FTS optimize thread exiting.
      10:23:20 0 [Note] InnoDB: Buffer pool(s) load aborted due to user instigated abort at XXXXXX10:23:20
      10:23:20 0 [Note] InnoDB: Dumping of buffer pool not started as load was incomplete
      10:23:20 0 [Note] InnoDB: Starting shutdown...
      10:23:20 0 [Note] InnoDB: Dumping of buffer pool not started as load was incomplete
      

      # systemctl start mariadb
      10:34:48 0 [Note] Starting MariaDB 10.11.7-MariaDB-log source revision 87e13722a95af5d9378d990caf48cb6874439347 as process 17288
      10:34:48 0 [ERROR] mariadbd: Can't lock aria control file '<somePath>/aria_log_control' for exclusive use, error: 11. Will retry for 30 seconds
      10:35:18 0 [ERROR] mariadbd: Got error 'Could not get an exclusive lock; file is probably in use by another process' when trying to use aria control file '<somePath>/aria_log_control'
      10:35:18 0 [ERROR] Plugin 'Aria' registration as a STORAGE ENGINE failed.
      10:35:18 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
      10:35:18 0 [Note] InnoDB: Number of transaction pools: 1
      10:35:18 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
      10:35:18 0 [Note] InnoDB: Using Linux native AIO
      10:35:18 0 [Note] InnoDB: Initializing buffer pool, total size = 3.032GiB, chunk size = 48.516MiB
      10:35:18 0 [Note] InnoDB: Completed initialization of buffer pool
      10:35:18 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes)
      10:35:18 0 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
      10:35:18 0 [Note] InnoDB: Check that you do not already have another mariadbd process using the same InnoDB data or log files.
      10:35:18 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
      10:35:18 0 [Note] InnoDB: Starting shutdown...
      10:35:18 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
      10:35:18 0 [Note] Plugin 'FEEDBACK' is disabled.
      10:35:18 0 [ERROR] Could not open mysql.plugin table: "Unknown storage engine 'Aria'". Some plugins may be not loaded
      10:35:18 0 [ERROR] Failed to initialize plugins.
      10:35:18 0 [ERROR] Aborting
      15:55:03 0 [Note] Starting MariaDB 10.11.7-MariaDB-log source revision 87e13722a95af5d9378d990caf48cb6874439347 as process 109521
      15:55:04 0 [ERROR] mariadbd: Can't lock aria control file '<somePath>/aria_log_control' for exclusive use, error: 11. Will retry for 30 seconds
      15:55:34 0 [ERROR] mariadbd: Got error 'Could not get an exclusive lock; file is probably in use by another process' when trying to use aria control file '<somePath>/aria_log_control'
      15:55:34 0 [ERROR] Plugin 'Aria' registration as a STORAGE ENGINE failed.
      15:55:34 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
      15:55:34 0 [Note] InnoDB: Number of transaction pools: 1
      15:55:34 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
      15:55:34 0 [Note] InnoDB: Using Linux native AIO
      15:55:34 0 [Note] InnoDB: Initializing buffer pool, total size = 3.032GiB, chunk size = 48.516MiB
      15:55:34 0 [Note] InnoDB: Completed initialization of buffer pool
      15:55:34 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes)
      15:55:34 0 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
      15:55:34 0 [Note] InnoDB: Check that you do not already have another mariadbd process using the same InnoDB data or log files.
      15:55:34 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
      15:55:34 0 [Note] InnoDB: Starting shutdown...
      15:55:34 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
      15:55:34 0 [Note] Plugin 'FEEDBACK' is disabled.
      15:55:34 0 [ERROR] Could not open mysql.plugin table: "Unknown storage engine 'Aria'". Some plugins may be not loaded
      15:55:34 0 [ERROR] Failed to initialize plugins.
      15:55:34 0 [ERROR] Aborting
      

      Attachments

        Activity

          People

            thiru Thirunarayanan Balathandayuthapani
            math.nao MathNao
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.