Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-14622

Running mysql_upgrade in background upon package installation or upgrade causes deadlocks and other side-effects

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • 10.1, 10.2, 10.3, 10.4, 10.5
    • 10.5.4
    • Platform Debian
    • None

    Description

      MariaDB Debian startup scripts, among other things, after server startup invoke debian-start, which starts a block of functions in background and exits. The block of functions includes mysql_upgrade script (from within upgrade_system_tables_if_necessary).

      trap "" SIGHUP
      (
        upgrade_system_tables_if_necessary;
        check_root_accounts;
        check_for_crashed_tables;
      ) >&2 &
      

      On non-systemd installations, this block proceeds after the main script ends, and eventually performs the necessary checks. It's a questionable decision, because the server might start being used before mysql_upgrade has been run. It would be more reasonable either wait till it finishes, if mysql_upgrade is really necessary, or not run it at all, if it's not. However, it's a minor problem.

      On systemd installations, when the main script finishes, the background block aborts too. So, mysql_upgrade has almost no chance to be executed. It remains unnoticed, the server reports that debian-start was executed successfully:

      $ sudo service mariadb status
      ● mariadb.service - MariaDB database server
         Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; vendor preset: enabled)
        Drop-In: /etc/systemd/system/mariadb.service.d
                 └─migrated-from-my.cnf-settings.conf
         Active: active (running) since Mon 2017-12-11 11:21:33 EST; 4s ago
        Process: 13680 ExecStartPost=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
        Process: 13678 ExecStartPost=/etc/mysql/debian-start (code=exited, status=0/SUCCESS)
        Process: 13622 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= ||   VAR=`/usr/bin/galera_recovery`; [ $? -eq 0 ]   && systemctl set-environment _WSR
        Process: 13618 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
        Process: 13615 ExecStartPre=/usr/bin/install -m 755 -o mysql -g root -d /var/run/mysqld (code=exited, status=0/SUCCESS)
       Main PID: 13647 (mysqld)
         Status: "Taking your SQL requests now..."
          Tasks: 30 (limit: 4915)
         CGroup: /system.slice/mariadb.service
                 └─13647 /usr/sbin/mysqld
       
      Dec 11 11:21:32 debian-9-stretch-amd64 mysqld[13647]: 2017-12-11 11:21:32 0 [Note] InnoDB: 5.7.19 started; log sequence number 1630818
      Dec 11 11:21:32 debian-9-stretch-amd64 mysqld[13647]: 2017-12-11 11:21:32 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
      Dec 11 11:21:32 debian-9-stretch-amd64 mysqld[13647]: 2017-12-11 11:21:32 0 [Note] Plugin 'FEEDBACK' is disabled.
      Dec 11 11:21:32 debian-9-stretch-amd64 mysqld[13647]: 2017-12-11 11:21:32 0 [Note] InnoDB: Buffer pool(s) load completed at 171211 11:21:32
      Dec 11 11:21:32 debian-9-stretch-amd64 mysqld[13647]: 2017-12-11 11:21:32 0 [Note] Server socket created on IP: '::'.
      Dec 11 11:21:32 debian-9-stretch-amd64 mysqld[13647]: 2017-12-11 11:21:32 0 [Note] Reading of all Master_info entries succeded
      Dec 11 11:21:32 debian-9-stretch-amd64 mysqld[13647]: 2017-12-11 11:21:32 0 [Note] Added new Master_info '' to hash table
      Dec 11 11:21:32 debian-9-stretch-amd64 mysqld[13647]: 2017-12-11 11:21:32 0 [Note] /usr/sbin/mysqld: ready for connections.
      Dec 11 11:21:32 debian-9-stretch-amd64 mysqld[13647]: Version: '10.3.2-MariaDB-10.3.2+maria~stretch-log'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  mariadb.org
      Dec 11 11:21:33 debian-9-stretch-amd64 systemd[1]: Started MariaDB database server.
      Warning: mariadb.service changed on disk. Run 'systemctl daemon-reload' to reload units.
      

      but both syslog and general log, if enabled, confirms that the background jobs have never been finished.

      Also, result of these jobs is ignored, even if mysql_upgrade fails, it is not reflected in the service status anyhow.

      Attachments

        Issue Links

          Activity

            People

              otto Otto Kekäläinen
              elenst Elena Stepanova
              Votes:
              2 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.