Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-34191

MariaDB 10.11.8 package scripts attempt to start service on upgrade, before configuration update

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Critical
    • Resolution: Fixed
    • 10.11.8
    • 10.11.9
    • Packaging
    • None
    • Ubuntu 20.04.6 LTS (Any Debian-based system)

    Description

      Problem Statement

      Attempting to upgrade to MariaDB 10.11.8 on a Debian-based system according to the recommended steps from OS vendor packages fails if there are incompatible configuration options before the upgrade. This happens because package scripts now attempt to start the mariadb.service.
      Likely the same behavior will be observed on upgrade from MariaDB packages as well.

      Steps to Reproduce

      This basically follows https://mariadb.com/kb/en/upgrading-from-mariadb-10-6-to-mariadb-10-11/

      1. Get Ubuntu 20.04 machine, with system mariadb-server installed.
      E.g. mariadb-server 1:10.3.39-0ubuntu0.20.04.2
      2. Ensure the configuration contains options incompatible with the future version, such as bind-address = ::ffff:127.0.0.1:

      # cat /etc/mysql/my.cnf
      # The MariaDB configuration file
      #
      # The MariaDB/MySQL tools read configuration files in the following order:
      # 1. "/etc/mysql/mariadb.cnf" (this file) to set global defaults,
      # 2. "/etc/mysql/conf.d/*.cnf" to set global options.
      # 3. "/etc/mysql/mariadb.conf.d/*.cnf" to set MariaDB-only options.
      # 4. "~/.my.cnf" to set user-specific options.
      #
      # If the same option is defined multiple times, the last one will apply.
      #
      # One can use all long options that the program supports.
      # Run program with --help to get a list of available options and with
      # --print-defaults to see which it would actually understand and use.
       
      #
      # This group is read both both by the client and the server
      # use it for options that affect everything
      #
      [client-server]
       
      # Import all .cnf files from configuration directory
      !includedir /etc/mysql/conf.d/
      !includedir /etc/mysql/mariadb.conf.d/
      [mysqld]
      sql_mode=ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
      bind-address = ::ffff:127.0.0.1
      local-infile=0
      

      3. Set up MariaDB 10.11 repositories, e.g. via:
      ./mariadb_repo_setup --mariadb-server-version mariadb-10.11 --skip-maxscale
      4. Stop and disable the service:
      systemctl stop mariadb.service
      systemctl disable mariadb.service
      5. Upgrade the packages:

      DEBIAN_FRONTEND=noninteractive LANG=C PATH=/usr/sbin:/usr/bin:/sbin:/bin \
          apt-get update --allow-releaseinfo-change
      DEBIAN_FRONTEND=noninteractive LANG=C PATH=/usr/sbin:/usr/bin:/sbin:/bin \
          apt-get --assume-yes \
              -o Dpkg::Options::=--force-confdef \
              -o Dpkg::Options::=--force-confold \
              -o Dpkg::Progress-Fancy="0" \
              -o Dpkg::Use-Pty="0" \
              -o APT::Install-Recommends=no \
              --purge \
              install mariadb-server mariadb-client
      

      Note that normally upgrade would include other further steps, such a modifying configuration and starting the service back up, but now it fails here.

      Actual Result

      The packages upgrade fails on an attempt to start the service:

      + DEBIAN_FRONTEND=noninteractive
      + LANG=C
      + PATH=/usr/sbin:/usr/bin:/sbin:/bin
      + apt-get --assume-yes -o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confold -o Dpkg::Progress-Fancy=0 -o Dpkg::Use-Pty=0 -o APT::Install-Recommends=no --purge install mariadb-server mariadb-client
      Reading package lists...
      Building dependency tree...
      Reading state information...
      The following additional packages will be installed:
        galera-4 libmariadb3 mariadb-client-core mariadb-common mariadb-server-core
      Suggested packages:
        mailx mariadb-test
      Recommended packages:
        libdbd-mariadb-perl | libdbd-mysql-perl libterm-readkey-perl
        libhtml-template-perl pv
      The following packages will be REMOVED:
        galera-3* mariadb-client-10.3* mariadb-client-core-10.3*
        mariadb-server-10.3* mariadb-server-core-10.3*
      The following NEW packages will be installed:
        galera-4 libmariadb3 mariadb-client mariadb-client-core mariadb-server-core
      The following packages will be upgraded:
        mariadb-common mariadb-server
      2 upgraded, 5 newly installed, 5 to remove and 14 not upgraded.
      Need to get 28.5 MB of archives.
      After this operation, 91.7 MB of additional disk space will be used.
      Get:1 https://dlm.mariadb.com/repo/mariadb-server/10.11/repo/ubuntu focal/main amd64 mariadb-common all 1:10.11.8+maria~ubu2004 [4036 B]
      Get:2 https://dlm.mariadb.com/repo/mariadb-server/10.11/repo/ubuntu focal/main amd64 mariadb-server amd64 1:10.11.8+maria~ubu2004 [3941 kB]
      Get:3 https://dlm.mariadb.com/repo/mariadb-server/10.11/repo/ubuntu focal/main amd64 galera-4 amd64 26.4.18-ubu2004 [12.9 MB]
      Get:4 https://dlm.mariadb.com/repo/mariadb-server/10.11/repo/ubuntu focal/main amd64 libmariadb3 amd64 1:10.11.8+maria~ubu2004 [161 kB]
      Get:5 https://dlm.mariadb.com/repo/mariadb-server/10.11/repo/ubuntu focal/main amd64 mariadb-client-core amd64 1:10.11.8+maria~ubu2004 [883 kB]
      Get:6 https://dlm.mariadb.com/repo/mariadb-server/10.11/repo/ubuntu focal/main amd64 mariadb-client amd64 1:10.11.8+maria~ubu2004 [3028 kB]
      Get:7 https://dlm.mariadb.com/repo/mariadb-server/10.11/repo/ubuntu focal/main amd64 mariadb-server-core amd64 1:10.11.8+maria~ubu2004 [7556 kB]
      Preconfiguring packages ...
      Fetched 28.5 MB in 7s (3829 kB/s)
      (Reading database ... 108507 files and directories currently installed.)
      Preparing to unpack .../mariadb-common_1%3a10.11.8+maria~ubu2004_all.deb ...
      Unpacking mariadb-common (1:10.11.8+maria~ubu2004) over (1:10.3.39-0ubuntu0.20.04.2) ...
      dpkg: mariadb-server-10.3: dependency problems, but removing anyway as you requested:
       mariadb-server depends on mariadb-server-10.3 (>= 1:10.3.39-0ubuntu0.20.04.2).
       
      (Reading database ... 108506 files and directories currently installed.)
      Removing mariadb-server-10.3 (1:10.3.39-0ubuntu0.20.04.2) ...
      Setting up mariadb-common (1:10.11.8+maria~ubu2004) ...
       
      Configuration file '/etc/mysql/mariadb.cnf'
       ==> Modified (by you or by a script) since installation.
       ==> Package distributor has shipped an updated version.
       ==> Keeping old config file as default.
      update-alternatives: warning: forcing reinstallation of alternative /etc/mysql/mariadb.cnf because link group my.cnf is broken
      update-alternatives: warning: not replacing /etc/mysql/my.cnf with a link
      (Reading database ... 108391 files and directories currently installed.)
      Preparing to unpack .../mariadb-server_1%3a10.11.8+maria~ubu2004_amd64.deb ...
      /var/lib/mysql: found previous version 10.3
      Unpacking mariadb-server (1:10.11.8+maria~ubu2004) over (1:10.3.39-0ubuntu0.20.04.2) ...
      (Reading database ... 108498 files and directories currently installed.)
      Removing galera-3 (25.3.29-1) ...
      Selecting previously unselected package galera-4.
      (Reading database ... 108490 files and directories currently installed.)
      Preparing to unpack .../galera-4_26.4.18-ubu2004_amd64.deb ...
      Unpacking galera-4 (26.4.18-ubu2004) ...
      (Reading database ... 108497 files and directories currently installed.)
      Removing mariadb-client-10.3 (1:10.3.39-0ubuntu0.20.04.2) ...
      Removing mariadb-client-core-10.3 (1:10.3.39-0ubuntu0.20.04.2) ...
      Removing mariadb-server-core-10.3 (1:10.3.39-0ubuntu0.20.04.2) ...
      Selecting previously unselected package libmariadb3:amd64.
      (Reading database ... 108349 files and directories currently installed.)
      Preparing to unpack .../libmariadb3_1%3a10.11.8+maria~ubu2004_amd64.deb ...
      Unpacking libmariadb3:amd64 (1:10.11.8+maria~ubu2004) ...
      Selecting previously unselected package mariadb-client-core.
      Preparing to unpack .../mariadb-client-core_1%3a10.11.8+maria~ubu2004_amd64.deb ...
      Unpacking mariadb-client-core (1:10.11.8+maria~ubu2004) ...
      Selecting previously unselected package mariadb-client.
      Preparing to unpack .../mariadb-client_1%3a10.11.8+maria~ubu2004_amd64.deb ...
      Unpacking mariadb-client (1:10.11.8+maria~ubu2004) ...
      Selecting previously unselected package mariadb-server-core.
      Preparing to unpack .../mariadb-server-core_1%3a10.11.8+maria~ubu2004_amd64.deb ...
      Unpacking mariadb-server-core (1:10.11.8+maria~ubu2004) ...
      Setting up galera-4 (26.4.18-ubu2004) ...
      Setting up libmariadb3:amd64 (1:10.11.8+maria~ubu2004) ...
      Setting up mariadb-server-core (1:10.11.8+maria~ubu2004) ...
      Setting up mariadb-client-core (1:10.11.8+maria~ubu2004) ...
      Setting up mariadb-client (1:10.11.8+maria~ubu2004) ...
      Installing new version of config file /etc/mysql/mariadb.conf.d/50-client.cnf ...
      Installing new version of config file /etc/mysql/mariadb.conf.d/50-mysql-clients.cnf ...
      Setting up mariadb-server (1:10.11.8+maria~ubu2004) ...
      Installing new version of config file /etc/mysql/debian-start ...
      Installing new version of config file /etc/mysql/mariadb.conf.d/50-mysqld_safe.cnf ...
      Installing new version of config file /etc/mysql/mariadb.conf.d/50-server.cnf ...
      renamed '/etc/logrotate.d/mysql-server' -> '/etc/logrotate.d/mysql-server.dpkg-bak'
      Job for mariadb.service failed because the control process exited with error code.
      See "systemctl status mariadb.service" and "journalctl -xe" for details.
      invoke-rc.d: initscript mariadb, action "start" failed.
      ● mariadb.service - MariaDB 10.11.8 database server
           Loaded: loaded (/lib/systemd/system/mariadb.service; disabled; vendor preset: enabled)
          Drop-In: /etc/systemd/system/mariadb.service.d
                   └─migrated-from-my.cnf-settings.conf
           Active: failed (Result: exit-code) since Fri 2024-05-17 11:47:00 UTC; 10ms ago
             Docs: man:mariadbd(8)
                   https://mariadb.com/kb/en/library/systemd/
          Process: 70270 ExecStartPre=/usr/bin/install -m 755 -o mysql -g root -d /var/run/mysqld (code=exited, status=0/SUCCESS)
          Process: 70271 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
          Process: 70273 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= ||   VAR=`cd /usr/bin/..; /usr/bin/galera_recovery`; [ $? -eq 0 ]   && systemctl set-environment _WSREP_START_POSITION=$VAR || exit 1 (code=exited, status=0/SUCCESS)
          Process: 70308 ExecStart=/usr/sbin/mariadbd $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION (code=exited, status=1/FAILURE)
         Main PID: 70308 (code=exited, status=1/FAILURE)
           Status: "MariaDB server is down"
       
      May 17 11:47:00 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:47:00 0 [Note] Plugin 'FEEDBACK' is disabled.
      May 17 11:47:00 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:47:00 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
      May 17 11:47:00 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:47:00 0 [Warning] You need to use --log-bin to make --expire-logs-days or --binlog-expire-logs-seconds work.
      May 17 11:47:00 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:47:00 0 [Note] Server socket created on IP: '127.0.0.1'.
      May 17 11:47:00 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:47:00 0 [ERROR] Can't start server: Bind on TCP/IP port. Got error: 22: Invalid argument
      May 17 11:47:00 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:47:00 0 [ERROR] Do you already have another server running on port: 3306 ?
      May 17 11:47:00 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:47:00 0 [ERROR] Aborting
      May 17 11:47:00 10-69-43-66.qa.plesk.tech systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE
      May 17 11:47:00 10-69-43-66.qa.plesk.tech systemd[1]: mariadb.service: Failed with result 'exit-code'.
      May 17 11:47:00 10-69-43-66.qa.plesk.tech systemd[1]: Failed to start MariaDB 10.11.8 database server.
      dpkg: error processing package mariadb-server (--configure):
       installed mariadb-server package post-installation script subprocess returned error exit status 1
      Processing triggers for systemd (245.4-4ubuntu3.23) ...
      Processing triggers for man-db (2.9.1-1) ...
      Processing triggers for libc-bin (2.31-0ubuntu9.15) ...
      Errors were encountered while processing:
       mariadb-server
      E: Sub-process /usr/bin/dpkg returned an error code (1)
      + return 100
      

      # journalctl -xeu mariadb.service
      May 17 11:46:59 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:46:59 0 [Note] Starting MariaDB 10.11.8-MariaDB-ubu2004 source revision 3a069644682e336e445039e48baae9693f9a08ee as process 70308
      May 17 11:46:59 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:46:59 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
      May 17 11:46:59 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:46:59 0 [Note] InnoDB: Number of transaction pools: 1
      May 17 11:46:59 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:46:59 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
      May 17 11:46:59 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:46:59 0 [Note] InnoDB: Using Linux native AIO
      May 17 11:46:59 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:46:59 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB
      May 17 11:46:59 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:46:59 0 [Note] InnoDB: Completed initialization of buffer pool
      May 17 11:46:59 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:46:59 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes)
      May 17 11:47:00 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:47:00 0 [Note] InnoDB: Upgrading redo log: 96.000MiB; LSN=5233374
      May 17 11:47:00 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:47:00 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes)
      May 17 11:47:00 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:47:00 0 [Note] InnoDB: 128 rollback segments are active.
      May 17 11:47:00 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:47:00 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
      May 17 11:47:00 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:47:00 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
      May 17 11:47:00 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:47:00 0 [Note] InnoDB: log sequence number 5233374; transaction id 6655
      May 17 11:47:00 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:47:00 0 [Note] Plugin 'FEEDBACK' is disabled.
      May 17 11:47:00 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:47:00 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
      May 17 11:47:00 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:47:00 0 [Warning] You need to use --log-bin to make --expire-logs-days or --binlog-expire-logs-seconds work.
      May 17 11:47:00 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:47:00 0 [Note] Server socket created on IP: '127.0.0.1'.
      May 17 11:47:00 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:47:00 0 [ERROR] Can't start server: Bind on TCP/IP port. Got error: 22: Invalid argument
      May 17 11:47:00 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:47:00 0 [ERROR] Do you already have another server running on port: 3306 ?
      May 17 11:47:00 10-69-43-66.qa.plesk.tech mariadbd[70308]: 2024-05-17 11:47:00 0 [ERROR] Aborting
      May 17 11:47:00 10-69-43-66.qa.plesk.tech systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE
      -- Subject: Unit process exited
      -- Defined-By: systemd
      -- Support: http://www.ubuntu.com/support
      --
      -- An ExecStart= process belonging to unit mariadb.service has exited.
      --
      -- The process' exit code is 'exited' and its exit status is 1.
      May 17 11:47:00 10-69-43-66.qa.plesk.tech systemd[1]: mariadb.service: Failed with result 'exit-code'.
      -- Subject: Unit failed
      -- Defined-By: systemd
      -- Support: http://www.ubuntu.com/support
      --
      -- The unit mariadb.service has entered the 'failed' state with result 'exit-code'.
      May 17 11:47:00 10-69-43-66.qa.plesk.tech systemd[1]: Failed to start MariaDB 10.11.8 database server.
      -- Subject: A start job for unit mariadb.service has failed
      -- Defined-By: systemd
      -- Support: http://www.ubuntu.com/support
      --
      -- A start job for unit mariadb.service has finished with a failure.
      --
      -- The job identifier is 18218 and the job result is failed.
      

      Changing bind-address = ::ffff:127.0.0.1 to bind-address = 127.0.0.1 in the configuration would allow the service to start.

      Expected Result

      Upgrade according to the steps from the guide succeeds.
      Package scripts do not attempt to start the service on upgrade, which it may have incompatible configuration.

      Additional Information

      Original MariaDB changes:

      Discussion in MariaDB community Slack: https://mariadb-community.slack.com/archives/C02R1CJ0LD6/p1715941300648099

      Found by Plesk web hosting control panel tests, internal id: PPP-65009

      Attachments

        Issue Links

          Activity

            People

              danblack Daniel Black
              nvizovitin Nikolay Vizovitin
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.