Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-15606

Galera can't perform SST in 10.2.13 if systemd in use due to timeout at startup

Details

    • Bug
    • Status: Closed (View Workflow)
    • Critical
    • Resolution: Duplicate
    • 10.2.13, 10.2.14, 10.3.6, 10.1(EOL)
    • N/A
    • Configuration
    • None
    • CentOS Linux release 7.4.1708 (Core)

    Description

      The second node can't join the first node because SST will get killed by systemd after the default timeout hits.

      systemctl show mariadb.service | grep Timeout will show timeout set to 1m 30s for startup, but an SST can last hours with large dataset and/or slow disks and/or slow networks.

      In fact, it is common for an SST to take several hours in production.

      Setting TimeoutSec=0 under Services in the mariadb.service config file under systemd fixes the problem.

      Right now, it is impossible to deploy Galera Cluster under 10.2.13 and CentOS 7 unless the above workaround is in place.

      Attachments

        Issue Links

          relates to

          Task - A task that needs to be done. MDEV-14705 systemd: EXTEND_TIMEOUT_USEC= to avoid startup and shutdown timeouts

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-17571 Make systemd timeout behavior more compatible with long Galera SSTs

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            View 4 older comments
            rpizzi Rick Pizzi (Inactive) added a comment -

            Guys, this needs a fix, just being bitten by this in a newly installed 10.2.14... please...

            rpizzi Rick Pizzi (Inactive) added a comment - Guys, this needs a fix, just being bitten by this in a newly installed 10.2.14... please...
            av Alex Vorona added a comment -

            Same problem affects 10.1 version

            av Alex Vorona added a comment - Same problem affects 10.1 version
            wworkman@neces.com Wayne Workman added a comment -

            These are the same:

            wworkman@neces.com Wayne Workman added a comment - These are the same: MDEV-16425 MDEV-15606
            jplindst Jan Lindström (Inactive) added a comment -

            MDEV-15607 should fix this issue.

            jplindst Jan Lindström (Inactive) added a comment - MDEV-15607 should fix this issue.
            brianryberg brianr added a comment - - edited

            This will not work any longer:

            echo 'TimeoutSec=0' >> /etc/systemd/system/mysqld.service
            systemctl daemon-reload

            systemd will apparently silently ignore the fact that it only reacts now, to "TimeoutSec=infinity" , not =0

            DAHMIKT

            brianryberg brianr added a comment - - edited This will not work any longer: echo 'TimeoutSec=0' >> /etc/systemd/system/mysqld.service systemctl daemon-reload systemd will apparently silently ignore the fact that it only reacts now, to "TimeoutSec=infinity" , not =0 DAHMIKT

            People

              jplindst Jan Lindström (Inactive)
              rpizzi Rick Pizzi (Inactive)
              Votes:
              4 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.