Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-17480

mysqld terminated before sst finish

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Not a Bug
    • Affects Version/s: 10.2.16, 10.2.17, 10.2.18
    • Fix Version/s: N/A
    • Component/s: Galera SST
    • Labels:
      None
    • Environment:
      FreeBSD11.2

      Description

      I am running galera cluster on 3 nodes including an arbitrator.
      Now every time I start the second node, I can see the SST processes but no mysqld processes
      In the error log, I can see mysqld terminated.

      2018-10-17 14:31:21 34424867584 [Note] WSREP: Quorum results:
          version    = 4,
          component  = PRIMARY,
          conf_id    = 10,
          members    = 2/3 (joined/total),
          act_id     = 405525295,
          last_appl. = -1,
          protocols  = 0/8/3 (gcs/repl/appl),
          group UUID = 7a79d7e6-7ca9-11e7-b980-ef84f0f16abf
      2018-10-17 14:31:21 34424867584 [Note] WSREP: Flow-control interval: [28, 28]
      2018-10-17 14:31:21 34424867584 [Note] WSREP: Trying to continue unpaused monitor
      2018-10-17 14:31:21 34424867584 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 405525295)
      2018-10-17 14:31:21 35616158976 [Note] WSREP: State transfer required:
          Group state: 7a79d7e6-7ca9-11e7-b980-ef84f0f16abf:405525295
          Local state: 00000000-0000-0000-0000-000000000000:-1
      2018-10-17 14:31:21 35616158976 [Note] WSREP: New cluster view: global state: 7a79d7e6-7ca9-11e7-b980-ef84f0f16abf:405525295, view# 11: Primary, number of nodes: 3, my index: 1, protocol version 3
      2018-10-17 14:31:21 35616158976 [Warning] WSREP: Gap in state sequence. Need state transfer.
      2018-10-17 14:31:21 35624464128 [Note] WSREP: Running: 'wsrep_sst_xtrabackup-v2 --role 'joiner' --address '203.29.62.201' --datadir '/var/mysql-data/'  --defaults-extra-file '/usr/local/etc/mysql/my.cnf'  --parent '12021'  '' '
      WSREP_SST: [INFO] Streaming with xbstream (20181017 14:31:21.N)
      WSREP_SST: [INFO] Using socat as streamer (20181017 14:31:21.N)
      WSREP_SST: [INFO] Stale sst_in_progress file: /var/mysql-data//sst_in_progress (20181017 14:31:21.N)
      Usage: timeout [--signal sig | -s sig] [--preserve-status] [--kill-after time | -k time] [--foreground] <duration> <command> <arg ...>
      WSREP_SST: [INFO] Evaluating timeout -s9 100 socat -u TCP-LISTEN:4444,reuseaddr stdio | xbstream -x; RC=( ${PIPESTATUS[@]} ) (20181017 14:31:21.N)
      encryption: using gcrypt 1.8.3
      2018-10-17 14:31:21 35616158976 [Note] WSREP: Prepared SST request: xtrabackup-v2|203.29.62.201:4444/xtrabackup_sst//203.29.62.201
      2018-10-17 14:31:21 35616158976 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
      2018-10-17 14:31:21 35616158976 [Note] WSREP: REPL Protocols: 8 (3, 2)
      2018-10-17 14:31:21 35616158976 [Note] WSREP: Assign initial position for certification: 405525295, protocol version: 3
      2018-10-17 14:31:21 34424865024 [Note] WSREP: Service thread queue flushed.
      2018-10-17 14:31:21 35616158976 [Warning] WSREP: Failed to prepare for incremental state transfer: Local state UUID (00000000-0000-0000-0000-000000000000) does not match group state UUID (7a79d7e6-7ca9-11e7-b980-ef84f0f16abf): 1 (Operation not permitted)
           at galera/src/replicator_str.cpp:prepare_for_IST():482. IST will be unavailable.
      2018-10-17 14:31:21 34424867584 [Note] WSREP: Member 1.0 (192.168.0.2) requested state transfer from '*any*'. Selected 2.0 (192.168.0.1)(SYNCED) as donor.
      2018-10-17 14:31:21 34424867584 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 405525305)
      2018-10-17 14:31:21 35616158976 [Note] WSREP: Requesting state transfer: success, donor: 2
      2018-10-17 14:31:21 35616158976 [Note] WSREP: GCache history reset: 00000000-0000-0000-0000-000000000000:0 -> 7a79d7e6-7ca9-11e7-b980-ef84f0f16abf:405525295
      2018-10-17 14:31:22 34424867584 [Warning] WSREP: 2.0 (192.168.0.1): State transfer to 1.0 (192.168.0.2) failed: -2 (No such file or directory)
      2018-10-17 14:31:22 34424867584 [ERROR] WSREP: gcs/src/gcs_group.cpp:gcs_group_handle_join_msg():737: Will never receive state. Need to abort.
      2018-10-17 14:31:22 34424867584 [Note] WSREP: gcomm: terminating thread
      2018-10-17 14:31:22 34424867584 [Note] WSREP: gcomm: joining thread
      2018-10-17 14:31:22 34424867584 [Note] WSREP: gcomm: closing backend
      2018-10-17 14:31:22 34424867584 [Note] WSREP: view(view_id(NON_PRIM,0ae9045c,11) memb {
          1d611ec6,0
      } joined {
      } left {
      } partitioned {
          0ae9045c,0
          4a399c73,0
      })
      2018-10-17 14:31:22 34424867584 [Note] WSREP: view((empty))
      2018-10-17 14:31:22 34424867584 [Note] WSREP: gcomm: closed
      2018-10-17 14:31:22 34424867584 [Note] WSREP: mysqld: Terminated.
      WSREP_SST: [ERROR] Possible timeout in receving first data from donor in gtid stage (20181017 14:33:01.N)
      WSREP_SST: [ERROR] Cleanup after exit with status:32 (20181017 14:33:01.N)
      
      

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              laocius TAO ZHOU
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: