[MDEV-15209]  mdb cluster node fails to start with wsrep_sst_mariabackup due to mariabackup not in path Created: 2018-02-05  Updated: 2021-12-23  Resolved: 2021-12-23

Status: Closed
Project: MariaDB Server
Component/s: Galera SST
Affects Version/s: 10.2
Fix Version/s: 10.2.42

Type: Bug Priority: Minor
Reporter: Zdravelina Sokolovska (Inactive) Assignee: Jan Lindström (Inactive)
Resolution: Fixed Votes: 0
Labels: None


 Description   

mdb cluster node fails to start with wsrep_sst_mariabackup due to mariabackup not in path

after installation of MariaDB-server ,setting up wsrep_sst_method=mariabackup
and enabling wsrep Node fails to start due to dependancy from MariaDB-backup package,

it would be expected that MariaDB-server is including mariabackup while the mariabackup
is a part of the supported wsrep_sst methods

2018-02-05 14:17:15 140016945899264 [Warning] WSREP: Gap in state sequence. Need state transfer.
2018-02-05 14:17:15 140016688793344 [Note] WSREP: Running: 'wsrep_sst_mariabackup --role 'joiner' --address '192.168.104.193' --datadir '/var/lib/mysql/'   --parent '27597'  '' '
which: no mariabackup in (/usr/sbin:/sbin:/usr//bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin)

[root@t4w6 ~]# yum install MariaDB-server MariaDB-client MariaDB-common
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: mirrors.netix.net
 * epel: mirrors.netix.net
 * extras: mirrors.netix.net
 * updates: mirrors.netix.net
Resolving Dependencies
--> Running transaction check
---> Package MariaDB-client.x86_64 0:10.2.12-1.el7.centos will be installed
---> Package MariaDB-common.x86_64 0:10.2.12-1.el7.centos will be installed
--> Processing Dependency: MariaDB-compat for package: MariaDB-common-10.2.12-1.el7.centos.x86_64
---> Package MariaDB-server.x86_64 0:10.2.12-1.el7.centos will be installed
--> Running transaction check
---> Package MariaDB-compat.x86_64 0:10.2.12-1.el7.centos will be installed
--> Finished Dependency Resolution
 
Dependencies Resolved
 
=============================================================================================================================================================
 Package                                Arch                           Version                                         Repository                       Size
=============================================================================================================================================================
Installing:
 MariaDB-client                         x86_64                         10.2.12-1.el7.centos                            mariadb                          48 M
 MariaDB-common                         x86_64                         10.2.12-1.el7.centos                            mariadb                         154 k
 MariaDB-server                         x86_64                         10.2.12-1.el7.centos                            mariadb                         110 M
Installing for dependencies:
 MariaDB-compat                         x86_64                         10.2.12-1.el7.centos                            mariadb                         2.8 M
 
Transaction Summary
=============================================================================================================================================================
Install  3 Packages (+1 Dependent package)
 
Total download size: 160 M
Installed size: 683 M
Is this ok [y/d/N]: y
Downloading packages:
(1/4): MariaDB-10.2.12-centos7-x86_64-common.rpm                                                                                      | 154 kB  00:00:00
(2/4): MariaDB-10.2.12-centos7-x86_64-compat.rpm                                                                                      | 2.8 MB  00:00:01
(3/4): MariaDB-10.2.12-centos7-x86_64-client.rpm                                                                                      |  48 MB  00:00:05
(4/4): MariaDB-10.2.12-centos7-x86_64-server.rpm                                                                                      | 110 MB  00:00:07
-------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                         16 MB/s | 160 MB  00:00:09
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : MariaDB-common-10.2.12-1.el7.centos.x86_64                                                                                                1/4
  Installing : MariaDB-compat-10.2.12-1.el7.centos.x86_64                                                                                                2/4
  Installing : MariaDB-client-10.2.12-1.el7.centos.x86_64                                                                                                3/4
  Installing : MariaDB-server-10.2.12-1.el7.centos.x86_64                                                                                                4/4
chown: cannot access ‘/var/lib/mysql’: No such file or directory
 
PLEASE REMEMBER TO SET A PASSWORD FOR THE MariaDB root USER !
To do so, start the server, then issue the following commands:
 
'/usr/bin/mysqladmin' -u root password 'new-password'
'/usr/bin/mysqladmin' -u root -h t4w6.xentio.lan password 'new-password'
 
Alternatively you can run:
'/usr/bin/mysql_secure_installation'
 
which will also give you the option of removing the test
databases and anonymous user created by default.  This is
strongly recommended for production servers.
 
See the MariaDB Knowledgebase at http://mariadb.com/kb or the
MySQL manual for more instructions.
 
Please report any problems at http://mariadb.org/jira
 
The latest information about MariaDB is available at http://mariadb.org/.
You can find additional information about the MySQL part at:
http://dev.mysql.com
Consider joining MariaDB's strong and vibrant community:
https://mariadb.org/get-involved/
 
  Verifying  : MariaDB-client-10.2.12-1.el7.centos.x86_64                                                                                                1/4
  Verifying  : MariaDB-compat-10.2.12-1.el7.centos.x86_64                                                                                                2/4
  Verifying  : MariaDB-common-10.2.12-1.el7.centos.x86_64                                                                                                3/4
  Verifying  : MariaDB-server-10.2.12-1.el7.centos.x86_64                                                                                                4/4
 
Installed:
  MariaDB-client.x86_64 0:10.2.12-1.el7.centos        MariaDB-common.x86_64 0:10.2.12-1.el7.centos        MariaDB-server.x86_64 0:10.2.12-1.el7.centos
 
Dependency Installed:
  MariaDB-compat.x86_64 0:10.2.12-1.el7.centos
 
Complete!
[root@t4w6 ~]#

[root@t4w3 log]# service mysql start --wsrep_cluster_address=gcomm://192.168.104.196
Starting mysql (via systemctl): Job for mariadb.service failed because the control process exited with error code. See "systemctl status mariadb.service" and "journalctl -xe" for details.
[FAILED]

2018-02-05 14:17:14 140017037990016 [Note] WSREP: Read nil XID from storage engines, skipping position init
2018-02-05 14:17:14 140017037990016 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib64/galera/libgalera_smm.so'
2018-02-05 14:17:14 140017037990016 [Note] WSREP: wsrep_load(): Galera 25.3.22(r3764) by Codership Oy <info@codership.com> loaded successfully.
2018-02-05 14:17:14 140017037990016 [Note] WSREP: CRC-32C: using "slicing-by-8" algorithm.
2018-02-05 14:17:14 140017037990016 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1, safe_to_bootstrap: 1
2018-02-05 14:17:14 140017037990016 [Note] WSREP: Passing config to GCS: base_dir = /var/lib/mysql/; base_host = 192.168.104.193; base_port = 4567; cert.log_conflicts = no; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.recover = no; gcache.size = 128M; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.checksum = false; p
2018-02-05 14:17:14 140017037990016 [Note] WSREP: GCache history reset: 00000000-0000-0000-0000-000000000000:0 -> 00000000-0000-0000-0000-000000000000:-1
2018-02-05 14:17:14 140017037990016 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1
2018-02-05 14:17:14 140017037990016 [Note] WSREP: wsrep_sst_grab()
2018-02-05 14:17:14 140017037990016 [Note] WSREP: Start replication
2018-02-05 14:17:14 140017037990016 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
2018-02-05 14:17:14 140017037990016 [Note] WSREP: protonet asio version 0
2018-02-05 14:17:14 140017037990016 [Note] WSREP: Using CRC-32C for message checksums.
2018-02-05 14:17:14 140017037990016 [Note] WSREP: backend: asio
2018-02-05 14:17:14 140017037990016 [Note] WSREP: gcomm thread scheduling priority set to other:0
2018-02-05 14:17:14 140017037990016 [Warning] WSREP: access file(/var/lib/mysql//gvwstate.dat) failed(No such file or directory)
2018-02-05 14:17:14 140017037990016 [Note] WSREP: restore pc from disk failed
2018-02-05 14:17:14 140017037990016 [Note] WSREP: GMCast version 0
2018-02-05 14:17:14 140017037990016 [Note] WSREP: (80294d65, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
2018-02-05 14:17:14 140017037990016 [Note] WSREP: (80294d65, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
2018-02-05 14:17:14 140017037990016 [Note] WSREP: EVS version 0
2018-02-05 14:17:14 140017037990016 [Note] WSREP: gcomm: connecting to group 'cluster1', peer '192.168.104.193:,192.168.104.196:'
2018-02-05 14:17:14 140017037990016 [Note] WSREP: (80294d65, 'tcp://0.0.0.0:4567') connection established to 80294d65 tcp://192.168.104.193:4567
2018-02-05 14:17:14 140017037990016 [Warning] WSREP: (80294d65, 'tcp://0.0.0.0:4567') address 'tcp://192.168.104.193:4567' points to own listening address, blacklisting
2018-02-05 14:17:14 140017037990016 [Note] WSREP: (80294d65, 'tcp://0.0.0.0:4567') connection established to 76c5a0e7 tcp://192.168.104.196:4567
2018-02-05 14:17:14 140017037990016 [Note] WSREP: (80294d65, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers:
2018-02-05 14:17:15 140017037990016 [Note] WSREP: declaring 76c5a0e7 at tcp://192.168.104.196:4567 stable
2018-02-05 14:17:15 140017037990016 [Note] WSREP: Node 76c5a0e7 state prim
2018-02-05 14:17:15 140017037990016 [Note] WSREP: view(view_id(PRIM,76c5a0e7,2) memb {
        76c5a0e7,0
        80294d65,0
} joined {
} left {
} partitioned {
})
2018-02-05 14:17:15 140017037990016 [Note] WSREP: save pc into disk
2018-02-05 14:17:15 140017037990016 [Note] WSREP: gcomm: connected
2018-02-05 14:17:15 140017037990016 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636
2018-02-05 14:17:15 140017037990016 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
2018-02-05 14:17:15 140017037990016 [Note] WSREP: Opened channel 'cluster1'
2018-02-05 14:17:15 140016697186048 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 2
2018-02-05 14:17:15 140016697186048 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.
2018-02-05 14:17:15 140017037990016 [Note] WSREP: Waiting for SST to complete.
2018-02-05 14:17:15 140016697186048 [Note] WSREP: STATE EXCHANGE: sent state msg: 88badf22-0a6e-11e8-91b4-bad85022d9a0
2018-02-05 14:17:15 140016697186048 [Note] WSREP: STATE EXCHANGE: got state msg: 88badf22-0a6e-11e8-91b4-bad85022d9a0 from 0 (t4w6)
2018-02-05 14:17:15 140016697186048 [Note] WSREP: STATE EXCHANGE: got state msg: 88badf22-0a6e-11e8-91b4-bad85022d9a0 from 1 (t4w3)
2018-02-05 14:17:15 140016697186048 [Note] WSREP: Quorum results:
        version    = 4,
        component  = PRIMARY,
        conf_id    = 1,
        members    = 1/2 (joined/total),
        act_id     = 0,
        last_appl. = -1,
        protocols  = 0/7/3 (gcs/repl/appl),
        group UUID = 376a9b49-0a68-11e8-abee-56e9cbc26fb1
2018-02-05 14:17:15 140016697186048 [Note] WSREP: Flow-control interval: [23, 23]
2018-02-05 14:17:15 140016697186048 [Note] WSREP: Trying to continue unpaused monitor
2018-02-05 14:17:15 140016697186048 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 0)
2018-02-05 14:17:15 140016945899264 [Note] WSREP: State transfer required:
        Group state: 376a9b49-0a68-11e8-abee-56e9cbc26fb1:0
        Local state: 00000000-0000-0000-0000-000000000000:-1
2018-02-05 14:17:15 140016945899264 [Note] WSREP: New cluster view: global state: 376a9b49-0a68-11e8-abee-56e9cbc26fb1:0, view# 2: Primary, number of nodes: 2, my index: 1, protocol version 3
2018-02-05 14:17:15 140016945899264 [Warning] WSREP: Gap in state sequence. Need state transfer.
2018-02-05 14:17:15 140016688793344 [Note] WSREP: Running: 'wsrep_sst_mariabackup --role 'joiner' --address '192.168.104.193' --datadir '/var/lib/mysql/'   --parent '27597'  '' '
which: no mariabackup in (/usr/sbin:/sbin:/usr//bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin)
WSREP_SST: [ERROR] mariabackup not in path: /usr/sbin:/sbin:/usr//bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin (20180205 14:17:15.807)
2018-02-05 14:17:15 140016688793344 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_mariabackup --role 'joiner' --address '192.168.104.193' --datadir '/var/lib/mysql/'   --parent '27597'  ''
        Read: '(null)'
2018-02-05 14:17:15 140016688793344 [ERROR] WSREP: Process completed with error: wsrep_sst_mariabackup --role 'joiner' --address '192.168.104.193' --datadir '/var/lib/mysql/'   --parent '27597'  '' : 2 (No such file or directory)
2018-02-05 14:17:15 140016945899264 [ERROR] WSREP: Failed to prepare for 'mariabackup' SST. Unrecoverable.
2018-02-05 14:17:15 140016945899264 [ERROR] Aborting
 
Error in my_thread_global_end(): 1 threads didn't exit
 


Generated at Thu Feb 08 08:19:30 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.