[MDEV-32158] wsrep_sst_mariabackup use /tmp dir during SST rather then user defined tmpdir Created: 2023-09-13  Updated: 2023-09-26

Status: Open
Project: MariaDB Server
Component/s: Galera SST
Affects Version/s: 10.5, 10.6
Fix Version/s: 10.5, 10.6

Type: Bug Priority: Major
Reporter: Pramod Mahto Assignee: Julius Goryavsky
Resolution: Unresolved Votes: 1
Labels: None


 Description   

During SST for joiner node , wsrep_sst_mariabackup consider /tmp as a working directory rather then the user defined.

My understanding during SST wsrep_sst_mariabackup should be using tmpdir defined under [mysqld] section not the* /tmp* dir.

Log from Donor while Joiner to trying to get sync using SST

 
2023-09-13  0:55:36 0 [Note] WSREP: (af83c624-a657, 'tcp://0.0.0.0:4567') connection established to c9e7c8ae-92d0 tcp://192.168.47.42:4567
2023-09-13  0:55:36 0 [Note] WSREP: (af83c624-a657, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers:
2023-09-13  0:55:37 0 [Note] WSREP: declaring c9e7c8ae-92d0 at tcp://192.168.47.42:4567 stable
2023-09-13  0:55:37 0 [Note] WSREP: Node af83c624-a657 state prim
2023-09-13  0:55:37 0 [Note] WSREP: view(view_id(PRIM,af83c624-a657,76) memb {
        af83c624-a657,0
        c9e7c8ae-92d0,0
} joined {
} left {
} partitioned {
})
2023-09-13  0:55:37 0 [Note] WSREP: save pc into disk
2023-09-13  0:55:37 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 2
2023-09-13  0:55:37 0 [Note] WSREP: STATE_EXCHANGE: sent state UUID: c7a87140-51f1-11ee-87cf-87f3f3a5fe55
2023-09-13  0:55:37 0 [Note] WSREP: STATE EXCHANGE: sent state msg: c7a87140-51f1-11ee-87cf-87f3f3a5fe55
2023-09-13  0:55:37 0 [Note] WSREP: STATE EXCHANGE: got state msg: c7a87140-51f1-11ee-87cf-87f3f3a5fe55 from 0 (db001)
2023-09-13  0:55:37 0 [Note] WSREP: STATE EXCHANGE: got state msg: c7a87140-51f1-11ee-87cf-87f3f3a5fe55 from 1 (db002)
2023-09-13  0:55:37 0 [Note] WSREP: Quorum results:
        version    = 6,
        component  = PRIMARY,
        conf_id    = 1,
        members    = 1/2 (joined/total),
        act_id     = 90,
        last_appl. = 89,
        protocols  = 2/10/4 (gcs/repl/appl),
        vote policy= 0,
        group UUID = 8f95c1cc-430f-11ee-8404-8f9a8663c992
2023-09-13  0:55:37 0 [Note] WSREP: Flow-control interval: [23, 23]
2023-09-13  0:55:37 2 [Note] WSREP: ####### processing CC 91, local, ordered
2023-09-13  0:55:37 2 [Note] WSREP: ####### My UUID: af83c624-4477-11ee-a657-0627ac9b29e9
2023-09-13  0:55:37 2 [Note] WSREP: Skipping cert index reset
2023-09-13  0:55:37 2 [Note] WSREP: REPL Protocols: 10 (5)
2023-09-13  0:55:37 2 [Note] WSREP: ####### Adjusting cert position: 90 -> 91
2023-09-13  0:55:37 0 [Note] WSREP: Service thread queue flushed.
2023-09-13  0:55:37 2 [Note] WSREP: ================================================
View:
  id: 8f95c1cc-430f-11ee-8404-8f9a8663c992:91
  status: primary
  protocol_version: 4
  capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
  final: no
  own_index: 0
  members(2):
        0: af83c624-4477-11ee-a657-0627ac9b29e9, db001
        1: c9e7c8ae-4476-11ee-92d0-f20d5682fc83, db002
=================================================
2023-09-13  0:55:37 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2023-09-13  0:55:37 2 [Note] WSREP: Lowest cert index boundary for CC from group: 91
2023-09-13  0:55:37 2 [Note] WSREP: Min available from gcache for CC from group: 1
2023-09-13  0:55:37 0 [Note] WSREP: Member 1.0 (db002) requested state transfer from '*any*'. Selected 0.0 (db001)(SYNCED) as donor.
2023-09-13  0:55:37 0 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 91)
2023-09-13  0:55:37 2 [Note] WSREP: Detected STR version: 1, req_len: 131, req: STRv1
2023-09-13  0:55:37 2 [Note] WSREP: Cert index preload: 91 -> 91
2023-09-13  0:55:37 2 [Note] WSREP: Server status change synced -> donor
2023-09-13  0:55:37 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2023-09-13  0:55:37 0 [Note] WSREP: async IST sender starting to serve tcp://192.168.47.42:4568 sending 91-91, preload starts from 91
2023-09-13  0:55:37 0 [Note] WSREP: IST sender 91 -> 91
2023-09-13  0:55:37 0 [Note] WSREP: Donor monitor thread started to monitor
2023-09-13  0:55:37 0 [Note] WSREP: Running: 'wsrep_sst_mariabackup --role 'donor' --address '192.168.47.42:4444/xtrabackup_sst//1' --local-port 3306 --socket '/var/lib/mysql/mysql.sock' --progress 0 --datadir '/var/lib/mysql/' --gtid '8f95c1cc-430f-11ee-8404-8f9a8663c992:91' --gtid-domain-id 0 --binlog '/var/lib/mysql/bin' --mysqld-args --wsrep-new-cluster --wsrep_start_position=8f95c1cc-430f-11ee-8404-8f9a8663c992:89,0-1-33'
2023-09-13  0:55:37 2 [Note] WSREP: sst_donor_thread signaled with 0
WSREP_SST: [INFO] mariabackup SST started on donor (20230913 00:55:37.688)
WSREP_SST: [INFO] SSL configuration: CA='', CAPATH='', CERT='', KEY='', MODE='DISABLED', encrypt='0' (20230913 00:55:37.727)
WSREP_SST: [INFO] Progress reporting tool pv not found in path: /usr//bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/bin:/sbin:/bin (20230913 00:55:37.824)
WSREP_SST: [INFO] Disabling all progress/rate-limiting (20230913 00:55:37.826)
WSREP_SST: [INFO] Streaming with mbstream (20230913 00:55:37.997)
WSREP_SST: [INFO] Using socat as streamer (20230913 00:55:37.999)
 
 
 
WSREP_SST: [INFO] Using '/var/lib/mysql/tmp.23l7f5jblf' as mariadb-backup temporary directory (20230913 00:55:38.011)
WSREP_SST: [INFO] Using '/tmp/tmp.t42TMS6WSR' as mariadb-abackup working directory (20230913 00:55:38.015)
 
 
 
..
.....
2023-09-13  0:55:50 0 [Note] WSREP: Member 0.0 (db001) synced with group.
2023-09-13  0:55:50 0 [Note] WSREP: Processing event queue:...100.0% (1/1 events) complete.
2023-09-13  0:55:50 0 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 91)
2023-09-13  0:55:50 2 [Note] WSREP: Server db001 synced with group
2023-09-13  0:55:50 2 [Note] WSREP: Server status change joined -> synced
2023-09-13  0:55:50 2 [Note] WSREP: Synchronized with group, ready for connections
2023-09-13  0:55:50 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
WSREP_SST: [INFO] Total time on donor: 0 seconds (20230913 00:55:50.414)
WSREP_SST: [INFO] mariabackup SST completed on donor (20230913 00:55:50.417)
WSREP_SST: [INFO] Cleaning up temporary directories (20230913 00:55:50.420)
2023-09-13  0:55:50 0 [Note] WSREP: Donor monitor thread ended with total time 13 sec
2023-09-13  0:55:51 0 [Note] WSREP: async IST sender served
2023-09-13  0:55:51 0 [Note] WSREP: 1.0 (db002): State transfer from 0.0 (db001) complete.
2023-09-13  0:55:51 0 [Note] WSREP: Member 1.0 (db002) synced with group.

https://github.com/MariaDB/server/blob/e987b9350cb83038c73a165922986441f345a3a9/scripts/wsrep_sst_mariabackup.sh#L1030

Donor node

 
[root@gcn-01 ~]# cat /etc/my.cnf|grep -i tmpdir
tmpdir                          = /var/lib/mysql/

Joiner node

 
[root@gcn-02 mysql]#  cat /etc/my.cnf|grep -i tmpdir
tmpdir                          = /var/lib/mysql/


Generated at Thu Feb 08 10:29:16 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.