[MDEV-29223] 485 GB of galera.page.xxxxx files never deleted!? Created: 2022-08-01  Updated: 2023-04-11

Status: Open
Project: MariaDB Server
Component/s: Galera
Affects Version/s: 10.6.7
Fix Version/s: 10.6

Type: Bug Priority: Major
Reporter: COUNOTTE CEDRIC Assignee: Alexey
Resolution: Unresolved Votes: 1
Labels: None
Environment:

Ubuntu 22.04.1


 Description   

This is a 2-node cluster (third node is down and is being entirely replaced). Both nodes are used for writing but one node has scripts running every 2, 3, 5 and 10 minutes updating data extensively.

On that node (running scripts), I found 485GB of galera.page.xxx files (not gcache.page.xxx) and the server keeps creating them every minute or so!

First, can I safely delete oldest one before disk space is completely consumed?

Any way to flush those without restarting server?

At that pace, the disk would be full in less than 2 days, so I tried to remove some oldest files and it seems ok however disk space is still consummed by those deleted files!?

Any way to resolve that without restarting server or at least that it stops happening after a restart (that I can do at night) ?

 Comments   
Comment by COUNOTTE CEDRIC [ 2022-08-01 ]

Here is the log showing when this server started creating galera.page.xxxx files without deleting any of them. It's just after starting server and joining the bootstrap galera node with an SST:

2022-07-30 10:11:22 0 [Note] WSREP: Loading provider /usr/lib/galera/libgalera_smm.so initial position: a07556ee-0cd8-11ed-b313-12051094cd1a:7734807
 
2022-07-30 10:11:22 0 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so'
 
2022-07-30 10:11:22 0 [Note] WSREP: wsrep_load(): Galera 26.4.11(r67341d07) by Codership Oy <info@codership.com> loaded successfully.
 
2022-07-30 10:11:22 0 [Note] WSREP: CRC-32C: using 64-bit x86 acceleration.
 
2022-07-30 10:11:22 0 [Note] WSREP: Found saved state: a07556ee-0cd8-11ed-b313-12051094cd1a:7734807, safe_to_bootstrap: 0
 
2022-07-30 10:11:22 0 [Note] WSREP: GCache DEBUG: opened preamble:
 
Version: 2
 
UUID: a07556ee-0cd8-11ed-b313-12051094cd1a
 
Seqno: 7701125 - 7734807
 
Offset: 103190648
 
Synced: 1
 
2022-07-30 10:11:22 0 [Note] WSREP: Recovering GCache ring buffer: version: 2, UUID: a07556ee-0cd8-11ed-b313-12051094cd1a, offset: 103190648
 
2022-07-30 10:11:22 0 [Note] WSREP: GCache::RingBuffer initial scan...  0.0% (        0/134217752 bytes) complete.
 
2022-07-30 10:11:22 0 [Note] WSREP: GCache::RingBuffer initial scan...100.0% (134217752/134217752 bytes) complete.
 
2022-07-30 10:11:22 0 [Note] WSREP: Recovering GCache ring buffer: found gapless sequence 7701125-7734807
 
2022-07-30 10:11:22 0 [Note] WSREP: GCache::RingBuffer unused buffers scan...  0.0% (       0/29060136 bytes) complete.
 
2022-07-30 10:11:22 0 [Note] WSREP: GCache DEBUG: RingBuffer::recover(): found 0/33683 locked buffers
 
2022-07-30 10:11:22 0 [Note] WSREP: GCache DEBUG: RingBuffer::recover(): free space: 105157592/134217728
 
2022-07-30 10:11:22 0 [Note] WSREP: GCache::RingBuffer unused buffers scan...100.0% (29060136/29060136 bytes) complete.
 
2022-07-30 10:11:22 0 [Warning] WSREP: Option 'gcs.fc_master_slave' is deprecated and will be removed in the future versions, please use 'gcs.fc_single_primary' instead.
 
2022-07-30 10:11:22 0 [Note] WSREP: Passing config to GCS: base_dir = /var/lib/mysql/; base_host = 192.168.0.1; base_port = 4567; cert.log_conflicts = no; cert.optimistic_pa = yes; debug = no; evs.auto_evict = 0;
 
 evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.
 
stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.keep_plaintext_size = 128M; gcac
 
he.mem_size = 0; gcache.name = galera.cache; gcache.page_size = 128M; gcache.recover = yes; gcache.size = 128M; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave
 
= no; gcs.fc_single_primary = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0;
 
2022-07-30 10:11:22 0 [Note] WSREP: Service thread queue flushed.
 
2022-07-30 10:11:22 0 [Note] WSREP: ####### Assign initial position for certification: a07556ee-0cd8-11ed-b313-12051094cd1a:7734807, protocol version: -1
 
2022-07-30 10:11:22 0 [Note] WSREP: Start replication
 
2022-07-30 10:11:22 0 [Note] WSREP: Connecting with bootstrap option: 0
 
2022-07-30 10:11:22 0 [Note] WSREP: Setting GCS initial position to a07556ee-0cd8-11ed-b313-12051094cd1a:7734807
 
2022-07-30 10:11:22 0 [Note] WSREP: protonet asio version 0
 
2022-07-30 10:11:22 0 [Note] WSREP: Using CRC-32C for message checksums.
 
2022-07-30 10:11:22 0 [Note] WSREP: backend: asio
 
2022-07-30 10:11:22 0 [Note] WSREP: gcomm thread scheduling priority set to other:0
 
2022-07-30 10:11:22 0 [Warning] WSREP: access file(/var/lib/mysql//gvwstate.dat) failed(No such file or directory)
 
2022-07-30 10:11:22 0 [Note] WSREP: restore pc from disk failed
 
2022-07-30 10:11:22 0 [Note] WSREP: GMCast version 0
 
2022-07-30 10:11:22 0 [Note] WSREP: (32e3e5d5-aa49, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
 
2022-07-30 10:11:22 0 [Note] WSREP: (32e3e5d5-aa49, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
 
2022-07-30 10:11:22 0 [Note] WSREP: EVS version 1
 
2022-07-30 10:11:22 0 [Note] WSREP: gcomm: connecting to group 'MariaDB Galera Cluster', peer '192.168.0.1:,192.168.0.6:'
 
2022-07-30 10:11:22 0 [Note] WSREP: (32e3e5d5-aa49, 'tcp://0.0.0.0:4567') Found matching local endpoint for a connection, blacklisting address tcp://192.168.0.1:4567
 
2022-07-30 10:11:22 0 [Note] WSREP: (32e3e5d5-aa49, 'tcp://0.0.0.0:4567') connection established to 2c964753-8f33 tcp://192.168.0.6:4567
 
2022-07-30 10:11:22 0 [Note] WSREP: (32e3e5d5-aa49, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers:
 
2022-07-30 10:11:22 0 [Note] WSREP: EVS version upgrade 0 -> 1
 
2022-07-30 10:11:22 0 [Note] WSREP: declaring 2c964753-8f33 at tcp://192.168.0.6:4567 stable
 
2022-07-30 10:11:22 0 [Note] WSREP: PC protocol upgrade 0 -> 1
 
2022-07-30 10:11:22 0 [Note] WSREP: Node 2c964753-8f33 state prim
 
2022-07-30 10:11:22 0 [Note] WSREP: view(view_id(PRIM,2c964753-8f33,2) memb {
 
        2c964753-8f33,0
 
        32e3e5d5-aa49,0
 
} joined {
 
} left {
 
} partitioned {
 
})
 
2022-07-30 10:11:22 0 [Note] WSREP: save pc into disk
 
2022-07-30 10:11:23 0 [Note] WSREP: gcomm: connected
 
2022-07-30 10:11:23 0 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636
 
2022-07-30 10:11:23 0 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
 
2022-07-30 10:11:23 0 [Note] WSREP: Opened channel 'MariaDB Galera Cluster'
 
2022-07-30 10:11:23 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 2
 
2022-07-30 10:11:23 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.
 
2022-07-30 10:11:23 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 3331e155-0fdf-11ed-86ce-5aa876e99e4c
 
2022-07-30 10:11:23 0 [Note] WSREP: STATE EXCHANGE: got state msg: 3331e155-0fdf-11ed-86ce-5aa876e99e4c from 0 (ovh6.1check.com)
 
2022-07-30 10:11:23 2 [Note] WSREP: Starting rollbacker thread 2
 
2022-07-30 10:11:23 1 [Note] WSREP: Starting applier thread 1
 
2022-07-30 10:11:23 0 [Note] WSREP: STATE EXCHANGE: got state msg: 3331e155-0fdf-11ed-86ce-5aa876e99e4c from 1 (ovh1.1check.com)
 
2022-07-30 10:11:23 0 [Note] WSREP: Quorum results:
 
        version    = 6,
 
        component  = PRIMARY,
 
        conf_id    = 1,
 
        members    = 1/2 (joined/total),
 
        act_id     = 20,
 
        last_appl. = 0,
 
        protocols  = 2/10/4 (gcs/repl/appl),
 
        vote policy= 0,
 
        group UUID = 2c96698a-0fdf-11ed-90d6-7ecc8fa70984
 
2022-07-30 10:11:23 0 [Note] WSREP: Flow-control interval: [23, 23]
 
2022-07-30 10:11:23 0 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 21)
 
2022-07-30 10:11:23 1 [Note] WSREP: ####### processing CC 21, local, ordered
 
2022-07-30 10:11:23 1 [Note] WSREP: Process first view: 2c96698a-0fdf-11ed-90d6-7ecc8fa70984 my uuid: 32e3e5d5-0fdf-11ed-aa49-3bcd410721b6
 
2022-07-30 10:11:23 1 [Note] WSREP: Server ovh1.1check.com connected to cluster at position 2c96698a-0fdf-11ed-90d6-7ecc8fa70984:21 with ID 32e3e5d5-0fdf-11ed-aa49-3bcd410721b6
 
2022-07-30 10:11:23 1 [Note] WSREP: Server status change disconnected -> connected
 
2022-07-30 10:11:23 1 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
 
2022-07-30 10:11:23 1 [Note] WSREP: ####### My UUID: 32e3e5d5-0fdf-11ed-aa49-3bcd410721b6
 
2022-07-30 10:11:23 1 [Note] WSREP: Cert index reset to 00000000-0000-0000-0000-000000000000:-1 (proto: 10), state transfer needed: yes
 
2022-07-30 10:11:23 0 [Note] WSREP: Service thread queue flushed.
 
2022-07-30 10:11:23 1 [Note] WSREP: ####### Assign initial position for certification: 00000000-0000-0000-0000-000000000000:-1, protocol version: -1
 
2022-07-30 10:11:23 1 [Note] WSREP: State transfer required:
 
        Group state: 2c96698a-0fdf-11ed-90d6-7ecc8fa70984:21
 
        Local state: a07556ee-0cd8-11ed-b313-12051094cd1a:7734807
 
2022-07-30 10:11:23 1 [Note] WSREP: Server status change connected -> joiner
 
2022-07-30 10:11:23 1 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
 
2022-07-30 10:11:23 0 [Note] WSREP: Running: 'wsrep_sst_mariabackup --role 'joiner' --address '192.168.0.1' --datadir '/var/lib/mysql/' --parent '795188' --binlog '/var/lib/mysql/mysql-bin' --binlog-index '/var/l
 
ib/mysql/mysql-bin.index' --mysqld-args --wsrep_start_position=a07556ee-0cd8-11ed-b313-12051094cd1a:7734807,100-2-311580'
 
2022-07-30 10:11:23 0 [Note] WSREP: Joiner monitor thread started to monitor
 
WSREP_SST: [INFO] SSL configuration: CA='', CAPATH='', CERT='', KEY='', MODE='DISABLED', encrypt='0' (20220730 10:11:23.487)
 
WSREP_SST: [INFO] Logging all stderr of SST/mariabackup to syslog (20220730 10:11:23.677)
 
2022-07-30 10:11:24 1 [Note] WSREP: ####### IST uuid:a07556ee-0cd8-11ed-b313-12051094cd1a f: 0, l: 21, STRv: 3
 
2022-07-30 10:11:24 1 [Note] WSREP: IST receiver addr using tcp://192.168.0.1:4568
 
2022-07-30 10:11:24 1 [Note] WSREP: Prepared IST receiver for 0-21, listening at: tcp://192.168.0.1:4568
 
2022-07-30 10:11:24 0 [Note] WSREP: Member 1.0 (ovh1.1check.com) requested state transfer from '*any*'. Selected 0.0 (ovh6.1check.com)(SYNCED) as donor.
 
2022-07-30 10:11:24 0 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 21)
 
2022-07-30 10:11:24 1 [Note] WSREP: Requesting state transfer: success, donor: 0
 
2022-07-30 10:11:24 1 [Note] WSREP: Resetting GCache seqno map due to different histories.
 
2022-07-30 10:11:24 1 [Note] WSREP: GCache history reset: a07556ee-0cd8-11ed-b313-12051094cd1a:7734807 -> 2c96698a-0fdf-11ed-90d6-7ecc8fa70984:21
 
2022-07-30 10:11:24 1 [Note] WSREP: GCache DEBUG: RingBuffer::seqno_reset(): discarded 29059952 bytes
 
2022-07-30 10:11:24 1 [Note] WSREP: GCache DEBUG: RingBuffer::seqno_reset(): found 1/2 locked buffers
 
2022-07-30 10:11:25 0 [Note] WSREP: (32e3e5d5-aa49, 'tcp://0.0.0.0:4567') turning message relay requesting off
 
2022-07-30 10:12:43 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000000 of size 134217696 bytes
 
2022-07-30 10:12:59 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000001 of size 134217696 bytes
 
2022-07-30 10:13:16 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000002 of size 134217696 bytes
 
2022-07-30 10:13:39 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000003 of size 134217696 bytes
 
2022-07-30 10:14:02 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000004 of size 134217696 bytes
 
2022-07-30 10:14:13 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000005 of size 134217696 bytes
 
2022-07-30 10:14:25 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000006 of size 134217696 bytes
 
2022-07-30 10:14:32 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000007 of size 134217696 bytes
 
2022-07-30 10:14:35 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000008 of size 134217696 bytes
 
2022-07-30 10:14:38 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000009 of size 134217696 bytes
 
2022-07-30 10:14:41 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000010 of size 134217696 bytes
 
2022-07-30 10:14:45 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000011 of size 134217696 bytes
 
2022-07-30 10:14:48 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000012 of size 134217696 bytes
 
2022-07-30 10:14:52 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000013 of size 134217696 bytes
 
2022-07-30 10:14:55 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000014 of size 134217696 bytes
 
2022-07-30 10:14:58 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000015 of size 134217696 bytes
 
2022-07-30 10:15:02 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000016 of size 134217696 bytes
 
2022-07-30 10:15:05 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000017 of size 134217696 bytes
 
2022-07-30 10:15:09 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000018 of size 134217696 bytes
 
2022-07-30 10:15:12 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000019 of size 134217696 bytes
 
2022-07-30 10:15:16 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000020 of size 134217696 bytes
 
2022-07-30 10:15:19 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000021 of size 134217696 bytes
 
2022-07-30 10:15:22 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000022 of size 134217696 bytes
 
2022-07-30 10:15:26 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000023 of size 134217696 bytes
 
2022-07-30 10:15:30 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000024 of size 134217696 bytes
 
2022-07-30 10:15:33 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000025 of size 134217696 bytes
 
2022-07-30 10:15:38 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000026 of size 134217696 bytes
 
2022-07-30 10:15:41 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000027 of size 134217696 bytes
 
2022-07-30 10:15:45 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000028 of size 134217696 bytes
 
2022-07-30 10:15:47 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000029 of size 134217696 bytes
 
2022-07-30 10:15:49 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000030 of size 134217696 bytes
 
2022-07-30 10:15:52 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000031 of size 134217696 bytes
 
2022-07-30 10:15:57 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000032 of size 134217696 bytes
 
2022-07-30 10:16:01 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000033 of size 134217696 bytes
 
2022-07-30 10:16:03 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000034 of size 134217696 bytes
 
2022-07-30 10:16:06 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000035 of size 134217696 bytes
 
2022-07-30 10:16:09 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000036 of size 134217696 bytes
 
2022-07-30 10:16:11 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000037 of size 134217696 bytes
 
2022-07-30 10:16:14 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000038 of size 134217696 bytes
 
2022-07-30 10:16:18 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000039 of size 134217696 bytes
 
2022-07-30 10:16:23 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000040 of size 134217696 bytes
 
2022-07-30 10:16:27 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000041 of size 134217696 bytes
 
2022-07-30 10:16:29 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000042 of size 134217696 bytes
 
2022-07-30 10:16:32 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000043 of size 134217696 bytes
 
2022-07-30 10:16:34 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000044 of size 134217696 bytes
 
2022-07-30 10:16:37 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000045 of size 134217696 bytes
 
2022-07-30 10:16:39 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000046 of size 134217696 bytes
 
2022-07-30 10:16:44 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000047 of size 134217696 bytes
 
2022-07-30 10:16:47 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000048 of size 134217696 bytes
 
2022-07-30 10:16:50 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000049 of size 134217696 bytes
 
2022-07-30 10:16:53 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000050 of size 134217696 bytes
 
2022-07-30 10:16:56 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000051 of size 134217696 bytes
 
2022-07-30 10:16:58 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000052 of size 134217696 bytes
 
2022-07-30 10:17:00 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000053 of size 134217696 bytes
 
2022-07-30 10:17:04 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000054 of size 134217696 bytes
 
2022-07-30 10:17:06 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000055 of size 134217696 bytes
 
2022-07-30 10:17:10 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000056 of size 134217696 bytes
 
2022-07-30 10:17:12 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000057 of size 134217696 bytes
 
2022-07-30 10:17:16 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000058 of size 134217696 bytes
 
2022-07-30 10:17:18 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000059 of size 134217696 bytes
 
2022-07-30 10:17:22 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000060 of size 134217696 bytes
 
2022-07-30 10:17:25 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000061 of size 134217696 bytes
 
2022-07-30 10:17:27 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000062 of size 134217696 bytes
 
2022-07-30 10:17:30 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000063 of size 134217696 bytes
 
2022-07-30 10:17:35 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000064 of size 134217696 bytes
 
2022-07-30 10:17:38 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000065 of size 134217696 bytes
 
2022-07-30 10:17:40 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000066 of size 134217696 bytes
 
2022-07-30 10:17:44 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000067 of size 134217696 bytes
 
2022-07-30 10:17:48 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000068 of size 134217696 bytes
 
2022-07-30 10:17:50 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000069 of size 134217696 bytes
 
2022-07-30 10:17:53 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000070 of size 134217696 bytes
 
2022-07-30 10:17:55 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000071 of size 134217696 bytes
 
2022-07-30 10:17:57 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000072 of size 134217696 bytes
 
2022-07-30 10:18:01 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000073 of size 134217696 bytes
 
2022-07-30 10:18:04 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000074 of size 134217696 bytes
 
2022-07-30 10:18:08 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000075 of size 134217696 bytes
 
2022-07-30 10:18:11 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000076 of size 134217696 bytes
 
2022-07-30 10:18:16 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000077 of size 134217696 bytes
 
2022-07-30 10:18:18 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000078 of size 134217696 bytes
 
2022-07-30 10:18:21 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000079 of size 134217696 bytes
 
2022-07-30 10:18:25 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000080 of size 134217696 bytes
 
2022-07-30 10:18:28 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000081 of size 134217696 bytes
 
2022-07-30 10:18:31 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000082 of size 134217696 bytes
 
2022-07-30 10:18:33 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000083 of size 134217696 bytes
 
2022-07-30 10:18:36 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000084 of size 134217696 bytes
 
2022-07-30 10:18:38 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000085 of size 134217696 bytes
 
2022-07-30 10:18:41 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000086 of size 134217696 bytes
 
2022-07-30 10:18:43 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000087 of size 134217696 bytes
 
2022-07-30 10:18:47 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000088 of size 134217696 bytes
 
2022-07-30 10:18:52 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000089 of size 134217696 bytes
 
2022-07-30 10:18:55 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000090 of size 134217696 bytes
 
2022-07-30 10:18:58 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000091 of size 134217696 bytes
 
2022-07-30 10:19:00 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000092 of size 134217696 bytes
 
2022-07-30 10:19:05 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000093 of size 134217696 bytes
 
2022-07-30 10:19:10 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000094 of size 134217696 bytes
 
2022-07-30 10:19:25 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000095 of size 134217696 bytes
 
2022-07-30 10:19:29 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000096 of size 134217696 bytes
 
2022-07-30 10:19:43 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000097 of size 134217696 bytes
 
2022-07-30 10:19:48 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000098 of size 134217696 bytes
 
2022-07-30 10:20:18 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000099 of size 249451680 bytes
 
2022-07-30 10:20:23 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000100 of size 134217696 bytes
 
2022-07-30 10:20:26 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000101 of size 134217696 bytes
 
2022-07-30 10:20:57 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000102 of size 249451600 bytes
 
2022-07-30 10:21:02 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000103 of size 134217696 bytes
 
2022-07-30 10:21:05 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000104 of size 134217696 bytes
 
2022-07-30 10:21:23 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000105 of size 134395072 bytes
 
2022-07-30 10:21:26 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000106 of size 134217696 bytes
 
2022-07-30 10:21:44 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000107 of size 134394992 bytes
 
2022-07-30 10:21:47 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000108 of size 134217696 bytes
 
2022-07-30 10:21:52 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000109 of size 134217696 bytes
 
2022-07-30 10:21:56 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000110 of size 134217696 bytes
 
2022-07-30 10:21:58 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000111 of size 134217696 bytes
 
2022-07-30 10:22:01 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000112 of size 134217696 bytes
 
2022-07-30 10:22:03 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000113 of size 134217696 bytes
 
2022-07-30 10:22:07 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000114 of size 134217696 bytes
 
2022-07-30 10:22:09 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000115 of size 134217696 bytes
 
2022-07-30 10:22:13 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000116 of size 134217696 bytes
 
2022-07-30 10:22:17 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000117 of size 134217696 bytes
 
2022-07-30 10:22:19 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000118 of size 134217696 bytes
 
2022-07-30 10:22:23 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000119 of size 134217696 bytes
 
2022-07-30 10:22:26 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000120 of size 134217696 bytes
 
2022-07-30 10:22:29 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000121 of size 134217696 bytes
 
2022-07-30 10:22:31 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000122 of size 134217696 bytes
 
2022-07-30 10:22:33 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000123 of size 134217696 bytes
 
2022-07-30 10:22:35 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000124 of size 134217696 bytes
 
2022-07-30 10:22:38 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000125 of size 134217696 bytes
 
2022-07-30 10:22:42 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000126 of size 134217696 bytes
 
2022-07-30 10:22:45 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000127 of size 134217696 bytes
 
2022-07-30 10:22:48 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000128 of size 134217696 bytes
 
2022-07-30 10:22:51 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000129 of size 134217696 bytes
 
2022-07-30 10:22:56 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000130 of size 134217696 bytes
 
2022-07-30 10:22:59 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000131 of size 134217696 bytes
 
2022-07-30 10:23:01 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000132 of size 134217696 bytes
 
2022-07-30 10:23:03 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000133 of size 134217696 bytes
 
2022-07-30 10:23:06 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000134 of size 134217696 bytes
 
2022-07-30 10:23:09 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000135 of size 134217696 bytes
 
2022-07-30 10:23:12 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000136 of size 134217696 bytes
 
2022-07-30 10:23:16 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000137 of size 134217696 bytes
 
2022-07-30 10:23:19 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000138 of size 134217696 bytes
 
2022-07-30 10:23:22 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000139 of size 134217696 bytes
 
2022-07-30 10:23:25 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000140 of size 134217696 bytes
 
2022-07-30 10:23:29 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000141 of size 134217696 bytes
 
2022-07-30 10:23:32 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000142 of size 134217696 bytes
 
2022-07-30 10:23:34 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000143 of size 134217696 bytes
 
2022-07-30 10:23:37 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000144 of size 134217696 bytes
 
2022-07-30 10:23:40 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000145 of size 134217696 bytes
 
2022-07-30 10:23:44 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000146 of size 134217696 bytes
 
2022-07-30 10:23:46 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000147 of size 134217696 bytes
 
2022-07-30 10:23:49 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000148 of size 134217696 bytes
 
2022-07-30 10:23:54 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000149 of size 134217696 bytes
 
2022-07-30 10:23:57 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000150 of size 134217696 bytes
 
2022-07-30 10:24:00 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000151 of size 134217696 bytes
 
2022-07-30 10:24:02 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000152 of size 134217696 bytes
 
2022-07-30 10:24:05 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000153 of size 134217696 bytes
 
2022-07-30 10:24:08 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000154 of size 134217696 bytes
 
2022-07-30 10:24:11 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000155 of size 134217696 bytes
 
2022-07-30 10:24:14 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000156 of size 134217696 bytes
 
2022-07-30 10:24:17 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000157 of size 134217696 bytes
 
2022-07-30 10:24:21 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000158 of size 134217696 bytes
 
2022-07-30 10:24:24 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000159 of size 134217696 bytes
 
2022-07-30 10:24:29 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000160 of size 134217696 bytes
 
2022-07-30 10:24:33 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000161 of size 134217696 bytes
 
2022-07-30 10:24:36 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000162 of size 134217696 bytes
 
2022-07-30 10:24:40 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000163 of size 134217696 bytes
 
2022-07-30 10:24:42 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000164 of size 134217696 bytes
 
2022-07-30 10:24:46 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000165 of size 134217696 bytes
 
2022-07-30 10:24:50 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000166 of size 134217696 bytes
 
2022-07-30 10:24:52 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000167 of size 134217696 bytes
 
2022-07-30 10:24:55 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000168 of size 134217696 bytes
 
2022-07-30 10:24:59 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000169 of size 134217696 bytes
 
2022-07-30 10:25:03 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000170 of size 134217696 bytes
 
2022-07-30 10:25:05 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000171 of size 134217696 bytes
 
2022-07-30 10:25:08 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000172 of size 134217696 bytes
 
2022-07-30 10:25:10 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000173 of size 134217696 bytes
 
2022-07-30 10:25:13 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000174 of size 134217696 bytes
 
2022-07-30 10:25:16 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000175 of size 134217696 bytes
 
2022-07-30 10:25:20 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000176 of size 134217696 bytes
 
2022-07-30 10:25:23 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000177 of size 134217696 bytes
 
2022-07-30 10:25:27 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000178 of size 134217696 bytes
 
2022-07-30 10:25:31 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000179 of size 134217696 bytes
 
2022-07-30 10:25:34 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000180 of size 134217696 bytes
 
2022-07-30 10:25:37 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000181 of size 134217696 bytes
 
2022-07-30 10:25:40 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000182 of size 134217696 bytes
 
2022-07-30 10:25:43 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000183 of size 134217696 bytes
 
2022-07-30 10:25:45 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000184 of size 134217696 bytes
 
2022-07-30 10:25:48 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000185 of size 134217696 bytes
 
2022-07-30 10:25:54 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000186 of size 134217696 bytes
 
2022-07-30 10:26:23 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000187 of size 134217696 bytes
 
ubuntu@ovh1:~$ tail -f -n 4000 /var/log/mysql/error.log  |more
 
2022-07-30 10:47:52 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000242 of size 134217696 bytes
 
2022-07-30 10:48:21 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000243 of size 134217696 bytes
 
2022-07-30 10:48:41 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000244 of size 134217696 bytes
 
2022-07-30 10:49:05 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000245 of size 134217696 bytes
 
2022-07-30 10:49:37 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000246 of size 134217696 bytes
 
2022-07-30 10:50:05 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000247 of size 134217696 bytes
 
2022-07-30 10:50:26 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000248 of size 134217696 bytes
 
2022-07-30 10:50:44 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000249 of size 134217696 bytes
 
2022-07-30 10:51:04 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000250 of size 134217696 bytes
 
2022-07-30 10:51:27 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000251 of size 134217696 bytes
 
2022-07-30 10:51:44 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000252 of size 134217696 bytes
 
2022-07-30 10:52:03 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000253 of size 134217696 bytes
 
2022-07-30 10:52:23 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000254 of size 134217696 bytes
 
2022-07-30 10:52:41 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000255 of size 134217696 bytes
 
2022-07-30 10:53:03 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000256 of size 134217696 bytes
 
2022-07-30 10:53:27 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000257 of size 134217696 bytes
 
2022-07-30 10:53:47 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000258 of size 134217696 bytes
 
2022-07-30 10:54:09 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000259 of size 134217696 bytes
 
2022-07-30 10:54:38 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000260 of size 134217696 bytes
 
2022-07-30 10:54:58 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000261 of size 134217696 bytes
 
2022-07-30 10:55:23 0 [Note] WSREP: 0.0 (ovh6.1check.com): State transfer to 1.0 (ovh1.1check.com) complete.
 
2022-07-30 10:55:23 0 [Note] WSREP: Member 0.0 (ovh6.1check.com) synced with group.
 
2022-07-30 10:55:24 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000262 of size 134217696 bytes
 
2022-07-30 10:55:46 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000263 of size 134217696 bytes
 
2022-07-30 10:56:03 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000264 of size 134217696 bytes
 
2022-07-30 10:56:20 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000265 of size 134217696 bytes
 
2022-07-30 10:56:26 0 [Warning] WSREP: Did not find domain ID from SST script output '2c96698a-0fdf-11ed-90d6-7ecc8fa70984:284340'. Domain ID must be set manually to keep binlog consistent
 
2022-07-30 10:56:26 3 [Note] WSREP: SST received
 
2022-07-30 10:56:26 3 [Note] WSREP: Server status change joiner -> initializing
 
2022-07-30 10:56:26 3 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
 
2022-07-30 10:56:26 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
 
2022-07-30 10:56:26 0 [Note] InnoDB: Number of pools: 1
 
2022-07-30 10:56:26 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
 
2022-07-30 10:56:26 0 [Note] InnoDB: Initializing buffer pool, total size = 103079215104, chunk size = 134217728
 
2022-07-30 10:56:26 0 [Note] InnoDB: Completed initialization of buffer pool
 
2022-07-30 10:56:26 0 [Note] InnoDB: Setting log file ./ib_logfile101 size to 1073741824 bytes
 
2022-07-30 10:56:26 0 [Note] InnoDB: Renaming log file ./ib_logfile101 to ./ib_logfile0
 
2022-07-30 10:56:26 0 [Note] InnoDB: New log file created, LSN=7810942519435
 
2022-07-30 10:56:26 0 [Note] InnoDB: 1 transaction(s) which must be rolled back or cleaned up in total 2 row operations to undo
 
2022-07-30 10:56:26 0 [Note] InnoDB: Trx id counter is 2190749400
 
2022-07-30 10:56:26 0 [Note] InnoDB: 128 rollback segments are active.
 
2022-07-30 10:56:27 0 [Note] InnoDB: Creating shared tablespace for temporary tables
 
2022-07-30 10:56:27 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
 
2022-07-30 10:56:27 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
 
2022-07-30 10:56:27 0 [Note] InnoDB: Starting in background the rollback of recovered transactions
 
2022-07-30 10:56:27 0 [Note] InnoDB: 10.6.7 started; log sequence number 0; transaction id 2190749401
 
2022-07-30 10:56:27 0 [Note] Plugin 'FEEDBACK' is disabled.
 
2022-07-30 10:56:27 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
 
2022-07-30 10:56:27 0 [Warning] 'innodb-adaptive-max-sleep-delay' was removed. It does nothing now and exists only for compatibility with old my.cnf files.
 
2022-07-30 10:56:27 0 [Warning] 'innodb-background-scrub-data-check-interval' was removed. It does nothing now and exists only for compatibility with old my.cnf files.
 
2022-07-30 10:56:27 0 [Warning] 'innodb-background-scrub-data-interval' was removed. It does nothing now and exists only for compatibility with old my.cnf files.
 
2022-07-30 10:56:27 0 [Warning] 'innodb-buffer-pool-instances' was removed. It does nothing now and exists only for compatibility with old my.cnf files.
 
2022-07-30 10:56:27 0 [Warning] 'innodb-concurrency-tickets' was removed. It does nothing now and exists only for compatibility with old my.cnf files.
 
2022-07-30 10:56:27 0 [Warning] 'innodb-buffer-pool-instances' was removed. It does nothing now and exists only for compatibility with old my.cnf files.
 
2022-07-30 10:56:27 0 [Note] Recovering after a crash using /var/lib/mysql/mysql-bin
 
2022-07-30 10:56:27 0 [Note] Starting table crash recovery...
 
2022-07-30 10:56:27 0 [Note] Crash table recovery finished.
 
2022-07-30 10:56:27 0 [Note] Server socket created on IP: '0.0.0.0'.
 
2022-07-30 10:56:27 0 [Warning] 'proxies_priv' entry '@% root@ovh2.1check.com' ignored in --skip-name-resolve mode.
 
2022-07-30 10:56:27 0 [Note] WSREP: wsrep_init_schema_and_SR 0x0
 
2022-07-30 10:56:27 0 [Note] InnoDB: Rolled back recovered transaction 2190749399
 
2022-07-30 10:56:27 0 [Note] InnoDB: Rollback of non-prepared transactions completed
 
2022-07-30 10:56:27 0 [Note] WSREP: Server initialized
 
2022-07-30 10:56:27 0 [Note] WSREP: Server status change initializing -> initialized
 
2022-07-30 10:56:27 0 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
 
2022-07-30 10:56:27 3 [Note] WSREP: Recovered position from storage: 2c96698a-0fdf-11ed-90d6-7ecc8fa70984:284340
 
2022-07-30 10:56:27 3 [Note] WSREP: Server status change initialized -> joined
 
2022-07-30 10:56:27 3 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
 
2022-07-30 10:56:27 6 [Note] WSREP: Starting applier thread 6
 
2022-07-30 10:56:27 13 [Note] WSREP: Starting applier thread 13
 
2022-07-30 10:56:27 9 [Note] WSREP: Starting applier thread 9
 
2022-07-30 10:56:27 7 [Note] WSREP: Starting applier thread 7
 
2022-07-30 10:56:27 15 [Note] WSREP: Starting applier thread 15
 
2022-07-30 10:56:27 8 [Note] WSREP: Starting applier thread 8
 
2022-07-30 10:56:27 11 [Note] WSREP: Starting applier thread 11
 
2022-07-30 10:56:27 12 [Note] WSREP: Starting applier thread 12
 
2022-07-30 10:56:27 10 [Note] WSREP: Starting applier thread 10
 
2022-07-30 10:56:27 14 [Note] WSREP: Starting applier thread 14
 
2022-07-30 10:56:27 16 [Note] WSREP: Starting applier thread 16
 
2022-07-30 10:56:27 0 [Note] /usr/sbin/mariadbd: ready for connections.
 
Version: '10.6.7-MariaDB-2ubuntu1.1-log'  socket: '/run/mysqld/mysqld.sock'  port: 3306  Ubuntu 22.04
 
2022-07-30 10:56:27 3 [Note] WSREP: Recovered view from SST:
 
  id: 2c96698a-0fdf-11ed-90d6-7ecc8fa70984:21
 
  status: primary
 
  protocol_version: 4
 
  capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
 
  final: no
 
  own_index: 1
 
  members(2):
 
        0: 2c964753-0fdf-11ed-8f33-fb3ea59a5a33, ovh6.1check.com
 
        1: 32e3e5d5-0fdf-11ed-aa49-3bcd410721b6, ovh1.1check.com
 
 
 
2022-07-30 10:56:27 3 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
 
2022-07-30 10:56:27 19 [Note] WSREP: Recovered cluster id 2c96698a-0fdf-11ed-90d6-7ecc8fa70984
 
2022-07-30 10:56:27 3 [Note] WSREP: SST received: 2c96698a-0fdf-11ed-90d6-7ecc8fa70984:284340
 
2022-07-30 10:56:27 3 [Note] WSREP: SST succeeded for position 2c96698a-0fdf-11ed-90d6-7ecc8fa70984:284340
 
2022-07-30 10:56:27 1 [Note] WSREP: Installed new state from SST: 2c96698a-0fdf-11ed-90d6-7ecc8fa70984:284340
 
2022-07-30 10:56:27 0 [Note] WSREP: Joiner monitor thread ended with total time 2704 sec
 
2022-07-30 10:56:27 1 [Note] WSREP: Cert. index preload up to 284340
 
2022-07-30 10:56:27 0 [Note] WSREP: ####### IST applying starts with 284341
 
2022-07-30 10:56:27 0 [Note] WSREP: ####### IST current seqno initialized to 2
 
2022-07-30 10:56:27 0 [Note] WSREP: Receiving IST...  0.0% ( 0/20 events) complete.
 
2022-07-30 10:56:27 0 [Note] WSREP: IST preload starting at 2
 
2022-07-30 10:56:27 0 [Note] WSREP: Service thread queue flushed.
 
2022-07-30 10:56:27 0 [Note] WSREP: ####### Assign initial position for certification: 00000000-0000-0000-0000-000000000000:1, protocol version: 5
 
2022-07-30 10:56:27 0 [Note] WSREP: REPL Protocols: 10 (5)
 
2022-07-30 10:56:27 0 [Note] WSREP: ####### Adjusting cert position: 20 -> 21
 
2022-07-30 10:56:27 0 [Note] WSREP: Service thread queue flushed.
 
2022-07-30 10:56:27 0 [Note] WSREP: Lowest cert index boundary for CC from preload: 2
 
2022-07-30 10:56:27 0 [Note] WSREP: Min available from gcache for CC from preload: 2
 
2022-07-30 10:56:27 0 [Note] WSREP: Receiving IST...100.0% (20/20 events) complete.
 
2022-07-30 10:56:27 1 [Note] WSREP: Cert. index preloaded up to 21
 
2022-07-30 10:56:27 1 [Note] WSREP: Lowest cert index boundary for CC from sst: 2
 
2022-07-30 10:56:27 1 [Note] WSREP: Min available from gcache for CC from sst: 2
 
2022-07-30 10:56:27 0 [Note] WSREP: 1.0 (ovh1.1check.com): State transfer from 0.0 (ovh6.1check.com) complete.
 
2022-07-30 10:56:27 0 [Note] WSREP: Shifting JOINER -> JOINED (TO: 286735)
 
2022-07-30 10:56:27 0 [Note] WSREP: Processing event queue:...  0.0% (     0/286701 events) complete.
 
2022-07-30 10:56:37 16 [Note] WSREP: Processing event queue:... 98.9% (284960/288214 events) complete.
 
2022-07-30 10:56:42 0 [Note] WSREP: Member 1.0 (ovh1.1check.com) synced with group.
 
2022-07-30 10:56:42 0 [Note] WSREP: Processing event queue:...100.0% (288277/288277 events) complete.
 
2022-07-30 10:56:42 0 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 288297)
 
2022-07-30 10:56:42 7 [Note] WSREP: Server ovh1.1check.com synced with group
 
2022-07-30 10:56:42 7 [Note] WSREP: Server status change joined -> synced
 
2022-07-30 10:56:42 7 [Note] WSREP: Synchronized with group, ready for connections
 
2022-07-30 10:56:42 7 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
 
2022-07-30 10:56:43 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000266 of size 134217696 bytes
 
2022-07-30 10:56:44 0 [Note] InnoDB: Buffer pool(s) load completed at 220730 10:56:44
 
2022-07-30 10:57:01 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000267 of size 134217696 bytes
 
2022-07-30 10:57:21 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000268 of size 134217696 bytes
 
2022-07-30 10:57:43 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000269 of size 134217696 bytes
 
2022-07-30 10:58:00 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000270 of size 134217696 bytes
 
2022-07-30 10:58:28 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000271 of size 134217696 bytes
 
2022-07-30 10:59:04 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000272 of size 134217696 bytes
 
2022-07-30 10:59:31 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000273 of size 134217696 bytes
 
2022-07-30 11:00:00 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000274 of size 134217696 bytes
 
2022-07-30 11:00:29 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000275 of size 134217696 bytes
 
2022-07-30 11:00:45 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000276 of size 134217696 bytes
 
2022-07-30 11:01:03 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000277 of size 134217696 bytes
 
2022-07-30 11:01:24 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000278 of size 134217696 bytes
 
2022-07-30 11:01:43 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000279 of size 134217696 bytes
 
2022-07-30 11:02:00 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000280 of size 134217696 bytes
 
2022-07-30 11:02:19 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000281 of size 134217696 bytes
 
2022-07-30 11:02:32 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000282 of size 134217696 bytes
 
2022-07-30 11:02:50 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000283 of size 134217696 bytes
 
2022-07-30 11:03:08 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000284 of size 134217696 bytes
 
2022-07-30 11:03:37 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000285 of size 134217696 bytes

It's still creating such pages at this time:

2022-08-01 13:45:00 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.003993 of size 134217696 bytes
 
2022-08-01 13:45:58 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.003994 of size 134217696 bytes
 
2022-08-01 13:46:32 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.003995 of size 134217696 bytes
 
2022-08-01 13:47:07 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.003996 of size 134217696 bytes
 
2022-08-01 13:47:48 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.003997 of size 134217696 bytes
 
2022-08-01 13:48:21 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.003998 of size 134217696 bytes
 
2022-08-01 13:48:55 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.003999 of size 134217696 bytes
 
2022-08-01 13:49:23 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.004000 of size 134217696 bytes

Comment by COUNOTTE CEDRIC [ 2022-08-01 ]

The server with this issue was restarted and it stopped creating new galera.page.xxxxxx files.

I had to remove all existing files manually and this time it released the disk space related to them. Earlier I deleted some but disk space was not reclaimed (assuming files where still locked/opened by server).

Comment by Alexey [ 2022-10-09 ]

jplindst
This is a very serious issue - resource leak. But seeing how diverged is MariaDB's Galera repo is from Codership's I find it unproductive to investigate it any further. Please update MariaDB galera repo to the actual Codership's code first.

Generated at Thu Feb 08 10:06:51 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.