[root@master1 nokiauser]# [root@master1 nokiauser]# [root@master1 nokiauser]# kgp $n Thu May 21 04:36:20 UTC 2026 NAME READY STATUS RESTARTS AGE nrd-db-mariadb-0 2/2 Running 3 (14h ago) 46h nrddb-v1-7b955f7ff4-wtl7w 4/4 Running 0 47h [root@master1 nokiauser]# k logs -p nrd-db-mariadb-0 $n Thu May 21 04:36:28 UTC 2026 Defaulted container "mariadb" out of: mariadb, metrics, init (init), decode-pwds (init) 2026-05-20 09:24:14 - Got member count [3] from cluster member state file 2026-05-20 09:24:14 - Cluster members are present, unsetting variables 2026-05-20 09:24:14 - Setting sequence no. to -1 to force SST now 2026-05-20 09:24:14-05:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 11.4.9 started. 2026-05-20 09:24:14-05:00 [Note] [Entrypoint]: MariaDB upgrade information missing, assuming required 2026-05-20 09:24:14-05:00 [Note] [Entrypoint]: MariaDB upgrade (mariadb-upgrade or creating healthcheck users) required, but skipped due to $MARIADB_AUTO_UPGRADE setting 2026-05-20 9:24:14 0 [Note] slave_connections_needed_for_purge changed to 0 because of Galera. Change it to 1 or higher if this Galera node is also Master in a normal replication setup 2026-05-20 9:24:14 0 [Note] Starting MariaDB 11.4.9-MariaDB-log source revision aa6a2e6bf05d43c3bb44db50566937a5f97a6580 server_uid htLuO0ZnVfV76kW33ZuI02Xr5+w= as process 12 2026-05-20 9:24:14 0 [Note] WSREP: Loading provider /usr/lib64/galera/libgalera_smm.so initial position: 00000000-0000-0000-0000-000000000000:-1 2026-05-20 9:24:14 0 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib64/galera/libgalera_smm.so' 2026-05-20 9:24:14 0 [Note] WSREP: wsrep_load(): Galera 26.4.25(r7387a566) by Codership Oy loaded successfully. 2026-05-20 9:24:14 0 [Note] WSREP: Initializing allowlist service v1 2026-05-20 9:24:14 0 [Note] WSREP: Initializing event service v1 2026-05-20 9:24:14 0 [Note] WSREP: Resolved symbol 'wsrep_node_isolation_mode_set_v1' 2026-05-20 9:24:14 0 [Note] WSREP: Resolved symbol 'wsrep_certify_v1' 2026-05-20 9:24:14 0 [Note] WSREP: CRC-32C: using 64-bit x86 acceleration. 2026-05-20 9:24:14 0 [Note] WSREP: Found saved state: 9ff7203e-5029-11f1-bf6b-2200d14b0348:-1, safe_to_bootstrap: 0 2026-05-20 9:24:14 0 [Note] WSREP: GCache DEBUG: opened preamble: Version: 2 UUID: 9ff7203e-5029-11f1-bf6b-2200d14b0348 Seqno: -1 - -1 Offset: -1 Synced: 0 2026-05-20 9:24:14 0 [Note] WSREP: Passing config to GCS: base_dir = /mariadb/data/; base_host = 11.127.6.37; base_port = 4567; cert.log_conflicts = no; cert.optimistic_pa = yes; debug = no; evs.auto_evict = 0; evs.causal_keepalive_period = PT1S; evs.debug_log_mask = 0x1; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.info_log_mask = 0; evs.join_retrans_period = PT1S; evs.keepalive_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.use_aggregate = true; evs.user_send_window = 2; evs.version = 1; evs.view_forget_timeout = PT24H; gcache.dir = /mariadb/data/; gcache.keep_pages_size = 512M; gcache.keep_plaintext_size = 256M; gcache.mem_size = 0; gcache.name = galera.cache; gcache.page_size = 256M; gcache.recover = no; gcache.size = 3G; gcomm.thread_prio = ; gcs.check_appl_proto = 1; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.fc_single_pri 2026-05-20 9:24:14 0 [Note] WSREP: Start replication 2026-05-20 9:24:14 0 [Note] WSREP: Connecting with bootstrap option: 0 2026-05-20 9:24:14 0 [Note] WSREP: Setting GCS initial position to 00000000-0000-0000-0000-000000000000:-1 2026-05-20 9:24:14 0 [Note] WSREP: protonet asio version 0 2026-05-20 9:24:14 0 [Note] WSREP: Using CRC-32C for message checksums. 2026-05-20 9:24:14 0 [Note] WSREP: backend: asio 2026-05-20 9:24:14 0 [Note] WSREP: gcomm thread scheduling priority set to other:0 2026-05-20 9:24:14 0 [Note] WSREP: restore pc from disk successfully 2026-05-20 9:24:14 0 [Note] WSREP: GMCast version 0 2026-05-20 9:24:14 0 [Note] WSREP: (efd61072-b0fd, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567 2026-05-20 9:24:14 0 [Note] WSREP: (efd61072-b0fd, 'tcp://0.0.0.0:4567') multicast: , ttl: 1 2026-05-20 9:24:14 0 [Note] WSREP: EVS version 1 2026-05-20 9:24:14 0 [Note] WSREP: gcomm: connecting to group 'nrdGalera', peer '11.127.4.37:,11.127.5.37:,11.127.6.37:' 2026-05-20 9:24:14 0 [Note] WSREP: (efd61072-b0fd, 'tcp://0.0.0.0:4567') Found matching local endpoint for a connection, blacklisting address tcp://11.127.6.37:4567 2026-05-20 9:24:14 0 [Note] WSREP: (efd61072-b0fd, 'tcp://0.0.0.0:4567') connection established to a2217745-a764 tcp://11.127.4.37:4567 2026-05-20 9:24:14 0 [Note] WSREP: (efd61072-b0fd, 'tcp://0.0.0.0:4567') connection established to 7b7a3560-a050 tcp://11.127.5.37:4567 2026-05-20 9:24:15 0 [Note] WSREP: EVS version upgrade 0 -> 1 2026-05-20 9:24:15 0 [Note] WSREP: declaring 7b7a3560-a050 at tcp://11.127.5.37:4567 stable 2026-05-20 9:24:15 0 [Note] WSREP: declaring a2217745-a764 at tcp://11.127.4.37:4567 stable 2026-05-20 9:24:15 0 [Note] WSREP: PC protocol upgrade 0 -> 1 2026-05-20 9:24:15 0 [Note] WSREP: Node 7b7a3560-a050 state prim 2026-05-20 9:24:15 0 [Note] WSREP: view(view_id(PRIM,7b7a3560-a050,147) memb { 7b7a3560-a050,0 a2217745-a764,0 efd61072-b0fd,0 } joined { } left { } partitioned { }) 2026-05-20 9:24:15 0 [Note] WSREP: save pc into disk 2026-05-20 9:24:15 0 [Note] WSREP: clear restored view 2026-05-20 9:24:15 0 [Note] WSREP: gcomm: connected 2026-05-20 9:24:15 0 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636 2026-05-20 9:24:15 0 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0) 2026-05-20 9:24:15 0 [Note] WSREP: Opened channel 'nrdGalera' 2026-05-20 9:24:15 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 2, memb_num = 3 2026-05-20 9:24:15 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. 2026-05-20 9:24:15 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 943ec31e-5457-11f1-b24f-cac71f2b53ae 2026-05-20 9:24:15 0 [Note] WSREP: Initializing config service v1 2026-05-20 9:24:15 0 [Note] WSREP: STATE EXCHANGE: got state msg: 943ec31e-5457-11f1-b24f-cac71f2b53ae from 0 (nrd-db-mariadb-0) 2026-05-20 9:24:15 0 [Note] WSREP: STATE EXCHANGE: got state msg: 943ec31e-5457-11f1-b24f-cac71f2b53ae from 1 (nrd-db-mariadb-0) 2026-05-20 9:24:15 1 [Note] WSREP: Starting rollbacker thread 1 2026-05-20 9:24:15 2 [Note] WSREP: Starting applier thread 2 2026-05-20 9:24:15 0 [Note] WSREP: Deinitializing config service v1 2026-05-20 9:24:15 0 [Note] WSREP: STATE EXCHANGE: got state msg: 943ec31e-5457-11f1-b24f-cac71f2b53ae from 2 (nrd-db-mariadb-0) 2026-05-20 9:24:15 0 [Note] WSREP: Quorum results: version = 6, component = PRIMARY, conf_id = 140, members = 2/3 (joined/total), act_id = 67362, last_appl. = 67337, protocols = 6/11/4 (gcs/repl/appl), vote policy= 0, group UUID = 9ff7203e-5029-11f1-bf6b-2200d14b0348 2026-05-20 9:24:15 0 [Note] WSREP: Flow-control interval: [28, 28] 2026-05-20 9:24:15 0 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 67363) 2026-05-20 9:24:15 2 [Note] WSREP: ####### processing CC 67363, local, ordered 2026-05-20 9:24:15 2 [Note] WSREP: Process first view: 9ff7203e-5029-11f1-bf6b-2200d14b0348 my uuid: efd61072-5344-11f1-b0fd-c713883b0e00 2026-05-20 9:24:15 2 [Note] WSREP: Server nrd-db-mariadb-0 connected to cluster at position 9ff7203e-5029-11f1-bf6b-2200d14b0348:67363 with ID efd61072-5344-11f1-b0fd-c713883b0e00 2026-05-20 9:24:15 2 [Note] WSREP: Server status change disconnected -> connected 2026-05-20 9:24:15 2 [Note] WSREP: ####### My UUID: efd61072-5344-11f1-b0fd-c713883b0e00 2026-05-20 9:24:15 2 [Note] WSREP: Cert index reset to 00000000-0000-0000-0000-000000000000:-1 (proto: 11), state transfer needed: yes 2026-05-20 9:24:15 0 [Note] WSREP: Service thread queue flushed. 2026-05-20 9:24:15 2 [Note] WSREP: ####### Assign initial position for certification: 00000000-0000-0000-0000-000000000000:-1, protocol version: -1 2026-05-20 9:24:15 2 [Note] WSREP: State transfer required: Group state: 9ff7203e-5029-11f1-bf6b-2200d14b0348:67363 Local state: 00000000-0000-0000-0000-000000000000:-1 2026-05-20 9:24:15 2 [Note] WSREP: Server status change connected -> joiner 2026-05-20 9:24:15 0 [Note] WSREP: Joiner monitor thread started to monitor 2026-05-20 9:24:15 0 [Note] WSREP: Running: 'wsrep_sst_mariabackup --role 'joiner' --address '11.127.6.37' --datadir '/mariadb/data/' --parent 12 --progress 0 --binlog 'mysql-bin' --mysqld-args --wsrep-cluster-address=gcomm://11.127.4.37,11.127.5.37,11.127.6.37 --wsrep-node-address=11.127.6.37' WSREP_SST: [INFO] mariabackup SST started on joiner (20260520 09:24:15.647) WSREP_SST: [INFO] SSL configuration: CA='', CAPATH='', CERT='', KEY='', MODE='DISABLED', encrypt='0' (20260520 09:24:15.743) WSREP_SST: [INFO] Streaming with mbstream (20260520 09:24:16.139) WSREP_SST: [INFO] Using socat as streamer (20260520 09:24:16.142) WSREP_SST: [INFO] Moving '/mariadb/data/mariabackup.prepare.log' to '/mariadb/sst/mariabackup.prepare.log.2026.05.20-09.24.16.170579911' (20260520 09:24:16.176) WSREP_SST: [INFO] Moving '/mariadb/data/mariabackup.move.log' to '/mariadb/sst/mariabackup.move.log.2026.05.20-09.24.16.170579911' (20260520 09:24:16.250) WSREP_SST: [INFO] 'ss' is selected as a socket information utility. (20260520 09:24:16.330) WSREP_SST: [INFO] Evaluating timeout -k 310 300 socat -u TCP-LISTEN:4444,reuseaddr stdio | '/usr/bin/mbstream' -x; RC=( ${PIPESTATUS[@]} ) (20260520 09:24:16.397) 2026-05-20 9:24:16 2 [Note] WSREP: ####### IST uuid:00000000-0000-0000-0000-000000000000 f: 0, l: 67363, STRv: 3 2026-05-20 9:24:16 2 [Note] WSREP: IST receiver addr using tcp://11.127.6.37:4568 2026-05-20 9:24:16 2 [Note] WSREP: Prepared IST receiver for 0-67363, listening at: tcp://11.127.6.37:4568 2026-05-20 9:24:16 0 [Note] WSREP: Member 2.0 (nrd-db-mariadb-0) requested state transfer from '*any*'. Selected 0.0 (nrd-db-mariadb-0)(SYNCED) as donor. 2026-05-20 9:24:16 0 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 67363) 2026-05-20 9:24:16 2 [Note] WSREP: Requesting state transfer: success, donor: 0 2026-05-20 9:24:16 2 [Note] WSREP: Resetting GCache seqno map due to different histories. 2026-05-20 9:24:16 2 [Note] WSREP: GCache history reset: 9ff7203e-5029-11f1-bf6b-2200d14b0348:0 -> 9ff7203e-5029-11f1-bf6b-2200d14b0348:67363 WSREP_SST: [INFO] Proceeding with SST (20260520 09:24:17.433) WSREP_SST: [INFO] Evaluating socat -u TCP-LISTEN:4444,reuseaddr stdio | '/usr/bin/mbstream' -x; RC=( ${PIPESTATUS[@]} ) (20260520 09:24:17.434) WSREP_SST: [INFO] Cleaning the old binary logs (20260520 09:24:17.453) WSREP_SST: [INFO] Cleaning the existing datadir and innodb-data/log directories (20260520 09:24:17.524) WSREP_SST: [INFO] Waiting for SST streaming to complete! (20260520 09:24:17.994) 2026-05-20 9:24:18 0 [Note] WSREP: (efd61072-b0fd, 'tcp://0.0.0.0:4567') turning message relay requesting off 2026-05-20 9:24:34 0 [Note] WSREP: 0.0 (nrd-db-mariadb-0): State transfer to 2.0 (nrd-db-mariadb-0) complete. 2026-05-20 9:24:34 0 [Note] WSREP: Member 0.0 (nrd-db-mariadb-0) synced with group. WSREP_SST: [INFO] Preparing the backup at /mariadb/data/.sst (20260520 09:24:34.242) WSREP_SST: [INFO] Evaluating /usr/bin/mariadb-backup --prepare --no-version-check --log-bin='mysql-bin' --use-memory=2G --target-dir='/mariadb/data/.sst' --datadir='/mariadb/data/.sst' --mysqld-args '--wsrep-cluster-address=gcomm://11.127.4.37,11.127.5.37,11.127.6.37' '--wsrep-node-address=11.127.6.37' > '/mariadb/data/mariabackup.prepare.log' 2>&1 (20260520 09:24:34.244) WSREP_SST: [INFO] Moving binary logs to . (20260520 09:24:36.265) WSREP_SST: [INFO] Moving the backup to /mariadb/data (20260520 09:24:36.275) WSREP_SST: [INFO] Evaluating /usr/bin/mariadb-backup --move-back --no-version-check --log-bin='mysql-bin' --use-memory=2G --galera-info --force-non-empty-directories --target-dir='/mariadb/data/.sst' --datadir='/mariadb/data' > '/mariadb/data/mariabackup.move.log' 2>&1 (20260520 09:24:36.277) WSREP_SST: [INFO] Move successful, removing /mariadb/data/.sst (20260520 09:24:37.009) WSREP_SST: [INFO] Galera co-ords from donor: 9ff7203e-5029-11f1-bf6b-2200d14b0348:67365 0 (20260520 09:24:37.029) WSREP_SST: [INFO] Total time on joiner: 0 seconds (20260520 09:24:37.032) WSREP_SST: [INFO] mariabackup SST completed on joiner (20260520 09:24:37.035) WSREP_SST: [INFO] Removing the sst_in_progress file (20260520 09:24:37.039) WSREP_SST: [INFO] Cleaning up temporary directories (20260520 09:24:37.045) 2026-05-20 9:24:37 3 [Note] WSREP: SST received 2026-05-20 9:24:37 3 [Note] WSREP: Server status change joiner -> initializing 2026-05-20 9:24:37 0 [Note] InnoDB: Compressed tables use zlib 1.2.11 2026-05-20 9:24:37 0 [Note] InnoDB: Using transactional memory 2026-05-20 9:24:37 0 [Note] InnoDB: Number of transaction pools: 1 2026-05-20 9:24:37 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions 2026-05-20 9:24:37 0 [Note] mariadbd: O_TMPFILE is not supported on /mariadb/tmp (disabling future attempts) 2026-05-20 9:24:37 0 [Warning] mariadbd: io_uring_queue_init() failed with ENOSYS: check seccomp filters, and the kernel version (newer than 5.1 required) create_uring failed: falling back to libaio 2026-05-20 9:24:37 0 [Note] InnoDB: Using Linux native AIO 2026-05-20 9:24:37 0 [Note] InnoDB: innodb_buffer_pool_size_max=2048m, innodb_buffer_pool_size=2048m 2026-05-20 9:24:37 0 [Note] InnoDB: Completed initialization of buffer pool 2026-05-20 9:24:37 0 [Note] InnoDB: Buffered log writes (block size=512 bytes) 2026-05-20 9:24:37 0 [Note] InnoDB: End of log at LSN=451324644 2026-05-20 9:24:37 0 [Note] InnoDB: Resizing redo log from 12.016KiB to 128.000MiB; LSN=451324644 2026-05-20 9:24:37 0 [Note] InnoDB: Buffered log writes (block size=512 bytes) 2026-05-20 9:24:37 0 [Note] InnoDB: Opened 3 undo tablespaces 2026-05-20 9:24:37 0 [Note] InnoDB: 128 rollback segments in 3 undo tablespaces are active. 2026-05-20 9:24:37 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ... 2026-05-20 9:24:37 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB. 2026-05-20 9:24:37 0 [Note] InnoDB: log sequence number 451324644; transaction id 139213 2026-05-20 9:24:37 0 [Note] Plugin 'FEEDBACK' is disabled. 2026-05-20 9:24:37 0 [Note] InnoDB: Loading buffer pool(s) from /mariadb/data/ib_buffer_pool 2026-05-20 9:24:37 0 [Note] InnoDB: Cannot open '/mariadb/data/ib_buffer_pool' for reading: No such file or directory 2026-05-20 9:24:37 0 [Warning] 'innodb-log-files-in-group' was removed. It does nothing now and exists only for compatibility with old my.cnf files. 2026-05-20 9:24:37 0 [Warning] 'innodb-file-format' was removed. It does nothing now and exists only for compatibility with old my.cnf files. 2026-05-20 9:24:37 0 [Note] Recovering after a crash using mysql-bin 2026-05-20 9:24:37 0 [Note] Starting table crash recovery... 2026-05-20 9:24:37 0 [Note] Crash table recovery finished. 2026-05-20 9:24:38 0 [Note] Server socket created on IP: '0.0.0.0', port: '3306'. 2026-05-20 9:24:39 0 [Note] mariadbd: Event Scheduler: Loaded 0 events 2026-05-20 9:24:39 0 [Note] WSREP: wsrep_init_schema_and_SR (nil) 2026-05-20 9:24:39 0 [Note] WSREP: Server initialized 2026-05-20 9:24:39 0 [Note] WSREP: Server status change initializing -> initialized 2026-05-20 9:24:39 3 [Note] WSREP: Recovered position from storage: 9ff7203e-5029-11f1-bf6b-2200d14b0348:67365 2026-05-20 9:24:39 3 [Note] WSREP: Server status change initialized -> joined 2026-05-20 9:24:39 7 [Note] WSREP: Starting applier thread 7 260520 9:24:39 [ERROR] mariadbd got signal 11 ; Sorry, we probably made a mistake, and this is a bug. Your assistance in bug reporting will enable us to fix this for the next release. To report this bug, see https://mariadb.com/kb/en/reporting-bugs about how to report a bug on https://jira.mariadb.org/. Please include the information from the server start above, to the end of the information below. Server version: 11.4.9-MariaDB-log source revision: aa6a2e6bf05d43c3bb44db50566937a5f97a6580 WSREP: Suppressing further logging WSREP: Shutting down network communications 2026-05-20 9:24:39 6 [Note] WSREP: Starting applier thread 6 The information page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mariadbd/ contains instructions to obtain a better version of the backtrace below. Following these instructions will help MariaDB developers provide a fix quicker. Attempting backtrace. Include this in the bug report. (note: Retrieving this information may fail) Thread pointer: 0x7f0d00000c58 stack_bottom = 0x7f0db17b4000 thread_stack 0x49000 2026-05-20 9:24:39 0 [Note] mariadbd: ready for connections. Version: '11.4.9-MariaDB-log' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server 2026-05-20 9:24:39 0 [Note] mariadbd (initiated by: unknown): Normal shutdown 2026-05-20 9:24:39 0 [Note] WSREP: Shutdown replication 2026-05-20 9:24:39 0 [Note] WSREP: Server status change joined -> disconnecting 2026-05-20 9:24:39 0 [Note] WSREP: Closing send monitor... 2026-05-20 9:24:39 0 [Note] WSREP: Closed send monitor. 2026-05-20 9:24:39 0 [Note] WSREP: gcomm: terminating thread 2026-05-20 9:24:39 0 [Note] WSREP: gcomm: joining thread 2026-05-20 9:24:39 0 [Note] WSREP: gcomm: closing backend 2026-05-20 9:24:39 3 [Note] WSREP: Recovered view from SST: id: 9ff7203e-5029-11f1-bf6b-2200d14b0348:67363 status: primary protocol_version: 4 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: no own_index: 2 members(3): 0: 7b7a3560-502d-11f1-a050-b3e4ffebfc35, nrd-db-mariadb-0 1: a2217745-5029-11f1-a764-8e04dd5ae922, nrd-db-mariadb-0 2: efd61072-5344-11f1-b0fd-c713883b0e00, nrd-db-mariadb-0 addr2line: 'mariadbd': No such file Printing to addr2line failed mariadbd(my_print_stacktrace+0x2e)[0x56471779314e] mariadbd(handle_fatal_signal+0x305)[0x5647172514b5] 2026-05-20 9:24:39 11 [Note] WSREP: Recovered cluster id 9ff7203e-5029-11f1-bf6b-2200d14b0348 /lib64/libc.so.6(+0x3fc30)[0x7f0ebd11ac30] 2026-05-20 9:24:39 3 [Note] WSREP: SST received: 9ff7203e-5029-11f1-bf6b-2200d14b0348:67365 2026-05-20 9:24:39 3 [Note] WSREP: SST succeeded for position 9ff7203e-5029-11f1-bf6b-2200d14b0348:67365 2026-05-20 9:24:39 0 [Note] WSREP: Joiner monitor thread ended with total time 24 sec 2026-05-20 9:24:39 2 [Note] WSREP: Installed new state from SST: 9ff7203e-5029-11f1-bf6b-2200d14b0348:67365 2026-05-20 9:24:39 2 [Note] WSREP: Cert. index preload up to 67365 2026-05-20 9:24:39 0 [Note] WSREP: ####### IST applying starts with 67366 2026-05-20 9:24:39 0 [Note] WSREP: ####### IST current seqno initialized to 67338 2026-05-20 9:24:39 0 [Note] WSREP: Receiving IST... 0.0% (0/26 events) complete. 2026-05-20 9:24:39 0 [Note] WSREP: IST preload starting at 67338 2026-05-20 9:24:39 0 [Note] WSREP: Service thread queue flushed. 2026-05-20 9:24:39 0 [Note] WSREP: ####### Assign initial position for certification: 00000000-0000-0000-0000-000000000000:67337, protocol version: 6 addr2line: 'mariadbd': No such file mariadbd(_ZN10Rpl_filter26table_rule_ent_hash_to_strEP6StringP7st_hashb+0x40)[0x564716efb500] mariadbd(_Z19copy_filter_settingP10Rpl_filterS0_+0xff)[0x5647170f9fef] mariadbd(_ZN11Master_infoC1EP25st_mysql_const_lex_stringb+0x2dd)[0x5647170fce0d] mariadbd(_ZN27Wsrep_high_priority_serviceC2EP3THD+0x22a)[0x5647174f1b5a] mariadbd(_ZN21Wsrep_applier_serviceC2EP3THD+0x1f)[0x5647174f262f] mariadbd(+0xdb8bf7)[0x564717510bf7] mariadbd(_Z15start_wsrep_THDPv+0x256)[0x5647174fed16] mariadbd(+0xd33f09)[0x56471748bf09] 2026-05-20 9:24:39 0 [Note] WSREP: REPL Protocols: 11 (6) 2026-05-20 9:24:39 0 [Note] WSREP: ####### Adjusting cert position: 67359 -> 67360 2026-05-20 9:24:39 0 [Note] WSREP: Service thread queue flushed. 2026-05-20 9:24:39 0 [Note] WSREP: Lowest cert index boundary for CC from preload: 67338 2026-05-20 9:24:39 0 [Note] WSREP: Min available from gcache for CC from preload: 67338 2026-05-20 9:24:39 0 [Note] WSREP: REPL Protocols: 11 (6) 2026-05-20 9:24:39 0 [Note] WSREP: ####### Adjusting cert position: 67362 -> 67363 2026-05-20 9:24:39 0 [Note] WSREP: Service thread queue flushed. 2026-05-20 9:24:39 0 [Note] WSREP: Lowest cert index boundary for CC from preload: 67338 2026-05-20 9:24:39 0 [Note] WSREP: Min available from gcache for CC from preload: 67338 2026-05-20 9:24:39 0 [Note] WSREP: Receiving IST... 100.0% (26/26 events) complete. 2026-05-20 9:24:39 2 [Note] WSREP: Cert. index preloaded up to 67363 /lib64/libc.so.6(+0x8b2ea)[0x7f0ebd1662ea] /lib64/libc.so.6(clone+0x44)[0x7f0ebd1ea5f4] Connection ID (thread ID): 9 Status: NOT_KILLED Query (0x0): (null) Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off,hash_join_cardinality=on,cset_narrowing=off,sargable_casefold=on Writing a core file... Working directory at /mariadb/data Resource Limits (excludes unlimited resources): Limit Soft Limit Hard Limit Units Max stack size 8388608 unlimited bytes Max open files 1048576 1048576 files Max locked memory 65536 65536 bytes Max pending signals 561462 561462 signals Max msgqueue size 819200 819200 bytes Max nice priority 0 0 Max realtime priority 0 0 Core pattern: | /usr/lib/systemd/systemd-coredump %p %u %g %s %t %e Kernel version: Linux version 4.18.0-348.el8.0.2.x86_64 (mockbuild@dal1-prod-builder001.bld.equ.rockylinux.org) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-3) (GCC)) #1 SMP Sun Nov 14 00:51:12 UTC 2021 bash: line 8: 12 Segmentation fault (core dumped) /usr/local/bin/docker-entrypoint.sh mariadbd --wsrep-cluster-address=$MARIADB_GALERA_CLUSTER_ADDRESS --wsrep-node-address=$MARIADB_GALERA_NODE_ADDRESS [root@master1 nokiauser]# kgp $n Thu May 21 04:59:33 UTC 2026 NAME READY STATUS RESTARTS AGE nrd-db-mariadb-0 2/2 Running 3 (14h ago) 47h nrddb-v1-7b955f7ff4-wtl7w 4/4 Running 0 47h [root@master1 nokiauser]# [root@master1 nokiauser]# [root@master1 nokiauser]# [root@master1 nokiauser]# [root@master1 nokiauser]# [root@master1 nokiauser]# [root@master1 nokiauser]# [root@master1 nokiauser]# [root@master1 nokiauser]# k describe po nrd-db-mariadb-0 $n Thu May 21 05:01:53 UTC 2026 Name: nrd-db-mariadb-0 Namespace: nrd-database Priority: 0 Service Account: nrd-db-mariadb Node: worker1vm.novalocal/11.127.146.121 Start Time: Tue, 19 May 2026 05:36:56 +0000 Labels: app=nrd_db chart=nrd_db-2607.13.0 component=master controller-revision-hash=nrd-db-mariadb-6bdbf89475 release=2607.13.0 statefulset.kubernetes.io/pod-name=nrd-db-mariadb-0 topologylab=nrdlabel Annotations: cni.projectcalico.org/containerID: fccdabd71cebb9b961c1379cfd44710c3528c68de9efae1a55ccdcec1f8ff2a2 cni.projectcalico.org/podIP: 10.244.158.145/32 cni.projectcalico.org/podIPs: 10.244.158.145/32 k8s.v1.cni.cncf.io/network-status: [{ "name": "", "ips": [ "10.244.158.145" ], "default": true, "dns": {} },{ "name": "nrd-database/nrd-db", "interface": "net1", "ips": [ "11.127.6.37" ], "mac": "fa:ac:a6:04:4d:00", "dns": {} }] k8s.v1.cni.cncf.io/networks: nrd-db k8s.v1.cni.cncf.io/networks-status: [{ "name": "", "ips": [ "10.244.158.145" ], "default": true, "dns": {} },{ "name": "nrd-database/nrd-db", "interface": "net1", "ips": [ "11.127.6.37" ], "mac": "fa:ac:a6:04:4d:00", "dns": {} }] sidecar.istio.io/inject: false Status: Running IP: 10.244.158.145 IPs: IP: 10.244.158.145 Controlled By: StatefulSet/nrd-db-mariadb Init Containers: init: Container ID: containerd://681e30a4f122c8a4ccbd54df82a5656b9fecb521ee3a3f4dcbc71e771365c583 Image: master1.novalocal:5000/nrd:NRD_26_7_B1-13 Image ID: master1.novalocal:5000/nrd@sha256:15da19c939be567efe91902f6d3d0452538bf5b3adfa1fd4b262a4a3d50b651f Port: Host Port: Command: /bin/sh -c sudo ip rule add from all to 11.127.6.37 table 1000; sudo ip rule add from 11.127.6.37 to all table 1000; sudo ip route del 11.127.6.0/24 dev net1; sudo ip route add 11.127.6.0/24 dev net1 proto kernel scope link src 11.127.6.37 table 1000; sudo ip route add default via 11.127.6.51 table 1000; State: Terminated Reason: Completed Exit Code: 0 Started: Tue, 19 May 2026 05:37:26 +0000 Finished: Tue, 19 May 2026 05:37:26 +0000 Ready: True Restart Count: 0 Limits: cpu: 1 memory: 1Gi Requests: cpu: 1 memory: 1Gi Environment: Mounts: /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-8mhrm (ro) decode-pwds: Container ID: containerd://944aaaf38180bab32f6ba895665589a4400b87366331770b3c9c85c71c10a5ad Image: master1.novalocal:5000/mariadb:NRDDB_26_7_B1-13 Image ID: master1.novalocal:5000/mariadb@sha256:5b86496a5369c9b58949a0856e071d36089c9ce6852a5f5a3b8b97a9139b7f8b Port: Host Port: Command: /bin/sh -c echo $MDBRP_ID > /opt/mariadb/rootpwd; echo $MDBUP_ID > /opt/mariadb/userpwd; touch /opt/mariadb/mariadb-root-password; chmod 777 /opt/mariadb/mariadb-root-password; touch /opt/mariadb/mariadb-password; chmod 777 /opt/mariadb/mariadb-password; echo "Decrypting root password"; /usr/bin/nrd_cipher "decrypt" "/opt/mariadb/rootpwd" "/opt/mariadb/mariadb-root-password"; echo "Decrypting user password"; /usr/bin/nrd_cipher "decrypt" "/opt/mariadb/userpwd" "/opt/mariadb/mariadb-password"; rm /opt/mariadb/rootpwd; rm /opt/mariadb/userpwd; echo "Password decryption complete"; State: Terminated Reason: Completed Exit Code: 0 Started: Tue, 19 May 2026 05:38:00 +0000 Finished: Tue, 19 May 2026 05:38:00 +0000 Ready: True Restart Count: 0 Limits: cpu: 1 memory: 1Gi Requests: cpu: 1 memory: 1Gi Environment: MDBUP_ID: ncmxc| MDBRP_ID: ncmxc| Mounts: /opt/mariadb from opt-data (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-8mhrm (ro) Containers: mariadb: Container ID: containerd://838381a65a99414aef1c179c513c7d5507def6adaac6c66b21a6e233c4b460eb Image: master1.novalocal:5000/mariadb:NRDDB_26_7_B1-13 Image ID: master1.novalocal:5000/mariadb@sha256:5b86496a5369c9b58949a0856e071d36089c9ce6852a5f5a3b8b97a9139b7f8b Ports: 3306/TCP, 4567/TCP, 4568/TCP, 4444/TCP Host Ports: 0/TCP, 0/TCP, 0/TCP, 0/TCP Command: bash -ec MARIADB_ROOT_PASSWORD=$(cat "$MARIADB_ROOT_PASSWORD_FILE") MARIADB_PASSWORD=$(cat "$MARIADB_PASSWORD_FILE") source /opt/mariadb/bootstrapenv.sh if [[ "$MARIADB_GALERA_CLUSTER_BOOTSTRAP" == "yes" ]]; then /usr/local/bin/docker-entrypoint.sh mariadbd --wsrep-cluster-address=$MARIADB_GALERA_CLUSTER_ADDRESS --wsrep-node-address=$MARIADB_GALERA_NODE_ADDRESS --wsrep-new-cluster else /usr/local/bin/docker-entrypoint.sh mariadbd --wsrep-cluster-address=$MARIADB_GALERA_CLUSTER_ADDRESS --wsrep-node-address=$MARIADB_GALERA_NODE_ADDRESS fi #tail -f /dev/null State: Running Started: Wed, 20 May 2026 14:25:06 +0000 Last State: Terminated Reason: Error Exit Code: 139 Started: Wed, 20 May 2026 14:24:14 +0000 Finished: Wed, 20 May 2026 14:24:39 +0000 Ready: True Restart Count: 3 Limits: cpu: 600m memory: 1600Mi Requests: cpu: 300m memory: 800Mi Environment: TZ: America/Chicago MARIADB_GALERA_CLUSTER_NAME: nrdGalera MARIADB_GALERA_CLUSTER_ADDRESS: gcomm://11.127.4.37,11.127.5.37,11.127.6.37 MARIADB_GALERA_NODE_ADDRESS: 11.127.6.37 MARIADB_GALERA_MARIABACKUP_USER: nrd MARIADB_GALERA_MARIABACKUP_PASSWORD_FILE: /mariadb/mariadb-root-password ALLOW_EMPTY_PASSWORD: true MYSQL_ALLOW_EMPTY_PASSWORD: true MARIADB_ROOT_PASSWORD_FILE: /opt/mariadb/mariadb-root-password TMPDIR: /mariadb/tmp MARIADB_USER: nrd MARIADB_PASSWORD_FILE: /opt/mariadb/mariadb-password MARIADB_DATABASE: my_database Mounts: /etc/my.cnf.d/nrd.cnf from nrd-config (rw,path="nrd.cnf") /mariadb from data (rw) /mariadb/logs from data (rw,path="logs") /mariadb/tmp from data (rw,path="tmp") /opt/mariadb from opt-data (rw) /opt/mariadb/bootstrapenv.sh from env-configmap (rw,path="bootstrapenv.sh") /opt/mariadb/mariadb-notify.sh from mariadb-notify-script (rw,path="mariadb-notify.sh") /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-8mhrm (ro) metrics: Container ID: containerd://311c49ba591e93e9156f8d8c6a54b4870f304c737f8315671e4a279ea3df1176 Image: master1.novalocal:5000/prom/mysqld-exporter:v0.18.0 Image ID: master1.novalocal:5000/prom/mysqld-exporter@sha256:60cbc468c116eab121c329e38a12a31191fd43a31aa0b7180090d4520252dbcb Port: 9104/TCP Host Port: 0/TCP Command: sh -c /bin/mysqld_exporter --config.my-cnf=/tmp/mysqld-exporter/exporter.cnf State: Running Started: Tue, 19 May 2026 05:38:16 +0000 Ready: True Restart Count: 0 Limits: cpu: 300m memory: 800Mi Requests: cpu: 100m memory: 100Mi Environment: TZ: America/Chicago MARIADB_ROOT_PASSWORD_FILE: /opt/mariadb/mariadb-root-password Mounts: /opt/mariadb from opt-data (rw) /tmp/mysqld-exporter/exporter.cnf from mysql-exporter-config (rw,path="exporter.cnf") /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-8mhrm (ro) Conditions: Type Status Initialized True Ready True ContainersReady True PodScheduled True Volumes: env-configmap: Type: ConfigMap (a volume populated by a ConfigMap) Name: env-configmap Optional: false mariadb-notify-script: Type: ConfigMap (a volume populated by a ConfigMap) Name: mariadb-notify-script Optional: false empty: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: empty-maxscale: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: empty-cdc: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: maxscale-config: Type: ConfigMap (a volume populated by a ConfigMap) Name: maxscale-conf Optional: false nrd-config: Type: ConfigMap (a volume populated by a ConfigMap) Name: nrd-db-mariadb Optional: false mysql-exporter-config: Type: Secret (a volume populated by a Secret) SecretName: mysql-exporter-config Optional: false opt-data: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: data: Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace) ClaimName: nrd-database-db-pvc ReadOnly: false kube-api-access-8mhrm: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: DownwardAPI: true QoS Class: Burstable Node-Selectors: Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 1s node.kubernetes.io/not-ready:NoSchedule op=Exists node.kubernetes.io/unreachable:NoExecute op=Exists for 1s node.kubernetes.io/unreachable:NoSchedule op=Exists Events: [root@master1 nokiauser]# [root@master1 nokiauser]# [root@master1 nokiauser]# [root@master1 nokiauser]# [root@master1 nokiauser]# [root@master1 nokiauser]# [root@master1 nokiauser]# helm history nrd-db $n REVISION UPDATED STATUS CHART APP VERSION DESCRIPTION 1 Fri May 15 07:29:08 2026 superseded nrd_db-2511.61.0 10.3.16 Install complete 2 Sat May 16 12:03:19 2026 superseded nrd_db-2603.47.0 10.3.16 Upgrade complete 3 Tue May 19 05:29:17 2026 deployed nrd_db-2607.13.0 10.3.16 Upgrade complete [root@master1 nokiauser]# [root@master1 nokiauser]# [root@master1 nokiauser]# [root@master1 nokiauser]# [root@master1 nokiauser]#