Details
Description
We have a 3-node Galera set up with binlog replication happening to another 3-node galera cluster. On the primary Galera cluster, mariadb process is aborted and restarted automatically in under 30 seconds, and it comes up fine with no issues.
Attaching the stack trace from the error log file:
mariadbd: /home/buildbot/buildbot/padding_for_CPACK_RPM_BUILD_SOURCE_DIRS_PREFIX/mariadb-10.6.11/wsrep-lib/include/wsrep/client_state.hpp:668: int wsrep::client_state::bf_abort(wsrep::seqno): Assertion `mode_ == m_local || transaction_.is_streaming()' failed.
230831 19:18:32 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
Server version: 10.6.11-MariaDB-log
key_buffer_size=134217728
read_buffer_size=131072
max_used_connections=18
max_threads=1002
thread_count=12
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 2337543 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x7fd470000c58
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7fdaf00fdbc8 thread_stack 0x49000
Can't start addr2line
/usr/sbin/mariadbd(my_print_stacktrace+0x2e)[0x5562edd0c33e]
/usr/sbin/mariadbd(handle_fatal_signal+0x485)[0x5562ed7ec295]
/lib64/libpthread.so.0(+0x12d40)[0x7fdb56753d40]
/lib64/libc.so.6(gsignal+0x10f)[0x7fdb55aafb8f]
/lib64/libc.so.6(abort+0x127)[0x7fdb55a82ea5]
/lib64/libc.so.6(+0x21d79)[0x7fdb55a82d79]
/lib64/libc.so.6(+0x474e6)[0x7fdb55aa84e6]
/usr/sbin/mariadbd(Z14wsrep_bf_abortP3THDS0+0x5f9)[0x5562eda90949]
/usr/sbin/mariadbd(wsrep_thd_bf_abort+0x19)[0x5562eda982a9]
/usr/sbin/mariadbd(+0xc7d64c)[0x5562edaba64c]
/usr/sbin/mariadbd(+0x6398fa)[0x5562ed4768fa]
/usr/sbin/mariadbd(+0x639964)[0x5562ed476964]
/usr/sbin/mariadbd(+0xd322af)[0x5562edb6f2af]
/usr/sbin/mariadbd(+0xd34731)[0x5562edb71731]
/usr/sbin/mariadbd(+0xc86da5)[0x5562edac3da5]
/usr/sbin/mariadbd(_ZN7handler12ha_write_rowEPKh+0x21d)[0x5562ed7fa8bd]
/usr/sbin/mariadbd(_ZN14Rows_log_event9write_rowEP14rpl_group_infob+0x175)[0x5562ed8f9365]
/usr/sbin/mariadbd(_ZN20Write_rows_log_event11do_exec_rowEP14rpl_group_info+0x10d)[0x5562ed8f99ad]
/usr/sbin/mariadbd(_ZN14Rows_log_event14do_apply_eventEP14rpl_group_info+0x27f)[0x5562ed8ee9ef]
/usr/sbin/mariadbd(_Z18wsrep_apply_eventsP3THDP14Relay_log_infoPKvm+0x1dc)[0x5562eda8f71c]
/usr/sbin/mariadbd(_ZN21Wsrep_applier_service15apply_write_setERKN5wsrep7ws_metaERKNS0_12const_bufferERNS0_14mutable_bufferE+0xa4)[0x5562eda75ff4]
/usr/sbin/mariadbd(+0xf5d89a)[0x5562edd9a89a]
/usr/sbin/mariadbd(_ZN5wsrep12server_state8on_applyERNS_21high_priority_serviceERKNS_9ws_handleERKNS_7ws_metaERKNS_12const_bufferE+0x15d)[0x5562edd9b42d]
/usr/sbin/mariadbd(+0xf71f34)[0x5562eddaef34]
/usr/lib64/galera-4/libgalera_smm.so(+0x57cef)[0x7fdb479e7cef]
/usr/lib64/galera-4/libgalera_smm.so(+0x69200)[0x7fdb479f9200]
/usr/lib64/galera-4/libgalera_smm.so(+0x6b03e)[0x7fdb479fb03e]
/usr/lib64/galera-4/libgalera_smm.so(+0x974eb)[0x7fdb47a274eb]
/usr/lib64/galera-4/libgalera_smm.so(+0x976a2)[0x7fdb47a276a2]
/usr/lib64/galera-4/libgalera_smm.so(+0x98171)[0x7fdb47a28171]
/usr/lib64/galera-4/libgalera_smm.so(+0x6bfc0)[0x7fdb479fbfc0]
/usr/lib64/galera-4/libgalera_smm.so(+0x47538)[0x7fdb479d7538]
/usr/sbin/mariadbd(_ZN5wsrep18wsrep_provider_v2611run_applierEPNS_21high_priority_serviceE+0xe)[0x5562eddafd0e]
/usr/sbin/mariadbd(+0xc546ed)[0x5562eda916ed]
/usr/sbin/mariadbd(_Z15start_wsrep_THDPv+0x28c)[0x5562eda8119c]
/usr/sbin/mariadbd(+0xbd113d)[0x5562eda0e13d]
/lib64/libpthread.so.0(+0x81da)[0x7fdb567491da]
/lib64/libc.so.6(clone+0x43)[0x7fdb55a9ae73]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x7fd404bb5493): LOAD DATA LOCAL INFILE ':SOURCE:21fc815f20034519b1f4a8327db39126' INTO TABLE TBL_LINKED_ORDERS_INPUT CHARACTER SET utf8mb4 FIELDS TERMINATED BY ' ' ESCAPED BY '
' LINES TERMINATED BY '
' IGNORE 0 LINES (`LEAD_JIN`,`LEAD_TYPE`,`LEAD_DATE`,`LEAD_STATUS`,`CHILD_JIN`,`CHILD_TYPE`,`CHILD_DATE`,`CHILD_STATUS`,`WMQUEUE`)
Connection ID (thread ID): 6
Status: NOT_KILLED
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off
The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
information that should help you find out what is causing the crash.
Writing a core file...
Working directory at /data/mariadata/mysql
Resource Limits:
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 127401 127401 processes
Max open files 32768 32768 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 127401 127401 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
Core pattern: |/opt/dynatrace/oneagent/agent/rdp -p %p -P %P -c %c -e %e -s %s -a %P %u %g %s %t %c %h %e
Kernel version: Linux version 4.18.0-477.15.1.el8_8.x86_64 (mockbuild@host-100-100-224-19) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-18.0.2) (GCC)) #1 SMP Thu Jul 20 11:31:48 PDT 2023
2023-08-31 19:18:47 0 [Note] WSREP: Loading provider /usr/lib64/galera-4/libgalera_smm.so initial position: c9a3c950-7d2f-11ed-9e25-5fc70c4f0d05:16049854
2023-08-31 19:18:47 0 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib64/galera-4/libgalera_smm.so'
2023-08-31 19:18:47 0 [Note] WSREP: wsrep_load(): Galera 26.4.13(rfe497aeb) by Codership Oy <info@codership.com> loaded successfully.
2023-08-31 19:18:47 0 [Note] WSREP: CRC-32C: using 64-bit x86 acceleration.
2023-08-31 19:18:47 0 [Note] WSREP: Found saved state: c9a3c950-7d2f-11ed-9e25-5fc70c4f0d05:-1, safe_to_bootstrap: 0
2023-08-31 19:18:47 0 [Note] WSREP: GCache DEBUG: opened preamble:
Version: 2
UUID: c9a3c950-7d2f-11ed-9e25-5fc70c4f0d05
Seqno: -1 - -1
Offset: -1
Synced: 0
2023-08-31 19:18:47 0 [Note] WSREP: Recovering GCache ring buffer: version: 2, UUID: c9a3c950-7d2f-11ed-9e25-5fc70c4f0d05, offset: -1
2023-08-31 19:18:47 0 [Note] WSREP: GCache::RingBuffer initial scan... 0.0% ( 0/524288024 bytes) complete.
2023-08-31 19:18:47 0 [Note] WSREP: GCache::RingBuffer initial scan...100.0% (524288024/524288024 bytes) complete.
2023-08-31 19:18:47 0 [Note] WSREP: Recovering GCache ring buffer: Recovery failed, need to do full reset.
2023-08-31 19:18:47 0 [Note] WSREP: Passing config to GCS: base_dir = /data/mariadata/mysql/; base_host = 10.13.237.29; base_port = 4567; cert.log_conflicts = no; cert.optimistic_pa = yes; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /data/mariadata/mysql/; gcache.keep_pages_size = 0; gcache.keep_plaintext_size = 500M; gcache.mem_size = 0; gcache.name = galera.cache; gcache.page_size = 500M; gcache.recover = yes; gcache.size = 500M; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.fc_single_primary = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcas
2023-08-31 19:18:47 0 [Note] WSREP: Service thread queue flushed.
2023-08-31 19:18:47 0 [Note] WSREP: ####### Assign initial position for certification: c9a3c950-7d2f-11ed-9e25-5fc70c4f0d05:16049854, protocol version: -1
2023-08-31 19:18:47 0 [Note] WSREP: GCache history reset: c9a3c950-7d2f-11ed-9e25-5fc70c4f0d05:0 -> c9a3c950-7d2f-11ed-9e25-5fc70c4f0d05:16049854
2023-08-31 19:18:47 0 [Note] WSREP: Start replication
2023-08-31 19:18:47 0 [Note] WSREP: Connecting with bootstrap option: 0
2023-08-31 19:18:47 0 [Note] WSREP: Setting GCS initial position to c9a3c950-7d2f-11ed-9e25-5fc70c4f0d05:16049854
2023-08-31 19:18:47 0 [Note] WSREP: protonet asio version 0
2023-08-31 19:18:47 0 [Note] WSREP: Using CRC-32C for message checksums.
2023-08-31 19:18:47 0 [Note] WSREP: backend: asio
2023-08-31 19:18:47 0 [Note] WSREP: gcomm thread scheduling priority set to other:0
2023-08-31 19:18:47 0 [Note] WSREP: restore pc from disk successfully
2023-08-31 19:18:47 0 [Note] WSREP: GMCast version 0
2023-08-31 19:18:47 0 [Note] WSREP: (213ecf6f-a3aa, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
2023-08-31 19:18:47 0 [Note] WSREP: (213ecf6f-a3aa, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
2023-08-31 19:18:47 0 [Note] WSREP: EVS version 1
2023-08-31 19:18:47 0 [Note] WSREP: gcomm: connecting to group 'galera-prod', peer '10.13.237.28:,10.13.237.29:,10.13.237.30:'
2023-08-31 19:18:47 0 [Note] WSREP: (213ecf6f-a3aa, 'tcp://0.0.0.0:4567') Found matching local endpoint for a connection, blacklisting address tcp://10.13.237.29:4567
2023-08-31 19:18:47 0 [Note] WSREP: (213ecf6f-a3aa, 'tcp://0.0.0.0:4567') connection established to a564b541-b726 tcp://10.13.237.28:4567
2023-08-31 19:18:47 0 [Note] WSREP: (213ecf6f-a3aa, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers:
2023-08-31 19:18:47 0 [Note] WSREP: (213ecf6f-a3aa, 'tcp://0.0.0.0:4567') connection established to 902a3da4-af60 tcp://10.13.237.30:4567
2023-08-31 19:18:47 0 [Note] WSREP: EVS version upgrade 0 -> 1
2023-08-31 19:18:47 0 [Note] WSREP: declaring 902a3da4-af60 at tcp://10.13.237.30:4567 stable
2023-08-31 19:18:47 0 [Note] WSREP: declaring a564b541-b726 at tcp://10.13.237.28:4567 stable
2023-08-31 19:18:47 0 [Note] WSREP: PC protocol upgrade 0 -> 1
2023-08-31 19:18:47 0 [Note] WSREP: Node 902a3da4-af60 state prim
2023-08-31 19:18:47 0 [Note] WSREP: view(view_id(PRIM,213ecf6f-a3aa,17) memb
joined {
} left {
} partitioned {
})
2023-08-31 19:18:47 0 [Note] WSREP: save pc into disk
2023-08-31 19:18:47 0 [Note] WSREP: clear restored view
2023-08-31 19:18:48 0 [Note] WSREP: gcomm: connected
2023-08-31 19:18:48 0 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636
2023-08-31 19:18:48 0 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
2023-08-31 19:18:48 0 [Note] WSREP: Opened channel 'galera-prod'
2023-08-31 19:18:48 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 3
2023-08-31 19:18:48 0 [Note] WSREP: STATE_EXCHANGE: sent state UUID: d44be2bc-482a-11ee-ae66-3773224befaa
2023-08-31 19:18:48 1 [Note] WSREP: Starting rollbacker thread 1
2023-08-31 19:18:48 2 [Note] WSREP: Starting applier thread 2
2023-08-31 19:18:48 0 [Note] WSREP: STATE EXCHANGE: sent state msg: d44be2bc-482a-11ee-ae66-3773224befaa
2023-08-31 19:18:48 0 [Note] WSREP: STATE EXCHANGE: got state msg: d44be2bc-482a-11ee-ae66-3773224befaa from 0 (blp21152017)
2023-08-31 19:18:48 0 [Note] WSREP: STATE EXCHANGE: got state msg: d44be2bc-482a-11ee-ae66-3773224befaa from 1 (blp21152018)
2023-08-31 19:18:48 0 [Note] WSREP: STATE EXCHANGE: got state msg: d44be2bc-482a-11ee-ae66-3773224befaa from 2 (blp21152016)
2023-08-31 19:18:48 0 [Note] WSREP: Quorum results:
version = 6,
component = PRIMARY,
conf_id = 16,
members = 2/3 (joined/total),
act_id = 16049865,
last_appl. = 16049855,
protocols = 2/10/4 (gcs/repl/appl),
vote policy= 0,
group UUID = c9a3c950-7d2f-11ed-9e25-5fc70c4f0d05
2023-08-31 19:18:48 0 [Note] WSREP: Flow-control interval: [28, 28]
2023-08-31 19:18:48 0 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 16049866)
2023-08-31 19:18:48 2 [Note] WSREP: ####### processing CC 16049866, local, ordered
2023-08-31 19:18:48 2 [Note] WSREP: Process first view: c9a3c950-7d2f-11ed-9e25-5fc70c4f0d05 my uuid: 213ecf6f-3f1b-11ee-a3aa-1b9e106b662b
2023-08-31 19:18:48 2 [Note] WSREP: Server blp21152017 connected to cluster at position c9a3c950-7d2f-11ed-9e25-5fc70c4f0d05:16049866 with ID 213ecf6f-3f1b-11ee-a3aa-1b9e106b662b
2023-08-31 19:18:48 2 [Note] WSREP: Server status change disconnected -> connected
2023-08-31 19:18:48 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2023-08-31 19:18:48 2 [Note] WSREP: ####### My UUID: 213ecf6f-3f1b-11ee-a3aa-1b9e106b662b
2023-08-31 19:18:48 2 [Note] WSREP: Cert index reset to 00000000-0000-0000-0000-000000000000:-1 (proto: 10), state transfer needed: yes
2023-08-31 19:18:48 0 [Note] WSREP: Service thread queue flushed.
2023-08-31 19:18:48 2 [Note] WSREP: ####### Assign initial position for certification: 00000000-0000-0000-0000-000000000000:-1, protocol version: -1
2023-08-31 19:18:48 2 [Note] WSREP: State transfer required:
Group state: c9a3c950-7d2f-11ed-9e25-5fc70c4f0d05:16049866
Local state: c9a3c950-7d2f-11ed-9e25-5fc70c4f0d05:16049854
2023-08-31 19:18:48 2 [Note] WSREP: Server status change connected -> joiner
2023-08-31 19:18:48 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2023-08-31 19:18:48 0 [Note] WSREP: Joiner monitor thread started to monitor
2023-08-31 19:18:48 0 [Note] WSREP: Running: 'wsrep_sst_rsync --role 'joiner' --address '10.13.237.29' --datadir '/data/mariadata/mysql/' --parent 3808279 --progress 0 --binlog '/data/mariadata/binlogs/mariadb-bin' --mysqld-args --wsrep_start_position=c9a3c950-7d2f-11ed-9e25-5fc70c4f0d05:16049854,5-1-16049240'
WSREP_SST: [INFO] rsync SST started on joiner (20230831 19:18:48.299)
2023-08-31 19:18:48 2 [Note] WSREP: ####### IST uuid:c9a3c950-7d2f-11ed-9e25-5fc70c4f0d05 f: 16049855, l: 16049866, STRv: 3
2023-08-31 19:18:48 2 [Note] WSREP: IST receiver addr using tcp://10.13.237.29:4568
2023-08-31 19:18:48 2 [Note] WSREP: Prepared IST receiver for 16049855-16049866, listening at: tcp://10.13.237.29:4568
2023-08-31 19:18:48 0 [Note] WSREP: Member 0.0 (blp21152017) requested state transfer from 'any'. Selected 2.0 (blp21152016)(SYNCED) as donor.
2023-08-31 19:18:48 0 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 16049866)
2023-08-31 19:18:48 2 [Note] WSREP: Requesting state transfer: success, donor: 2
2023-08-31 19:18:48 0 [Note] WSREP: 2.0 (blp21152016): State transfer to 0.0 (blp21152017) complete.
2023-08-31 19:18:48 0 [Note] WSREP: Member 2.0 (blp21152016) synced with group.
WSREP_SST: [INFO] Galera co-ords from recovery: c9a3c950-7d2f-11ed-9e25-5fc70c4f0d05:16049854 5 (20230831 19:18:49.560)
WSREP_SST: [INFO] rsync IST completed on joiner (20230831 19:18:49.563)
WSREP_SST: [INFO] Joiner cleanup: rsync PID=3808409, stunnel PID=0 (20230831 19:18:49.567)
WSREP_SST: [INFO] Joiner cleanup done. (20230831 19:18:50.111)
2023-08-31 19:18:50 3 [Note] WSREP: SST received
2023-08-31 19:18:50 3 [Note] WSREP: Server status change joiner -> initializing
2023-08-31 19:18:50 3 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2023-08-31 19:18:50 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2023-08-31 19:18:50 0 [Note] InnoDB: Number of pools: 1
2023-08-31 19:18:50 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
2023-08-31 19:18:50 0 [Note] InnoDB: Using Linux native AIO
2023-08-31 19:18:50 0 [Note] InnoDB: Initializing buffer pool, total size = 25769803776, chunk size = 134217728
2023-08-31 19:18:50 0 [Note] InnoDB: Completed initialization of buffer pool
2023-08-31 19:18:50 0 [Note] InnoDB: 128 rollback segments are active.
2023-08-31 19:18:50 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2023-08-31 19:18:50 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2023-08-31 19:18:50 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2023-08-31 19:18:50 0 [Note] InnoDB: 10.6.11 started; log sequence number 1484110312804; transaction id 60758841
2023-08-31 19:18:50 0 [Note] InnoDB: Loading buffer pool(s) from /data/mariadata/mysql/ib_buffer_pool
2023-08-31 19:18:50 0 [Note] Plugin 'FEEDBACK' is disabled.
2023-08-31 19:18:50 0 [Note] Server socket created on IP: '0.0.0.0'.
2023-08-31 19:18:50 0 [Note] WSREP: wsrep_init_schema_and_SR (nil)
2023-08-31 19:18:50 0 [Note] WSREP: Server initialized
2023-08-31 19:18:50 0 [Note] WSREP: Server status change initializing -> initialized
2023-08-31 19:18:50 0 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2023-08-31 19:18:50 3 [Note] WSREP: Recovered position from storage: c9a3c950-7d2f-11ed-9e25-5fc70c4f0d05:16049854
2023-08-31 19:18:50 3 [Note] WSREP: Recovered view from SST:
id: c9a3c950-7d2f-11ed-9e25-5fc70c4f0d05:16024761
status: primary
protocol_version: 4
capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
final: no
own_index: -1
members(3):
0: 213ecf6f-3f1b-11ee-a3a9-1b9e106b662b, blp21152017
1: 902a3da4-3f15-11ee-af60-b706fe188821, blp21152018
2: a564b541-3eca-11ee-b726-a2aa4392ef31, blp21152016
2023-08-31 19:18:50 6 [Note] WSREP: Starting applier thread 6
2023-08-31 19:18:50 3 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2023-08-31 19:18:50 0 [Note] /usr/sbin/mariadbd: ready for connections.
Version: '10.6.11-MariaDB-log' socket: '/data/mariadata/mysql/mysql.sock' port: 3306 MariaDB Server
2023-08-31 19:18:50 9 [Note] WSREP: Starting applier thread 9
2023-08-31 19:18:50 11 [Note] WSREP: Recovered cluster id c9a3c950-7d2f-11ed-9e25-5fc70c4f0d05
2023-08-31 19:18:50 7 [Note] WSREP: Starting applier thread 7
2023-08-31 19:18:50 3 [Note] WSREP: SST received: c9a3c950-7d2f-11ed-9e25-5fc70c4f0d05:16049854
2023-08-31 19:18:50 3 [Note] WSREP: SST succeeded for position c9a3c950-7d2f-11ed-9e25-5fc70c4f0d05:16049854
2023-08-31 19:18:50 0 [Note] WSREP: Joiner monitor thread ended with total time 2 sec
2023-08-31 19:18:50 2 [Note] WSREP: Installed new state from SST: c9a3c950-7d2f-11ed-9e25-5fc70c4f0d05:16049854
2023-08-31 19:18:50 2 [Note] WSREP: Receiving IST: 12 writesets, seqnos 16049855-16049866
2023-08-31 19:18:50 0 [Note] WSREP: ####### IST applying starts with 16049855
2023-08-31 19:18:50 0 [Note] WSREP: ####### IST current seqno initialized to 16049855
2023-08-31 19:18:50 0 [Note] WSREP: Receiving IST... 0.0% ( 0/12 events) complete.
2023-08-31 19:18:50 0 [Note] WSREP: IST preload starting at 16049855
2023-08-31 19:18:50 0 [Note] WSREP: Service thread queue flushed.
2023-08-31 19:18:50 0 [Note] WSREP: ####### Assign initial position for certification: 00000000-0000-0000-0000-000000000000:16049854, protocol version: 5
2023-08-31 19:18:50 0 [Note] InnoDB: Buffer pool(s) load completed at 230831 19:18:50
2023-08-31 19:18:51 0 [Note] WSREP: (213ecf6f-a3aa, 'tcp://0.0.0.0:4567') turning message relay requesting off
2023-08-31 19:18:51 0 [Note] WSREP: REPL Protocols: 10 (5)
2023-08-31 19:18:51 0 [Note] WSREP: ####### Adjusting cert position: 16049862 -> 16049863
2023-08-31 19:18:51 0 [Note] WSREP: Service thread queue flushed.
2023-08-31 19:18:51 0 [Note] WSREP: Lowest cert index boundary for CC from ist: 16049855
2023-08-31 19:18:51 0 [Note] WSREP: Min available from gcache for CC from ist: 16049855
2023-08-31 19:18:51 9 [Note] WSREP: ================================================
View:
id: c9a3c950-7d2f-11ed-9e25-5fc70c4f0d05:16049863
status: primary
protocol_version: 4
capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
final: no
own_index: -1
members(2):
0: 902a3da4-3f15-11ee-af60-b706fe188821, blp21152018
1: a564b541-3eca-11ee-b726-a2aa4392ef31, blp21152016
=================================================
2023-08-31 19:18:51 9 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2023-08-31 19:18:51 0 [Note] WSREP: REPL Protocols: 10 (5)
2023-08-31 19:18:51 0 [Note] WSREP: ####### Adjusting cert position: 16049865 -> 16049866
2023-08-31 19:18:51 0 [Note] WSREP: Service thread queue flushed.
2023-08-31 19:18:51 0 [Note] WSREP: Lowest cert index boundary for CC from ist: 16049855
2023-08-31 19:18:51 0 [Note] WSREP: Min available from gcache for CC from ist: 16049855
2023-08-31 19:18:51 0 [Note] WSREP: Receiving IST...100.0% (12/12 events) complete.
2023-08-31 19:18:51 6 [Note] WSREP: ================================================
View:
id: c9a3c950-7d2f-11ed-9e25-5fc70c4f0d05:16049866
status: primary
protocol_version: 4
capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
final: no
own_index: 0
members(3):
0: 213ecf6f-3f1b-11ee-a3aa-1b9e106b662b, blp21152017
1: 902a3da4-3f15-11ee-af60-b706fe188821, blp21152018
2: a564b541-3eca-11ee-b726-a2aa4392ef31, blp21152016
=================================================
2023-08-31 19:18:51 6 [Note] WSREP: Server status change initialized -> joined
2023-08-31 19:18:51 6 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2023-08-31 19:18:51 6 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2023-08-31 19:18:51 2 [Note] WSREP: Draining apply monitors after IST up to 16049866
2023-08-31 19:18:51 2 [Note] WSREP: IST received: c9a3c950-7d2f-11ed-9e25-5fc70c4f0d05:16049866
2023-08-31 19:18:51 2 [Note] WSREP: Lowest cert index boundary for CC from sst: 16049855
2023-08-31 19:18:51 2 [Note] WSREP: Min available from gcache for CC from sst: 16049855
2023-08-31 19:18:51 0 [Note] WSREP: 0.0 (blp21152017): State transfer from 2.0 (blp21152016) complete.
2023-08-31 19:18:51 0 [Note] WSREP: Shifting JOINER -> JOINED (TO: 16049867)
2023-08-31 19:18:51 0 [Note] WSREP: Processing event queue:... -nan% (0/0 events) complete.
2023-08-31 19:18:51 0 [Note] WSREP: Member 0.0 (blp21152017) synced with group.
2023-08-31 19:18:51 0 [Note] WSREP: Processing event queue:...100.0% (1/1 events) complete.
2023-08-31 19:18:51 0 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 16049867)
2023-08-31 19:18:52 7 [Note] WSREP: Server blp21152017 synced with group
2023-08-31 19:18:52 7 [Note] WSREP: Server status change joined -> synced
2023-08-31 19:18:52 7 [Note] WSREP: Synchronized with group, ready for connections
2023-08-31 19:18:52 7 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
Attachments
Issue Links
- duplicates
-
MDEV-30217 Assertion `mode_ == m_local || transaction_.is_streaming()' failed in int wsrep::client_state::bf_abort(wsrep::seqno)
- Closed