-- Logs begin at Fri 2020-11-13 06:21:20 CET, end at Tue 2021-03-23 11:44:47 CET. -- Mar 23 09:02:09 srv3 mysqld[1522]: 2021-03-23 9:02:09 0 [Note] WSREP: (fa61149d-9ae2, 'ssl://0.0.0.0:4567') turning message relay requesting on, nonlive peers: ssl://10.0.1.3:4567 Mar 23 09:02:10 srv3 mysqld[1522]: 2021-03-23 9:02:10 0 [Note] WSREP: (fa61149d-9ae2, 'ssl://0.0.0.0:4567') reconnecting to 6552b06c-a6b1 (ssl://10.0.1.3:4567), attempt 0 Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 0 [Note] WSREP: evs::proto(fa61149d-9ae2, GATHER, view_id(REG,6552b06c-a6b1,158)) suspecting node: 6552b06c-a6b1 Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 0 [Note] WSREP: evs::proto(fa61149d-9ae2, GATHER, view_id(REG,6552b06c-a6b1,158)) suspected node without join message, declaring inactive Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 0 [Note] WSREP: declaring 7d447799-915b at ssl://10.0.1.2:4567 stable Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 0 [Note] WSREP: Node 7d447799-915b state prim Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 0 [Note] WSREP: view(view_id(PRIM,7d447799-915b,159) memb { Mar 23 09:02:14 srv3 mysqld[1522]: 7d447799-915b,0 Mar 23 09:02:14 srv3 mysqld[1522]: fa61149d-9ae2,0 Mar 23 09:02:14 srv3 mysqld[1522]: } joined { Mar 23 09:02:14 srv3 mysqld[1522]: } left { Mar 23 09:02:14 srv3 mysqld[1522]: } partitioned { Mar 23 09:02:14 srv3 mysqld[1522]: 6552b06c-a6b1,0 Mar 23 09:02:14 srv3 mysqld[1522]: }) Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 0 [Note] WSREP: save pc into disk Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 0 [Note] WSREP: forgetting 6552b06c-a6b1 (ssl://10.0.1.3:4567) Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 0 [Note] WSREP: (fa61149d-9ae2, 'ssl://0.0.0.0:4567') turning message relay requesting off Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 2 Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 0 [Warning] WSREP: Action message in non-primary configuration from member 0 Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 145f9ae8-8bae-11eb-acb7-3f1beaea1490 Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: 145f9ae8-8bae-11eb-acb7-3f1beaea1490 from 0 (srv1) Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: 145f9ae8-8bae-11eb-acb7-3f1beaea1490 from 1 (srv3) Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 0 [Note] WSREP: Quorum results: Mar 23 09:02:14 srv3 mysqld[1522]: version = 6, Mar 23 09:02:14 srv3 mysqld[1522]: component = PRIMARY, Mar 23 09:02:14 srv3 mysqld[1522]: conf_id = 3, Mar 23 09:02:14 srv3 mysqld[1522]: members = 2/2 (joined/total), Mar 23 09:02:14 srv3 mysqld[1522]: act_id = 1647665871, Mar 23 09:02:14 srv3 mysqld[1522]: last_appl. = 1647665736, Mar 23 09:02:14 srv3 mysqld[1522]: protocols = 2/10/4 (gcs/repl/appl), Mar 23 09:02:14 srv3 mysqld[1522]: vote policy= 0, Mar 23 09:02:14 srv3 mysqld[1522]: group UUID = bd401d28-f285-11ea-9c06-4e99d1c864f6 Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 0 [Note] WSREP: Flow-control interval: [23, 23] Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 25 [Note] WSREP: ####### processing CC 1647665872, local, ordered Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 25 [Note] WSREP: ####### My UUID: fa61149d-87ed-11eb-9ae2-be4d05c83bf4 Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 25 [Note] WSREP: Skipping cert index reset Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 25 [Note] WSREP: REPL Protocols: 10 (5) Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 25 [Note] WSREP: ####### Adjusting cert position: 1647665871 -> 1647665872 Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 0 [Note] WSREP: Service thread queue flushed. Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 25 [Note] WSREP: ================================================ Mar 23 09:02:14 srv3 mysqld[1522]: View: Mar 23 09:02:14 srv3 mysqld[1522]: id: bd401d28-f285-11ea-9c06-4e99d1c864f6:1647665872 Mar 23 09:02:14 srv3 mysqld[1522]: status: primary Mar 23 09:02:14 srv3 mysqld[1522]: protocol_version: 4 Mar 23 09:02:14 srv3 mysqld[1522]: capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO Mar 23 09:02:14 srv3 mysqld[1522]: final: no Mar 23 09:02:14 srv3 mysqld[1522]: own_index: 1 Mar 23 09:02:14 srv3 mysqld[1522]: members(2): Mar 23 09:02:14 srv3 mysqld[1522]: 0: 7d447799-8b22-11eb-915b-468cb230a09e, srv1 Mar 23 09:02:14 srv3 mysqld[1522]: 1: fa61149d-87ed-11eb-9ae2-be4d05c83bf4, srv3 Mar 23 09:02:14 srv3 mysqld[1522]: ================================================= Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 25 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 25 [Note] WSREP: Lowest cert index boundary for CC from group: 1647665737 Mar 23 09:02:14 srv3 mysqld[1522]: 2021-03-23 9:02:14 25 [Note] WSREP: Min available from gcache for CC from group: 1644973750 Mar 23 09:02:17 srv3 mysqld[1522]: 2021-03-23 9:02:17 0 [Note] WSREP: cleaning up 6552b06c-a6b1 (ssl://10.0.1.3:4567) Mar 23 09:02:33 srv3 mysqld[1522]: 2021-03-23 9:02:33 0 [Note] WSREP: (fa61149d-9ae2, 'ssl://0.0.0.0:4567') connection established to 6552b06c-a6b2 ssl://10.0.1.3:4567 Mar 23 09:02:33 srv3 mysqld[1522]: 2021-03-23 9:02:33 0 [Note] WSREP: (fa61149d-9ae2, 'ssl://0.0.0.0:4567') turning message relay requesting on, nonlive peers: Mar 23 09:02:33 srv3 mysqld[1522]: 2021-03-23 9:02:33 0 [Note] WSREP: declaring 6552b06c-a6b2 at ssl://10.0.1.3:4567 stable Mar 23 09:02:33 srv3 mysqld[1522]: 2021-03-23 9:02:33 0 [Note] WSREP: declaring 7d447799-915b at ssl://10.0.1.2:4567 stable Mar 23 09:02:33 srv3 mysqld[1522]: 2021-03-23 9:02:33 0 [Note] WSREP: Node 7d447799-915b state prim Mar 23 09:02:33 srv3 mysqld[1522]: 2021-03-23 9:02:33 0 [Note] WSREP: view(view_id(PRIM,6552b06c-a6b2,160) memb { Mar 23 09:02:33 srv3 mysqld[1522]: 6552b06c-a6b2,0 Mar 23 09:02:33 srv3 mysqld[1522]: 7d447799-915b,0 Mar 23 09:02:33 srv3 mysqld[1522]: fa61149d-9ae2,0 Mar 23 09:02:33 srv3 mysqld[1522]: } joined { Mar 23 09:02:33 srv3 mysqld[1522]: } left { Mar 23 09:02:33 srv3 mysqld[1522]: } partitioned { Mar 23 09:02:33 srv3 mysqld[1522]: }) Mar 23 09:02:33 srv3 mysqld[1522]: 2021-03-23 9:02:33 0 [Note] WSREP: save pc into disk Mar 23 09:02:33 srv3 mysqld[1522]: 2021-03-23 9:02:33 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 2, memb_num = 3 Mar 23 09:02:33 srv3 mysqld[1522]: 2021-03-23 9:02:33 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. Mar 23 09:02:34 srv3 mysqld[1522]: 2021-03-23 9:02:34 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 20039b70-8bae-11eb-936b-5abdaa1c6f49 Mar 23 09:02:34 srv3 mysqld[1522]: 2021-03-23 9:02:34 0 [Note] WSREP: STATE EXCHANGE: got state msg: 20039b70-8bae-11eb-936b-5abdaa1c6f49 from 0 (srv2) Mar 23 09:02:34 srv3 mysqld[1522]: 2021-03-23 9:02:34 0 [Note] WSREP: STATE EXCHANGE: got state msg: 20039b70-8bae-11eb-936b-5abdaa1c6f49 from 1 (srv1) Mar 23 09:02:34 srv3 mysqld[1522]: 2021-03-23 9:02:34 0 [Note] WSREP: STATE EXCHANGE: got state msg: 20039b70-8bae-11eb-936b-5abdaa1c6f49 from 2 (srv3) Mar 23 09:02:34 srv3 mysqld[1522]: 2021-03-23 9:02:34 0 [Note] WSREP: Quorum results: Mar 23 09:02:34 srv3 mysqld[1522]: version = 6, Mar 23 09:02:34 srv3 mysqld[1522]: component = PRIMARY, Mar 23 09:02:34 srv3 mysqld[1522]: conf_id = 4, Mar 23 09:02:34 srv3 mysqld[1522]: members = 2/3 (joined/total), Mar 23 09:02:34 srv3 mysqld[1522]: act_id = 1647668374, Mar 23 09:02:34 srv3 mysqld[1522]: last_appl. = 1647665736, Mar 23 09:02:34 srv3 mysqld[1522]: protocols = 2/10/4 (gcs/repl/appl), Mar 23 09:02:34 srv3 mysqld[1522]: vote policy= 0, Mar 23 09:02:34 srv3 mysqld[1522]: group UUID = bd401d28-f285-11ea-9c06-4e99d1c864f6 Mar 23 09:02:34 srv3 mysqld[1522]: 2021-03-23 9:02:34 0 [Note] WSREP: Flow-control interval: [28, 28] Mar 23 09:02:34 srv3 mysqld[1522]: 2021-03-23 9:02:34 13 [Note] WSREP: ####### processing CC 1647668375, local, ordered Mar 23 09:02:34 srv3 mysqld[1522]: 2021-03-23 9:02:34 13 [Note] WSREP: ####### My UUID: fa61149d-87ed-11eb-9ae2-be4d05c83bf4 Mar 23 09:02:34 srv3 mysqld[1522]: 2021-03-23 9:02:34 13 [Note] WSREP: Skipping cert index reset Mar 23 09:02:34 srv3 mysqld[1522]: 2021-03-23 9:02:34 13 [Note] WSREP: REPL Protocols: 10 (5) Mar 23 09:02:34 srv3 mysqld[1522]: 2021-03-23 9:02:34 13 [Note] WSREP: ####### Adjusting cert position: 1647668374 -> 1647668375 Mar 23 09:02:34 srv3 mysqld[1522]: 2021-03-23 9:02:34 0 [Note] WSREP: Service thread queue flushed. Mar 23 09:02:34 srv3 mysqld[1522]: 2021-03-23 9:02:34 13 [Note] WSREP: ================================================ Mar 23 09:02:34 srv3 mysqld[1522]: View: Mar 23 09:02:34 srv3 mysqld[1522]: id: bd401d28-f285-11ea-9c06-4e99d1c864f6:1647668375 Mar 23 09:02:34 srv3 mysqld[1522]: status: primary Mar 23 09:02:34 srv3 mysqld[1522]: protocol_version: 4 Mar 23 09:02:34 srv3 mysqld[1522]: capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO Mar 23 09:02:34 srv3 mysqld[1522]: final: no Mar 23 09:02:34 srv3 mysqld[1522]: own_index: 2 Mar 23 09:02:34 srv3 mysqld[1522]: members(3): Mar 23 09:02:34 srv3 mysqld[1522]: 0: 6552b06c-87fd-11eb-a6b2-670c0c1acbe7, srv2 Mar 23 09:02:34 srv3 mysqld[1522]: 1: 7d447799-8b22-11eb-915b-468cb230a09e, srv1 Mar 23 09:02:34 srv3 mysqld[1522]: 2: fa61149d-87ed-11eb-9ae2-be4d05c83bf4, srv3 Mar 23 09:02:34 srv3 mysqld[1522]: ================================================= Mar 23 09:02:34 srv3 mysqld[1522]: 2021-03-23 9:02:34 13 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. Mar 23 09:02:34 srv3 mysqld[1522]: 210323 9:02:34 [ERROR] mysqld got signal 11 ; Mar 23 09:02:34 srv3 mysqld[1522]: This could be because you hit a bug. It is also possible that this binary Mar 23 09:02:34 srv3 mysqld[1522]: or one of the libraries it was linked against is corrupt, improperly built, Mar 23 09:02:34 srv3 mysqld[1522]: or misconfigured. This error can also be caused by malfunctioning hardware. Mar 23 09:02:34 srv3 mysqld[1522]: To report this bug, see https://mariadb.com/kb/en/reporting-bugs Mar 23 09:02:34 srv3 mysqld[1522]: We will try our best to scrape up some info that will hopefully help Mar 23 09:02:34 srv3 mysqld[1522]: diagnose the problem, but since we have already crashed, Mar 23 09:02:34 srv3 mysqld[1522]: something is definitely wrong and this may fail. Mar 23 09:02:34 srv3 mysqld[1522]: Server version: 10.4.18-MariaDB-1:10.4.18+maria~buster-log Mar 23 09:02:34 srv3 mysqld[1522]: key_buffer_size=134217728 Mar 23 09:02:34 srv3 mysqld[1522]: read_buffer_size=2097152 Mar 23 09:02:34 srv3 mysqld[1522]: max_used_connections=82 Mar 23 09:02:34 srv3 mysqld[1522]: max_threads=302 Mar 23 09:02:34 srv3 mysqld[1522]: thread_count=105 Mar 23 09:02:34 srv3 mysqld[1522]: It is possible that mysqld could use up to Mar 23 09:02:34 srv3 mysqld[1522]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1994007 K bytes of memory Mar 23 09:02:34 srv3 mysqld[1522]: Hope that's ok; if not, decrease some variables in the equation. Mar 23 09:02:34 srv3 mysqld[1522]: Thread pointer: 0x7f4088000c08 Mar 23 09:02:34 srv3 mysqld[1522]: Attempting backtrace. You can use the following information to find out Mar 23 09:02:34 srv3 mysqld[1522]: where mysqld died. If you see no messages after this, something went Mar 23 09:02:34 srv3 mysqld[1522]: terribly wrong... Mar 23 09:02:34 srv3 mysqld[1522]: stack_bottom = 0x7f41087f7e08 thread_stack 0x100000 Mar 23 09:02:34 srv3 mysqld[1522]: /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x55b02736b7fe] Mar 23 09:02:34 srv3 mysqld[1522]: /usr/sbin/mysqld(handle_fatal_signal+0x54d)[0x55b026e5eb0d] Mar 23 09:02:34 srv3 mysqld[1522]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x12730)[0x7f4c519ea730] Mar 23 09:02:34 srv3 mysqld[1522]: /usr/sbin/mysqld(+0x64819d)[0x55b026bee19d] Mar 23 09:02:34 srv3 mysqld[1522]: /usr/sbin/mysqld(_Z11lock_tablesP3THDP10TABLE_LISTjj+0x4a8)[0x55b026bf3f78] Mar 23 09:02:34 srv3 mysqld[1522]: /usr/sbin/mysqld(_Z20open_and_lock_tablesP3THDRK14DDL_options_stP10TABLE_LISTbjP19Prelocking_strategy+0x8a)[0x55b026bf56ba] Mar 23 09:02:34 srv3 mysqld[1522]: /usr/sbin/mysqld(_Z24open_n_lock_single_tableP3THDP10TABLE_LIST13thr_lock_typejP19Prelocking_strategy+0x41)[0x55b026bf5761] Mar 23 09:02:34 srv3 mysqld[1522]: /usr/sbin/mysqld(+0x8370c5)[0x55b026ddd0c5] Mar 23 09:02:34 srv3 mysqld[1522]: /usr/sbin/mysqld(_ZNK12Wsrep_schema12restore_viewEP3THDRKN5wsrep2idE+0xf8)[0x55b026ddfd88] Mar 23 09:02:34 srv3 mysqld[1522]: /usr/sbin/mysqld(_ZN20Wsrep_server_service8log_viewEPN5wsrep21high_priority_serviceERKNS0_4viewE+0x4d4)[0x55b026fdb0d4] Mar 23 09:02:34 srv3 mysqld[1522]: /usr/sbin/mysqld(_ZN5wsrep12server_state7on_viewERKNS_4viewEPNS_21high_priority_serviceE+0x39b)[0x55b0273e395b] Mar 23 09:02:34 srv3 mysqld[1522]: /usr/sbin/mysqld(+0xe4facc)[0x55b0273f5acc] Mar 23 09:02:34 srv3 mysqld[1522]: /usr/lib/libgalera_smm.so(+0x5ccd9)[0x7f4c50268cd9] Mar 23 09:02:34 srv3 mysqld[1522]: /usr/lib/libgalera_smm.so(+0x5cd18)[0x7f4c50268d18] Mar 23 09:02:34 srv3 mysqld[1522]: /usr/lib/libgalera_smm.so(+0x68398)[0x7f4c50274398] Mar 23 09:02:34 srv3 mysqld[1522]: /usr/lib/libgalera_smm.so(+0x685ae)[0x7f4c502745ae] Mar 23 09:02:34 srv3 mysqld[1522]: /usr/lib/libgalera_smm.so(+0x92dd8)[0x7f4c5029edd8] Mar 23 09:02:34 srv3 mysqld[1522]: /usr/lib/libgalera_smm.so(+0x93061)[0x7f4c5029f061] Mar 23 09:02:34 srv3 mysqld[1522]: /usr/lib/libgalera_smm.so(+0x66cb0)[0x7f4c50272cb0] Mar 23 09:02:34 srv3 mysqld[1522]: 2021-03-23 9:02:34 0 [Note] WSREP: Member 0.0 (srv2) requested state transfer from '*any*'. Selected 1.0 (srv1)(SYNCED) as donor. Mar 23 09:02:34 srv3 mysqld[1522]: /usr/lib/libgalera_smm.so(+0x4248b)[0x7f4c5024e48b] Mar 23 09:02:34 srv3 mysqld[1522]: /usr/sbin/mysqld(_ZN5wsrep18wsrep_provider_v2611run_applierEPNS_21high_priority_serviceE+0xe)[0x55b0273f34fe] Mar 23 09:02:35 srv3 mysqld[1522]: /usr/sbin/mysqld(+0x836dff)[0x55b026ddcdff] Mar 23 09:02:35 srv3 mysqld[1522]: /usr/sbin/mysqld(_Z15start_wsrep_THDPv+0x388)[0x55b026dceac8] Mar 23 09:02:35 srv3 mysqld[1522]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x7fa3)[0x7f4c519dffa3] Mar 23 09:02:35 srv3 mysqld[1522]: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7f4c50ef44cf] Mar 23 09:02:35 srv3 mysqld[1522]: Trying to get some variables. Mar 23 09:02:35 srv3 mysqld[1522]: Some pointers may be invalid and cause the dump to abort. Mar 23 09:02:35 srv3 mysqld[1522]: Query (0x0): (null) Mar 23 09:02:35 srv3 mysqld[1522]: Connection ID (thread ID): 13 Mar 23 09:02:35 srv3 mysqld[1522]: Status: NOT_KILLED Mar 23 09:02:35 srv3 mysqld[1522]: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on Mar 23 09:02:35 srv3 mysqld[1522]: The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains Mar 23 09:02:35 srv3 mysqld[1522]: information that should help you find out what is causing the crash. Mar 23 09:02:35 srv3 mysqld[1522]: We think the query pointer is invalid, but we will try to print it anyway. Mar 23 09:02:35 srv3 mysqld[1522]: Query: Mar 23 09:02:35 srv3 mysqld[1522]: Writing a core file... Mar 23 09:02:35 srv3 mysqld[1522]: Working directory at /var/lib/mysql Mar 23 09:02:35 srv3 mysqld[1522]: Resource Limits: Mar 23 09:02:35 srv3 mysqld[1522]: Limit Soft Limit Hard Limit Units Mar 23 09:02:35 srv3 mysqld[1522]: Max cpu time unlimited unlimited seconds Mar 23 09:02:35 srv3 mysqld[1522]: Max file size unlimited unlimited bytes Mar 23 09:02:35 srv3 mysqld[1522]: Max data size unlimited unlimited bytes Mar 23 09:02:35 srv3 mysqld[1522]: Max stack size 8388608 unlimited bytes Mar 23 09:02:35 srv3 mysqld[1522]: Max core file size 0 unlimited bytes Mar 23 09:02:35 srv3 mysqld[1522]: Max resident set unlimited unlimited bytes Mar 23 09:02:35 srv3 mysqld[1522]: Max processes 257361 257361 processes Mar 23 09:02:35 srv3 mysqld[1522]: Max open files 16384 16384 files Mar 23 09:02:35 srv3 mysqld[1522]: Max locked memory 65536 65536 bytes Mar 23 09:02:35 srv3 mysqld[1522]: Max address space unlimited unlimited bytes Mar 23 09:02:35 srv3 mysqld[1522]: Max file locks unlimited unlimited locks Mar 23 09:02:35 srv3 mysqld[1522]: Max pending signals 257361 257361 signals Mar 23 09:02:35 srv3 mysqld[1522]: Max msgqueue size 819200 819200 bytes Mar 23 09:02:35 srv3 mysqld[1522]: Max nice priority 0 0 Mar 23 09:02:35 srv3 mysqld[1522]: Max realtime priority 0 0 Mar 23 09:02:35 srv3 mysqld[1522]: Max realtime timeout unlimited unlimited us Mar 23 09:02:35 srv3 mysqld[1522]: Core pattern: core Mar 23 09:02:35 srv3 systemd[1]: mariadb.service: Main process exited, code=killed, status=11/SEGV Mar 23 09:02:35 srv3 systemd[1]: mariadb.service: Failed with result 'signal'. Mar 23 09:02:40 srv3 systemd[1]: mariadb.service: Service RestartSec=5s expired, scheduling restart. Mar 23 09:02:40 srv3 systemd[1]: mariadb.service: Scheduled restart job, restart counter is at 1. Mar 23 09:02:40 srv3 systemd[1]: Stopped MariaDB 10.4.18 database server. Mar 23 09:02:40 srv3 systemd[1]: Starting MariaDB 10.4.18 database server... Mar 23 09:02:58 srv3 mysqld[30290]: WSREP: Recovered position bd401d28-f285-11ea-9c06-4e99d1c864f6:1647668374 Mar 23 09:02:58 srv3 mysqld[30597]: 2021-03-23 9:02:58 0 [Note] /usr/sbin/mysqld (mysqld 10.4.18-MariaDB-1:10.4.18+maria~buster-log) starting as process 30597 ... Mar 23 09:02:58 srv3 mysqld[30597]: 2021-03-23 9:02:58 0 [Note] WSREP: Loading provider /usr/lib/libgalera_smm.so initial position: bd401d28-f285-11ea-9c06-4e99d1c864f6:1647668374 Mar 23 09:02:58 srv3 mysqld[30597]: 2021-03-23 9:02:58 0 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/libgalera_smm.so' Mar 23 09:02:58 srv3 mysqld[30597]: 2021-03-23 9:02:58 0 [Note] WSREP: wsrep_load(): Galera 4.7(ree4f10fc) by Codership Oy loaded successfully. Mar 23 09:02:58 srv3 mysqld[30597]: 2021-03-23 9:02:58 0 [Note] WSREP: CRC-32C: using 64-bit x86 acceleration. Mar 23 09:02:58 srv3 mysqld[30597]: 2021-03-23 9:02:58 0 [Note] WSREP: Found saved state: bd401d28-f285-11ea-9c06-4e99d1c864f6:-1, safe_to_bootstrap: 0 Mar 23 09:02:58 srv3 mysqld[30597]: 2021-03-23 9:02:58 0 [Note] WSREP: GCache DEBUG: opened preamble: Mar 23 09:02:58 srv3 mysqld[30597]: Version: 2 Mar 23 09:02:58 srv3 mysqld[30597]: UUID: bd401d28-f285-11ea-9c06-4e99d1c864f6 Mar 23 09:02:58 srv3 mysqld[30597]: Seqno: -1 - -1 Mar 23 09:02:58 srv3 mysqld[30597]: Offset: -1 Mar 23 09:02:58 srv3 mysqld[30597]: Synced: 0 Mar 23 09:02:58 srv3 mysqld[30597]: 2021-03-23 9:02:58 0 [Note] WSREP: Recovering GCache ring buffer: version: 2, UUID: bd401d28-f285-11ea-9c06-4e99d1c864f6, offset: -1 Mar 23 09:02:58 srv3 mysqld[30597]: 2021-03-23 9:02:58 0 [Note] WSREP: GCache::RingBuffer initial scan... 0.0% ( 0/4294967320 bytes) complete. Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: GCache::RingBuffer initial scan...100.0% (4294967320/4294967320 bytes) complete. Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: Recovering GCache ring buffer: found gapless sequence 1644976253-1647668375 Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: GCache::RingBuffer unused buffers scan... 0.0% ( 0/4294961016 bytes) complete. Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: GCache::RingBuffer unused buffers scan...100.0% (4294961016/4294961016 bytes) complete. Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: GCache DEBUG: RingBuffer::recover(): found 0/2692123 locked buffers Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: GCache DEBUG: RingBuffer::recover(): free space: 6280/4294967296 Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: Passing config to GCS: base_dir = /var/lib/mysql/; base_host = 10.0.1.4; base_port = 4567; cert.log_conflicts = no; cert.optimistic_pa = yes; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = galera.cache; gcache.page_size = 128M; gcache.recover = yes; gcache.size = 4G; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.checksum = Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: Service thread queue flushed. Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: ####### Assign initial position for certification: bd401d28-f285-11ea-9c06-4e99d1c864f6:1647668374, protocol version: -1 Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: Start replication Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: Connecting with bootstrap option: 0 Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: Setting GCS initial position to bd401d28-f285-11ea-9c06-4e99d1c864f6:1647668374 Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: protonet asio version 0 Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: Using CRC-32C for message checksums. Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: backend: asio Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: gcomm thread scheduling priority set to other:0 Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: restore pc from disk successfully Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: GMCast version 0 Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: (fa61149d-9ae3, 'ssl://0.0.0.0:4567') listening at ssl://0.0.0.0:4567 Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: (fa61149d-9ae3, 'ssl://0.0.0.0:4567') multicast: , ttl: 1 Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: EVS version 1 Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: gcomm: connecting to group 'DS1', peer '10.0.1.2:,10.0.1.3:,10.0.1.4:' Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: (fa61149d-9ae3, 'ssl://0.0.0.0:4567') Found matching local endpoint for a connection, blacklisting address ssl://10.0.1.4:4567 Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: (fa61149d-9ae3, 'ssl://0.0.0.0:4567') connection established to 7d447799-915b ssl://10.0.1.2:4567 Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: (fa61149d-9ae3, 'ssl://0.0.0.0:4567') turning message relay requesting on, nonlive peers: Mar 23 09:03:01 srv3 mysqld[30597]: 2021-03-23 9:03:01 0 [Note] WSREP: (fa61149d-9ae3, 'ssl://0.0.0.0:4567') connection established to 6552b06c-a6b3 ssl://10.0.1.3:4567 Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: EVS version upgrade 0 -> 1 Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: declaring 6552b06c-a6b3 at ssl://10.0.1.3:4567 stable Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: declaring 7d447799-915b at ssl://10.0.1.2:4567 stable Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: PC protocol upgrade 0 -> 1 Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: re-bootstrapping prim from partitioned components Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: view(view_id(PRIM,6552b06c-a6b3,163) memb { Mar 23 09:03:02 srv3 mysqld[30597]: 6552b06c-a6b3,0 Mar 23 09:03:02 srv3 mysqld[30597]: 7d447799-915b,0 Mar 23 09:03:02 srv3 mysqld[30597]: fa61149d-9ae3,0 Mar 23 09:03:02 srv3 mysqld[30597]: } joined { Mar 23 09:03:02 srv3 mysqld[30597]: } left { Mar 23 09:03:02 srv3 mysqld[30597]: } partitioned { Mar 23 09:03:02 srv3 mysqld[30597]: 6552b06c-a6b2,0 Mar 23 09:03:02 srv3 mysqld[30597]: fa61149d-9ae2,0 Mar 23 09:03:02 srv3 mysqld[30597]: }) Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: save pc into disk Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: clear restored view Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: gcomm: connected Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636 Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0) Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: Opened channel 'DS1' Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 2, memb_num = 3 Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 30a0e386-8bae-11eb-96b2-5a30bedc374f Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 1 [Note] WSREP: Starting rollbacker thread 1 Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: STATE EXCHANGE: got state msg: 30a0e386-8bae-11eb-96b2-5a30bedc374f from 0 (srv2) Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: STATE EXCHANGE: got state msg: 30a0e386-8bae-11eb-96b2-5a30bedc374f from 1 (srv1) Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 2 [Note] WSREP: Starting applier thread 2 Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: STATE EXCHANGE: got state msg: 30a0e386-8bae-11eb-96b2-5a30bedc374f from 2 (srv3) Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Warning] WSREP: Quorum: No node with complete state: Mar 23 09:03:02 srv3 mysqld[30597]: Version : 6 Mar 23 09:03:02 srv3 mysqld[30597]: Flags : 0x1 Mar 23 09:03:02 srv3 mysqld[30597]: Protocols : 2 / 10 / 4 Mar 23 09:03:02 srv3 mysqld[30597]: State : NON-PRIMARY Mar 23 09:03:02 srv3 mysqld[30597]: Desync count : 0 Mar 23 09:03:02 srv3 mysqld[30597]: Prim state : NON-PRIMARY Mar 23 09:03:02 srv3 mysqld[30597]: Prim UUID : 00000000-0000-0000-0000-000000000000 Mar 23 09:03:02 srv3 mysqld[30597]: Prim seqno : -1 Mar 23 09:03:02 srv3 mysqld[30597]: First seqno : 1644976252 Mar 23 09:03:02 srv3 mysqld[30597]: Last seqno : 1647668374 Mar 23 09:03:02 srv3 mysqld[30597]: Commit cut : 1647668374 Mar 23 09:03:02 srv3 mysqld[30597]: Last vote : -1.0 Mar 23 09:03:02 srv3 mysqld[30597]: Vote policy : 0 Mar 23 09:03:02 srv3 mysqld[30597]: Prim JOINED : 0 Mar 23 09:03:02 srv3 mysqld[30597]: State UUID : 30a0e386-8bae-11eb-96b2-5a30bedc374f Mar 23 09:03:02 srv3 mysqld[30597]: Group UUID : bd401d28-f285-11ea-9c06-4e99d1c864f6 Mar 23 09:03:02 srv3 mysqld[30597]: Name : 'srv2' Mar 23 09:03:02 srv3 mysqld[30597]: Incoming addr: 'AUTO' Mar 23 09:03:02 srv3 mysqld[30597]: Version : 6 Mar 23 09:03:02 srv3 mysqld[30597]: Flags : 0x2 Mar 23 09:03:02 srv3 mysqld[30597]: Protocols : 2 / 10 / 4 Mar 23 09:03:02 srv3 mysqld[30597]: State : NON-PRIMARY Mar 23 09:03:02 srv3 mysqld[30597]: Desync count : 0 Mar 23 09:03:02 srv3 mysqld[30597]: Prim state : JOINED Mar 23 09:03:02 srv3 mysqld[30597]: Prim UUID : 20039b70-8bae-11eb-936b-5abdaa1c6f49 Mar 23 09:03:02 srv3 mysqld[30597]: Prim seqno : 5 Mar 23 09:03:02 srv3 mysqld[30597]: First seqno : 1644976267 Mar 23 09:03:02 srv3 mysqld[30597]: Last seqno : 1647668380 Mar 23 09:03:02 srv3 mysqld[30597]: Commit cut : 1647665736 Mar 23 09:03:02 srv3 mysqld[30597]: Last vote : -1.0 Mar 23 09:03:02 srv3 mysqld[30597]: Vote policy : 0 Mar 23 09:03:02 srv3 mysqld[30597]: Prim JOINED : 2 Mar 23 09:03:02 srv3 mysqld[30597]: State UUID : 30a0e386-8bae-11eb-96b2-5a30bedc374f Mar 23 09:03:02 srv3 mysqld[30597]: Group UUID : bd401d28-f285-11ea-9c06-4e99d1c864f6 Mar 23 09:03:02 srv3 mysqld[30597]: Name : 'srv1' Mar 23 09:03:02 srv3 mysqld[30597]: Incoming addr: 'AUTO' Mar 23 09:03:02 srv3 mysqld[30597]: Version : 6 Mar 23 09:03:02 srv3 mysqld[30597]: Flags : 00 Mar 23 09:03:02 srv3 mysqld[30597]: Protocols : 2 / 10 / 4 Mar 23 09:03:02 srv3 mysqld[30597]: State : NON-PRIMARY Mar 23 09:03:02 srv3 mysqld[30597]: Desync count : 0 Mar 23 09:03:02 srv3 mysqld[30597]: Prim state : NON-PRIMARY Mar 23 09:03:02 srv3 mysqld[30597]: Prim UUID : 00000000-0000-0000-0000-000000000000 Mar 23 09:03:02 srv3 mysqld[30597]: Prim seqno : -1 Mar 23 09:03:02 srv3 mysqld[30597]: First seqno : 1644976253 Mar 23 09:03:02 srv3 mysqld[30597]: Last seqno : 1647668374 Mar 23 09:03:02 srv3 mysqld[30597]: Commit cut : 1647668374 Mar 23 09:03:02 srv3 mysqld[30597]: Last vote : -1.0 Mar 23 09:03:02 srv3 mysqld[30597]: Vote policy : 0 Mar 23 09:03:02 srv3 mysqld[30597]: Prim JOINED : 0 Mar 23 09:03:02 srv3 mysqld[30597]: State UUID : 30a0e386-8bae-11eb-96b2-5a30bedc374f Mar 23 09:03:02 srv3 mysqld[30597]: Group UUID : bd401d28-f285-11ea-9c06-4e99d1c864f6 Mar 23 09:03:02 srv3 mysqld[30597]: Name : 'srv3' Mar 23 09:03:02 srv3 mysqld[30597]: Incoming addr: 'AUTO' Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: Partial re-merge of primary 20039b70-8bae-11eb-936b-5abdaa1c6f49 found: 1 of 2. Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: Quorum results: Mar 23 09:03:02 srv3 mysqld[30597]: version = 6, Mar 23 09:03:02 srv3 mysqld[30597]: component = PRIMARY, Mar 23 09:03:02 srv3 mysqld[30597]: conf_id = 5, Mar 23 09:03:02 srv3 mysqld[30597]: members = 1/3 (joined/total), Mar 23 09:03:02 srv3 mysqld[30597]: act_id = 1647668380, Mar 23 09:03:02 srv3 mysqld[30597]: last_appl. = 1647665736, Mar 23 09:03:02 srv3 mysqld[30597]: protocols = 2/10/4 (gcs/repl/appl), Mar 23 09:03:02 srv3 mysqld[30597]: vote policy= 0, Mar 23 09:03:02 srv3 mysqld[30597]: group UUID = bd401d28-f285-11ea-9c06-4e99d1c864f6 Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: Flow-control interval: [28, 28] Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 1647668381) Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 2 [Note] WSREP: ####### processing CC 1647668381, local, ordered Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 2 [Note] WSREP: Process first view: bd401d28-f285-11ea-9c06-4e99d1c864f6 my uuid: fa61149d-87ed-11eb-9ae3-be4d05c83bf4 Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 2 [Note] WSREP: Server srv3 connected to cluster at position bd401d28-f285-11ea-9c06-4e99d1c864f6:1647668381 with ID fa61149d-87ed-11eb-9ae3-be4d05c83bf4 Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 2 [Note] WSREP: Server status change disconnected -> connected Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 2 [Note] WSREP: ####### My UUID: fa61149d-87ed-11eb-9ae3-be4d05c83bf4 Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 2 [Note] WSREP: Cert index reset to 00000000-0000-0000-0000-000000000000:-1 (proto: 10), state transfer needed: yes Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: Service thread queue flushed. Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 2 [Note] WSREP: ####### Assign initial position for certification: 00000000-0000-0000-0000-000000000000:-1, protocol version: -1 Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 2 [Note] WSREP: State transfer required: Mar 23 09:03:02 srv3 mysqld[30597]: Group state: bd401d28-f285-11ea-9c06-4e99d1c864f6:1647668381 Mar 23 09:03:02 srv3 mysqld[30597]: Local state: bd401d28-f285-11ea-9c06-4e99d1c864f6:1647668374 Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 2 [Note] WSREP: Server status change connected -> joiner Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: Joiner monitor thread started to monitor Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: Running: 'wsrep_sst_mariabackup --role 'joiner' --address '10.0.1.4' --datadir '/var/lib/mysql/' --parent '30597' --binlog '/var/log/mysql/mariadb-bin' --binlog-index '/var/log/mysql/mariadb-bin.index' --mysqld-args --wsrep_start_position=bd401d28-f285-11ea-9c06-4e99d1c864f6:1647668374' Mar 23 09:03:02 srv3 mysqld[30597]: 2021-03-23 9:03:02 0 [Note] WSREP: Member 1.0 (srv1) synced with group. Mar 23 09:03:02 srv3 mysqld[30597]: WSREP_SST: [INFO] Logging all stderr of SST/Innobackupex to syslog (20210323 09:03:02.793) Mar 23 09:03:03 srv3 mysqld[30597]: 2021-03-23 9:03:03 0 [Note] WSREP: Member 0.0 (srv2) requested state transfer from '*any*'. Selected 1.0 (srv1)(SYNCED) as donor. Mar 23 09:03:03 srv3 mysqld[30597]: 2021-03-23 9:03:03 2 [Note] WSREP: Prepared SST request: mariabackup|10.0.1.4:4444/xtrabackup_sst//1 Mar 23 09:03:03 srv3 mysqld[30597]: 2021-03-23 9:03:03 2 [Note] WSREP: ####### IST uuid:bd401d28-f285-11ea-9c06-4e99d1c864f6 f: 1647668375, l: 1647668381, STRv: 3 Mar 23 09:03:03 srv3 mysqld[30597]: 2021-03-23 9:03:03 2 [Note] WSREP: IST receiver addr using ssl://10.0.1.4:4568 Mar 23 09:03:03 srv3 mysqld[30597]: 2021-03-23 9:03:03 2 [Note] WSREP: IST receiver using ssl Mar 23 09:03:03 srv3 mysqld[30597]: 2021-03-23 9:03:03 2 [Note] WSREP: Prepared IST receiver for 1647668375-1647668381, listening at: ssl://10.0.1.4:4568 Mar 23 09:03:03 srv3 mysqld[30597]: 2021-03-23 9:03:03 0 [Warning] WSREP: Member 2.0 (srv3) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable Mar 23 09:03:03 srv3 mysqld[30597]: 2021-03-23 9:03:03 2 [Note] WSREP: Requesting state transfer failed: -11(Resource temporarily unavailable). Will keep retrying every 1 second(s) Mar 23 09:03:03 srv3 mysqld[30597]: 2021-03-23 9:03:03 0 [Note] WSREP: 1.0 (srv1): State transfer to 0.0 (srv2) complete. Mar 23 09:03:03 srv3 mysqld[30597]: 2021-03-23 9:03:03 0 [Note] WSREP: Member 1.0 (srv1) synced with group. Mar 23 09:03:04 srv3 mysqld[30597]: 2021-03-23 9:03:04 0 [Note] WSREP: Member 2.0 (srv3) requested state transfer from '*any*'. Selected 1.0 (srv1)(SYNCED) as donor. Mar 23 09:03:04 srv3 mysqld[30597]: 2021-03-23 9:03:04 0 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 1647668381) Mar 23 09:03:04 srv3 mysqld[30597]: 2021-03-23 9:03:04 2 [Note] WSREP: Requesting state transfer: success after 2 tries, donor: 1 Mar 23 09:03:04 srv3 mysqld[30597]: 2021-03-23 9:03:04 0 [Note] WSREP: 1.0 (srv1): State transfer to 2.0 (srv3) complete. Mar 23 09:03:04 srv3 mysqld[30597]: 2021-03-23 9:03:04 0 [Note] WSREP: Member 1.0 (srv1) synced with group. Mar 23 09:03:04 srv3 mysqld[30597]: 2021-03-23 9:03:04 3 [Note] WSREP: SST received Mar 23 09:03:04 srv3 mysqld[30597]: 2021-03-23 9:03:04 3 [Note] WSREP: Server status change joiner -> initializing Mar 23 09:03:04 srv3 mysqld[30597]: 2021-03-23 9:03:04 3 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. Mar 23 09:03:04 srv3 mysqld[30597]: 2021-03-23 9:03:04 0 [Note] InnoDB: Using Linux native AIO Mar 23 09:03:04 srv3 mysqld[30597]: 2021-03-23 9:03:04 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins Mar 23 09:03:04 srv3 mysqld[30597]: 2021-03-23 9:03:04 0 [Note] InnoDB: Uses event mutexes Mar 23 09:03:04 srv3 mysqld[30597]: 2021-03-23 9:03:04 0 [Note] InnoDB: Compressed tables use zlib 1.2.11 Mar 23 09:03:04 srv3 mysqld[30597]: 2021-03-23 9:03:04 0 [Note] InnoDB: Number of pools: 1 Mar 23 09:03:04 srv3 mysqld[30597]: 2021-03-23 9:03:04 0 [Note] InnoDB: Using SSE2 crc32 instructions Mar 23 09:03:04 srv3 mysqld[30597]: 2021-03-23 9:03:04 0 [Note] mysqld: O_TMPFILE is not supported on /tmp (disabling future attempts) Mar 23 09:03:04 srv3 mysqld[30597]: 2021-03-23 9:03:04 0 [Note] InnoDB: Initializing buffer pool, total size = 40G, instances = 40, chunk size = 128M Mar 23 09:03:04 srv3 mysqld[30597]: 2021-03-23 9:03:04 0 [Note] WSREP: 0.0 (srv2): State transfer from 1.0 (srv1) complete. Mar 23 09:03:04 srv3 mysqld[30597]: 2021-03-23 9:03:04 0 [Note] WSREP: Member 0.0 (srv2) synced with group. Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] InnoDB: Completed initialization of buffer pool Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] InnoDB: Setting log file ./ib_logfile101 size to 4294967296 bytes Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] InnoDB: Renaming log file ./ib_logfile101 to ./ib_logfile0 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] InnoDB: New log files created, LSN=19512069651915 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: (fa61149d-9ae3, 'ssl://0.0.0.0:4567') turning message relay requesting off Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] InnoDB: 128 out of 128 rollback segments are active. Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] InnoDB: Creating shared tablespace for temporary tables Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB. Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] InnoDB: 10.4.18 started; log sequence number 19512069651980; transaction id 26894856468 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] Plugin 'FEEDBACK' is disabled. Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [ERROR] mysqld: Plugin 'innodb' already installed Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Warning] Plugin 'FEDERATED' is of maturity level gamma while the server is stable Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] Server socket created on IP: '0.0.0.0'. Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: wsrep_init_schema_and_SR (nil) Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: Server initialized Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: Server status change initializing -> initialized Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 3 [Note] WSREP: Server status change initialized -> joined Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 3 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 3 [Note] WSREP: Recovered position from storage: bd401d28-f285-11ea-9c06-4e99d1c864f6:1647668374 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 12 [Note] WSREP: Starting applier thread 12 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 16 [Note] WSREP: Starting applier thread 16 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 15 [Note] WSREP: Starting applier thread 15 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 17 [Note] WSREP: Starting applier thread 17 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 25 [Note] WSREP: Starting applier thread 25 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 20 [Note] WSREP: Starting applier thread 20 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 22 [Note] WSREP: Starting applier thread 22 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 23 [Note] WSREP: Starting applier thread 23 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 14 [Note] WSREP: Starting applier thread 14 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] Reading of all Master_info entries succeeded Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 19 [Note] WSREP: Starting applier thread 19 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 21 [Note] WSREP: Starting applier thread 21 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 11 [Note] WSREP: Starting applier thread 11 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 24 [Note] WSREP: Starting applier thread 24 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 18 [Note] WSREP: Starting applier thread 18 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 13 [Note] WSREP: Starting applier thread 13 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] Added new Master_info '' to hash table Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] /usr/sbin/mysqld: ready for connections. Mar 23 09:03:05 srv3 mysqld[30597]: Version: '10.4.18-MariaDB-1:10.4.18+maria~buster-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 3 [Note] WSREP: Recovered view from SST: Mar 23 09:03:05 srv3 mysqld[30597]: id: bd401d28-f285-11ea-9c06-4e99d1c864f6:1647665872 Mar 23 09:03:05 srv3 mysqld[30597]: status: primary Mar 23 09:03:05 srv3 mysqld[30597]: protocol_version: 4 Mar 23 09:03:05 srv3 mysqld[30597]: capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO Mar 23 09:03:05 srv3 mysqld[30597]: final: no Mar 23 09:03:05 srv3 mysqld[30597]: own_index: -1 Mar 23 09:03:05 srv3 mysqld[30597]: members(2): Mar 23 09:03:05 srv3 mysqld[30597]: 0: 7d447799-8b22-11eb-915b-468cb230a09e, srv1 Mar 23 09:03:05 srv3 mysqld[30597]: 1: fa61149d-87ed-11eb-9ae2-be4d05c83bf4, srv3 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 3 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 28 [Note] WSREP: Recovered cluster id bd401d28-f285-11ea-9c06-4e99d1c864f6 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 3 [Note] WSREP: SST received: bd401d28-f285-11ea-9c06-4e99d1c864f6:1647668374 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 3 [Note] WSREP: SST succeeded for position bd401d28-f285-11ea-9c06-4e99d1c864f6:1647668374 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: Joiner monitor thread ended with total time 3 sec Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 2 [Note] WSREP: Installed new state from SST: bd401d28-f285-11ea-9c06-4e99d1c864f6:1647668374 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 2 [Note] WSREP: Receiving IST: 7 writesets, seqnos 1647668375-1647668381 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: ####### IST applying starts with 1647668375 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: ####### IST current seqno initialized to 1647665737 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: Receiving IST... 0.0% ( 0/2645 events) complete. Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: IST preload starting at 1647665737 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: Service thread queue flushed. Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: ####### Assign initial position for certification: 00000000-0000-0000-0000-000000000000:1647665736, protocol version: 5 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: REPL Protocols: 10 (5) Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: ####### Adjusting cert position: 1647665871 -> 1647665872 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: Service thread queue flushed. Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: Lowest cert index boundary for CC from preload: 1647665737 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: Min available from gcache for CC from preload: 1644976253 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: REPL Protocols: 10 (5) Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: ####### Adjusting cert position: 1647668374 -> 1647668375 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: Service thread queue flushed. Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: Lowest cert index boundary for CC from ist: 1647665737 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: Min available from gcache for CC from ist: 1644976253 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 12 [Note] WSREP: ================================================ Mar 23 09:03:05 srv3 mysqld[30597]: View: Mar 23 09:03:05 srv3 mysqld[30597]: id: bd401d28-f285-11ea-9c06-4e99d1c864f6:1647668375 Mar 23 09:03:05 srv3 mysqld[30597]: status: primary Mar 23 09:03:05 srv3 mysqld[30597]: protocol_version: 4 Mar 23 09:03:05 srv3 mysqld[30597]: capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO Mar 23 09:03:05 srv3 mysqld[30597]: final: no Mar 23 09:03:05 srv3 mysqld[30597]: own_index: -1 Mar 23 09:03:05 srv3 mysqld[30597]: members(3): Mar 23 09:03:05 srv3 mysqld[30597]: 0: 6552b06c-87fd-11eb-a6b2-670c0c1acbe7, srv2 Mar 23 09:03:05 srv3 mysqld[30597]: 1: 7d447799-8b22-11eb-915b-468cb230a09e, srv1 Mar 23 09:03:05 srv3 mysqld[30597]: 2: fa61149d-87ed-11eb-9ae2-be4d05c83bf4, srv3 Mar 23 09:03:05 srv3 mysqld[30597]: ================================================= Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 12 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. Mar 23 09:03:05 srv3 systemd[1]: Started MariaDB 10.4.18 database server. Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: REPL Protocols: 10 (5) Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: ####### Adjusting cert position: 1647668380 -> 1647668381 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: Service thread queue flushed. Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: Lowest cert index boundary for CC from ist: 1647665737 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: Min available from gcache for CC from ist: 1644976260 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 22 [Note] WSREP: ================================================ Mar 23 09:03:05 srv3 mysqld[30597]: View: Mar 23 09:03:05 srv3 mysqld[30597]: id: bd401d28-f285-11ea-9c06-4e99d1c864f6:1647668381 Mar 23 09:03:05 srv3 mysqld[30597]: status: primary Mar 23 09:03:05 srv3 mysqld[30597]: protocol_version: 4 Mar 23 09:03:05 srv3 mysqld[30597]: capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO Mar 23 09:03:05 srv3 mysqld[30597]: final: no Mar 23 09:03:05 srv3 mysqld[30597]: own_index: 2 Mar 23 09:03:05 srv3 mysqld[30597]: members(3): Mar 23 09:03:05 srv3 mysqld[30597]: 0: 6552b06c-87fd-11eb-a6b3-670c0c1acbe7, srv2 Mar 23 09:03:05 srv3 mysqld[30597]: 1: 7d447799-8b22-11eb-915b-468cb230a09e, srv1 Mar 23 09:03:05 srv3 mysqld[30597]: 2: fa61149d-87ed-11eb-9ae3-be4d05c83bf4, srv3 Mar 23 09:03:05 srv3 mysqld[30597]: ================================================= Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: Receiving IST...100.0% (2645/2645 events) complete. Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 22 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 2 [Note] WSREP: Draining apply monitors after IST up to 1647668381 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 2 [Note] WSREP: IST received: bd401d28-f285-11ea-9c06-4e99d1c864f6:1647668381 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 2 [Note] WSREP: Lowest cert index boundary for CC from sst: 1647665737 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 2 [Note] WSREP: Min available from gcache for CC from sst: 1644976260 Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: 2.0 (srv3): State transfer from 1.0 (srv1) complete. Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: Shifting JOINER -> JOINED (TO: 1647668384) Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: Member 2.0 (srv3) synced with group. Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 0 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 1647668384) Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 2 [Note] WSREP: Server srv3 synced with group Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 2 [Note] WSREP: Server status change joined -> synced Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 2 [Note] WSREP: Synchronized with group, ready for connections Mar 23 09:03:05 srv3 mysqld[30597]: 2021-03-23 9:03:05 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.