[MDEV-33908] WSREP: MDL BF-BF conflict - Jira

Daniel Czadek created issue - 2024-04-15 06:00

Daniel Czadek made changes - 2024-04-17 09:46

Field	Original Value	New Value
Summary	MDL BF-BF conflict	WSREP: MDL BF-BF conflict

Daniel Czadek made changes - 2024-04-17 09:47

Description

Sometimes we got an error on the second MariaDB-Galera-Node which get dropped then from the cluster with an MDL BF-BF conflict.
What could that be?

Here is our central config for all mariadb servers:

{code:shell}
# MANAGED BY ANSIBLE, ALL CHANGES WILL BE OVERWRITTEN!
#
# These groups are read by MariaDB server.
# Use it for options that only the server (but not clients) should see
#
# See the examples of server my.cnf files in /usr/share/mysql

# this is read by the standalone daemon and embedded servers
[server]

# this is only for the mysqld standalone daemon
[mysqld]

#
# * Basic Settings
#
user = mysql
pid-file = /run/mysqld/mysqld.pid
socket = /run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /data/pr-be-mdb-galera02-02
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
#skip-external-locking
skip-name-resolve

# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address = 0.0.0.0

#
# * Fine Tuning
#
#key_buffer_size = 16M
max_allowed_packet = 128M
#thread_stack = 192K
#thread_cache_size = 8
# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched
#myisam_recover_options = BACKUP
max_connections = 1250
#table_cache = 64
#thread_concurrency = 10

sql_mode = ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
key_buffer_size = 256M
table_open_cache = 8192
table_definition_cache = 8192
thread_cache_size = 32
max_user_connections = 0
tmp_table_size = 32M
max_heap_table_size = 32M
join_buffer_size = 2M
sort_buffer_size = 2M
read_rnd_buffer_size = 1M

#
# * Performance schema for better diagnostics
#
performance_schema = ON

#
# * Query Cache Configuration
#
query_cache_limit = 1M
query_cache_size = 0
query_cache_type = 0

#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
# As of 5.1 you can enable the log at runtime!
#general_log_file = /var/log/mysql/mysql.log
#general_log = 1
#
# Error log - should be very few entries.
#
log_error = /var/log/mysql/error.log
#
# Enable the slow query log to see queries with especially long duration
#slow_query_log_file = /var/log/mysql/mariadb-slow.log
#long_query_time = 10
#log_slow_rate_limit = 1000
#log_slow_verbosity = query_plan
#log-queries-not-using-indexes
#
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
# other settings you may need to change.
#server-id = 1

expire_logs_days = 10
#max_binlog_size = 100M
#binlog_do_db = include_database_name
#binlog_ignore_db = exclude_database_name

#
# * Security Features
#
# Read the manual, too, if you want chroot!
#chroot = /var/lib/mysql/
#
# For generating SSL certificates you can use for example the GUI tool "tinyca".
#
#ssl-ca = /etc/mysql/cacert.pem
#ssl-cert = /etc/mysql/server-cert.pem
#ssl-key = /etc/mysql/server-key.pem
#
# Accept only connections using the latest and most secure TLS protocol version.
# ..when MariaDB is compiled with OpenSSL:
#ssl-cipher = TLSv1.2
# ..when MariaDB is compiled with YaSSL (default in Debian):
#ssl = on

#
# * Character sets
#
# MySQL/MariaDB default is Latin1, but in Debian we rather default to the full
# utf8 4-byte character set. See also client.cnf
#
character-set-server = utf8mb4
collation-server = utf8mb4_general_ci

#
# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
innodb_buffer_pool_size = 16G
innodb_buffer_pool_instances = 16
innodb_thread_concurrency = 8
innodb_log_file_size = 6G
innodb_log_buffer_size = 16M
innodb_flush_log_at_trx_commit = 0
innodb_lock_wait_timeout = 300
innodb_io_capacity = 4000
innodb_io_capacity_max = 8000
innodb_read_io_threads = 8
innodb_write_io_threads = 8

#
# * Unix socket authentication plugin is built-in since 10.0.22-6
#
# Needed so the root database user can authenticate without a password but
# only when running as the unix root user.
#
# Also available for other users if required.
# See https://mariadb.com/kb/en/unix_socket-authentication-plugin/

# this is only for embedded server
[embedded]

# This group is only read by MariaDB servers, not by MySQL.
# If you use the same .cnf file for MySQL and MariaDB,
# you can put MariaDB-only options here
[mariadb]

# This group is only read by MariaDB-10.3 servers.
# If you use the same .cnf file for MariaDB of different versions,
# use this group for options that older servers don't understand
[mariadb-10.3]
{code}

{code:shell}
[mysqld]
####################################
# mysql/mariadb settings

binlog_format = ROW
default-storage-engine = innodb
innodb_autoinc_lock_mode = 2
innodb_file_per_table = on
binlog-row-image = minimal

innodb_doublewrite = 1

####################################
# galera settings

wsrep_on = ON
wsrep_provider = /usr/lib/galera/libgalera_smm.so
wsrep_cluster_name = "sql-gc02"
wsrep_cluster_address = "gcomm://10.49.65.120,10.49.65.121"
wsrep_node_address = "10.49.65.121"
wsrep_node_name = "pr-be-mdb-galera02-02"
wsrep_sst_method = "mariabackup"
wsrep_sst_auth = xxxxxxxxxxxx:xxxxxxxxxxxxxxxxxxxxxxx
ignore_db_dirs = .snapshot
wsrep_slave_threads = 8
{code}

Sometimes we got an error on the second MariaDB-Galera-Node which get dropped then from the cluster with an MDL BF-BF conflict.
What could that be?

{code:shell}
2024-04-10 8:02:54 10 [Note] WSREP: Server pr-be-mdb-galera02-02 synced with group
2024-04-12 10:14:11 860965 [Note] WSREP: Desyncing and pausing the provider
2024-04-12 10:14:11 0 [Note] WSREP: Member 1.0 (pr-be-mdb-galera02-02) desyncs itself from group
2024-04-12 10:14:11 0 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 449986329)
2024-04-12 10:14:11 860965 [Note] WSREP: pause
2024-04-12 10:14:11 860965 [Note] WSREP: Provider paused at 2b276be9-b04a-11ee-9634-0ffee8933331:449986329 (205127771)
2024-04-12 10:14:11 860965 [Note] WSREP: Provider paused at: 449986329
2024-04-12 10:14:28 860965 [Note] WSREP: Resuming and resyncing the provider
2024-04-12 10:14:28 860965 [Note] WSREP: resume
2024-04-12 10:14:28 860965 [Note] WSREP: resuming provider at 205127771
2024-04-12 10:14:28 860965 [Note] WSREP: Provider resumed.
2024-04-12 10:14:28 0 [Note] WSREP: Member 1.0 (pr-be-mdb-galera02-02) resyncs itself to group.
2024-04-12 10:14:28 0 [Note] WSREP: Shifting DONOR/DESYNCED -> JOINED (TO: 449987886)
2024-04-12 10:14:28 0 [Note] WSREP: Processing event queue:... 0.0% ( 0/1549 events) complete.
2024-04-12 10:14:29 0 [Note] WSREP: Member 1.0 (pr-be-mdb-galera02-02) synced with group.
2024-04-12 10:14:29 0 [Note] WSREP: Processing event queue:...100.0% (1641/1641 events) complete.
2024-04-12 10:14:29 0 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 449987963)
2024-04-12 10:14:29 7 [Note] WSREP: Server pr-be-mdb-galera02-02 synced with group
2024-04-13 0:23:56 7 [Note] WSREP: MDL BF-BF conflict
schema: bs_actionpress
request: (7 seqno 455199246 wsrep (toi, exec, committed) cmd 0 45 OPTIMIZE TABLE exportdb_sqls)
granted: (2 seqno 455199247 wsrep (high priority, exec, committing) cmd 0 161 (null))
2024-04-13 0:23:56 7 [ERROR] Aborting
{code}

Here is our central config for all mariadb servers:

{code:shell}
# MANAGED BY ANSIBLE, ALL CHANGES WILL BE OVERWRITTEN!
#
# These groups are read by MariaDB server.
# Use it for options that only the server (but not clients) should see
#
# See the examples of server my.cnf files in /usr/share/mysql

# this is read by the standalone daemon and embedded servers
[server]

# this is only for the mysqld standalone daemon
[mysqld]

#
# * Basic Settings
#
user = mysql
pid-file = /run/mysqld/mysqld.pid
socket = /run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /data/pr-be-mdb-galera02-02
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
#skip-external-locking
skip-name-resolve

# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address = 0.0.0.0

#
# * Fine Tuning
#
#key_buffer_size = 16M
max_allowed_packet = 128M
#thread_stack = 192K
#thread_cache_size = 8
# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched
#myisam_recover_options = BACKUP
max_connections = 1250
#table_cache = 64
#thread_concurrency = 10

sql_mode = ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
key_buffer_size = 256M
table_open_cache = 8192
table_definition_cache = 8192
thread_cache_size = 32
max_user_connections = 0
tmp_table_size = 32M
max_heap_table_size = 32M
join_buffer_size = 2M
sort_buffer_size = 2M
read_rnd_buffer_size = 1M

#
# * Performance schema for better diagnostics
#
performance_schema = ON

#
# * Query Cache Configuration
#
query_cache_limit = 1M
query_cache_size = 0
query_cache_type = 0

#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
# As of 5.1 you can enable the log at runtime!
#general_log_file = /var/log/mysql/mysql.log
#general_log = 1
#
# Error log - should be very few entries.
#
log_error = /var/log/mysql/error.log
#
# Enable the slow query log to see queries with especially long duration
#slow_query_log_file = /var/log/mysql/mariadb-slow.log
#long_query_time = 10
#log_slow_rate_limit = 1000
#log_slow_verbosity = query_plan
#log-queries-not-using-indexes
#
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
# other settings you may need to change.
#server-id = 1

expire_logs_days = 10
#max_binlog_size = 100M
#binlog_do_db = include_database_name
#binlog_ignore_db = exclude_database_name

#
# * Security Features
#
# Read the manual, too, if you want chroot!
#chroot = /var/lib/mysql/
#
# For generating SSL certificates you can use for example the GUI tool "tinyca".
#
#ssl-ca = /etc/mysql/cacert.pem
#ssl-cert = /etc/mysql/server-cert.pem
#ssl-key = /etc/mysql/server-key.pem
#
# Accept only connections using the latest and most secure TLS protocol version.
# ..when MariaDB is compiled with OpenSSL:
#ssl-cipher = TLSv1.2
# ..when MariaDB is compiled with YaSSL (default in Debian):
#ssl = on

#
# * Character sets
#
# MySQL/MariaDB default is Latin1, but in Debian we rather default to the full
# utf8 4-byte character set. See also client.cnf
#
character-set-server = utf8mb4
collation-server = utf8mb4_general_ci

#
# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
innodb_buffer_pool_size = 16G
innodb_buffer_pool_instances = 16
innodb_thread_concurrency = 8
innodb_log_file_size = 6G
innodb_log_buffer_size = 16M
innodb_flush_log_at_trx_commit = 0
innodb_lock_wait_timeout = 300
innodb_io_capacity = 4000
innodb_io_capacity_max = 8000
innodb_read_io_threads = 8
innodb_write_io_threads = 8

#
# * Unix socket authentication plugin is built-in since 10.0.22-6
#
# Needed so the root database user can authenticate without a password but
# only when running as the unix root user.
#
# Also available for other users if required.
# See https://mariadb.com/kb/en/unix_socket-authentication-plugin/

# this is only for embedded server
[embedded]

# This group is only read by MariaDB servers, not by MySQL.
# If you use the same .cnf file for MySQL and MariaDB,
# you can put MariaDB-only options here
[mariadb]

# This group is only read by MariaDB-10.3 servers.
# If you use the same .cnf file for MariaDB of different versions,
# use this group for options that older servers don't understand
[mariadb-10.3]
{code}

{code:shell}
[mysqld]
####################################
# mysql/mariadb settings

binlog_format = ROW
default-storage-engine = innodb
innodb_autoinc_lock_mode = 2
innodb_file_per_table = on
binlog-row-image = minimal

innodb_doublewrite = 1

####################################
# galera settings

wsrep_on = ON
wsrep_provider = /usr/lib/galera/libgalera_smm.so
wsrep_cluster_name = "sql-gc02"
wsrep_cluster_address = "gcomm://10.49.65.120,10.49.65.121"
wsrep_node_address = "10.49.65.121"
wsrep_node_name = "pr-be-mdb-galera02-02"
wsrep_sst_method = "mariabackup"
wsrep_sst_auth = xxxxxxxxxxxx:xxxxxxxxxxxxxxxxxxxxxxx
ignore_db_dirs = .snapshot
wsrep_slave_threads = 8
{code}

Daniel Czadek made changes - 2024-04-17 09:51

Description

Sometimes we got an error on the second MariaDB-Galera-Node which get dropped then from the cluster with an MDL BF-BF conflict.
What could that be?

{code:shell}
2024-04-10 8:02:54 10 [Note] WSREP: Server pr-be-mdb-galera02-02 synced with group
2024-04-12 10:14:11 860965 [Note] WSREP: Desyncing and pausing the provider
2024-04-12 10:14:11 0 [Note] WSREP: Member 1.0 (pr-be-mdb-galera02-02) desyncs itself from group
2024-04-12 10:14:11 0 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 449986329)
2024-04-12 10:14:11 860965 [Note] WSREP: pause
2024-04-12 10:14:11 860965 [Note] WSREP: Provider paused at 2b276be9-b04a-11ee-9634-0ffee8933331:449986329 (205127771)
2024-04-12 10:14:11 860965 [Note] WSREP: Provider paused at: 449986329
2024-04-12 10:14:28 860965 [Note] WSREP: Resuming and resyncing the provider
2024-04-12 10:14:28 860965 [Note] WSREP: resume
2024-04-12 10:14:28 860965 [Note] WSREP: resuming provider at 205127771
2024-04-12 10:14:28 860965 [Note] WSREP: Provider resumed.
2024-04-12 10:14:28 0 [Note] WSREP: Member 1.0 (pr-be-mdb-galera02-02) resyncs itself to group.
2024-04-12 10:14:28 0 [Note] WSREP: Shifting DONOR/DESYNCED -> JOINED (TO: 449987886)
2024-04-12 10:14:28 0 [Note] WSREP: Processing event queue:... 0.0% ( 0/1549 events) complete.
2024-04-12 10:14:29 0 [Note] WSREP: Member 1.0 (pr-be-mdb-galera02-02) synced with group.
2024-04-12 10:14:29 0 [Note] WSREP: Processing event queue:...100.0% (1641/1641 events) complete.
2024-04-12 10:14:29 0 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 449987963)
2024-04-12 10:14:29 7 [Note] WSREP: Server pr-be-mdb-galera02-02 synced with group
2024-04-13 0:23:56 7 [Note] WSREP: MDL BF-BF conflict
schema: bs_actionpress
request: (7 seqno 455199246 wsrep (toi, exec, committed) cmd 0 45 OPTIMIZE TABLE exportdb_sqls)
granted: (2 seqno 455199247 wsrep (high priority, exec, committing) cmd 0 161 (null))
2024-04-13 0:23:56 7 [ERROR] Aborting
{code}

Here is our central config for all mariadb servers:

{code:shell}
# MANAGED BY ANSIBLE, ALL CHANGES WILL BE OVERWRITTEN!
#
# These groups are read by MariaDB server.
# Use it for options that only the server (but not clients) should see
#
# See the examples of server my.cnf files in /usr/share/mysql

# this is read by the standalone daemon and embedded servers
[server]

# this is only for the mysqld standalone daemon
[mysqld]

#
# * Basic Settings
#
user = mysql
pid-file = /run/mysqld/mysqld.pid
socket = /run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /data/pr-be-mdb-galera02-02
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
#skip-external-locking
skip-name-resolve

# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address = 0.0.0.0

#
# * Fine Tuning
#
#key_buffer_size = 16M
max_allowed_packet = 128M
#thread_stack = 192K
#thread_cache_size = 8
# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched
#myisam_recover_options = BACKUP
max_connections = 1250
#table_cache = 64
#thread_concurrency = 10

sql_mode = ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
key_buffer_size = 256M
table_open_cache = 8192
table_definition_cache = 8192
thread_cache_size = 32
max_user_connections = 0
tmp_table_size = 32M
max_heap_table_size = 32M
join_buffer_size = 2M
sort_buffer_size = 2M
read_rnd_buffer_size = 1M

#
# * Performance schema for better diagnostics
#
performance_schema = ON

#
# * Query Cache Configuration
#
query_cache_limit = 1M
query_cache_size = 0
query_cache_type = 0

#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
# As of 5.1 you can enable the log at runtime!
#general_log_file = /var/log/mysql/mysql.log
#general_log = 1
#
# Error log - should be very few entries.
#
log_error = /var/log/mysql/error.log
#
# Enable the slow query log to see queries with especially long duration
#slow_query_log_file = /var/log/mysql/mariadb-slow.log
#long_query_time = 10
#log_slow_rate_limit = 1000
#log_slow_verbosity = query_plan
#log-queries-not-using-indexes
#
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
# other settings you may need to change.
#server-id = 1

expire_logs_days = 10
#max_binlog_size = 100M
#binlog_do_db = include_database_name
#binlog_ignore_db = exclude_database_name

#
# * Security Features
#
# Read the manual, too, if you want chroot!
#chroot = /var/lib/mysql/
#
# For generating SSL certificates you can use for example the GUI tool "tinyca".
#
#ssl-ca = /etc/mysql/cacert.pem
#ssl-cert = /etc/mysql/server-cert.pem
#ssl-key = /etc/mysql/server-key.pem
#
# Accept only connections using the latest and most secure TLS protocol version.
# ..when MariaDB is compiled with OpenSSL:
#ssl-cipher = TLSv1.2
# ..when MariaDB is compiled with YaSSL (default in Debian):
#ssl = on

#
# * Character sets
#
# MySQL/MariaDB default is Latin1, but in Debian we rather default to the full
# utf8 4-byte character set. See also client.cnf
#
character-set-server = utf8mb4
collation-server = utf8mb4_general_ci

#
# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
innodb_buffer_pool_size = 16G
innodb_buffer_pool_instances = 16
innodb_thread_concurrency = 8
innodb_log_file_size = 6G
innodb_log_buffer_size = 16M
innodb_flush_log_at_trx_commit = 0
innodb_lock_wait_timeout = 300
innodb_io_capacity = 4000
innodb_io_capacity_max = 8000
innodb_read_io_threads = 8
innodb_write_io_threads = 8

#
# * Unix socket authentication plugin is built-in since 10.0.22-6
#
# Needed so the root database user can authenticate without a password but
# only when running as the unix root user.
#
# Also available for other users if required.
# See https://mariadb.com/kb/en/unix_socket-authentication-plugin/

# this is only for embedded server
[embedded]

# This group is only read by MariaDB servers, not by MySQL.
# If you use the same .cnf file for MySQL and MariaDB,
# you can put MariaDB-only options here
[mariadb]

# This group is only read by MariaDB-10.3 servers.
# If you use the same .cnf file for MariaDB of different versions,
# use this group for options that older servers don't understand
[mariadb-10.3]
{code}

{code:shell}
[mysqld]
####################################
# mysql/mariadb settings

binlog_format = ROW
default-storage-engine = innodb
innodb_autoinc_lock_mode = 2
innodb_file_per_table = on
binlog-row-image = minimal

innodb_doublewrite = 1

####################################
# galera settings

wsrep_on = ON
wsrep_provider = /usr/lib/galera/libgalera_smm.so
wsrep_cluster_name = "sql-gc02"
wsrep_cluster_address = "gcomm://10.49.65.120,10.49.65.121"
wsrep_node_address = "10.49.65.121"
wsrep_node_name = "pr-be-mdb-galera02-02"
wsrep_sst_method = "mariabackup"
wsrep_sst_auth = xxxxxxxxxxxx:xxxxxxxxxxxxxxxxxxxxxxx
ignore_db_dirs = .snapshot
wsrep_slave_threads = 8
{code}

Sometimes we got an error on the second MariaDB-Galera-Node which get dropped then from the cluster with an MDL BF-BF conflict.
What could that be?

{code:shell}
2024-04-10 8:02:54 10 [Note] WSREP: Server pr-be-mdb-galera02-02 synced with group
2024-04-12 10:14:11 860965 [Note] WSREP: Desyncing and pausing the provider
2024-04-12 10:14:11 0 [Note] WSREP: Member 1.0 (pr-be-mdb-galera02-02) desyncs itself from group
2024-04-12 10:14:11 0 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 449986329)
2024-04-12 10:14:11 860965 [Note] WSREP: pause
2024-04-12 10:14:11 860965 [Note] WSREP: Provider paused at 2b276be9-b04a-11ee-9634-0ffee8933331:449986329 (205127771)
2024-04-12 10:14:11 860965 [Note] WSREP: Provider paused at: 449986329
2024-04-12 10:14:28 860965 [Note] WSREP: Resuming and resyncing the provider
2024-04-12 10:14:28 860965 [Note] WSREP: resume
2024-04-12 10:14:28 860965 [Note] WSREP: resuming provider at 205127771
2024-04-12 10:14:28 860965 [Note] WSREP: Provider resumed.
2024-04-12 10:14:28 0 [Note] WSREP: Member 1.0 (pr-be-mdb-galera02-02) resyncs itself to group.
2024-04-12 10:14:28 0 [Note] WSREP: Shifting DONOR/DESYNCED -> JOINED (TO: 449987886)
2024-04-12 10:14:28 0 [Note] WSREP: Processing event queue:... 0.0% ( 0/1549 events) complete.
2024-04-12 10:14:29 0 [Note] WSREP: Member 1.0 (pr-be-mdb-galera02-02) synced with group.
2024-04-12 10:14:29 0 [Note] WSREP: Processing event queue:...100.0% (1641/1641 events) complete.
2024-04-12 10:14:29 0 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 449987963)
2024-04-12 10:14:29 7 [Note] WSREP: Server pr-be-mdb-galera02-02 synced with group
2024-04-13 0:23:56 7 [Note] WSREP: MDL BF-BF conflict
schema: bs_actionpress
request: (7 seqno 455199246 wsrep (toi, exec, committed) cmd 0 45 OPTIMIZE TABLE exportdb_sqls)
granted: (2 seqno 455199247 wsrep (high priority, exec, committing) cmd 0 161 (null))
2024-04-13 0:23:56 7 [ERROR] Aborting
{code}
{code:shell}
2024-04-13 0:05:15 108629708 [Warning] Aborted connection 108629708 to db: 'bs_actionpress' user: 'bs_actionpress' host: '10.49.97.138' (Got an error reading communication packets)
2024-04-13 0:05:15 108629692 [Warning] Aborted connection 108629692 to db: 'bs_actionpress' user: 'bs_actionpress' host: '10.49.97.138' (Got an error reading communication packets)
2024-04-13 0:05:15 108629695 [Warning] Aborted connection 108629695 to db: 'bs_actionpress' user: 'bs_actionpress' host: '10.49.97.138' (Got an error reading communication packets)
2024-04-13 0:05:15 108629726 [Warning] Aborted connection 108629726 to db: 'bs_actionpress' user: 'bs_actionpress' host: '10.49.97.138' (Got an error reading communication packets)
2024-04-13 0:05:15 108629703 [Warning] Aborted connection 108629703 to db: 'bs_actionpress' user: 'bs_actionpress' host: '10.49.97.138' (Got an error reading communication packets)
2024-04-13 0:05:15 108629697 [Warning] Aborted connection 108629697 to db: 'bs_actionpress' user: 'bs_actionpress' host: '10.49.97.138' (Got an error reading communication packets)
2024-04-13 0:05:15 108629731 [Warning] Aborted connection 108629731 to db: 'bs_actionpress' user: 'bs_actionpress' host: '10.49.97.138' (Got an error reading communication packets)
2024-04-13 0:05:15 108629705 [Warning] Aborted connection 108629705 to db: 'bs_actionpress' user: 'bs_actionpress' host: '10.49.97.138' (Got an error reading communication packets)
2024-04-13 0:08:05 108637615 [Warning] Aborted connection 108637615 to db: 'bs_actionpress' user: 'bs_actionpress' host: '10.49.97.138' (Got an error reading communication packets)
2024-04-13 0:23:57 0 [Note] WSREP: Deferred close timer started for socket with remote endpoint: tcp://10.49.65.121:57538
2024-04-13 0:23:57 0 [Note] WSREP: forgetting d6a7f0d5-a6b5 (tcp://10.49.65.121:4567)
2024-04-13 0:23:57 0 [Note] WSREP: Node 8e8022d1-9aaa state prim
2024-04-13 0:23:57 0 [Note] WSREP: view(view_id(PRIM,8e8022d1-9aaa,29) memb {
        8e8022d1-9aaa,0
} joined {
} left {
} partitioned {
        d6a7f0d5-a6b5,0
})
2024-04-13 0:23:57 0 [Note] WSREP: save pc into disk
2024-04-13 0:23:57 0 [Note] WSREP: forgetting d6a7f0d5-a6b5 (tcp://10.49.65.121:4567)
2024-04-13 0:23:57 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 1
2024-04-13 0:23:57 0 [Note] WSREP: STATE_EXCHANGE: sent state UUID: 5a650c2e-f91b-11ee-9241-5eaa8b49759e
2024-04-13 0:23:57 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 5a650c2e-f91b-11ee-9241-5eaa8b49759e
2024-04-13 0:23:57 0 [Note] WSREP: STATE EXCHANGE: got state msg: 5a650c2e-f91b-11ee-9241-5eaa8b49759e from 0 (pr-be-mdb-galera02-01)
2024-04-13 0:23:57 0 [Note] WSREP: Quorum results:
        version = 6,
        component = PRIMARY,
        conf_id = 28,
        members = 1/1 (joined/total),
        act_id = 455199278,
        last_appl. = 455199146,
        protocols = 2/10/4 (gcs/repl/appl),
        vote policy= 0,
        group UUID = 2b276be9-b04a-11ee-9634-0ffee8933331
2024-04-13 0:23:57 0 [Note] WSREP: Flow-control interval: [16, 16]
2024-04-13 0:23:57 6 [Note] WSREP: ####### processing CC 455199279, local, ordered
2024-04-13 0:23:57 0 [Note] WSREP: Deferred close timer handle_wait Operation aborted. for 0x5626dfa643f0
2024-04-13 0:23:57 6 [Note] WSREP: ####### My UUID: 8e8022d1-c97c-11ee-9aaa-c66638f704cf
2024-04-13 0:23:57 6 [Note] WSREP: Skipping cert index reset
2024-04-13 0:23:57 0 [Note] WSREP: Deferred close timer destruct
2024-04-13 0:23:57 6 [Note] WSREP: REPL Protocols: 10 (5)
2024-04-13 0:23:57 6 [Note] WSREP: ####### Adjusting cert position: 455199278 -> 455199279
2024-04-13 0:23:57 0 [Note] WSREP: Service thread queue flushed.
2024-04-13 0:23:57 6 [Note] WSREP: ================================================
View:
  id: 2b276be9-b04a-11ee-9634-0ffee8933331:455199279
  status: primary
  protocol_version: 4
  capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
  final: no
  own_index: 0
  members(1):
        0: 8e8022d1-c97c-11ee-9aaa-c66638f704cf, pr-be-mdb-galera02-01
=================================================
2024-04-13 0:23:57 6 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2024-04-13 0:23:57 6 [Note] WSREP: Lowest cert index boundary for CC from group: 455199147
2024-04-13 0:23:57 6 [Note] WSREP: Min available from gcache for CC from group: 455084811
2024-04-13 0:24:02 0 [Note] WSREP: cleaning up d6a7f0d5-a6b5 (tcp://10.49.65.121:4567)
{code}

Here is our central config for all mariadb servers:

{code:shell}
# MANAGED BY ANSIBLE, ALL CHANGES WILL BE OVERWRITTEN!
#
# These groups are read by MariaDB server.
# Use it for options that only the server (but not clients) should see
#
# See the examples of server my.cnf files in /usr/share/mysql

# this is read by the standalone daemon and embedded servers
[server]

# this is only for the mysqld standalone daemon
[mysqld]

#
# * Basic Settings
#
user = mysql
pid-file = /run/mysqld/mysqld.pid
socket = /run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /data/pr-be-mdb-galera02-02
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
#skip-external-locking
skip-name-resolve

# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address = 0.0.0.0

#
# * Fine Tuning
#
#key_buffer_size = 16M
max_allowed_packet = 128M
#thread_stack = 192K
#thread_cache_size = 8
# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched
#myisam_recover_options = BACKUP
max_connections = 1250
#table_cache = 64
#thread_concurrency = 10

sql_mode = ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
key_buffer_size = 256M
table_open_cache = 8192
table_definition_cache = 8192
thread_cache_size = 32
max_user_connections = 0
tmp_table_size = 32M
max_heap_table_size = 32M
join_buffer_size = 2M
sort_buffer_size = 2M
read_rnd_buffer_size = 1M

#
# * Performance schema for better diagnostics
#
performance_schema = ON

#
# * Query Cache Configuration
#
query_cache_limit = 1M
query_cache_size = 0
query_cache_type = 0

#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
# As of 5.1 you can enable the log at runtime!
#general_log_file = /var/log/mysql/mysql.log
#general_log = 1
#
# Error log - should be very few entries.
#
log_error = /var/log/mysql/error.log
#
# Enable the slow query log to see queries with especially long duration
#slow_query_log_file = /var/log/mysql/mariadb-slow.log
#long_query_time = 10
#log_slow_rate_limit = 1000
#log_slow_verbosity = query_plan
#log-queries-not-using-indexes
#
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
# other settings you may need to change.
#server-id = 1

expire_logs_days = 10
#max_binlog_size = 100M
#binlog_do_db = include_database_name
#binlog_ignore_db = exclude_database_name

#
# * Security Features
#
# Read the manual, too, if you want chroot!
#chroot = /var/lib/mysql/
#
# For generating SSL certificates you can use for example the GUI tool "tinyca".
#
#ssl-ca = /etc/mysql/cacert.pem
#ssl-cert = /etc/mysql/server-cert.pem
#ssl-key = /etc/mysql/server-key.pem
#
# Accept only connections using the latest and most secure TLS protocol version.
# ..when MariaDB is compiled with OpenSSL:
#ssl-cipher = TLSv1.2
# ..when MariaDB is compiled with YaSSL (default in Debian):
#ssl = on

#
# * Character sets
#
# MySQL/MariaDB default is Latin1, but in Debian we rather default to the full
# utf8 4-byte character set. See also client.cnf
#
character-set-server = utf8mb4
collation-server = utf8mb4_general_ci

#
# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
innodb_buffer_pool_size = 16G
innodb_buffer_pool_instances = 16
innodb_thread_concurrency = 8
innodb_log_file_size = 6G
innodb_log_buffer_size = 16M
innodb_flush_log_at_trx_commit = 0
innodb_lock_wait_timeout = 300
innodb_io_capacity = 4000
innodb_io_capacity_max = 8000
innodb_read_io_threads = 8
innodb_write_io_threads = 8

#
# * Unix socket authentication plugin is built-in since 10.0.22-6
#
# Needed so the root database user can authenticate without a password but
# only when running as the unix root user.
#
# Also available for other users if required.
# See https://mariadb.com/kb/en/unix_socket-authentication-plugin/

# this is only for embedded server
[embedded]

# This group is only read by MariaDB servers, not by MySQL.
# If you use the same .cnf file for MySQL and MariaDB,
# you can put MariaDB-only options here
[mariadb]

# This group is only read by MariaDB-10.3 servers.
# If you use the same .cnf file for MariaDB of different versions,
# use this group for options that older servers don't understand
[mariadb-10.3]
{code}

{code:shell}
[mysqld]
####################################
# mysql/mariadb settings

binlog_format = ROW
default-storage-engine = innodb
innodb_autoinc_lock_mode = 2
innodb_file_per_table = on
binlog-row-image = minimal

innodb_doublewrite = 1

####################################
# galera settings

wsrep_on = ON
wsrep_provider = /usr/lib/galera/libgalera_smm.so
wsrep_cluster_name = "sql-gc02"
wsrep_cluster_address = "gcomm://10.49.65.120,10.49.65.121"
wsrep_node_address = "10.49.65.121"
wsrep_node_name = "pr-be-mdb-galera02-02"
wsrep_sst_method = "mariabackup"
wsrep_sst_auth = xxxxxxxxxxxx:xxxxxxxxxxxxxxxxxxxxxxx
ignore_db_dirs = .snapshot
wsrep_slave_threads = 8
{code}

Jan Lindström made changes - 2024-04-29 06:21

Assignee

Jan Lindström [ JIRAUSER53125 ]

Jan Lindström added a comment - 2024-04-29 06:23

dcz01 Do your system have network problems? I see that node desyncs and syncs to cluster several times before MDL BF-BF problem.There is some critical fixes on more recent version of MariaDB and Galera library, maybe you could try to upgrade. Also if you could provide error log with --wsrep-debug=1 setting, it could help.

Jan Lindström added a comment - 2024-04-29 06:23 dcz01 Do your system have network problems? I see that node desyncs and syncs to cluster several times before MDL BF-BF problem.There is some critical fixes on more recent version of MariaDB and Galera library, maybe you could try to upgrade. Also if you could provide error log with --wsrep-debug=1 setting, it could help.

Jan Lindström made changes - 2024-04-29 06:23

Status

Open [ 1 ]

Needs Feedback [ 10501 ]

Daniel Czadek added a comment - 2024-04-29 06:30

Hi Jan,

Thanks for your answer.
Well the systems don't have network problems but i encountered that this desync and resync is normal if taking a backup of the databases with mariabackup. So thats normally would i say.
I will provide better logs with the debug option active.
We're already thinking about using a newer version of mariadb (not the shipped one with Debian 11).

Greetings

Daniel Czadek added a comment - 2024-04-29 06:30 Hi Jan, Thanks for your answer. Well the systems don't have network problems but i encountered that this desync and resync is normal if taking a backup of the databases with mariabackup. So thats normally would i say. I will provide better logs with the debug option active. We're already thinking about using a newer version of mariadb (not the shipped one with Debian 11). Greetings

Daniel Czadek made changes - 2024-04-29 08:42

Attachment

error.log [ 73373 ]

Daniel Czadek made changes - 2024-04-29 08:48

Description

Sometimes we got an error on the second MariaDB-Galera-Node which get dropped then from the cluster with an MDL BF-BF conflict.
What could that be?

{code:shell}
2024-04-10 8:02:54 10 [Note] WSREP: Server pr-be-mdb-galera02-02 synced with group
2024-04-12 10:14:11 860965 [Note] WSREP: Desyncing and pausing the provider
2024-04-12 10:14:11 0 [Note] WSREP: Member 1.0 (pr-be-mdb-galera02-02) desyncs itself from group
2024-04-12 10:14:11 0 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 449986329)
2024-04-12 10:14:11 860965 [Note] WSREP: pause
2024-04-12 10:14:11 860965 [Note] WSREP: Provider paused at 2b276be9-b04a-11ee-9634-0ffee8933331:449986329 (205127771)
2024-04-12 10:14:11 860965 [Note] WSREP: Provider paused at: 449986329
2024-04-12 10:14:28 860965 [Note] WSREP: Resuming and resyncing the provider
2024-04-12 10:14:28 860965 [Note] WSREP: resume
2024-04-12 10:14:28 860965 [Note] WSREP: resuming provider at 205127771
2024-04-12 10:14:28 860965 [Note] WSREP: Provider resumed.
2024-04-12 10:14:28 0 [Note] WSREP: Member 1.0 (pr-be-mdb-galera02-02) resyncs itself to group.
2024-04-12 10:14:28 0 [Note] WSREP: Shifting DONOR/DESYNCED -> JOINED (TO: 449987886)
2024-04-12 10:14:28 0 [Note] WSREP: Processing event queue:... 0.0% ( 0/1549 events) complete.
2024-04-12 10:14:29 0 [Note] WSREP: Member 1.0 (pr-be-mdb-galera02-02) synced with group.
2024-04-12 10:14:29 0 [Note] WSREP: Processing event queue:...100.0% (1641/1641 events) complete.
2024-04-12 10:14:29 0 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 449987963)
2024-04-12 10:14:29 7 [Note] WSREP: Server pr-be-mdb-galera02-02 synced with group
2024-04-13 0:23:56 7 [Note] WSREP: MDL BF-BF conflict
schema: bs_actionpress
request: (7 seqno 455199246 wsrep (toi, exec, committed) cmd 0 45 OPTIMIZE TABLE exportdb_sqls)
granted: (2 seqno 455199247 wsrep (high priority, exec, committing) cmd 0 161 (null))
2024-04-13 0:23:56 7 [ERROR] Aborting
{code}
{code:shell}
2024-04-13 0:05:15 108629708 [Warning] Aborted connection 108629708 to db: 'bs_actionpress' user: 'bs_actionpress' host: '10.49.97.138' (Got an error reading communication packets)
2024-04-13 0:05:15 108629692 [Warning] Aborted connection 108629692 to db: 'bs_actionpress' user: 'bs_actionpress' host: '10.49.97.138' (Got an error reading communication packets)
2024-04-13 0:05:15 108629695 [Warning] Aborted connection 108629695 to db: 'bs_actionpress' user: 'bs_actionpress' host: '10.49.97.138' (Got an error reading communication packets)
2024-04-13 0:05:15 108629726 [Warning] Aborted connection 108629726 to db: 'bs_actionpress' user: 'bs_actionpress' host: '10.49.97.138' (Got an error reading communication packets)
2024-04-13 0:05:15 108629703 [Warning] Aborted connection 108629703 to db: 'bs_actionpress' user: 'bs_actionpress' host: '10.49.97.138' (Got an error reading communication packets)
2024-04-13 0:05:15 108629697 [Warning] Aborted connection 108629697 to db: 'bs_actionpress' user: 'bs_actionpress' host: '10.49.97.138' (Got an error reading communication packets)
2024-04-13 0:05:15 108629731 [Warning] Aborted connection 108629731 to db: 'bs_actionpress' user: 'bs_actionpress' host: '10.49.97.138' (Got an error reading communication packets)
2024-04-13 0:05:15 108629705 [Warning] Aborted connection 108629705 to db: 'bs_actionpress' user: 'bs_actionpress' host: '10.49.97.138' (Got an error reading communication packets)
2024-04-13 0:08:05 108637615 [Warning] Aborted connection 108637615 to db: 'bs_actionpress' user: 'bs_actionpress' host: '10.49.97.138' (Got an error reading communication packets)
2024-04-13 0:23:57 0 [Note] WSREP: Deferred close timer started for socket with remote endpoint: tcp://10.49.65.121:57538
2024-04-13 0:23:57 0 [Note] WSREP: forgetting d6a7f0d5-a6b5 (tcp://10.49.65.121:4567)
2024-04-13 0:23:57 0 [Note] WSREP: Node 8e8022d1-9aaa state prim
2024-04-13 0:23:57 0 [Note] WSREP: view(view_id(PRIM,8e8022d1-9aaa,29) memb {
        8e8022d1-9aaa,0
} joined {
} left {
} partitioned {
        d6a7f0d5-a6b5,0
})
2024-04-13 0:23:57 0 [Note] WSREP: save pc into disk
2024-04-13 0:23:57 0 [Note] WSREP: forgetting d6a7f0d5-a6b5 (tcp://10.49.65.121:4567)
2024-04-13 0:23:57 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 1
2024-04-13 0:23:57 0 [Note] WSREP: STATE_EXCHANGE: sent state UUID: 5a650c2e-f91b-11ee-9241-5eaa8b49759e
2024-04-13 0:23:57 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 5a650c2e-f91b-11ee-9241-5eaa8b49759e
2024-04-13 0:23:57 0 [Note] WSREP: STATE EXCHANGE: got state msg: 5a650c2e-f91b-11ee-9241-5eaa8b49759e from 0 (pr-be-mdb-galera02-01)
2024-04-13 0:23:57 0 [Note] WSREP: Quorum results:
        version = 6,
        component = PRIMARY,
        conf_id = 28,
        members = 1/1 (joined/total),
        act_id = 455199278,
        last_appl. = 455199146,
        protocols = 2/10/4 (gcs/repl/appl),
        vote policy= 0,
        group UUID = 2b276be9-b04a-11ee-9634-0ffee8933331
2024-04-13 0:23:57 0 [Note] WSREP: Flow-control interval: [16, 16]
2024-04-13 0:23:57 6 [Note] WSREP: ####### processing CC 455199279, local, ordered
2024-04-13 0:23:57 0 [Note] WSREP: Deferred close timer handle_wait Operation aborted. for 0x5626dfa643f0
2024-04-13 0:23:57 6 [Note] WSREP: ####### My UUID: 8e8022d1-c97c-11ee-9aaa-c66638f704cf
2024-04-13 0:23:57 6 [Note] WSREP: Skipping cert index reset
2024-04-13 0:23:57 0 [Note] WSREP: Deferred close timer destruct
2024-04-13 0:23:57 6 [Note] WSREP: REPL Protocols: 10 (5)
2024-04-13 0:23:57 6 [Note] WSREP: ####### Adjusting cert position: 455199278 -> 455199279
2024-04-13 0:23:57 0 [Note] WSREP: Service thread queue flushed.
2024-04-13 0:23:57 6 [Note] WSREP: ================================================
View:
  id: 2b276be9-b04a-11ee-9634-0ffee8933331:455199279
  status: primary
  protocol_version: 4
  capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
  final: no
  own_index: 0
  members(1):
        0: 8e8022d1-c97c-11ee-9aaa-c66638f704cf, pr-be-mdb-galera02-01
=================================================
2024-04-13 0:23:57 6 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2024-04-13 0:23:57 6 [Note] WSREP: Lowest cert index boundary for CC from group: 455199147
2024-04-13 0:23:57 6 [Note] WSREP: Min available from gcache for CC from group: 455084811
2024-04-13 0:24:02 0 [Note] WSREP: cleaning up d6a7f0d5-a6b5 (tcp://10.49.65.121:4567)
{code}

Here is our central config for all mariadb servers:

{code:shell}
# MANAGED BY ANSIBLE, ALL CHANGES WILL BE OVERWRITTEN!
#
# These groups are read by MariaDB server.
# Use it for options that only the server (but not clients) should see
#
# See the examples of server my.cnf files in /usr/share/mysql

# this is read by the standalone daemon and embedded servers
[server]

# this is only for the mysqld standalone daemon
[mysqld]

#
# * Basic Settings
#
user = mysql
pid-file = /run/mysqld/mysqld.pid
socket = /run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /data/pr-be-mdb-galera02-02
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
#skip-external-locking
skip-name-resolve

# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address = 0.0.0.0

#
# * Fine Tuning
#
#key_buffer_size = 16M
max_allowed_packet = 128M
#thread_stack = 192K
#thread_cache_size = 8
# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched
#myisam_recover_options = BACKUP
max_connections = 1250
#table_cache = 64
#thread_concurrency = 10

sql_mode = ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
key_buffer_size = 256M
table_open_cache = 8192
table_definition_cache = 8192
thread_cache_size = 32
max_user_connections = 0
tmp_table_size = 32M
max_heap_table_size = 32M
join_buffer_size = 2M
sort_buffer_size = 2M
read_rnd_buffer_size = 1M

#
# * Performance schema for better diagnostics
#
performance_schema = ON

#
# * Query Cache Configuration
#
query_cache_limit = 1M
query_cache_size = 0
query_cache_type = 0

#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
# As of 5.1 you can enable the log at runtime!
#general_log_file = /var/log/mysql/mysql.log
#general_log = 1
#
# Error log - should be very few entries.
#
log_error = /var/log/mysql/error.log
#
# Enable the slow query log to see queries with especially long duration
#slow_query_log_file = /var/log/mysql/mariadb-slow.log
#long_query_time = 10
#log_slow_rate_limit = 1000
#log_slow_verbosity = query_plan
#log-queries-not-using-indexes
#
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
# other settings you may need to change.
#server-id = 1

expire_logs_days = 10
#max_binlog_size = 100M
#binlog_do_db = include_database_name
#binlog_ignore_db = exclude_database_name

#
# * Security Features
#
# Read the manual, too, if you want chroot!
#chroot = /var/lib/mysql/
#
# For generating SSL certificates you can use for example the GUI tool "tinyca".
#
#ssl-ca = /etc/mysql/cacert.pem
#ssl-cert = /etc/mysql/server-cert.pem
#ssl-key = /etc/mysql/server-key.pem
#
# Accept only connections using the latest and most secure TLS protocol version.
# ..when MariaDB is compiled with OpenSSL:
#ssl-cipher = TLSv1.2
# ..when MariaDB is compiled with YaSSL (default in Debian):
#ssl = on

#
# * Character sets
#
# MySQL/MariaDB default is Latin1, but in Debian we rather default to the full
# utf8 4-byte character set. See also client.cnf
#
character-set-server = utf8mb4
collation-server = utf8mb4_general_ci

#
# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
innodb_buffer_pool_size = 16G
innodb_buffer_pool_instances = 16
innodb_thread_concurrency = 8
innodb_log_file_size = 6G
innodb_log_buffer_size = 16M
innodb_flush_log_at_trx_commit = 0
innodb_lock_wait_timeout = 300
innodb_io_capacity = 4000
innodb_io_capacity_max = 8000
innodb_read_io_threads = 8
innodb_write_io_threads = 8

#
# * Unix socket authentication plugin is built-in since 10.0.22-6
#
# Needed so the root database user can authenticate without a password but
# only when running as the unix root user.
#
# Also available for other users if required.
# See https://mariadb.com/kb/en/unix_socket-authentication-plugin/

# this is only for embedded server
[embedded]

# This group is only read by MariaDB servers, not by MySQL.
# If you use the same .cnf file for MySQL and MariaDB,
# you can put MariaDB-only options here
[mariadb]

# This group is only read by MariaDB-10.3 servers.
# If you use the same .cnf file for MariaDB of different versions,
# use this group for options that older servers don't understand
[mariadb-10.3]
{code}

{code:shell}
[mysqld]
####################################
# mysql/mariadb settings

binlog_format = ROW
default-storage-engine = innodb
innodb_autoinc_lock_mode = 2
innodb_file_per_table = on
binlog-row-image = minimal

innodb_doublewrite = 1

####################################
# galera settings

wsrep_on = ON
wsrep_provider = /usr/lib/galera/libgalera_smm.so
wsrep_cluster_name = "sql-gc02"
wsrep_cluster_address = "gcomm://10.49.65.120,10.49.65.121"
wsrep_node_address = "10.49.65.121"
wsrep_node_name = "pr-be-mdb-galera02-02"
wsrep_sst_method = "mariabackup"
wsrep_sst_auth = xxxxxxxxxxxx:xxxxxxxxxxxxxxxxxxxxxxx
ignore_db_dirs = .snapshot
wsrep_slave_threads = 8
{code}

Sometimes we got an error on the second MariaDB-Galera-Node which get dropped then from the cluster with an MDL BF-BF conflict.
What could that be?

{code:shell}
2024-04-10 8:02:54 10 [Note] WSREP: Server server2 synced with group
2024-04-12 10:14:11 860965 [Note] WSREP: Desyncing and pausing the provider
2024-04-12 10:14:11 0 [Note] WSREP: Member 1.0 (server2) desyncs itself from group
2024-04-12 10:14:11 0 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 449986329)
2024-04-12 10:14:11 860965 [Note] WSREP: pause
2024-04-12 10:14:11 860965 [Note] WSREP: Provider paused at 2b276be9-b04a-11ee-9634-0ffee8933331:449986329 (205127771)
2024-04-12 10:14:11 860965 [Note] WSREP: Provider paused at: 449986329
2024-04-12 10:14:28 860965 [Note] WSREP: Resuming and resyncing the provider
2024-04-12 10:14:28 860965 [Note] WSREP: resume
2024-04-12 10:14:28 860965 [Note] WSREP: resuming provider at 205127771
2024-04-12 10:14:28 860965 [Note] WSREP: Provider resumed.
2024-04-12 10:14:28 0 [Note] WSREP: Member 1.0 (server2) resyncs itself to group.
2024-04-12 10:14:28 0 [Note] WSREP: Shifting DONOR/DESYNCED -> JOINED (TO: 449987886)
2024-04-12 10:14:28 0 [Note] WSREP: Processing event queue:... 0.0% ( 0/1549 events) complete.
2024-04-12 10:14:29 0 [Note] WSREP: Member 1.0 (server2) synced with group.
2024-04-12 10:14:29 0 [Note] WSREP: Processing event queue:...100.0% (1641/1641 events) complete.
2024-04-12 10:14:29 0 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 449987963)
2024-04-12 10:14:29 7 [Note] WSREP: Server server2 synced with group
2024-04-13 0:23:56 7 [Note] WSREP: MDL BF-BF conflict
schema: xxxxxxxxxxxxx
request: (7 seqno 455199246 wsrep (toi, exec, committed) cmd 0 45 OPTIMIZE TABLE exportdb_sqls)
granted: (2 seqno 455199247 wsrep (high priority, exec, committing) cmd 0 161 (null))
2024-04-13 0:23:56 7 [ERROR] Aborting
{code}
{code:shell}
2024-04-13 0:23:57 0 [Note] WSREP: Deferred close timer started for socket with remote endpoint: tcp://server2:57538
2024-04-13 0:23:57 0 [Note] WSREP: forgetting d6a7f0d5-a6b5 (tcp://server2:4567)
2024-04-13 0:23:57 0 [Note] WSREP: Node 8e8022d1-9aaa state prim
2024-04-13 0:23:57 0 [Note] WSREP: view(view_id(PRIM,8e8022d1-9aaa,29) memb {
        8e8022d1-9aaa,0
} joined {
} left {
} partitioned {
        d6a7f0d5-a6b5,0
})
2024-04-13 0:23:57 0 [Note] WSREP: save pc into disk
2024-04-13 0:23:57 0 [Note] WSREP: forgetting d6a7f0d5-a6b5 (tcp://server2:4567)
2024-04-13 0:23:57 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 1
2024-04-13 0:23:57 0 [Note] WSREP: STATE_EXCHANGE: sent state UUID: 5a650c2e-f91b-11ee-9241-5eaa8b49759e
2024-04-13 0:23:57 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 5a650c2e-f91b-11ee-9241-5eaa8b49759e
2024-04-13 0:23:57 0 [Note] WSREP: STATE EXCHANGE: got state msg: 5a650c2e-f91b-11ee-9241-5eaa8b49759e from 0 (server1)
2024-04-13 0:23:57 0 [Note] WSREP: Quorum results:
        version = 6,
        component = PRIMARY,
        conf_id = 28,
        members = 1/1 (joined/total),
        act_id = 455199278,
        last_appl. = 455199146,
        protocols = 2/10/4 (gcs/repl/appl),
        vote policy= 0,
        group UUID = 2b276be9-b04a-11ee-9634-0ffee8933331
2024-04-13 0:23:57 0 [Note] WSREP: Flow-control interval: [16, 16]
2024-04-13 0:23:57 6 [Note] WSREP: ####### processing CC 455199279, local, ordered
2024-04-13 0:23:57 0 [Note] WSREP: Deferred close timer handle_wait Operation aborted. for 0x5626dfa643f0
2024-04-13 0:23:57 6 [Note] WSREP: ####### My UUID: 8e8022d1-c97c-11ee-9aaa-c66638f704cf
2024-04-13 0:23:57 6 [Note] WSREP: Skipping cert index reset
2024-04-13 0:23:57 0 [Note] WSREP: Deferred close timer destruct
2024-04-13 0:23:57 6 [Note] WSREP: REPL Protocols: 10 (5)
2024-04-13 0:23:57 6 [Note] WSREP: ####### Adjusting cert position: 455199278 -> 455199279
2024-04-13 0:23:57 0 [Note] WSREP: Service thread queue flushed.
2024-04-13 0:23:57 6 [Note] WSREP: ================================================
View:
  id: 2b276be9-b04a-11ee-9634-0ffee8933331:455199279
  status: primary
  protocol_version: 4
  capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
  final: no
  own_index: 0
  members(1):
        0: 8e8022d1-c97c-11ee-9aaa-c66638f704cf, server1
=================================================
2024-04-13 0:23:57 6 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2024-04-13 0:23:57 6 [Note] WSREP: Lowest cert index boundary for CC from group: 455199147
2024-04-13 0:23:57 6 [Note] WSREP: Min available from gcache for CC from group: 455084811
2024-04-13 0:24:02 0 [Note] WSREP: cleaning up d6a7f0d5-a6b5 (tcp://server2:4567)
{code}

Here is our central config for all mariadb servers:

{code:shell}
# MANAGED BY ANSIBLE, ALL CHANGES WILL BE OVERWRITTEN!
#
# These groups are read by MariaDB server.
# Use it for options that only the server (but not clients) should see
#
# See the examples of server my.cnf files in /usr/share/mysql

# this is read by the standalone daemon and embedded servers
[server]

# this is only for the mysqld standalone daemon
[mysqld]

#
# * Basic Settings
#
user = mysql
pid-file = /run/mysqld/mysqld.pid
socket = /run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /data/server2
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
#skip-external-locking
skip-name-resolve

# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address = 0.0.0.0

#
# * Fine Tuning
#
#key_buffer_size = 16M
max_allowed_packet = 128M
#thread_stack = 192K
#thread_cache_size = 8
# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched
#myisam_recover_options = BACKUP
max_connections = 1250
#table_cache = 64
#thread_concurrency = 10

sql_mode = ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
key_buffer_size = 256M
table_open_cache = 8192
table_definition_cache = 8192
thread_cache_size = 32
max_user_connections = 0
tmp_table_size = 32M
max_heap_table_size = 32M
join_buffer_size = 2M
sort_buffer_size = 2M
read_rnd_buffer_size = 1M

#
# * Performance schema for better diagnostics
#
performance_schema = ON

#
# * Query Cache Configuration
#
query_cache_limit = 1M
query_cache_size = 0
query_cache_type = 0

#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
# As of 5.1 you can enable the log at runtime!
#general_log_file = /var/log/mysql/mysql.log
#general_log = 1
#
# Error log - should be very few entries.
#
log_error = /var/log/mysql/error.log
#
# Enable the slow query log to see queries with especially long duration
#slow_query_log_file = /var/log/mysql/mariadb-slow.log
#long_query_time = 10
#log_slow_rate_limit = 1000
#log_slow_verbosity = query_plan
#log-queries-not-using-indexes
#
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
# other settings you may need to change.
#server-id = 1

expire_logs_days = 10
#max_binlog_size = 100M
#binlog_do_db = include_database_name
#binlog_ignore_db = exclude_database_name

#
# * Security Features
#
# Read the manual, too, if you want chroot!
#chroot = /var/lib/mysql/
#
# For generating SSL certificates you can use for example the GUI tool "tinyca".
#
#ssl-ca = /etc/mysql/cacert.pem
#ssl-cert = /etc/mysql/server-cert.pem
#ssl-key = /etc/mysql/server-key.pem
#
# Accept only connections using the latest and most secure TLS protocol version.
# ..when MariaDB is compiled with OpenSSL:
#ssl-cipher = TLSv1.2
# ..when MariaDB is compiled with YaSSL (default in Debian):
#ssl = on

#
# * Character sets
#
# MySQL/MariaDB default is Latin1, but in Debian we rather default to the full
# utf8 4-byte character set. See also client.cnf
#
character-set-server = utf8mb4
collation-server = utf8mb4_general_ci

#
# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
innodb_buffer_pool_size = 16G
innodb_buffer_pool_instances = 16
innodb_thread_concurrency = 8
innodb_log_file_size = 6G
innodb_log_buffer_size = 16M
innodb_flush_log_at_trx_commit = 0
innodb_lock_wait_timeout = 300
innodb_io_capacity = 4000
innodb_io_capacity_max = 8000
innodb_read_io_threads = 8
innodb_write_io_threads = 8

#
# * Unix socket authentication plugin is built-in since 10.0.22-6
#
# Needed so the root database user can authenticate without a password but
# only when running as the unix root user.
#
# Also available for other users if required.
# See https://mariadb.com/kb/en/unix_socket-authentication-plugin/

# this is only for embedded server
[embedded]

# This group is only read by MariaDB servers, not by MySQL.
# If you use the same .cnf file for MySQL and MariaDB,
# you can put MariaDB-only options here
[mariadb]

# This group is only read by MariaDB-10.3 servers.
# If you use the same .cnf file for MariaDB of different versions,
# use this group for options that older servers don't understand
[mariadb-10.3]
{code}

{code:shell}
[mysqld]
####################################
# mysql/mariadb settings

binlog_format = ROW
default-storage-engine = innodb
innodb_autoinc_lock_mode = 2
innodb_file_per_table = on
binlog-row-image = minimal

innodb_doublewrite = 1

####################################
# galera settings

wsrep_on = ON
wsrep_provider = /usr/lib/galera/libgalera_smm.so
wsrep_cluster_name = "xxxxxxxxx"
wsrep_cluster_address = "gcomm://server1,server2"
wsrep_node_address = "server2"
wsrep_node_name = "server2"
wsrep_sst_method = "mariabackup"
wsrep_sst_auth = xxxxxxxxxxxx:xxxxxxxxxxxxxxxxxxxxxxx
ignore_db_dirs = .snapshot
wsrep_slave_threads = 8
{code}

Jan Lindström added a comment - 2024-04-30 05:55

This is most likely duplicate of https://jira.mariadb.org/browse/MDEV-28452 that is under work.

Jan Lindström added a comment - 2024-04-30 05:55 This is most likely duplicate of https://jira.mariadb.org/browse/MDEV-28452 that is under work.

Jan Lindström made changes - 2024-04-30 06:14

Link

This issue is duplicated by ~~MDEV-28452~~ [ ~~MDEV-28452~~ ]

Jan Lindström made changes - 2024-04-30 06:14

Fix Version/s

10.5 [ 23123 ]

Daniel Czadek added a comment - 2024-05-06 06:39

Hi Jan,

now i got some more log with the debug option set:

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_commit_empty for 2 client_state exec client_mode high priority trans_state executing sql NULL

2024-05-04  0:02:06 2 [Note] WSREP: assigned new next trx id: 34687670

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_before_commit: 1, 880471116

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_after_commit: 1, 1, 880471116, 0

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_after_apply 2

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_thd_binlog_reset

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_commit_empty for 2 client_state exec client_mode high priority trans_state executing sql NULL

2024-05-04  0:02:06 2 [Note] WSREP: assigned new next trx id: 34687671

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_before_commit: 1, 880471117

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_after_commit: 1, 1, 880471117, 0

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_after_apply 2

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_thd_binlog_reset

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_commit_empty for 2 client_state exec client_mode high priority trans_state executing sql NULL

2024-05-04  0:02:06 2 [Note] WSREP: assigned new next trx id: 34687672

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_before_commit: 1, 880471118

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_after_commit: 1, 1, 880471118, 0

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_after_apply 2

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_thd_binlog_reset

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_commit_empty for 2 client_state exec client_mode high priority trans_state executing sql NULL

2024-05-04  0:02:06 2 [Note] WSREP: assigned new next trx id: 34687673

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_before_commit: 1, 880471119

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_after_commit: 1, 1, 880471119, 0

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_after_apply 2

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_thd_binlog_reset

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_commit_empty for 2 client_state exec client_mode high priority trans_state executing sql NULL

2024-05-04  0:02:06 2 [Note] WSREP: assigned new next trx id: 34687674

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_before_commit: 1, 880471120

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_after_commit: 1, 1, 880471120, 0

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_after_apply 2

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_thd_binlog_reset

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_commit_empty for 2 client_state exec client_mode high priority trans_state executing sql NULL

2024-05-04  0:02:06 2 [Note] WSREP: assigned new next trx id: 34687675

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_before_commit: 1, 880471121

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_after_commit: 1, 1, 880471121, 0

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_after_apply 2

2024-05-04  0:02:06 2 [Note] WSREP: wsrep_thd_binlog_reset

2024-05-04  0:02:06 2 [Note] WSREP: Wsrep_high_priority_service::apply_toi: 880471122

2024-05-04  0:02:06 2 [ERROR] Error in Log_event::read_log_event(): 'Found invalid event in binary log', data_len: 42, event_type: -94

2024-05-04  0:02:06 2 [ERROR] WSREP: applier could not read binlog event, seqno: 880471122, len: 102

2024-05-04  0:02:06 2 [Note] WSREP: Error buffer for thd 2 seqno 880471122, 0 bytes: '(null)'

2024-05-04  0:02:06 2 [Note] WSREP: Set WSREPXid for InnoDB:  5ad80727-7fad-11ee-a05e-4e9bdce1cc77:880471122

2024-05-04  0:02:06 0 [Note] WSREP: Member 1(server1) initiates vote on 5ad80727-7fad-11ee-a05e-4e9bdce1cc77:880471122,bab5c85874658610:

2024-05-04  0:02:06 0 [Note] WSREP: Votes over 5ad80727-7fad-11ee-a05e-4e9bdce1cc77:880471122:

   bab5c85874658610:   1/2

Waiting for more votes.

2024-05-04  0:02:06 0 [Note] WSREP: Member 0(server2) responds to vote on 5ad80727-7fad-11ee-a05e-4e9bdce1cc77:880471122,0000000000000000: Success

2024-05-04  0:02:06 0 [Note] WSREP: Votes over 5ad80727-7fad-11ee-a05e-4e9bdce1cc77:880471122:

   0000000000000000:   1/2

   bab5c85874658610:   1/2

Winner: 0000000000000000

2024-05-04  0:02:06 2 [ERROR] WSREP: Inconsistency detected: Inconsistent by consensus on 5ad80727-7fad-11ee-a05e-4e9bdce1cc77:880471122

         at ./galera/src/replicator_smm.cpp:process_apply_error():1351

2024-05-04  0:02:06 2 [Note] WSREP: Closing send monitor...

2024-05-04  0:02:06 2 [Note] WSREP: Closed send monitor.

2024-05-04  0:02:06 2 [Note] WSREP: gcomm: terminating thread

2024-05-04  0:02:06 2 [Note] WSREP: gcomm: joining thread

2024-05-04  0:02:06 2 [Note] WSREP: gcomm: closing backend

2024-05-04  0:02:07 2 [Note] WSREP: view(view_id(NON_PRIM,bf0f2119-9863,12) memb {

        c619ca8c-bbc3,0

} joined {

} left {

} partitioned {

        bf0f2119-9863,0

})

2024-05-04  0:02:07 2 [Note] WSREP: PC protocol downgrade 1 -> 0

2024-05-04  0:02:07 2 [Note] WSREP: view((empty))

2024-05-04  0:02:07 2 [Note] WSREP: gcomm: closed

2024-05-04  0:02:07 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1

2024-05-04  0:02:07 0 [Note] WSREP: Flow-control interval: [16, 16]

2024-05-04  0:02:07 0 [Note] WSREP: Received NON-PRIMARY.

2024-05-04  0:02:07 0 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 880471122)

2024-05-04  0:02:07 0 [Note] WSREP: New SELF-LEAVE.

2024-05-04  0:02:07 0 [Note] WSREP: Flow-control interval: [0, 0]

2024-05-04  0:02:07 0 [Note] WSREP: Received SELF-LEAVE. Closing connection.

2024-05-04  0:02:07 0 [Note] WSREP: Shifting OPEN -> CLOSED (TO: 880471122)

2024-05-04  0:02:07 0 [Note] WSREP: RECV thread exiting 0: Success

2024-05-04  0:02:07 2 [Note] WSREP: recv_thread() joined.

2024-05-04  0:02:07 2 [Note] WSREP: Closing replication queue.

2024-05-04  0:02:07 2 [Note] WSREP: Closing slave action queue.

2024-05-04  0:02:07 2 [Note] WSREP: ================================================

View:

  id: 5ad80727-7fad-11ee-a05e-4e9bdce1cc77:880471122

  status: non-primary

  protocol_version: 4

  capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO

  final: no

  own_index: 0

  members(1):

        0: c619ca8c-05f2-11ef-bbc3-1af43f4299e5, server1

=================================================

2024-05-04  0:02:07 2 [Note] WSREP: Non-primary view

2024-05-04  0:02:07 2 [Note] WSREP: server server1 state change: synced -> connected

2024-05-04  0:02:07 2 [Note] WSREP: Server status change synced -> connected

2024-05-04  0:02:07 2 [Note] WSREP: Setting wsrep_ready to 0

2024-05-04  0:02:07 2 [Note] WSREP: wsrep_notify_status server not yet ready : wsrep_ready=0 status 3

2024-05-04  0:02:07 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.

2024-05-04  0:02:07 2 [Note] WSREP: ================================================

View:

  id: 5ad80727-7fad-11ee-a05e-4e9bdce1cc77:880471122

  status: non-primary

  protocol_version: 4

  capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO

  final: yes

  own_index: -1

  members(0):

=================================================

2024-05-04  0:02:07 2 [Note] WSREP: Non-primary view

2024-05-04  0:02:07 2 [Note] WSREP: server server1 state change: connected -> disconnected

2024-05-04  0:02:07 2 [Note] WSREP: Server status change connected -> disconnected

2024-05-04  0:02:07 2 [Note] WSREP: Setting wsrep_ready to 0

2024-05-04  0:02:07 2 [Note] WSREP: wsrep_notify_status server not yet ready : wsrep_ready=0 status 0

2024-05-04  0:02:07 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.

2024-05-04  0:02:07 0 [Note] WSREP: Service thread queue flushed.

2024-05-04  0:02:07 2 [Note] WSREP: ####### Assign initial position for certification: 00000000-0000-0000-0000-000000000000:-1, protocol version: 5

2024-05-04  0:02:07 2 [Note] WSREP: Applier thread exiting ret: 0 thd: 2

2024-05-04  0:02:07 2 [Note] WSREP: closing applier 2

2024-05-04  0:02:07 2 [Note] WSREP: wsrep system thread: 2, 0x7f4a18000c58 closing

Daniel Czadek added a comment - 2024-05-06 06:39 Hi Jan, now i got some more log with the debug option set: 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_commit_empty for 2 client_state exec client_mode high priority trans_state executing sql NULL 2024-05-04 0:02:06 2 [Note] WSREP: assigned new next trx id: 34687670 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_before_commit: 1, 880471116 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_after_commit: 1, 1, 880471116, 0 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_after_apply 2 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_thd_binlog_reset 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_commit_empty for 2 client_state exec client_mode high priority trans_state executing sql NULL 2024-05-04 0:02:06 2 [Note] WSREP: assigned new next trx id: 34687671 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_before_commit: 1, 880471117 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_after_commit: 1, 1, 880471117, 0 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_after_apply 2 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_thd_binlog_reset 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_commit_empty for 2 client_state exec client_mode high priority trans_state executing sql NULL 2024-05-04 0:02:06 2 [Note] WSREP: assigned new next trx id: 34687672 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_before_commit: 1, 880471118 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_after_commit: 1, 1, 880471118, 0 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_after_apply 2 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_thd_binlog_reset 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_commit_empty for 2 client_state exec client_mode high priority trans_state executing sql NULL 2024-05-04 0:02:06 2 [Note] WSREP: assigned new next trx id: 34687673 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_before_commit: 1, 880471119 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_after_commit: 1, 1, 880471119, 0 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_after_apply 2 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_thd_binlog_reset 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_commit_empty for 2 client_state exec client_mode high priority trans_state executing sql NULL 2024-05-04 0:02:06 2 [Note] WSREP: assigned new next trx id: 34687674 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_before_commit: 1, 880471120 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_after_commit: 1, 1, 880471120, 0 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_after_apply 2 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_thd_binlog_reset 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_commit_empty for 2 client_state exec client_mode high priority trans_state executing sql NULL 2024-05-04 0:02:06 2 [Note] WSREP: assigned new next trx id: 34687675 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_before_commit: 1, 880471121 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_after_commit: 1, 1, 880471121, 0 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_after_apply 2 2024-05-04 0:02:06 2 [Note] WSREP: wsrep_thd_binlog_reset 2024-05-04 0:02:06 2 [Note] WSREP: Wsrep_high_priority_service::apply_toi: 880471122 2024-05-04 0:02:06 2 [ERROR] Error in Log_event::read_log_event(): 'Found invalid event in binary log', data_len: 42, event_type: -94 2024-05-04 0:02:06 2 [ERROR] WSREP: applier could not read binlog event, seqno: 880471122, len: 102 2024-05-04 0:02:06 2 [Note] WSREP: Error buffer for thd 2 seqno 880471122, 0 bytes: '(null)' 2024-05-04 0:02:06 2 [Note] WSREP: Set WSREPXid for InnoDB: 5ad80727-7fad-11ee-a05e-4e9bdce1cc77:880471122 2024-05-04 0:02:06 0 [Note] WSREP: Member 1(server1) initiates vote on 5ad80727-7fad-11ee-a05e-4e9bdce1cc77:880471122,bab5c85874658610: 2024-05-04 0:02:06 0 [Note] WSREP: Votes over 5ad80727-7fad-11ee-a05e-4e9bdce1cc77:880471122: bab5c85874658610: 1/2 Waiting for more votes. 2024-05-04 0:02:06 0 [Note] WSREP: Member 0(server2) responds to vote on 5ad80727-7fad-11ee-a05e-4e9bdce1cc77:880471122,0000000000000000: Success 2024-05-04 0:02:06 0 [Note] WSREP: Votes over 5ad80727-7fad-11ee-a05e-4e9bdce1cc77:880471122: 0000000000000000: 1/2 bab5c85874658610: 1/2 Winner: 0000000000000000 2024-05-04 0:02:06 2 [ERROR] WSREP: Inconsistency detected: Inconsistent by consensus on 5ad80727-7fad-11ee-a05e-4e9bdce1cc77:880471122 at ./galera/src/replicator_smm.cpp:process_apply_error():1351 2024-05-04 0:02:06 2 [Note] WSREP: Closing send monitor... 2024-05-04 0:02:06 2 [Note] WSREP: Closed send monitor. 2024-05-04 0:02:06 2 [Note] WSREP: gcomm: terminating thread 2024-05-04 0:02:06 2 [Note] WSREP: gcomm: joining thread 2024-05-04 0:02:06 2 [Note] WSREP: gcomm: closing backend 2024-05-04 0:02:07 2 [Note] WSREP: view(view_id(NON_PRIM,bf0f2119-9863,12) memb { c619ca8c-bbc3,0 } joined { } left { } partitioned { bf0f2119-9863,0 }) 2024-05-04 0:02:07 2 [Note] WSREP: PC protocol downgrade 1 -> 0 2024-05-04 0:02:07 2 [Note] WSREP: view((empty)) 2024-05-04 0:02:07 2 [Note] WSREP: gcomm: closed 2024-05-04 0:02:07 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1 2024-05-04 0:02:07 0 [Note] WSREP: Flow-control interval: [16, 16] 2024-05-04 0:02:07 0 [Note] WSREP: Received NON-PRIMARY. 2024-05-04 0:02:07 0 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 880471122) 2024-05-04 0:02:07 0 [Note] WSREP: New SELF-LEAVE. 2024-05-04 0:02:07 0 [Note] WSREP: Flow-control interval: [0, 0] 2024-05-04 0:02:07 0 [Note] WSREP: Received SELF-LEAVE. Closing connection. 2024-05-04 0:02:07 0 [Note] WSREP: Shifting OPEN -> CLOSED (TO: 880471122) 2024-05-04 0:02:07 0 [Note] WSREP: RECV thread exiting 0: Success 2024-05-04 0:02:07 2 [Note] WSREP: recv_thread() joined. 2024-05-04 0:02:07 2 [Note] WSREP: Closing replication queue. 2024-05-04 0:02:07 2 [Note] WSREP: Closing slave action queue. 2024-05-04 0:02:07 2 [Note] WSREP: ================================================ View: id: 5ad80727-7fad-11ee-a05e-4e9bdce1cc77:880471122 status: non-primary protocol_version: 4 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: no own_index: 0 members(1): 0: c619ca8c-05f2-11ef-bbc3-1af43f4299e5, server1 ================================================= 2024-05-04 0:02:07 2 [Note] WSREP: Non-primary view 2024-05-04 0:02:07 2 [Note] WSREP: server server1 state change: synced -> connected 2024-05-04 0:02:07 2 [Note] WSREP: Server status change synced -> connected 2024-05-04 0:02:07 2 [Note] WSREP: Setting wsrep_ready to 0 2024-05-04 0:02:07 2 [Note] WSREP: wsrep_notify_status server not yet ready : wsrep_ready=0 status 3 2024-05-04 0:02:07 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2024-05-04 0:02:07 2 [Note] WSREP: ================================================ View: id: 5ad80727-7fad-11ee-a05e-4e9bdce1cc77:880471122 status: non-primary protocol_version: 4 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: yes own_index: -1 members(0): ================================================= 2024-05-04 0:02:07 2 [Note] WSREP: Non-primary view 2024-05-04 0:02:07 2 [Note] WSREP: server server1 state change: connected -> disconnected 2024-05-04 0:02:07 2 [Note] WSREP: Server status change connected -> disconnected 2024-05-04 0:02:07 2 [Note] WSREP: Setting wsrep_ready to 0 2024-05-04 0:02:07 2 [Note] WSREP: wsrep_notify_status server not yet ready : wsrep_ready=0 status 0 2024-05-04 0:02:07 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2024-05-04 0:02:07 0 [Note] WSREP: Service thread queue flushed. 2024-05-04 0:02:07 2 [Note] WSREP: ####### Assign initial position for certification: 00000000-0000-0000-0000-000000000000:-1, protocol version: 5 2024-05-04 0:02:07 2 [Note] WSREP: Applier thread exiting ret: 0 thd: 2 2024-05-04 0:02:07 2 [Note] WSREP: closing applier 2 2024-05-04 0:02:07 2 [Note] WSREP: wsrep system thread: 2, 0x7f4a18000c58 closing

Jan Lindström made changes - 2024-05-06 07:40

Status

Needs Feedback [ 10501 ]

Open [ 1 ]

Jan Lindström added a comment - 2024-05-06 07:57

dcz01 Last error log suggest network problem as event in binary log is corrupted.

Jan Lindström added a comment - 2024-05-06 07:57 dcz01 Last error log suggest network problem as event in binary log is corrupted.

Daniel Czadek added a comment - 2024-05-06 08:29

But normally there isn't any network problem and this just only happens once per week.
I think there would be a problem with the OPTIMIZE TABLE changes coming from mysql 5.7 with replication for migration to new galera cluster.
What can be done then?

Daniel Czadek added a comment - 2024-05-06 08:29 But normally there isn't any network problem and this just only happens once per week. I think there would be a problem with the OPTIMIZE TABLE changes coming from mysql 5.7 with replication for migration to new galera cluster. What can be done then?

Jan Lindström added a comment - 2024-05-06 08:38

dcz01 Do you really need to run OPTIMIZE TABLE? If yes why not do it using SET SESSION wsrep_OSU_method='RSU'; ?

Jan Lindström added a comment - 2024-05-06 08:38 dcz01 Do you really need to run OPTIMIZE TABLE? If yes why not do it using SET SESSION wsrep_OSU_method='RSU'; ?

Daniel Czadek added a comment - 2024-05-06 08:43

Well yes we need to do that every weekend because that are really big databases with a lot of changes every time/day.
So should this change be used always just before the optimisation and then changed back or can it be kept forever?

Daniel Czadek added a comment - 2024-05-06 08:43 Well yes we need to do that every weekend because that are really big databases with a lot of changes every time/day. So should this change be used always just before the optimisation and then changed back or can it be kept forever?

Jan Lindström added a comment - 2024-05-06 09:56

dcz01 This is workaround for current problem. Please read https://galeracluster.com/library/documentation/schema-upgrades.html#rsu. You will then have to do it for every node separately. Or wait for fix to actual issue.

Jan Lindström added a comment - 2024-05-06 09:56 dcz01 This is workaround for current problem. Please read https://galeracluster.com/library/documentation/schema-upgrades.html#rsu . You will then have to do it for every node separately. Or wait for fix to actual issue.

Daniel Czadek added a comment - 2024-05-14 05:30

Now i got some more debug information and i hope it can help.

2024-05-14  7:13:52 1726 [Note] WSREP: cleanup_enter

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local

    trx_id: -1, seqno: -1, flags: 0

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac41eb700

2024-05-14  7:13:52 1726 [Note] WSREP: wsrep_thd_binlog_reset

2024-05-14  7:13:52 1726 [Note] WSREP: cleanup_leave

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local

    trx_id: -1, seqno: -1, flags: 0

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac41eb700

2024-05-14  7:13:52 1726 [Note] WSREP: after_statement_leave

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local

    trx_id: -1, seqno: -1, flags: 0

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac41eb700

2024-05-14  7:13:52 1726 [Note] WSREP: after_statement: success(1726,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:52 1726 [Note] WSREP: after_command_before_result: enter(1726,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:52 1726 [Note] WSREP: after_command_before_result: leave(1726,result,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:52 1726 [Note] WSREP: after_command_after_result_enter(1726,result,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:52 1726 [Note] WSREP: after_command_after_result: leave(1726,idle,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:53 1726 [Note] WSREP: before_command: enter(1726,idle,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:53 1726 [Note] WSREP: before_command: success(1726,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:53 1726 [Note] WSREP: assigned new next trx id: 3452

2024-05-14  7:13:53 1726 [Note] WSREP: start_transaction enter

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local

    trx_id: -1, seqno: -1, flags: 0

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac41eb700

2024-05-14  7:13:53 1726 [Note] WSREP: start_transaction success

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local

    trx_id: 3452, seqno: -1, flags: 1

    state: executing, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac41eb700

2024-05-14  7:13:53 1726 [Note] WSREP: wsrep_commit_empty for 1726 client_state exec client_mode local trans_state executing sql show variables like '%defragment%'

2024-05-14  7:13:53 1726 [Note] WSREP: before_rollback_enter

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local

    trx_id: 3452, seqno: -1, flags: 1

    state: executing, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac41eb700

2024-05-14  7:13:53 1726 [Note] WSREP: client: 1726 txc: 3452 state: executing -> aborting

2024-05-14  7:13:53 1726 [Note] WSREP: before_rollback_leave

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local

    trx_id: 3452, seqno: -1, flags: 1

    state: aborting, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac41eb700

2024-05-14  7:13:53 1726 [Note] WSREP: after_rollback_enter

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local

    trx_id: 3452, seqno: -1, flags: 1

    state: aborting, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac41eb700

2024-05-14  7:13:53 1726 [Note] WSREP: client: 1726 txc: 3452 state: aborting -> aborted

2024-05-14  7:13:53 1726 [Note] WSREP: after_rollback_leave

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local

    trx_id: 3452, seqno: -1, flags: 1

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac41eb700

2024-05-14  7:13:53 1726 [Note] WSREP: wsrep_after_statement for 1726 client_state exec  client_mode local trans_state aborted

2024-05-14  7:13:53 1726 [Note] WSREP: after_statement: enter(1726,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:53 1726 [Note] WSREP: after_statement_enter

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local

    trx_id: 3452, seqno: -1, flags: 1

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac41eb700

2024-05-14  7:13:53 1726 [Note] WSREP: cleanup_enter

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local

    trx_id: 3452, seqno: -1, flags: 1

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac41eb700

2024-05-14  7:13:53 1726 [Note] WSREP: wsrep_thd_binlog_reset

2024-05-14  7:13:53 1726 [Note] WSREP: cleanup_leave

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local

    trx_id: -1, seqno: -1, flags: 0

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac41eb700

2024-05-14  7:13:53 1726 [Note] WSREP: after_statement_leave

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local

    trx_id: -1, seqno: -1, flags: 0

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac41eb700

2024-05-14  7:13:53 1726 [Note] WSREP: after_statement: success(1726,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:53 1726 [Note] WSREP: wsrep_after_statement for 1726 client_state exec  client_mode local trans_state aborted

2024-05-14  7:13:53 1726 [Note] WSREP: after_statement: enter(1726,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:53 1726 [Note] WSREP: after_statement_enter

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local

    trx_id: -1, seqno: -1, flags: 0

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac41eb700

2024-05-14  7:13:53 1726 [Note] WSREP: cleanup_enter

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local

    trx_id: -1, seqno: -1, flags: 0

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac41eb700

2024-05-14  7:13:53 1726 [Note] WSREP: wsrep_thd_binlog_reset

2024-05-14  7:13:53 1726 [Note] WSREP: cleanup_leave

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local

    trx_id: -1, seqno: -1, flags: 0

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac41eb700

2024-05-14  7:13:53 1726 [Note] WSREP: after_statement_leave

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local

    trx_id: -1, seqno: -1, flags: 0

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac41eb700

2024-05-14  7:13:53 1726 [Note] WSREP: after_statement: success(1726,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:53 1726 [Note] WSREP: after_command_before_result: enter(1726,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:53 1726 [Note] WSREP: after_command_before_result: leave(1726,result,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:53 1726 [Note] WSREP: after_command_after_result_enter(1726,result,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:53 1726 [Note] WSREP: after_command_after_result: leave(1726,idle,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:55 1731 [Note] WSREP: open: enter(1730,none,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:55 1731 [Note] WSREP: open: leave(1731,idle,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:55 1731 [Note] WSREP: before_command: enter(1731,idle,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:55 1731 [Note] WSREP: before_command: success(1731,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:55 1731 [Note] WSREP: assigned new next trx id: 3453

2024-05-14  7:13:55 1731 [Note] WSREP: start_transaction enter

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local

    trx_id: -1, seqno: -1, flags: 0

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:13:55 1731 [Note] WSREP: start_transaction success

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local

    trx_id: 3453, seqno: -1, flags: 1

    state: executing, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:13:55 1731 [Note] WSREP: wsrep_commit_empty for 1731 client_state exec client_mode local trans_state executing sql SHOW GLOBAL STATUS LIKE 'wsrep_cluster_size'

2024-05-14  7:13:55 1731 [Note] WSREP: before_rollback_enter

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local

    trx_id: 3453, seqno: -1, flags: 1

    state: executing, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:13:55 1731 [Note] WSREP: client: 1731 txc: 3453 state: executing -> aborting

2024-05-14  7:13:55 1731 [Note] WSREP: before_rollback_leave

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local

    trx_id: 3453, seqno: -1, flags: 1

    state: aborting, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:13:55 1731 [Note] WSREP: after_rollback_enter

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local

    trx_id: 3453, seqno: -1, flags: 1

    state: aborting, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:13:55 1731 [Note] WSREP: client: 1731 txc: 3453 state: aborting -> aborted

2024-05-14  7:13:55 1731 [Note] WSREP: after_rollback_leave

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local

    trx_id: 3453, seqno: -1, flags: 1

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:13:55 1731 [Note] WSREP: wsrep_after_statement for 1731 client_state exec  client_mode local trans_state aborted

2024-05-14  7:13:55 1731 [Note] WSREP: after_statement: enter(1731,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:55 1731 [Note] WSREP: after_statement_enter

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local

    trx_id: 3453, seqno: -1, flags: 1

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:13:55 1731 [Note] WSREP: cleanup_enter

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local

    trx_id: 3453, seqno: -1, flags: 1

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:13:55 1731 [Note] WSREP: wsrep_thd_binlog_reset

2024-05-14  7:13:55 1731 [Note] WSREP: cleanup_leave

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local

    trx_id: -1, seqno: -1, flags: 0

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:13:55 1731 [Note] WSREP: after_statement_leave

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local

    trx_id: -1, seqno: -1, flags: 0

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:13:55 1731 [Note] WSREP: after_statement: success(1731,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:55 1731 [Note] WSREP: wsrep_after_statement for 1731 client_state exec  client_mode local trans_state aborted

2024-05-14  7:13:55 1731 [Note] WSREP: after_statement: enter(1731,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:55 1731 [Note] WSREP: after_statement_enter

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local

    trx_id: -1, seqno: -1, flags: 0

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:13:55 1731 [Note] WSREP: cleanup_enter

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local

    trx_id: -1, seqno: -1, flags: 0

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:13:55 1731 [Note] WSREP: wsrep_thd_binlog_reset

2024-05-14  7:13:55 1731 [Note] WSREP: cleanup_leave

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local

    trx_id: -1, seqno: -1, flags: 0

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:13:55 1731 [Note] WSREP: after_statement_leave

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local

    trx_id: -1, seqno: -1, flags: 0

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:13:55 1731 [Note] WSREP: after_statement: success(1731,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:55 1731 [Note] WSREP: after_command_before_result: enter(1731,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:55 1731 [Note] WSREP: after_command_before_result: leave(1731,result,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:55 1731 [Note] WSREP: after_command_after_result_enter(1731,result,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:55 1731 [Note] WSREP: after_command_after_result: leave(1731,idle,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:55 1731 [Note] WSREP: before_command: enter(1731,idle,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:55 1731 [Note] WSREP: before_command: success(1731,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:55 1731 [Note] WSREP: assigned new next trx id: 3454

2024-05-14  7:13:55 1731 [Note] WSREP: after_command_before_result: enter(1731,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:55 1731 [Note] WSREP: after_command_before_result: leave(1731,result,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:55 1731 [Note] WSREP: after_command_after_result_enter(1731,result,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:55 1731 [Note] WSREP: after_command_after_result: leave(1731,idle,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:55 1731 [Note] WSREP: close: enter(1731,idle,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:55 1731 [Note] WSREP: Disabling streaming

2024-05-14  7:13:55 1731 [Note] WSREP: close: leave(1731,quit,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:55 1731 [Note] WSREP: cleanup: enter(1731,quit,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:13:55 1731 [Note] WSREP: cleanup: leave(1731,none,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:04 1726 [Note] WSREP: before_command: enter(1726,idle,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:04 1726 [Note] WSREP: before_command: success(1726,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:04 1726 [Note] WSREP: assigned new next trx id: 3455

2024-05-14  7:14:04 1726 [Note] WSREP: after_command_before_result: enter(1726,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:04 1726 [Note] WSREP: after_command_before_result: leave(1726,result,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:04 1726 [Note] WSREP: after_command_after_result_enter(1726,result,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:04 1726 [Note] WSREP: after_command_after_result: leave(1726,idle,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:04 1726 [Note] WSREP: close: enter(1726,idle,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:04 1726 [Note] WSREP: Disabling streaming

2024-05-14  7:14:04 1726 [Note] WSREP: close: leave(1726,quit,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:04 1726 [Note] WSREP: cleanup: enter(1726,quit,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:04 1726 [Note] WSREP: cleanup: leave(1726,none,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: open: enter(1731,none,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: open: leave(1732,idle,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: before_command: enter(1732,idle,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: before_command: success(1732,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: assigned new next trx id: 3456

2024-05-14  7:14:16 1732 [Note] WSREP: after_command_before_result: enter(1732,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: after_command_before_result: leave(1732,result,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: after_command_after_result_enter(1732,result,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: after_command_after_result: leave(1732,idle,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: before_command: enter(1732,idle,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: before_command: success(1732,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: assigned new next trx id: 3456

2024-05-14  7:14:16 1732 [Note] WSREP: start_transaction enter

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local

    trx_id: -1, seqno: -1, flags: 0

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:14:16 1732 [Note] WSREP: start_transaction success

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local

    trx_id: 3456, seqno: -1, flags: 1

    state: executing, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:14:16 1732 [Note] WSREP: wsrep_commit_empty for 1732 client_state exec client_mode local trans_state executing sql show global status

2024-05-14  7:14:16 1732 [Note] WSREP: before_rollback_enter

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local

    trx_id: 3456, seqno: -1, flags: 1

    state: executing, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:14:16 1732 [Note] WSREP: client: 1732 txc: 3456 state: executing -> aborting

2024-05-14  7:14:16 1732 [Note] WSREP: before_rollback_leave

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local

    trx_id: 3456, seqno: -1, flags: 1

    state: aborting, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:14:16 1732 [Note] WSREP: after_rollback_enter

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local

    trx_id: 3456, seqno: -1, flags: 1

    state: aborting, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:14:16 1732 [Note] WSREP: client: 1732 txc: 3456 state: aborting -> aborted

2024-05-14  7:14:16 1732 [Note] WSREP: after_rollback_leave

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local

    trx_id: 3456, seqno: -1, flags: 1

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:14:16 1732 [Note] WSREP: wsrep_after_statement for 1732 client_state exec  client_mode local trans_state aborted

2024-05-14  7:14:16 1732 [Note] WSREP: after_statement: enter(1732,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: after_statement_enter

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local

    trx_id: 3456, seqno: -1, flags: 1

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:14:16 1732 [Note] WSREP: cleanup_enter

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local

    trx_id: 3456, seqno: -1, flags: 1

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:14:16 1732 [Note] WSREP: wsrep_thd_binlog_reset

2024-05-14  7:14:16 1732 [Note] WSREP: cleanup_leave

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local

    trx_id: -1, seqno: -1, flags: 0

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:14:16 1732 [Note] WSREP: after_statement_leave

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local

    trx_id: -1, seqno: -1, flags: 0

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:14:16 1732 [Note] WSREP: after_statement: success(1732,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: wsrep_after_statement for 1732 client_state exec  client_mode local trans_state aborted

2024-05-14  7:14:16 1732 [Note] WSREP: after_statement: enter(1732,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: after_statement_enter

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local

    trx_id: -1, seqno: -1, flags: 0

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:14:16 1732 [Note] WSREP: cleanup_enter

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local

    trx_id: -1, seqno: -1, flags: 0

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:14:16 1732 [Note] WSREP: wsrep_thd_binlog_reset

2024-05-14  7:14:16 1732 [Note] WSREP: cleanup_leave

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local

    trx_id: -1, seqno: -1, flags: 0

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:14:16 1732 [Note] WSREP: after_statement_leave

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local

    trx_id: -1, seqno: -1, flags: 0

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac4ff8700

2024-05-14  7:14:16 1732 [Note] WSREP: after_statement: success(1732,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: after_command_before_result: enter(1732,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: after_command_before_result: leave(1732,result,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: after_command_after_result_enter(1732,result,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: after_command_after_result: leave(1732,idle,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: before_command: enter(1732,idle,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: before_command: success(1732,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: assigned new next trx id: 3457

2024-05-14  7:14:16 1732 [Note] WSREP: after_command_before_result: enter(1732,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: after_command_before_result: leave(1732,result,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: after_command_after_result_enter(1732,result,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: after_command_after_result: leave(1732,idle,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: close: enter(1732,idle,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: Disabling streaming

2024-05-14  7:14:16 1732 [Note] WSREP: close: leave(1732,quit,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: cleanup: enter(1732,quit,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:14:16 1732 [Note] WSREP: cleanup: leave(1732,none,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:15:00 2 [Note] WSREP: enter_toi_mode: enter(2,exec,high priority,success,0,toi: -1,nbo: -1)

2024-05-14  7:15:00 2 [Note] WSREP: enter_toi_mode: leave(2,exec,toi,success,0,toi: 13,nbo: -1)

2024-05-14  7:15:00 2 [Note] WSREP: Wsrep_high_priority_service::apply_toi: 13

2024-05-14  7:15:00 2 [ERROR] Error in Log_event::read_log_event(): 'Found invalid event in binary log', data_len: 42, event_type: -94

2024-05-14  7:15:00 2 [ERROR] WSREP: applier could not read binlog event, seqno: 13, len: 123

2024-05-14  7:15:00 2 [Note] WSREP: Error buffer for thd 2 seqno 13, 0 bytes: '(null)'

2024-05-14  7:15:01 2 [Note] WSREP: Set WSREPXid for InnoDB:  2f8df556-1124-11ef-8583-27a3a59319ba:13

2024-05-14  7:15:01 2 [Note] WSREP: leave_toi_mode: enter(2,exec,toi,success,0,toi: 13,nbo: -1)

2024-05-14  7:15:01 2 [Note] WSREP: leave_toi_mode: leave(2,exec,high priority,success,0,toi: -1,nbo: -1)

2024-05-14  7:15:01 0 [Note] WSREP: Member 1(server1) initiates vote on 2f8df556-1124-11ef-8583-27a3a59319ba:13,8ee6eaad8a44af47:

2024-05-14  7:15:01 0 [Note] WSREP: Votes over 2f8df556-1124-11ef-8583-27a3a59319ba:13:

   8ee6eaad8a44af47:   1/2

Waiting for more votes.

2024-05-14  7:15:01 0 [Note] WSREP: Member 0(server2) responds to vote on 2f8df556-1124-11ef-8583-27a3a59319ba:13,0000000000000000: Success

2024-05-14  7:15:01 0 [Note] WSREP: Votes over 2f8df556-1124-11ef-8583-27a3a59319ba:13:

   0000000000000000:   1/2

   8ee6eaad8a44af47:   1/2

Winner: 0000000000000000

2024-05-14  7:15:01 2 [ERROR] WSREP: Inconsistency detected: Inconsistent by consensus on 2f8df556-1124-11ef-8583-27a3a59319ba:13

         at ./galera/src/replicator_smm.cpp:process_apply_error():1351

2024-05-14  7:15:01 2 [Note] WSREP: Closing send monitor...

2024-05-14  7:15:01 2 [Note] WSREP: Closed send monitor.

2024-05-14  7:15:01 2 [Note] WSREP: gcomm: terminating thread

2024-05-14  7:15:01 2 [Note] WSREP: gcomm: joining thread

2024-05-14  7:15:01 2 [Note] WSREP: gcomm: closing backend

2024-05-14  7:15:02 2 [Note] WSREP: view(view_id(NON_PRIM,2f8d24c7-bcd9,53) memb {

        f0261506-9e8b,0

} joined {

} left {

} partitioned {

        2f8d24c7-bcd9,0

})

2024-05-14  7:15:02 2 [Note] WSREP: PC protocol downgrade 1 -> 0

2024-05-14  7:15:02 2 [Note] WSREP: view((empty))

2024-05-14  7:15:02 2 [Note] WSREP: gcomm: closed

2024-05-14  7:15:02 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1

2024-05-14  7:15:02 0 [Note] WSREP: Flow-control interval: [16, 16]

2024-05-14  7:15:02 0 [Note] WSREP: Received NON-PRIMARY.

2024-05-14  7:15:02 0 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 14)

2024-05-14  7:15:02 0 [Note] WSREP: New SELF-LEAVE.

2024-05-14  7:15:02 0 [Note] WSREP: Flow-control interval: [0, 0]

2024-05-14  7:15:02 0 [Note] WSREP: Received SELF-LEAVE. Closing connection.

2024-05-14  7:15:02 0 [Note] WSREP: Shifting OPEN -> CLOSED (TO: 14)

2024-05-14  7:15:02 0 [Note] WSREP: RECV thread exiting 0: Success

2024-05-14  7:15:02 2 [Note] WSREP: recv_thread() joined.

2024-05-14  7:15:02 2 [Note] WSREP: Closing replication queue.

2024-05-14  7:15:02 2 [Note] WSREP: Closing slave action queue.

2024-05-14  7:15:02 2 [Note] WSREP: ================================================

View:

  id: 2f8df556-1124-11ef-8583-27a3a59319ba:14

  status: non-primary

  protocol_version: 4

  capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO

  final: no

  own_index: 0

  members(1):

        0: f0261506-114c-11ef-9e8b-aff268655bd6, server1

=================================================

2024-05-14  7:15:02 2 [Note] WSREP: Non-primary view

2024-05-14  7:15:02 2 [Note] WSREP: server server1 state change: synced -> connected

2024-05-14  7:15:02 2 [Note] WSREP: Server status change synced -> connected

2024-05-14  7:15:02 2 [Note] WSREP: Setting wsrep_ready to 0

2024-05-14  7:15:02 2 [Note] WSREP: wsrep_notify_status server not yet ready : wsrep_ready=0 status 3

2024-05-14  7:15:02 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.

2024-05-14  7:15:02 2 [Note] WSREP: ================================================

View:

  id: 2f8df556-1124-11ef-8583-27a3a59319ba:14

  status: non-primary

  protocol_version: 4

  capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO

  final: yes

  own_index: -1

  members(0):

=================================================

2024-05-14  7:15:02 2 [Note] WSREP: Non-primary view

2024-05-14  7:15:02 2 [Note] WSREP: server server1 state change: connected -> disconnected

2024-05-14  7:15:02 2 [Note] WSREP: Server status change connected -> disconnected

2024-05-14  7:15:02 2 [Note] WSREP: Setting wsrep_ready to 0

2024-05-14  7:15:02 2 [Note] WSREP: wsrep_notify_status server not yet ready : wsrep_ready=0 status 0

2024-05-14  7:15:02 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.

2024-05-14  7:15:02 0 [Note] WSREP: Service thread queue flushed.

2024-05-14  7:15:02 2 [Note] WSREP: ####### Assign initial position for certification: 00000000-0000-0000-0000-000000000000:-1, protocol version: 5

2024-05-14  7:15:02 2 [Note] WSREP: Applier thread exiting ret: 0 thd: 2

2024-05-14  7:15:02 2 [Note] WSREP: closing applier 2

2024-05-14  7:15:02 2 [Note] WSREP: after_command_before_result: enter(2,exec,high priority,success,0,toi: -1,nbo: -1)

2024-05-14  7:15:02 2 [Note] WSREP: after_command_before_result: leave(2,result,high priority,success,0,toi: -1,nbo: -1)

2024-05-14  7:15:02 2 [Note] WSREP: after_command_after_result_enter(2,result,high priority,success,0,toi: -1,nbo: -1)

2024-05-14  7:15:02 2 [Note] WSREP: after_command_after_result: leave(2,idle,high priority,success,0,toi: -1,nbo: -1)

2024-05-14  7:15:02 2 [Note] WSREP: close: enter(2,idle,high priority,success,0,toi: -1,nbo: -1)

2024-05-14  7:15:02 2 [Note] WSREP: close: leave(2,quit,high priority,success,0,toi: -1,nbo: -1)

2024-05-14  7:15:02 2 [Note] WSREP: cleanup: enter(2,quit,high priority,success,0,toi: -1,nbo: -1)

2024-05-14  7:15:02 2 [Note] WSREP: cleanup: leave(2,none,high priority,success,0,toi: -1,nbo: -1)

2024-05-14  7:15:02 2 [Note] WSREP: wsrep system thread: 2, 0x7f7ab0000c58 closing

2024-05-14  7:15:02 2 [Warning] Aborted connection 2 to db: 'unconnected' user: 'unauthenticated' host: '' (This connection closed normally without authentication)

2024-05-14  7:15:02 2 [Note] WSREP: wsrep running threads now: 1

2024-05-14  7:15:14 1733 [Note] WSREP: open: enter(1726,none,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:15:14 1733 [Note] WSREP: open: leave(1733,idle,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:15:14 1733 [Note] WSREP: before_command: enter(1733,idle,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:15:14 1733 [Note] WSREP: before_command: success(1733,exec,local,success,0,toi: -1,nbo: -1)

2024-05-14  7:15:14 1733 [Note] WSREP: assigned new next trx id: 3458

2024-05-14  7:15:14 1733 [Note] WSREP: start_transaction enter

    server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1733, state: exec, mode: local

    trx_id: -1, seqno: -1, flags: 0

    state: aborted, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac41eb700

2024-05-14  7:15:14 1733 [Note] WSREP: start_transaction success

    server: 00000000-0000-0000-0000-000000000000, client: 1733, state: exec, mode: local

    trx_id: 3458, seqno: -1, flags: 1

    state: executing, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

    own: 1 thread_id: 7f7ac41eb700

2024-05-14  7:15:14 1733 [Note] WSREP: wsrep_commit_empty for 1733 client_state exec client_mode local trans_state executing sql show /*!40003 GLOBAL */ variables

2024-05-14  7:15:14 1733 [Note] WSREP: before_rollback_enter

    server: 00000000-0000-0000-0000-000000000000, client: 1733, state: exec, mode: local

    trx_id: 3458, seqno: -1, flags: 1

    state: executing, bfa_state: executing, error: success, status: 0

    is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

Daniel Czadek added a comment - 2024-05-14 05:30 Now i got some more debug information and i hope it can help. 2024-05-14 7:13:52 1726 [Note] WSREP: cleanup_enter server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local trx_id: -1, seqno: -1, flags: 0 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac41eb700 2024-05-14 7:13:52 1726 [Note] WSREP: wsrep_thd_binlog_reset 2024-05-14 7:13:52 1726 [Note] WSREP: cleanup_leave server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local trx_id: -1, seqno: -1, flags: 0 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac41eb700 2024-05-14 7:13:52 1726 [Note] WSREP: after_statement_leave server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local trx_id: -1, seqno: -1, flags: 0 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac41eb700 2024-05-14 7:13:52 1726 [Note] WSREP: after_statement: success(1726,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:52 1726 [Note] WSREP: after_command_before_result: enter(1726,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:52 1726 [Note] WSREP: after_command_before_result: leave(1726,result,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:52 1726 [Note] WSREP: after_command_after_result_enter(1726,result,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:52 1726 [Note] WSREP: after_command_after_result: leave(1726,idle,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:53 1726 [Note] WSREP: before_command: enter(1726,idle,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:53 1726 [Note] WSREP: before_command: success(1726,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:53 1726 [Note] WSREP: assigned new next trx id: 3452 2024-05-14 7:13:53 1726 [Note] WSREP: start_transaction enter server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local trx_id: -1, seqno: -1, flags: 0 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac41eb700 2024-05-14 7:13:53 1726 [Note] WSREP: start_transaction success server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local trx_id: 3452, seqno: -1, flags: 1 state: executing, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac41eb700 2024-05-14 7:13:53 1726 [Note] WSREP: wsrep_commit_empty for 1726 client_state exec client_mode local trans_state executing sql show variables like '%defragment%' 2024-05-14 7:13:53 1726 [Note] WSREP: before_rollback_enter server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local trx_id: 3452, seqno: -1, flags: 1 state: executing, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac41eb700 2024-05-14 7:13:53 1726 [Note] WSREP: client: 1726 txc: 3452 state: executing -> aborting 2024-05-14 7:13:53 1726 [Note] WSREP: before_rollback_leave server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local trx_id: 3452, seqno: -1, flags: 1 state: aborting, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac41eb700 2024-05-14 7:13:53 1726 [Note] WSREP: after_rollback_enter server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local trx_id: 3452, seqno: -1, flags: 1 state: aborting, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac41eb700 2024-05-14 7:13:53 1726 [Note] WSREP: client: 1726 txc: 3452 state: aborting -> aborted 2024-05-14 7:13:53 1726 [Note] WSREP: after_rollback_leave server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local trx_id: 3452, seqno: -1, flags: 1 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac41eb700 2024-05-14 7:13:53 1726 [Note] WSREP: wsrep_after_statement for 1726 client_state exec client_mode local trans_state aborted 2024-05-14 7:13:53 1726 [Note] WSREP: after_statement: enter(1726,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:53 1726 [Note] WSREP: after_statement_enter server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local trx_id: 3452, seqno: -1, flags: 1 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac41eb700 2024-05-14 7:13:53 1726 [Note] WSREP: cleanup_enter server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local trx_id: 3452, seqno: -1, flags: 1 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac41eb700 2024-05-14 7:13:53 1726 [Note] WSREP: wsrep_thd_binlog_reset 2024-05-14 7:13:53 1726 [Note] WSREP: cleanup_leave server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local trx_id: -1, seqno: -1, flags: 0 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac41eb700 2024-05-14 7:13:53 1726 [Note] WSREP: after_statement_leave server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local trx_id: -1, seqno: -1, flags: 0 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac41eb700 2024-05-14 7:13:53 1726 [Note] WSREP: after_statement: success(1726,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:53 1726 [Note] WSREP: wsrep_after_statement for 1726 client_state exec client_mode local trans_state aborted 2024-05-14 7:13:53 1726 [Note] WSREP: after_statement: enter(1726,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:53 1726 [Note] WSREP: after_statement_enter server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local trx_id: -1, seqno: -1, flags: 0 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac41eb700 2024-05-14 7:13:53 1726 [Note] WSREP: cleanup_enter server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local trx_id: -1, seqno: -1, flags: 0 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac41eb700 2024-05-14 7:13:53 1726 [Note] WSREP: wsrep_thd_binlog_reset 2024-05-14 7:13:53 1726 [Note] WSREP: cleanup_leave server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local trx_id: -1, seqno: -1, flags: 0 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac41eb700 2024-05-14 7:13:53 1726 [Note] WSREP: after_statement_leave server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1726, state: exec, mode: local trx_id: -1, seqno: -1, flags: 0 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac41eb700 2024-05-14 7:13:53 1726 [Note] WSREP: after_statement: success(1726,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:53 1726 [Note] WSREP: after_command_before_result: enter(1726,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:53 1726 [Note] WSREP: after_command_before_result: leave(1726,result,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:53 1726 [Note] WSREP: after_command_after_result_enter(1726,result,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:53 1726 [Note] WSREP: after_command_after_result: leave(1726,idle,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:55 1731 [Note] WSREP: open: enter(1730,none,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:55 1731 [Note] WSREP: open: leave(1731,idle,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:55 1731 [Note] WSREP: before_command: enter(1731,idle,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:55 1731 [Note] WSREP: before_command: success(1731,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:55 1731 [Note] WSREP: assigned new next trx id: 3453 2024-05-14 7:13:55 1731 [Note] WSREP: start_transaction enter server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local trx_id: -1, seqno: -1, flags: 0 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:13:55 1731 [Note] WSREP: start_transaction success server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local trx_id: 3453, seqno: -1, flags: 1 state: executing, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:13:55 1731 [Note] WSREP: wsrep_commit_empty for 1731 client_state exec client_mode local trans_state executing sql SHOW GLOBAL STATUS LIKE 'wsrep_cluster_size' 2024-05-14 7:13:55 1731 [Note] WSREP: before_rollback_enter server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local trx_id: 3453, seqno: -1, flags: 1 state: executing, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:13:55 1731 [Note] WSREP: client: 1731 txc: 3453 state: executing -> aborting 2024-05-14 7:13:55 1731 [Note] WSREP: before_rollback_leave server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local trx_id: 3453, seqno: -1, flags: 1 state: aborting, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:13:55 1731 [Note] WSREP: after_rollback_enter server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local trx_id: 3453, seqno: -1, flags: 1 state: aborting, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:13:55 1731 [Note] WSREP: client: 1731 txc: 3453 state: aborting -> aborted 2024-05-14 7:13:55 1731 [Note] WSREP: after_rollback_leave server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local trx_id: 3453, seqno: -1, flags: 1 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:13:55 1731 [Note] WSREP: wsrep_after_statement for 1731 client_state exec client_mode local trans_state aborted 2024-05-14 7:13:55 1731 [Note] WSREP: after_statement: enter(1731,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:55 1731 [Note] WSREP: after_statement_enter server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local trx_id: 3453, seqno: -1, flags: 1 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:13:55 1731 [Note] WSREP: cleanup_enter server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local trx_id: 3453, seqno: -1, flags: 1 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:13:55 1731 [Note] WSREP: wsrep_thd_binlog_reset 2024-05-14 7:13:55 1731 [Note] WSREP: cleanup_leave server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local trx_id: -1, seqno: -1, flags: 0 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:13:55 1731 [Note] WSREP: after_statement_leave server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local trx_id: -1, seqno: -1, flags: 0 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:13:55 1731 [Note] WSREP: after_statement: success(1731,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:55 1731 [Note] WSREP: wsrep_after_statement for 1731 client_state exec client_mode local trans_state aborted 2024-05-14 7:13:55 1731 [Note] WSREP: after_statement: enter(1731,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:55 1731 [Note] WSREP: after_statement_enter server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local trx_id: -1, seqno: -1, flags: 0 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:13:55 1731 [Note] WSREP: cleanup_enter server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local trx_id: -1, seqno: -1, flags: 0 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:13:55 1731 [Note] WSREP: wsrep_thd_binlog_reset 2024-05-14 7:13:55 1731 [Note] WSREP: cleanup_leave server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local trx_id: -1, seqno: -1, flags: 0 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:13:55 1731 [Note] WSREP: after_statement_leave server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1731, state: exec, mode: local trx_id: -1, seqno: -1, flags: 0 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:13:55 1731 [Note] WSREP: after_statement: success(1731,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:55 1731 [Note] WSREP: after_command_before_result: enter(1731,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:55 1731 [Note] WSREP: after_command_before_result: leave(1731,result,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:55 1731 [Note] WSREP: after_command_after_result_enter(1731,result,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:55 1731 [Note] WSREP: after_command_after_result: leave(1731,idle,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:55 1731 [Note] WSREP: before_command: enter(1731,idle,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:55 1731 [Note] WSREP: before_command: success(1731,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:55 1731 [Note] WSREP: assigned new next trx id: 3454 2024-05-14 7:13:55 1731 [Note] WSREP: after_command_before_result: enter(1731,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:55 1731 [Note] WSREP: after_command_before_result: leave(1731,result,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:55 1731 [Note] WSREP: after_command_after_result_enter(1731,result,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:55 1731 [Note] WSREP: after_command_after_result: leave(1731,idle,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:55 1731 [Note] WSREP: close: enter(1731,idle,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:55 1731 [Note] WSREP: Disabling streaming 2024-05-14 7:13:55 1731 [Note] WSREP: close: leave(1731,quit,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:55 1731 [Note] WSREP: cleanup: enter(1731,quit,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:13:55 1731 [Note] WSREP: cleanup: leave(1731,none,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:04 1726 [Note] WSREP: before_command: enter(1726,idle,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:04 1726 [Note] WSREP: before_command: success(1726,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:04 1726 [Note] WSREP: assigned new next trx id: 3455 2024-05-14 7:14:04 1726 [Note] WSREP: after_command_before_result: enter(1726,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:04 1726 [Note] WSREP: after_command_before_result: leave(1726,result,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:04 1726 [Note] WSREP: after_command_after_result_enter(1726,result,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:04 1726 [Note] WSREP: after_command_after_result: leave(1726,idle,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:04 1726 [Note] WSREP: close: enter(1726,idle,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:04 1726 [Note] WSREP: Disabling streaming 2024-05-14 7:14:04 1726 [Note] WSREP: close: leave(1726,quit,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:04 1726 [Note] WSREP: cleanup: enter(1726,quit,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:04 1726 [Note] WSREP: cleanup: leave(1726,none,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: open: enter(1731,none,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: open: leave(1732,idle,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: before_command: enter(1732,idle,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: before_command: success(1732,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: assigned new next trx id: 3456 2024-05-14 7:14:16 1732 [Note] WSREP: after_command_before_result: enter(1732,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: after_command_before_result: leave(1732,result,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: after_command_after_result_enter(1732,result,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: after_command_after_result: leave(1732,idle,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: before_command: enter(1732,idle,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: before_command: success(1732,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: assigned new next trx id: 3456 2024-05-14 7:14:16 1732 [Note] WSREP: start_transaction enter server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local trx_id: -1, seqno: -1, flags: 0 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:14:16 1732 [Note] WSREP: start_transaction success server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local trx_id: 3456, seqno: -1, flags: 1 state: executing, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:14:16 1732 [Note] WSREP: wsrep_commit_empty for 1732 client_state exec client_mode local trans_state executing sql show global status 2024-05-14 7:14:16 1732 [Note] WSREP: before_rollback_enter server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local trx_id: 3456, seqno: -1, flags: 1 state: executing, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:14:16 1732 [Note] WSREP: client: 1732 txc: 3456 state: executing -> aborting 2024-05-14 7:14:16 1732 [Note] WSREP: before_rollback_leave server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local trx_id: 3456, seqno: -1, flags: 1 state: aborting, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:14:16 1732 [Note] WSREP: after_rollback_enter server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local trx_id: 3456, seqno: -1, flags: 1 state: aborting, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:14:16 1732 [Note] WSREP: client: 1732 txc: 3456 state: aborting -> aborted 2024-05-14 7:14:16 1732 [Note] WSREP: after_rollback_leave server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local trx_id: 3456, seqno: -1, flags: 1 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:14:16 1732 [Note] WSREP: wsrep_after_statement for 1732 client_state exec client_mode local trans_state aborted 2024-05-14 7:14:16 1732 [Note] WSREP: after_statement: enter(1732,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: after_statement_enter server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local trx_id: 3456, seqno: -1, flags: 1 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:14:16 1732 [Note] WSREP: cleanup_enter server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local trx_id: 3456, seqno: -1, flags: 1 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:14:16 1732 [Note] WSREP: wsrep_thd_binlog_reset 2024-05-14 7:14:16 1732 [Note] WSREP: cleanup_leave server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local trx_id: -1, seqno: -1, flags: 0 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:14:16 1732 [Note] WSREP: after_statement_leave server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local trx_id: -1, seqno: -1, flags: 0 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:14:16 1732 [Note] WSREP: after_statement: success(1732,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: wsrep_after_statement for 1732 client_state exec client_mode local trans_state aborted 2024-05-14 7:14:16 1732 [Note] WSREP: after_statement: enter(1732,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: after_statement_enter server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local trx_id: -1, seqno: -1, flags: 0 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:14:16 1732 [Note] WSREP: cleanup_enter server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local trx_id: -1, seqno: -1, flags: 0 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:14:16 1732 [Note] WSREP: wsrep_thd_binlog_reset 2024-05-14 7:14:16 1732 [Note] WSREP: cleanup_leave server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local trx_id: -1, seqno: -1, flags: 0 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:14:16 1732 [Note] WSREP: after_statement_leave server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1732, state: exec, mode: local trx_id: -1, seqno: -1, flags: 0 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac4ff8700 2024-05-14 7:14:16 1732 [Note] WSREP: after_statement: success(1732,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: after_command_before_result: enter(1732,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: after_command_before_result: leave(1732,result,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: after_command_after_result_enter(1732,result,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: after_command_after_result: leave(1732,idle,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: before_command: enter(1732,idle,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: before_command: success(1732,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: assigned new next trx id: 3457 2024-05-14 7:14:16 1732 [Note] WSREP: after_command_before_result: enter(1732,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: after_command_before_result: leave(1732,result,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: after_command_after_result_enter(1732,result,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: after_command_after_result: leave(1732,idle,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: close: enter(1732,idle,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: Disabling streaming 2024-05-14 7:14:16 1732 [Note] WSREP: close: leave(1732,quit,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: cleanup: enter(1732,quit,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:14:16 1732 [Note] WSREP: cleanup: leave(1732,none,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:15:00 2 [Note] WSREP: enter_toi_mode: enter(2,exec,high priority,success,0,toi: -1,nbo: -1) 2024-05-14 7:15:00 2 [Note] WSREP: enter_toi_mode: leave(2,exec,toi,success,0,toi: 13,nbo: -1) 2024-05-14 7:15:00 2 [Note] WSREP: Wsrep_high_priority_service::apply_toi: 13 2024-05-14 7:15:00 2 [ERROR] Error in Log_event::read_log_event(): 'Found invalid event in binary log', data_len: 42, event_type: -94 2024-05-14 7:15:00 2 [ERROR] WSREP: applier could not read binlog event, seqno: 13, len: 123 2024-05-14 7:15:00 2 [Note] WSREP: Error buffer for thd 2 seqno 13, 0 bytes: '(null)' 2024-05-14 7:15:01 2 [Note] WSREP: Set WSREPXid for InnoDB: 2f8df556-1124-11ef-8583-27a3a59319ba:13 2024-05-14 7:15:01 2 [Note] WSREP: leave_toi_mode: enter(2,exec,toi,success,0,toi: 13,nbo: -1) 2024-05-14 7:15:01 2 [Note] WSREP: leave_toi_mode: leave(2,exec,high priority,success,0,toi: -1,nbo: -1) 2024-05-14 7:15:01 0 [Note] WSREP: Member 1(server1) initiates vote on 2f8df556-1124-11ef-8583-27a3a59319ba:13,8ee6eaad8a44af47: 2024-05-14 7:15:01 0 [Note] WSREP: Votes over 2f8df556-1124-11ef-8583-27a3a59319ba:13: 8ee6eaad8a44af47: 1/2 Waiting for more votes. 2024-05-14 7:15:01 0 [Note] WSREP: Member 0(server2) responds to vote on 2f8df556-1124-11ef-8583-27a3a59319ba:13,0000000000000000: Success 2024-05-14 7:15:01 0 [Note] WSREP: Votes over 2f8df556-1124-11ef-8583-27a3a59319ba:13: 0000000000000000: 1/2 8ee6eaad8a44af47: 1/2 Winner: 0000000000000000 2024-05-14 7:15:01 2 [ERROR] WSREP: Inconsistency detected: Inconsistent by consensus on 2f8df556-1124-11ef-8583-27a3a59319ba:13 at ./galera/src/replicator_smm.cpp:process_apply_error():1351 2024-05-14 7:15:01 2 [Note] WSREP: Closing send monitor... 2024-05-14 7:15:01 2 [Note] WSREP: Closed send monitor. 2024-05-14 7:15:01 2 [Note] WSREP: gcomm: terminating thread 2024-05-14 7:15:01 2 [Note] WSREP: gcomm: joining thread 2024-05-14 7:15:01 2 [Note] WSREP: gcomm: closing backend 2024-05-14 7:15:02 2 [Note] WSREP: view(view_id(NON_PRIM,2f8d24c7-bcd9,53) memb { f0261506-9e8b,0 } joined { } left { } partitioned { 2f8d24c7-bcd9,0 }) 2024-05-14 7:15:02 2 [Note] WSREP: PC protocol downgrade 1 -> 0 2024-05-14 7:15:02 2 [Note] WSREP: view((empty)) 2024-05-14 7:15:02 2 [Note] WSREP: gcomm: closed 2024-05-14 7:15:02 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1 2024-05-14 7:15:02 0 [Note] WSREP: Flow-control interval: [16, 16] 2024-05-14 7:15:02 0 [Note] WSREP: Received NON-PRIMARY. 2024-05-14 7:15:02 0 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 14) 2024-05-14 7:15:02 0 [Note] WSREP: New SELF-LEAVE. 2024-05-14 7:15:02 0 [Note] WSREP: Flow-control interval: [0, 0] 2024-05-14 7:15:02 0 [Note] WSREP: Received SELF-LEAVE. Closing connection. 2024-05-14 7:15:02 0 [Note] WSREP: Shifting OPEN -> CLOSED (TO: 14) 2024-05-14 7:15:02 0 [Note] WSREP: RECV thread exiting 0: Success 2024-05-14 7:15:02 2 [Note] WSREP: recv_thread() joined. 2024-05-14 7:15:02 2 [Note] WSREP: Closing replication queue. 2024-05-14 7:15:02 2 [Note] WSREP: Closing slave action queue. 2024-05-14 7:15:02 2 [Note] WSREP: ================================================ View: id: 2f8df556-1124-11ef-8583-27a3a59319ba:14 status: non-primary protocol_version: 4 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: no own_index: 0 members(1): 0: f0261506-114c-11ef-9e8b-aff268655bd6, server1 ================================================= 2024-05-14 7:15:02 2 [Note] WSREP: Non-primary view 2024-05-14 7:15:02 2 [Note] WSREP: server server1 state change: synced -> connected 2024-05-14 7:15:02 2 [Note] WSREP: Server status change synced -> connected 2024-05-14 7:15:02 2 [Note] WSREP: Setting wsrep_ready to 0 2024-05-14 7:15:02 2 [Note] WSREP: wsrep_notify_status server not yet ready : wsrep_ready=0 status 3 2024-05-14 7:15:02 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2024-05-14 7:15:02 2 [Note] WSREP: ================================================ View: id: 2f8df556-1124-11ef-8583-27a3a59319ba:14 status: non-primary protocol_version: 4 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: yes own_index: -1 members(0): ================================================= 2024-05-14 7:15:02 2 [Note] WSREP: Non-primary view 2024-05-14 7:15:02 2 [Note] WSREP: server server1 state change: connected -> disconnected 2024-05-14 7:15:02 2 [Note] WSREP: Server status change connected -> disconnected 2024-05-14 7:15:02 2 [Note] WSREP: Setting wsrep_ready to 0 2024-05-14 7:15:02 2 [Note] WSREP: wsrep_notify_status server not yet ready : wsrep_ready=0 status 0 2024-05-14 7:15:02 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2024-05-14 7:15:02 0 [Note] WSREP: Service thread queue flushed. 2024-05-14 7:15:02 2 [Note] WSREP: ####### Assign initial position for certification: 00000000-0000-0000-0000-000000000000:-1, protocol version: 5 2024-05-14 7:15:02 2 [Note] WSREP: Applier thread exiting ret: 0 thd: 2 2024-05-14 7:15:02 2 [Note] WSREP: closing applier 2 2024-05-14 7:15:02 2 [Note] WSREP: after_command_before_result: enter(2,exec,high priority,success,0,toi: -1,nbo: -1) 2024-05-14 7:15:02 2 [Note] WSREP: after_command_before_result: leave(2,result,high priority,success,0,toi: -1,nbo: -1) 2024-05-14 7:15:02 2 [Note] WSREP: after_command_after_result_enter(2,result,high priority,success,0,toi: -1,nbo: -1) 2024-05-14 7:15:02 2 [Note] WSREP: after_command_after_result: leave(2,idle,high priority,success,0,toi: -1,nbo: -1) 2024-05-14 7:15:02 2 [Note] WSREP: close: enter(2,idle,high priority,success,0,toi: -1,nbo: -1) 2024-05-14 7:15:02 2 [Note] WSREP: close: leave(2,quit,high priority,success,0,toi: -1,nbo: -1) 2024-05-14 7:15:02 2 [Note] WSREP: cleanup: enter(2,quit,high priority,success,0,toi: -1,nbo: -1) 2024-05-14 7:15:02 2 [Note] WSREP: cleanup: leave(2,none,high priority,success,0,toi: -1,nbo: -1) 2024-05-14 7:15:02 2 [Note] WSREP: wsrep system thread: 2, 0x7f7ab0000c58 closing 2024-05-14 7:15:02 2 [Warning] Aborted connection 2 to db: 'unconnected' user: 'unauthenticated' host: '' (This connection closed normally without authentication) 2024-05-14 7:15:02 2 [Note] WSREP: wsrep running threads now: 1 2024-05-14 7:15:14 1733 [Note] WSREP: open: enter(1726,none,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:15:14 1733 [Note] WSREP: open: leave(1733,idle,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:15:14 1733 [Note] WSREP: before_command: enter(1733,idle,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:15:14 1733 [Note] WSREP: before_command: success(1733,exec,local,success,0,toi: -1,nbo: -1) 2024-05-14 7:15:14 1733 [Note] WSREP: assigned new next trx id: 3458 2024-05-14 7:15:14 1733 [Note] WSREP: start_transaction enter server: f0261506-114c-11ef-9e8b-aff268655bd6, client: 1733, state: exec, mode: local trx_id: -1, seqno: -1, flags: 0 state: aborted, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac41eb700 2024-05-14 7:15:14 1733 [Note] WSREP: start_transaction success server: 00000000-0000-0000-0000-000000000000, client: 1733, state: exec, mode: local trx_id: 3458, seqno: -1, flags: 1 state: executing, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0 own: 1 thread_id: 7f7ac41eb700 2024-05-14 7:15:14 1733 [Note] WSREP: wsrep_commit_empty for 1733 client_state exec client_mode local trans_state executing sql show /*!40003 GLOBAL */ variables 2024-05-14 7:15:14 1733 [Note] WSREP: before_rollback_enter server: 00000000-0000-0000-0000-000000000000, client: 1733, state: exec, mode: local trx_id: 3458, seqno: -1, flags: 1 state: executing, bfa_state: executing, error: success, status: 0 is_sr: 0, frags: 0, frags size: 0, unit: 0, size: 0, counter: 0, log_pos: 0, sr_rb: 0

Jan Lindström added a comment - 2024-05-14 06:46

dcz01 Can you attach full error log, node configuration files and binlog files from all nodes.

Jan Lindström added a comment - 2024-05-14 06:46 dcz01 Can you attach full error log, node configuration files and binlog files from all nodes.

Jan Lindström made changes - 2024-05-14 06:46

Status

Open [ 1 ]

Needs Feedback [ 10501 ]

Daniel Czadek added a comment - 2024-05-14 09:33

Well now i tested again on our test cluster and it seems that the crash or disconnect from wsrep comes from the OPTIMIZE TABLE:

xxx.erp_calculationbase_types_fra

note     : Table does not support optimize, doing recreate + analyze instead

status   : OK

xxx.erp_calculationbase_types_lcs

note     : Table does not support optimize, doing recreate + analyze instead

status   : OK

xxx.erp_date_types

note     : Table does not support optimize, doing recreate + analyze instead

status   : OK

xxx.erp_date_types_deu

note     : Table does not support optimize, doing recreate + analyze instead

status   : OK

xxx.erp_date_types_eng

note     : Table does not support optimize, doing recreate + analyze instead

status   : OK

Daniel Czadek added a comment - 2024-05-14 09:33 Well now i tested again on our test cluster and it seems that the crash or disconnect from wsrep comes from the OPTIMIZE TABLE: xxx.erp_calculationbase_types_fra note : Table does not support optimize, doing recreate + analyze instead status : OK xxx.erp_calculationbase_types_lcs note : Table does not support optimize, doing recreate + analyze instead status : OK xxx.erp_date_types note : Table does not support optimize, doing recreate + analyze instead status : OK xxx.erp_date_types_deu note : Table does not support optimize, doing recreate + analyze instead status : OK xxx.erp_date_types_eng note : Table does not support optimize, doing recreate + analyze instead status : OK

Sergei Golubchik made changes - 2024-06-24 09:24

Status

Needs Feedback [ 10501 ]

Open [ 1 ]

Jan Lindström made changes - 2024-07-29 12:12

Status

Open [ 1 ]

Needs Feedback [ 10501 ]

Daniel Czadek added a comment - 2024-07-29 12:49

Hi,

I think we can close this case because i mean we could solve the problem for the first by removing "wsrep_slave_threads = 8" and using default value like told in MDEV-28452.

Greetings

Daniel Czadek added a comment - 2024-07-29 12:49 Hi, I think we can close this case because i mean we could solve the problem for the first by removing "wsrep_slave_threads = 8" and using default value like told in MDEV-28452 . Greetings

Sergei Golubchik made changes - 2024-08-26 18:57

Fix Version/s		N/A [ 14700 ]
Fix Version/s	10.5 [ 23123 ]
Resolution		Incomplete [ 4 ]
Status	Needs Feedback [ 10501 ]	Closed [ 6 ]

MariaDB Server

WSREP: MDL BF-BF conflict

Details

Description

Attachments

Issue Links

Activity

People

Dates

Git Integration