Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Not a Bug
-
10.1.18, 10.2.14
-
None
-
3 nodes running mysql Ver 15.1 Distrib 10.1.18-MariaDB, for Linux (x86_64) using readline 5.1.
- Hosts: Physical Hosts running CentOS 7.4.1708 (Core), 32 CPUs, 64 GB RAM
1 node is the master and the others are slave.
# MariaDB Configuration:
[mysqld]
datadir=/mysql/data/mysql
socket=/mysql/data/mysql/mysql.sock
open_files_limit = 65535
max_connections = 500
user=mysql
binlog_format=ROW
default_storage_engine=InnoDB
innodb_autoinc_lock_mode=2
innodb_flush_log_at_trx_commit=0
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_provider_options="gcache.size=50G; gcache.page_size=1024M; gcs.fc_limit=1000000; gcs.fc_master_slave=yes"
wsrep_cluster_name="zabbixdb_cluster"
wsrep_cluster_address="gcomm://10.1.1.1,10.1.1.2,10.1.1.3"
wsrep_on=ON
wsrep_node_name=AAAAAA
wsrep_slave_threads=32
wsrep_sst_auth=root:xxxxxxx
wsrep_sst_method=xtrabackup
innodb_autoextend_increment = 256
innodb_buffer_pool_instances = 52
innodb_buffer_pool_size = 52G
innodb_concurrency_tickets = 5000
innodb_file_per_table = 1
innodb_flush_method = O_DIRECT
innodb_log_file_size = 512M
innodb_log_files_in_group = 4
innodb_old_blocks_time = 1000
innodb_open_files = 2048
innodb_stats_on_metadata = OFF
innodb_lock_wait_timeout = 50
innodb_io_capacity = 2000
optimizer_switch = 'index_condition_pushdown=off'
large-pages
binlog-row-event-max-size = 8192
character_set_server = utf8
collation_server = utf8_bin
expire_logs_days = 1
join_buffer_size = 262144
max_allowed_packet = 32M
max_connect_errors = 10000
max_heap_table_size = 134217728
query_cache_type = 0
query_cache_size = 0
slow-query-log = ON
table_open_cache = 2048
thread_cache_size = 64
tmp_table_size = 134217728
wait_timeout = 864003 nodes running mysql Ver 15.1 Distrib 10.1.18-MariaDB, for Linux (x86_64) using readline 5.1. - Hosts: Physical Hosts running CentOS 7.4.1708 (Core), 32 CPUs, 64 GB RAM 1 node is the master and the others are slave. # MariaDB Configuration: [mysqld] datadir=/mysql/data/mysql socket=/mysql/data/mysql/mysql.sock open_files_limit = 65535 max_connections = 500 user=mysql binlog_format=ROW default_storage_engine=InnoDB innodb_autoinc_lock_mode=2 innodb_flush_log_at_trx_commit=0 wsrep_provider=/usr/lib64/galera/libgalera_smm.so wsrep_provider_options="gcache.size=50G; gcache.page_size=1024M; gcs.fc_limit=1000000; gcs.fc_master_slave=yes" wsrep_cluster_name="zabbixdb_cluster" wsrep_cluster_address=" gcomm://10.1.1.1,10.1.1.2,10.1.1.3 " wsrep_on=ON wsrep_node_name=AAAAAA wsrep_slave_threads=32 wsrep_sst_auth=root:xxxxxxx wsrep_sst_method=xtrabackup innodb_autoextend_increment = 256 innodb_buffer_pool_instances = 52 innodb_buffer_pool_size = 52G innodb_concurrency_tickets = 5000 innodb_file_per_table = 1 innodb_flush_method = O_DIRECT innodb_log_file_size = 512M innodb_log_files_in_group = 4 innodb_old_blocks_time = 1000 innodb_open_files = 2048 innodb_stats_on_metadata = OFF innodb_lock_wait_timeout = 50 innodb_io_capacity = 2000 optimizer_switch = 'index_condition_pushdown=off' large-pages binlog-row-event-max-size = 8192 character_set_server = utf8 collation_server = utf8_bin expire_logs_days = 1 join_buffer_size = 262144 max_allowed_packet = 32M max_connect_errors = 10000 max_heap_table_size = 134217728 query_cache_type = 0 query_cache_size = 0 slow-query-log = ON table_open_cache = 2048 thread_cache_size = 64 tmp_table_size = 134217728 wait_timeout = 86400
Description
Every node in the cluster is logging warnings similar to this:
[Warning] WSREP: Failed to report last committed 211651504, -4 (Interrupted system call)
At this point, one node starts the flow-control mechanism and the my application stops.
I can see, running the myq_status too, that the "Queue down" on one node achieve the fc_limit and starts the flow control.
I must stop the MariaDB on this node to application becomes up again.
I have one master node e two slave nodes. I could see this behavior on the slave nodes only.
I couln´t find why one node is starting the flow control. On the /var/log/messages I can see only the "WSREP: Failed to report last committed" messages. Nothing more.
Any suggestion?
Attachments
Issue Links
- relates to
-
MDEV-17550 Improve WSREP's "Failed to report last committed" warning message
- Closed