IST is failing after 10.3 > 10.4 upgrade

2022-04-07 12:20:37 0 [Note] /home/ramesh/framework/GAL_MD070422-mariadb-10.4.25-linux-x86_64-opt/bin/mysqld: ready for connections.

Version: '10.4.25-MariaDB-log'  socket: '/home/ramesh/framework/node3/mysql.sock'  port: 5700  MariaDB Server

2022-04-07 12:20:37 14 [ERROR] WSREP: Receiving IST failed, node restart required: IST receiver reported failure: 4 (Interrupted system call)

	 at /test/mtest/10.4_galera_opt/galera/src/replicator_smm.hpp:pop_front():336. Null event.

2022-04-07 12:20:37 1 [Note] WSREP: Lowest cert index boundary for CC from sst: 16414

2022-04-07 12:20:37 1 [Note] WSREP: Min available from gcache for CC from sst: -1 

/home/ramesh/framework/GAL_MD070422-mariadb-10.3.35-linux-x86_64-opt/scripts/mysql_install_db --no-defaults --force --basedir=/home/ramesh/framework/GAL_MD070422-mariadb-10.3.35-linux-x86_64-opt --datadir=/home/ramesh/framework/node1 > /home/ramesh/framework/log/startup1.log 2>&1

/home/ramesh/framework/GAL_MD070422-mariadb-10.3.35-linux-x86_64-opt/scripts/mysql_install_db --no-defaults --force --basedir=/home/ramesh/framework/GAL_MD070422-mariadb-10.3.35-linux-x86_64-opt --datadir=/home/ramesh/framework/node2 > /home/ramesh/framework/log/startup2.log 2>&1

/home/ramesh/framework/GAL_MD070422-mariadb-10.3.35-linux-x86_64-opt/scripts/mysql_install_db --no-defaults --force --basedir=/home/ramesh/framework/GAL_MD070422-mariadb-10.3.35-linux-x86_64-opt --datadir=/home/ramesh/framework/node3 > /home/ramesh/framework/log/startup3.log 2>&1

/home/ramesh/framework/GAL_MD070422-mariadb-10.3.35-linux-x86_64-opt/bin/mysqld --defaults-file=/home/ramesh/framework/conf/node1.cnf  --wsrep-new-cluster > /home/ramesh/framework/node1/node1.err 2>&1 &

/home/ramesh/framework/GAL_MD070422-mariadb-10.3.35-linux-x86_64-opt/bin/mysql --user=root --socket=/home/ramesh/framework/node1/mysql.sock -Bse"SET SESSION sql_log_bin=0;delete from mysql.user where user='';" > /dev/null 2>&1

/home/ramesh/framework/GAL_MD070422-mariadb-10.3.35-linux-x86_64-opt/bin/mysqld --defaults-file=/home/ramesh/framework/conf/node2.cnf  > /home/ramesh/framework/node2/node2.err 2>&1 &

/home/ramesh/framework/GAL_MD070422-mariadb-10.3.35-linux-x86_64-opt/bin/mysqld --defaults-file=/home/ramesh/framework/conf/node3.cnf  > /home/ramesh/framework/node3/node3.err 2>&1 &

/home/ramesh/framework/GAL_MD070422-mariadb-10.3.35-linux-x86_64-opt/bin/mysql --user=root --socket=/home/ramesh/framework/node1/mysql.sock -e"drop database if exists test; create database test;" > /dev/null 2>&1

/home/ramesh/framework/GAL_MD070422-mariadb-10.3.35-linux-x86_64-opt/bin/mysql --user=root --socket=/home/ramesh/framework/node1/mysql.sock -e"create user if not exists sysbench@'localhost' identified  by 'sysbench';grant all on *.* to sysbench@'localhost';" > /dev/null 2>&1

sysbench data load 

sysbench /usr/share/sysbench/oltp_insert.lua --table-size=10000 --tables=10 --threads=10 --mysql-db=test  --mysql-user=sysbench --mysql-password=sysbench --db-driver=mysql  --mysql-socket=/home/ramesh/framework/node1/mysql.sock prepare >/home/ramesh/framework/log/sysbench_prepare.log

Initiate multi master sysbench oltp load in background 

sysbench /usr/share/sysbench/oltp_read_write.lua --table-size=10000 --tables=10 --threads=10 --mysql-db=test  --mysql-user=sysbench --mysql-password=sysbench --db-driver=mysql  --mysql-socket=/home/ramesh/framework/node1/mysql.sock --time=1000 --db-ps-mode=disable run >> /home/ramesh/framework/log/sysbench_read_write_10.log & 

sysbench /usr/share/sysbench/oltp_read_write.lua --table-size=10000 --tables=10 --threads=10 --mysql-db=test  --mysql-user=sysbench --mysql-password=sysbench --db-driver=mysql  --mysql-socket=/home/ramesh/framework/node2/mysql.sock --time=1000 --db-ps-mode=disable run >> /home/ramesh/framework/log/sysbench_read_write_10.log & 

sysbench /usr/share/sysbench/oltp_read_write.lua --table-size=10000 --tables=10 --threads=10 --mysql-db=test  --mysql-user=sysbench --mysql-password=sysbench --db-driver=mysql  --mysql-socket=/home/ramesh/framework/node3/mysql.sock --time=1000 --db-ps-mode=disable run >> /home/ramesh/framework/log/sysbench_read_write_10.log & 

Terminate OLTP sysbench run initiated on node3

/home/ramesh/framework/GAL_MD070422-mariadb-10.3.35-linux-x86_64-opt/bin/mysqladmin --user=root --socket=/home/ramesh/framework/node3/mysql.sock shutdown > /dev/null 2>&1

Initiate DATA load on node1

/home/ramesh/framework/GAL_MD070422-mariadb-10.3.35-linux-x86_64-opt/bin/mysql --user=root --socket=/home/ramesh/framework/node1/mysql.sock -e"drop database if exists test_one; create database test_one;" > /dev/null 2>&1

/home/ramesh/framework/GAL_MD070422-mariadb-10.3.35-linux-x86_64-opt/bin/mysql --user=root --socket=/home/ramesh/framework/node1/mysql.sock -e"create user if not exists sysbench@'localhost' identified  by 'sysbench';grant all on *.* to sysbench@'localhost';" > /dev/null 2>&1

/home/ramesh/framework/GAL_MD070422-mariadb-10.3.35-linux-x86_64-opt/bin/mysql --user=root --socket=/home/ramesh/framework/node1/mysql.sock -e"drop database if exists test_two; create database test_two;" > /dev/null 2>&1

/home/ramesh/framework/GAL_MD070422-mariadb-10.3.35-linux-x86_64-opt/bin/mysql --user=root --socket=/home/ramesh/framework/node1/mysql.sock -e"create user if not exists sysbench@'localhost' identified  by 'sysbench';grant all on *.* to sysbench@'localhost';" > /dev/null 2>&1

/home/ramesh/framework/GAL_MD070422-mariadb-10.3.35-linux-x86_64-opt/bin/mysql --user=root --socket=/home/ramesh/framework/node1/mysql.sock -e"drop database if exists test_three; create database test_three;" > /dev/null 2>&1

/home/ramesh/framework/GAL_MD070422-mariadb-10.3.35-linux-x86_64-opt/bin/mysql --user=root --socket=/home/ramesh/framework/node1/mysql.sock -e"create user if not exists sysbench@'localhost' identified  by 'sysbench';grant all on *.* to sysbench@'localhost';" > /dev/null 2>&1

sysbench /usr/share/sysbench/oltp_insert.lua --table-size=100000 --tables=10 --threads=10 --mysql-db=test_one  --mysql-user=sysbench --mysql-password=sysbench --db-driver=mysql  --mysql-socket=/home/ramesh/framework/node1/mysql.sock prepare >/home/ramesh/framework/log/sysbench_prepare.log

sysbench /usr/share/sysbench/oltp_insert.lua --table-size=100000 --tables=10 --threads=10 --mysql-db=test_two  --mysql-user=sysbench --mysql-password=sysbench --db-driver=mysql  --mysql-socket=/home/ramesh/framework/node1/mysql.sock prepare >/home/ramesh/framework/log/sysbench_prepare.log

sysbench /usr/share/sysbench/oltp_insert.lua --table-size=100000 --tables=10 --threads=10 --mysql-db=test_three  --mysql-user=sysbench --mysql-password=sysbench --db-driver=mysql  --mysql-socket=/home/ramesh/framework/node1/mysql.sock prepare >/home/ramesh/framework/log/sysbench_prepare.log

Start node3

/home/ramesh/framework/GAL_MD070422-mariadb-10.4.25-linux-x86_64-opt/bin/mysqld --defaults-file=/home/ramesh/framework/conf/node3.cnf --wsrep-provider=/home/ramesh/framework/GAL_MD070422-mariadb-10.4.25-linux-x86_64-opt/lib/libgalera_smm.so --datadir=/home/ramesh/framework/node3 --basedir=/home/ramesh/framework/GAL_MD070422-mariadb-10.4.25-linux-x86_64-opt >> /home/ramesh/framework/node3/node3.err 2>&1 &

run mysql upgrade on node3

/home/ramesh/framework/GAL_MD070422-mariadb-10.4.25-linux-x86_64-opt/bin/mysql_upgrade -uroot --socket=/home/ramesh/framework/node3/mysql.sock --skip-write-binlog > /home/ramesh/framework/log/node3_upgrade.log 2>&1

0

Check IST log in node3 error log