from application server logs where it was using .net connector --------------------------------------------------------------- MySql.Data.MySqlClient.MySqlProtocolException HResult=0x80131509 Message=Packet received out-of-order. Expected 1; got 3. Source=mscorlib StackTrace: at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at MySqlConnector.Protocol.Serialization.ProtocolUtility.DoReadPayloadAsync(BufferedByteReader bufferedByteReader, IByteHandler byteHandler, Func`1 getNextSequenceNumber, ArraySegmentHolder`1 previousPayloads, ProtocolErrorBehavior protocolErrorBehavior, IOBehavior ioBehavior) in C:\projects\mysqlconnector\src\MySqlConnector\Protocol\Serialization\ProtocolUtility.cs:line 470 at MySqlConnector.Protocol.Serialization.StandardPayloadHandler.ReadPayloadAsync(ArraySegmentHolder`1 cache, ProtocolErrorBehavior protocolErrorBehavior, IOBehavior ioBehavior) in C:\projects\mysqlconnector\src\MySqlConnector\Protocol\Serialization\StandardPayloadHandler.cs:line 42 at MySqlConnector.Core.ServerSession.ReceiveReplyAsync(IOBehavior ioBehavior, CancellationToken cancellationToken) in C:\projects\mysqlconnector\src\MySqlConnector\Core\ServerSession.cs:line 734 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at MySqlConnector.Core.ServerSession.d__78.MoveNext() in C:\projects\mysqlconnector\src\MySqlConnector\Core\ServerSession.cs:line 769 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Threading.Tasks.ValueTask`1.get_Result() at MySqlConnector.Core.ServerSession.d__69.MoveNext() in C:\projects\mysqlconnector\src\MySqlConnector\Core\ServerSession.cs:line 494 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task) at MySqlConnector.Core.ConnectionPool.d__10.MoveNext() in C:\projects\mysqlconnector\src\MySqlConnector\Core\ConnectionPool.cs:line 72 at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at MySqlConnector.Core.ConnectionPool.d__10.MoveNext() in C:\projects\mysqlconnector\src\MySqlConnector\Core\ConnectionPool.cs:line 141 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Threading.Tasks.ValueTask`1.get_Result() at MySql.Data.MySqlClient.MySqlConnection.d__109.MoveNext() in C:\projects\mysqlconnector\src\MySqlConnector\MySql.Data.MySqlClient\MySqlConnection.cs:line 645 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at MySql.Data.MySqlClient.MySqlConnection.d__25.MoveNext() in C:\projects\mysqlconnector\src\MySqlConnector\MySql.Data.MySqlClient\MySqlConnection.cs:line 312 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at MySql.Data.MySqlClient.MySqlConnection.Open() in C:\projects\mysqlconnector\src\MySqlConnector\MySql.Data.MySqlClient\MySqlConnection.cs:line 280 at MPP.SessionMgr.GlobalSessionSettings.GetDbSessionSettings() in C:\Users\ballenal\source\repos\mpp_web_2_0\SessionMgr\GlobaSessionSettings.cs:line 64 at MPP.SessionMgr.GlobalSessionSettings..ctor() in C:\Users\ballenal\source\repos\mpp_web_2_0\SessionMgr\GlobaSessionSettings.cs:line 25 at MPP.SessionMgr.SessionManager..ctor() in C:\Users\ballenal\source\repos\mpp_web_2_0\SessionMgr\SessionManager.cs:line 30 This exception was originally thrown at this call stack: [External Code] MPP.SessionMgr.GlobalSessionSettings.GetDbSessionSettings() in GlobaSessionSettings.cs MPP.SessionMgr.GlobalSessionSettings.GlobalSessionSettings() in GlobaSessionSettings.cs MPP.SessionMgr.SessionManager.SessionManager() in SessionManager.cs from Maxscale log, we found below errors during this testing ------------------------------------------------------------ From maxscale log: 2021-03-01 14:44:25 notice : (ReadWriteSplit-Listener2) Listening for connections at [10.21.65.48]:12866 2021-03-01 14:44:25 notice : Service 'ReadWriteSplit-Service2' started (1/2) 2021-03-01 14:44:25 notice : (ReadWriteSplit-Listener) Listening for connections at [10.21.65.48]:12865 2021-03-01 14:44:25 notice : Service 'ReadWriteSplit-Service' started (2/2) 2021-03-01 14:44:25 notice : Read 42 user@host entries from 'master-node2' for service 'ReadWriteSplit-Service2'. 2021-03-01 14:44:25 notice : Read 47 user@host entries from 'master-node' for service 'ReadWriteSplit-Service'. 2021-03-01 14:44:29 info : Server variables loaded from 'master-node', next update in 600 seconds. 2021-03-01 14:44:29 info : Server variables loaded from 'slave-node', next update in 600 seconds. 2021-03-01 14:44:30 info : Server variables loaded from 'master-node2', next update in 600 seconds. 2021-03-01 14:44:30 info : Server variables loaded from 'slave-node2', next update in 600 seconds. 2021-03-01 14:45:18 info : (1) Found matching user 'svc_mpp_web'@'10.%' for client 'svc_mpp_web'@'10.84.120.185' with sufficient privileges. 2021-03-01 14:45:18 info : (1) [readwritesplit] (ReadWriteSplit-Service) Target connection counts: 2021-03-01 14:45:18 info : (1) [readwritesplit] (ReadWriteSplit-Service) current operations : 0 in master-node Master, Running 2021-03-01 14:45:18 info : (1) [readwritesplit] (ReadWriteSplit-Service) current operations : 0 in slave-node Slave, Running 2021-03-01 14:45:18 info : (1) [readwritesplit] (ReadWriteSplit-Service) Connected to 'master-node' 2021-03-01 14:45:18 info : (1) [readwritesplit] (ReadWriteSplit-Service) Selected Master: master-node 2021-03-01 14:45:18 info : (1) [readwritesplit] (ReadWriteSplit-Service) Connected to 'slave-node' 2021-03-01 14:45:18 info : (1) [readwritesplit] (ReadWriteSplit-Service) Selected Slave: slave-node 2021-03-01 14:45:18 info : (1) Started ReadWriteSplit-Service client session [1] for 'svc_mpp_web' from 10.84.120.185 2021-03-01 14:45:18 info : (1) Connected to 'master-node' with thread id 108 2021-03-01 14:45:18 info : (1) Connected to 'slave-node' with thread id 153 2021-03-01 14:45:18 info : (1) Authentication to 'master-node' succeeded. 2021-03-01 14:45:18 info : (1) Authentication to 'slave-node' succeeded. 2021-03-01 14:45:18 info : (1) (ReadWriteSplit-Service) > Autocommit: [enabled], trx is [not open], cmd: (0x03) COM_QUERY, plen: 23, type: QUERY_TYPE_SESSION_WRITE, stmt: SET NAMES utf8mb4; 2021-03-01 14:45:18 info : (1) [readwritesplit] (ReadWriteSplit-Service) Session write, routing to all servers. 2021-03-01 14:45:18 info : (1) (ReadWriteSplit-Service) Execute sescmd #1 on 'master-node': [COM_QUERY] SET NAMES utf8mb4; 2021-03-01 14:45:18 info : (1) [readwritesplit] (ReadWriteSplit-Service) Route query to master: master-node 2021-03-01 14:45:18 info : (1) (ReadWriteSplit-Service) Execute sescmd #1 on 'slave-node': [COM_QUERY] SET NAMES utf8mb4; 2021-03-01 14:45:18 info : (1) [readwritesplit] (ReadWriteSplit-Service) Route query to slave: slave-node 2021-03-01 14:45:18 info : (1) [readwritesplit] (ReadWriteSplit-Service) Will return response from 'master-node' to the client 2021-03-01 14:45:18 info : (1) [readwritesplit] (ReadWriteSplit-Service) Reply complete, last reply from master-node 2021-03-01 14:45:18 info : (1) [readwritesplit] (ReadWriteSplit-Service) Reply complete, last reply from slave-node 2021-03-01 14:45:18 info : (1) (ReadWriteSplit-Service) > Autocommit: [enabled], trx is [not open], cmd: (0x03) COM_QUERY, plen: 116, type: QUERY_TYPE_READ, stmt: SELECT param_list, returns FROM mysql.proc WHERE db = 'MPIE' AND name = 'nsp_mppweb_Session_GetGlobalSettings'; 2021-03-01 14:45:18 info : (1) [readwritesplit] (ReadWriteSplit-Service) Route query to slave: slave-node < 2021-03-01 14:45:18 info : (1) [readwritesplit] (ReadWriteSplit-Service) Reply complete, last reply from slave-node 2021-03-01 14:45:19 info : (1) (ReadWriteSplit-Service) Multi-statement query or stored procedure call, routing all future queries to master. 2021-03-01 14:45:19 info : (1) (ReadWriteSplit-Service) > Autocommit: [enabled], trx is [not open], cmd: (0x03) COM_QUERY, plen: 56, type: QUERY_TYPE_WRITE, stmt: CALL nsp_mppweb_Session_GetGlobalSettings('ADMIN'); 2021-03-01 14:45:19 info : (1) [readwritesplit] (ReadWriteSplit-Service) Route query to master: master-node < 2021-03-01 14:45:19 info : (1) [readwritesplit] (ReadWriteSplit-Service) Reply complete, last reply from master-node 2021-03-01 14:45:21 info : (1) [mariadbclient] Client 'svc_mpp_web'@'10.84.120.185' is attempting a COM_CHANGE_USER to 'svc_mpp_web'. 2021-03-01 14:45:21 info : (1) Found matching user 'svc_mpp_web'@'10.%' for client 'svc_mpp_web'@'10.84.120.185' with sufficient privileges. 2021-03-01 14:45:21 info : (1) [mariadbclient] COM_CHANGE_USER from 'svc_mpp_web'@'10.84.120.185' to 'svc_mpp_web' in service 'ReadWriteSplit-Service' succeeded. 2021-03-01 14:45:21 info : (1) (ReadWriteSplit-Service) > Autocommit: [enabled], trx is [not open], cmd: (0x11) COM_CHANGE_USER, plen: 239, type: QUERY_TYPE_SESSION_WRITE, stmt: 2021-03-01 14:45:21 info : (1) [readwritesplit] (ReadWriteSplit-Service) Session write, routing to all servers. 2021-03-01 14:45:21 info : (1) (ReadWriteSplit-Service) Execute sescmd #2 on 'master-node': [COM_CHANGE_USER] 2021-03-01 14:45:21 info : (1) [readwritesplit] (ReadWriteSplit-Service) Route query to master: master-node 2021-03-01 14:45:21 info : (1) (ReadWriteSplit-Service) Execute sescmd #2 on 'slave-node': [COM_CHANGE_USER] 2021-03-01 14:45:21 info : (1) [readwritesplit] (ReadWriteSplit-Service) Route query to slave: slave-node 2021-03-01 14:45:21 info : (1) [readwritesplit] (ReadWriteSplit-Service) Will return response from 'master-node' to the client 2021-03-01 14:45:21 info : (1) [readwritesplit] (ReadWriteSplit-Service) Reply complete, last reply from master-node 2021-03-01 14:45:21 info : (1) [readwritesplit] (ReadWriteSplit-Service) Resetting session command history to position 2 2021-03-01 14:45:21 info : (1) [readwritesplit] (ReadWriteSplit-Service) Reply complete, last reply from slave-node 2021-03-01 14:45:21 info : (1) (ReadWriteSplit-Service) > Autocommit: [enabled], trx is [not open], cmd: (0x01) COM_QUIT, plen: 5, type: QUERY_TYPE_SESSION_WRITE, stmt: 2021-03-01 14:45:21 info : (1) [readwritesplit] (ReadWriteSplit-Service) Session write, routing to all servers. 2021-03-01 14:45:21 info : (1) (ReadWriteSplit-Service) Execute sescmd #3 on 'master-node': [COM_QUIT] 2021-03-01 14:45:21 info : (1) [readwritesplit] (ReadWriteSplit-Service) Route query to master: master-node 2021-03-01 14:45:21 info : (1) (ReadWriteSplit-Service) Execute sescmd #3 on 'slave-node': [COM_QUIT] 2021-03-01 14:45:21 info : (1) [readwritesplit] (ReadWriteSplit-Service) Route query to slave: slave-node 2021-03-01 14:45:21 info : Stopped ReadWriteSplit-Service client session [1] 2021-03-01 14:45:21 info : (2) Found matching user 'svc_mpp_web'@'10.%' for client 'svc_mpp_web'@'10.84.120.185' with sufficient privileges. 2021-03-01 14:45:21 info : (2) [readwritesplit] (ReadWriteSplit-Service) Target connection counts: 2021-03-01 14:45:21 info : (2) [readwritesplit] (ReadWriteSplit-Service) current operations : 0 in master-node Master, Running 2021-03-01 14:45:21 info : (2) [readwritesplit] (ReadWriteSplit-Service) current operations : 0 in slave-node Slave, Running 2021-03-01 14:45:21 info : (2) [readwritesplit] (ReadWriteSplit-Service) Connected to 'master-node' 2021-03-01 14:45:21 info : (2) [readwritesplit] (ReadWriteSplit-Service) Selected Master: master-node 2021-03-01 14:45:21 info : (2) [readwritesplit] (ReadWriteSplit-Service) Connected to 'slave-node' 2021-03-01 14:45:21 info : (2) [readwritesplit] (ReadWriteSplit-Service) Selected Slave: slave-node 2021-03-01 14:45:21 info : (2) Started ReadWriteSplit-Service client session [2] for 'svc_mpp_web' from 10.84.120.185 2021-03-01 14:45:21 info : (2) Connected to 'master-node' with thread id 109 2021-03-01 14:45:21 info : (2) Connected to 'slave-node' with thread id 154 2021-03-01 14:45:21 info : (2) Authentication to 'master-node' succeeded. 2021-03-01 14:45:21 info : (2) Authentication to 'slave-node' succeeded. 2021-03-01 14:45:21 info : (2) (ReadWriteSplit-Service) > Autocommit: [enabled], trx is [not open], cmd: (0x03) COM_QUERY, plen: 23, type: QUERY_TYPE_SESSION_WRITE, stmt: SET NAMES utf8mb4; 2021-03-01 14:45:21 info : (2) [readwritesplit] (ReadWriteSplit-Service) Session write, routing to all servers. 2021-03-01 14:45:21 info : (2) (ReadWriteSplit-Service) Execute sescmd #1 on 'master-node': [COM_QUERY] SET NAMES utf8mb4; 2021-03-01 14:45:21 info : (2) [readwritesplit] (ReadWriteSplit-Service) Route query to master: master-node 2021-03-01 14:45:21 info : (2) (ReadWriteSplit-Service) Execute sescmd #1 on 'slave-node': [COM_QUERY] SET NAMES utf8mb4; 2021-03-01 14:45:21 info : (2) [readwritesplit] (ReadWriteSplit-Service) Route query to slave: slave-node 2021-03-01 14:45:21 info : (2) [readwritesplit] (ReadWriteSplit-Service) Will return response from 'master-node' to the client 2021-03-01 14:45:21 info : (2) [readwritesplit] (ReadWriteSplit-Service) Reply complete, last reply from master-node 2021-03-01 14:45:21 info : (2) [readwritesplit] (ReadWriteSplit-Service) Reply complete, last reply from slave-node 2021-03-01 14:45:21 info : (2) (ReadWriteSplit-Service) Multi-statement query or stored procedure call, routing all future queries to master. 2021-03-01 14:45:21 info : (2) (ReadWriteSplit-Service) > Autocommit: [enabled], trx is [not open], cmd: (0x03) COM_QUERY, plen: 56, type: QUERY_TYPE_WRITE, stmt: CALL nsp_mppweb_Session_GetGlobalSettings('ADMIN'); 2021-03-01 14:45:21 info : (2) [readwritesplit] (ReadWriteSplit-Service) Route query to master: master-node < 2021-03-01 14:45:21 info : (2) [readwritesplit] (ReadWriteSplit-Service) Reply complete, last reply from master-node 2021-03-01 14:46:16 info : (2) [mariadbclient] Client 'svc_mpp_web'@'10.84.120.185' is attempting a COM_CHANGE_USER to 'svc_mpp_web'. 2021-03-01 14:46:16 info : (2) Found matching user 'svc_mpp_web'@'10.%' for client 'svc_mpp_web'@'10.84.120.185' with sufficient privileges. 2021-03-01 14:46:16 info : (2) [mariadbclient] COM_CHANGE_USER from 'svc_mpp_web'@'10.84.120.185' to 'svc_mpp_web' in service 'ReadWriteSplit-Service' succeeded. 2021-03-01 14:46:16 info : (2) (ReadWriteSplit-Service) > Autocommit: [enabled], trx is [not open], cmd: (0x11) COM_CHANGE_USER, plen: 239, type: QUERY_TYPE_SESSION_WRITE, stmt: 2021-03-01 14:46:16 info : (2) [readwritesplit] (ReadWriteSplit-Service) Session write, routing to all servers. 2021-03-01 14:46:16 info : (2) (ReadWriteSplit-Service) Execute sescmd #2 on 'master-node': [COM_CHANGE_USER] 2021-03-01 14:46:16 info : (2) [readwritesplit] (ReadWriteSplit-Service) Route query to master: master-node 2021-03-01 14:46:16 info : (2) (ReadWriteSplit-Service) Execute sescmd #2 on 'slave-node': [COM_CHANGE_USER] 2021-03-01 14:46:16 info : (2) [readwritesplit] (ReadWriteSplit-Service) Route query to slave: slave-node 2021-03-01 14:46:16 info : (2) [readwritesplit] (ReadWriteSplit-Service) Will return response from 'master-node' to the client 2021-03-01 14:46:16 info : (2) [readwritesplit] (ReadWriteSplit-Service) Reply complete, last reply from master-node 2021-03-01 14:46:16 info : (2) [readwritesplit] (ReadWriteSplit-Service) Resetting session command history to position 2 2021-03-01 14:46:16 info : (2) [readwritesplit] (ReadWriteSplit-Service) Reply complete, last reply from slave-node 2021-03-01 14:46:16 info : (2) (ReadWriteSplit-Service) > Autocommit: [enabled], trx is [not open], cmd: (0x01) COM_QUIT, plen: 5, type: QUERY_TYPE_SESSION_WRITE, stmt: 2021-03-01 14:46:16 info : (2) [readwritesplit] (ReadWriteSplit-Service) Session write, routing to all servers. 2021-03-01 14:46:16 info : (2) (ReadWriteSplit-Service) Execute sescmd #3 on 'master-node': [COM_QUIT] 2021-03-01 14:46:16 info : (2) [readwritesplit] (ReadWriteSplit-Service) Route query to master: master-node 2021-03-01 14:46:16 info : (2) (ReadWriteSplit-Service) Execute sescmd #3 on 'slave-node': [COM_QUIT] 2021-03-01 14:46:16 info : (2) [readwritesplit] (ReadWriteSplit-Service) Route query to slave: slave-node 2021-03-01 14:46:16 info : Stopped ReadWriteSplit-Service client session [2]