[MXS-4389] Crash in handleError Created: 2022-11-11  Updated: 2022-12-02  Resolved: 2022-11-15

Status: Closed
Project: MariaDB MaxScale
Component/s: Core
Affects Version/s: 6.4.2, 22.08.2
Fix Version/s: 6.4.4, 22.08.3

Type: Bug Priority: Major
Reporter: markus makela Assignee: markus makela
Resolution: Fixed Votes: 1
Labels: None


 Description   

The session command handler seems to end up doing something which it shouldn't do as the pointer to the router session is null.

Thread 1 (Thread 0x7fb717fff700 (LWP 3174926)):
 
#0  0x00007fb7229ba08b in ServiceEndpoint::handleError (this=0x7fb70caebac0, type=maxscale::ErrorType::PERMANENT, error=0x7fb70cb10b20, down=<optimized out>, reply=...) at /home/timofey_turenko_mariadb_com/MaxScale/server/core/service.cc:1718
 
#1  0x00007fb7229a7dcb in ServerEndpoint::handleError (this=0x7fb70caebbe0, type=maxscale::ErrorType::PERMANENT, error=0x7fb70cb10b20, down=<optimized out>, reply=...) at /home/timofey_turenko_mariadb_com/MaxScale/server/core/server.cc:1207
 
#2  0x00007fb7229e7d6d in MariaDBBackendConnection::do_handle_error (this=0x7fb70cb13de0, dcb=0x7fb70cb14570, errmsg=..., type=maxscale::ErrorType::PERMANENT) at /home/timofey_turenko_mariadb_com/MaxScale/server/modules/protocol/MariaDB/mariadb_backend.cc:584
 
#3  0x00007fb7229eb43c in operator() (__closure=0x7fb717ff9370) at /home/timofey_turenko_mariadb_com/MaxScale/server/modules/protocol/MariaDB/mariadb_backend.cc:959
 
#4  std::__invoke_impl<void, MariaDBBackendConnection::compare_responses()::<lambda()>&> (__f=...) at /usr/include/c++/10/bits/invoke.h:60
 
#5  std::__invoke_r<void, MariaDBBackendConnection::compare_responses()::<lambda()>&> (__fn=...) at /usr/include/c++/10/bits/invoke.h:153
 
#6  std::_Function_handler<void(), MariaDBBackendConnection::compare_responses()::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/10/bits/std_function.h:291
 
#7  0x00007fb7229fb375 in std::function<void ()>::operator()() const (this=0x7fb717ff9370) at /usr/include/c++/10/bits/std_function.h:622
 
#8  MariaDBClientConnection::process_normal_read (this=0x7fb70c119120) at /home/timofey_turenko_mariadb_com/MaxScale/server/modules/protocol/MariaDB/mariadb_client.cc:1364
 
#9  0x00007fb7229fb798 in MariaDBClientConnection::ready_for_reading (this=0x7fb70c119120, event_dcb=<optimized out>) at /home/timofey_turenko_mariadb_com/MaxScale/server/modules/protocol/MariaDB/mariadb_client.cc:1548
 
#10 0x00007fb722931134 in DCB::process_events (events=1, this=0x7fb70caec010) at /home/timofey_turenko_mariadb_com/MaxScale/server/core/dcb.cc:1311
 
#11 DCB::process_events (this=0x7fb70caec010, events=1) at /home/timofey_turenko_mariadb_com/MaxScale/server/core/dcb.cc:1245
 
#12 0x00007fb722931231 in DCB::event_handler (dcb=0x7fb70caec010, events=<optimized out>) at /home/timofey_turenko_mariadb_com/MaxScale/server/core/dcb.cc:1363
 
#13 0x00007fb722933922 in DCB::FakeEventTask::execute (worker=..., this=0x7fb70c854c10) at /home/timofey_turenko_mariadb_com/MaxScale/server/core/dcb.cc:1434
 
#14 DCB::FakeEventTask::execute (this=0x7fb70c854c10, worker=...) at /home/timofey_turenko_mariadb_com/MaxScale/server/core/dcb.cc:1421
 
#15 0x00007fb722a2db99 in maxbase::Worker::handle_message (this=0x56378c366a80, queue=..., msg=...) at /home/timofey_turenko_mariadb_com/MaxScale/maxutils/maxbase/src/worker.cc:663
 
#16 0x00007fb722a30dc8 in maxbase::MessageQueue::handle_poll_events (pWorker=<optimized out>, events=<optimized out>, this=0x56378c364a60) at /home/timofey_turenko_mariadb_com/MaxScale/maxutils/maxbase/src/messagequeue.cc:307
 
#17 maxbase::MessageQueue::handle_poll_events (this=0x56378c364a60, pWorker=<optimized out>, events=<optimized out>) at /home/timofey_turenko_mariadb_com/MaxScale/maxutils/maxbase/src/messagequeue.cc:286
 
#18 0x00007fb722a2f4b2 in maxbase::Worker::poll_waitevents (this=this@entry=0x56378c366a80) at /home/timofey_turenko_mariadb_com/MaxScale/maxutils/maxbase/src/worker.cc:848
 
#19 0x00007fb722a2f70f in maxbase::Worker::run (this=0x56378c366a80, pSem=0x7fff2f6b7020) at /home/timofey_turenko_mariadb_com/MaxScale/maxutils/maxbase/src/worker.cc:554
 
#20 0x00007fb7222b0ed0 in ?? ()
 
#21 0x0000000000000000 in ?? ()
 
(gdb) p m_router_session
 
$1 = (maxscale::RouterSession *) 0x0



 Comments   
Comment by markus makela [ 2022-11-11 ]

I think I managed to reproduce it:

2022-11-11 16:06:20   error  : (13) (server2); debug assert at /home/markusjm/MaxScale/server/core/server.cc:1206 failed: is_open()
 
debug assert at /home/markusjm/MaxScale/server/core/server.cc:1206 failed: is_open()
 
alert  : MaxScale 6.4.4 received fatal signal 6. Commit ID: 81237c3e62a8a0a42f50d1fcdfd265be1ab6ed41 System name: Linux Release string: NAME="Fedora Linux"
 
 
 
 
 
2022-11-11 16:06:46   alert  : (13) (server2); MaxScale 6.4.4 received fatal signal 6. Commit ID: 81237c3e62a8a0a42f50d1fcdfd265be1ab6ed41 System name: Linux Release string: NAME="Fedora Linux"
 
2022-11-11 16:06:46   alert  : (13) (server2); Statement currently being classified: none/unknown
 
2022-11-11 16:06:46   alert  : (13) (server2); Session: 13 Service: SchemaRouter-Router
 
2022-11-11 16:06:46   notice : (13) (server2); For a more detailed stacktrace, install GDB and add 'debug=gdb-stacktrace' under the [maxscale] section.
 
alert  :   /lib64/libc.so.6(+0x8ec0c): ??:?
 
  /lib64/libc.so.6(raise+0x16): :?
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN14ServerEndpoint11handleErrorEN8maxscale9ErrorTypeEP5GWBUFPNS0_8EndpointERKNS0_5ReplyE+0xfe): server/core/server.cc:1207
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN24MariaDBBackendConnection15do_handle_errorEP3DCBRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEN8maxscale9ErrorTypeE+0x2cf): server/modules/protocol/MariaDB/mariadb_backend.cc:584 (discriminator 4)
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(+0x633c75): server/modules/protocol/MariaDB/mariadb_backend.cc:959 (discriminator 4)
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(+0x63c74e): /usr/include/c++/12/bits/invoke.h:61
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(+0x63c5c6): /usr/include/c++/12/bits/invoke.h:156
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(+0x63c412): /usr/include/c++/12/bits/std_function.h:291
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZNKSt8functionIFvvEEclEv+0x32): /usr/include/c++/12/bits/std_function.h:591
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN23MariaDBClientConnection19process_normal_readEv+0x1c2): server/modules/protocol/MariaDB/mariadb_client.cc:1365 (discriminator 2)
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN23MariaDBClientConnection17ready_for_readingEP3DCB+0x20d): server/modules/protocol/MariaDB/mariadb_client.cc:1548
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN3DCB14process_eventsEj+0x55f): server/core/dcb.cc:1311
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN3DCB13event_handlerEPS_j+0x2f): server/core/dcb.cc:1363
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN3DCB13FakeEventTask7executeERN7maxbase6WorkerE+0x21d): server/core/dcb.cc:1435
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker14handle_messageERNS_12MessageQueueERKNS_19MessageQueueMessageE+0x16b): maxutils/maxbase/src/worker.cc:664
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase12MessageQueue18handle_poll_eventsEPNS_6WorkerEj+0x1ef): maxutils/maxbase/src/messagequeue.cc:307
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase12MessageQueue12poll_handlerEP13MXB_POLL_DATAP10MXB_WORKERj+0x31): maxutils/maxbase/src/messagequeue.cc:343
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker15poll_waiteventsEv+0x4b1): maxutils/maxbase/src/worker.cc:848
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker3runEPNS_9SemaphoreE+0x12c): maxutils/maxbase/src/worker.cc:556
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker11thread_mainEPS0_PNS_9SemaphoreE+0x23): maxutils/maxbase/src/worker.cc:682
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZSt13__invoke_implIvPFvPN7maxbase6WorkerEPNS0_9SemaphoreEEJS2_S4_EET_St14__invoke_otherOT0_DpOT1_+0x4d): /usr/include/c++/12/bits/invoke.h:61
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZSt8__invokeIPFvPN7maxbase6WorkerEPNS0_9SemaphoreEEJS2_S4_EENSt15__invoke_resultIT_JDpT0_EE4typeEOS8_DpOS9_+0x4f): /usr/include/c++/12/bits/invoke.h:97
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZNSt6thread8_InvokerISt5tupleIJPFvPN7maxbase6WorkerEPNS2_9SemaphoreEES4_S6_EEE9_M_invokeIJLm0ELm1ELm2EEEEvSt12_Index_tupleIJXspT_EEE+0x5f): /usr/include/c++/12/bits/std_thread.h:252
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZNSt6thread8_InvokerISt5tupleIJPFvPN7maxbase6WorkerEPNS2_9SemaphoreEES4_S6_EEEclEv+0x18): /usr/include/c++/12/bits/std_thread.h:259
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJPFvPN7maxbase6WorkerEPNS3_9SemaphoreEES5_S7_EEEEE6_M_runEv+0x1c): /usr/include/c++/12/bits/std_thread.h:210
 
  /lib64/libstdc++.so.6(+0xdbb03): ??:?
 
  /lib64/libc.so.6(+0x8cded): ??:?
 
  /lib64/libc.so.6(+0x112370): :?
 
 
 
 
 
  /lib64/libc.so.6(+0x8ec0c): ??:?
 
  /lib64/libc.so.6(raise+0x16): :?
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN14ServerEndpoint11handleErrorEN8maxscale9ErrorTypeEP5GWBUFPNS0_8EndpointERKNS0_5ReplyE+0xfe): server/core/server.cc:1207
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN24MariaDBBackendConnection15do_handle_errorEP3DCBRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEN8maxscale9ErrorTypeE+0x2cf): server/modules/protocol/MariaDB/mariadb_backend.cc:584 (discriminator 4)
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(+0x633c75): server/modules/protocol/MariaDB/mariadb_backend.cc:959 (discriminator 4)
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(+0x63c74e): /usr/include/c++/12/bits/invoke.h:61
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(+0x63c5c6): /usr/include/c++/12/bits/invoke.h:156
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(+0x63c412): /usr/include/c++/12/bits/std_function.h:291
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZNKSt8functionIFvvEEclEv+0x32): /usr/include/c++/12/bits/std_function.h:591
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN23MariaDBClientConnection19process_normal_readEv+0x1c2): server/modules/protocol/MariaDB/mariadb_client.cc:1365 (discriminator 2)
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN23MariaDBClientConnection17ready_for_readingEP3DCB+0x20d): server/modules/protocol/MariaDB/mariadb_client.cc:1548
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN3DCB14process_eventsEj+0x55f): server/core/dcb.cc:1311
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN3DCB13event_handlerEPS_j+0x2f): server/core/dcb.cc:1363
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN3DCB13FakeEventTask7executeERN7maxbase6WorkerE+0x21d): server/core/dcb.cc:1435
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker14handle_messageERNS_12MessageQueueERKNS_19MessageQueueMessageE+0x16b): maxutils/maxbase/src/worker.cc:664
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase12MessageQueue18handle_poll_eventsEPNS_6WorkerEj+0x1ef): maxutils/maxbase/src/messagequeue.cc:307
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase12MessageQueue12poll_handlerEP13MXB_POLL_DATAP10MXB_WORKERj+0x31): maxutils/maxbase/src/messagequeue.cc:343
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker15poll_waiteventsEv+0x4b1): maxutils/maxbase/src/worker.cc:848
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker3runEPNS_9SemaphoreE+0x12c): maxutils/maxbase/src/worker.cc:556
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker11thread_mainEPS0_PNS_9SemaphoreE+0x23): maxutils/maxbase/src/worker.cc:682
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZSt13__invoke_implIvPFvPN7maxbase6WorkerEPNS0_9SemaphoreEEJS2_S4_EET_St14__invoke_otherOT0_DpOT1_+0x4d): /usr/include/c++/12/bits/invoke.h:61
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZSt8__invokeIPFvPN7maxbase6WorkerEPNS0_9SemaphoreEEJS2_S4_EENSt15__invoke_resultIT_JDpT0_EE4typeEOS8_DpOS9_+0x4f): /usr/include/c++/12/bits/invoke.h:97
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZNSt6thread8_InvokerISt5tupleIJPFvPN7maxbase6WorkerEPNS2_9SemaphoreEES4_S6_EEE9_M_invokeIJLm0ELm1ELm2EEEEvSt12_Index_tupleIJXspT_EEE+0x5f): /usr/include/c++/12/bits/std_thread.h:252
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZNSt6thread8_InvokerISt5tupleIJPFvPN7maxbase6WorkerEPNS2_9SemaphoreEES4_S6_EEEclEv+0x18): /usr/include/c++/12/bits/std_thread.h:259
 
  /home/markusjm/build-develop/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJPFvPN7maxbase6WorkerEPNS3_9SemaphoreEES5_S7_EEEEE6_M_runEv+0x1c): /usr/include/c++/12/bits/std_thread.h:210
 
  /lib64/libstdc++.so.6(+0xdbb03): ??:?
 
  /lib64/libc.so.6(+0x8cded): ??:?
 
  /lib64/libc.so.6(+0x112370): :?
 
alert  : Writing core dump.

I appears to be related to a session command being queued for execution for a backend for which the authentication will fail.

Comment by markus makela [ 2022-11-11 ]

For future reference, here's the complete log:

2022-11-11 16:06:20   info   : (13) Found matching user 'bob'@'%' for client 'bob'@'::ffff:127.0.0.1' with sufficient privileges.
 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router2); Target connection counts:
 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router2); current operations : 0 in      server3 Master, Running
 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router2); current operations : 0 in      server4 Slave, Running
 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router2); Connected to 'server3'
 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router2); Selected Master: server3
 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router2); Connected to 'server4'
 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router2); Selected Slave: server4
 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router); Target connection counts:
 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router); current operations : 0 in       server1 Master, Running
 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router); current operations : 0 in       server2 Slave, Running
 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router); Connected to 'server1'
 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router); Selected Master: server1
 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router); Connected to 'server2'
 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router); Selected Slave: server2
 
2022-11-11 16:06:20   info   : (13) [schemarouter] (SchemaRouter-Router); Connected Master, Running in  'RW-Split-Router2'
 
2022-11-11 16:06:20   info   : (13) [schemarouter] (SchemaRouter-Router); Connected Master, Running in  'RW-Split-Router'
 
2022-11-11 16:06:20   info   : (13) Started SchemaRouter-Router client session [13] for 'bob' from ::ffff:127.0.0.1
 
2022-11-11 16:06:20   info   : (13) Connected to 'server1' with thread id 56
 
2022-11-11 16:06:20   info   : (13) Connected to 'server2' with thread id 53
 
2022-11-11 16:06:20   info   : (13) Authentication to 'server1' succeeded.
 
2022-11-11 16:06:20   info   : (13) Authentication to 'server2' succeeded.
 
2022-11-11 16:06:20   info   : (13) > Autocommit: [enabled], trx is [not open], cmd: (0x03) COM_QUERY, plen: 15, type: QUERY_TYPE_SESSION_WRITE|QUERY_TYPE_USERVAR_WRITE, stmt: set @a = 1 
2022-11-11 16:06:20   info   : (13) [schemarouter] (SchemaRouter-Router); > Command: COM_QUERY, stmt: set @a = 1 
2022-11-11 16:06:20   info   : (13) (RW-Split-Router2); > Autocommit: [enabled], trx is [not open], cmd: (0x03) COM_QUERY, plen: 15, type: QUERY_TYPE_SESSION_WRITE|QUERY_TYPE_USERVAR_WRITE, stmt: set @a = 1 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router2); Session write, routing to all servers.
 
2022-11-11 16:06:20   info   : (13) (server3); Storing COM_QUERY while in state 'Handshaking': set @a = 1
 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router2); Route query to master: server3
 
2022-11-11 16:06:20   info   : (13) (server4); Storing COM_QUERY while in state 'Handshaking': set @a = 1
 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router2); Route query to slave: server4
 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router2); Will return response from 'server3' to the client
 
2022-11-11 16:06:20   info   : (13) [schemarouter] (SchemaRouter-Router); Route query to master: RW-Split-Router2
 
2022-11-11 16:06:20   info   : (13) (RW-Split-Router); > Autocommit: [enabled], trx is [not open], cmd: (0x03) COM_QUERY, plen: 15, type: QUERY_TYPE_SESSION_WRITE|QUERY_TYPE_USERVAR_WRITE, stmt: set @a = 1 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router); Session write, routing to all servers.
 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router); Route query to master: server1
 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router); Route query to slave: server2
 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router); Will return response from 'server1' to the client
 
2022-11-11 16:06:20   info   : (13) [schemarouter] (SchemaRouter-Router); Route query to master: RW-Split-Router
 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router); Reply complete from 'server2', discarding it.
 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router); Reply complete from 'server1' (OK: 0 warnings)
 
2022-11-11 16:06:20   info   : (13) [schemarouter] (SchemaRouter-Router); Reply complete from 'RW-Split-Router'
 
2022-11-11 16:06:20   info   : (13) Connected to 'server4' with thread id 55
 
2022-11-11 16:06:20   info   : (13) Connected to 'server3' with thread id 68
 
2022-11-11 16:06:20   error  : (13) Authentication to 'server4' failed: 1045, #28000: Access denied for user 'bob'@'127.0.0.1' (using password: YES)
 
2022-11-11 16:06:20   warning: (13) User accounts have been recently updated, cannot update again for 'bob'@'::ffff:127.0.0.1'.
 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router2); Slave 'server4' failed: #HY000: Authentication to 'server4' failed: 1045, #28000: Access denied for user 'bob'@'127.0.0.1' (using password: YES) (server4)
 
2022-11-11 16:06:20   error  : (13) Authentication to 'server3' failed: 1045, #28000: Access denied for user 'bob'@'127.0.0.1' (using password: YES)
 
2022-11-11 16:06:20   warning: (13) User accounts have been recently updated, cannot update again for 'bob'@'::ffff:127.0.0.1'.
 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router2); Master 'server3' failed: #HY000: Authentication to 'server3' failed: 1045, #28000: Access denied for user 'bob'@'127.0.0.1' (using password: YES) (server3)
 
2022-11-11 16:06:20   error  : (13) [readwritesplit] (RW-Split-Router2); Lost connection to the master server, closing session. Lost connection to master server while waiting for a result. Connection has been idle for 0 seconds. Error caused by: #HY000: Authentication to 'server3' failed: 1045, #28000: Access denied for user 'bob'@'127.0.0.1' (using password: YES) (server3). Last close reason: <none>. Last error: 
2022-11-11 16:06:20   info   : (13) [MariaDBProtocol] (SchemaRouter-Router); Added COM_QUERY to history with ID 1: set @a = 1 (result: )
 
2022-11-11 16:06:20   info   : (13) [readwritesplit] (RW-Split-Router); Master 'server1' failed: #HY000: Response from server 'server1' differs from the expected response to COM_QUERY. Closing connection due to inconsistent session state. (server1)
 
2022-11-11 16:06:20   error  : (13) [readwritesplit] (RW-Split-Router); Lost connection to the master server, closing session. Lost connection to master server while connection was idle. Connection has been idle for 0 seconds. Error caused by: #HY000: Response from server 'server1' differs from the expected response to COM_QUERY. Closing connection due to inconsistent session state. (server1). Last close reason: <none>. Last error: 
2022-11-11 16:06:20   error  : (13) (server2); debug assert at /home/markusjm/MaxScale/server/core/server.cc:1206 failed: is_open()

Generated at Thu Feb 08 04:28:17 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.