[MXS-1938]  More clear logs related to commands inconsistency between master and slave Created: 2018-06-22  Updated: 2018-07-03  Resolved: 2018-07-03

Status: Closed
Project: MariaDB MaxScale
Component/s: readwritesplit
Affects Version/s: 2.2.9
Fix Version/s: 2.2.10

Type: New Feature Priority: Major
Reporter: Wagner Bianchi (Inactive) Assignee: markus makela
Resolution: Fixed Votes: 0
Labels: None


 Description   

Folks,

After moving to 2.2.9 from 2.2.4, I started seeing the below warning events that got me a bit worried. Firstly because it shows that the transactions the applications are sending to Maxscale to then, route it to the backend are being inconsistent among master and slaves, and the connections are being closed - the main concern here is that, where didn't run on slaves and the connection was closed?

2018-06-22 20:20:36.904   warning: (56482) [readwritesplit] (discard_if_response_differs): Slave server 'DB05': response (0xff) differs from master's response(0x00). Closing slave connection due to inconsistent session state.
2018-06-22 20:20:36.904   warning: (56482) [readwritesplit] (discard_if_response_differs): Slave server 'DB02': response (0xff) differs from master's response(0x00). Closing slave connection due to inconsistent session state.
2018-06-22 20:20:36.904   warning: (56482) [readwritesplit] (discard_if_response_differs): Slave server 'DB04': response (0xff) differs from master's response(0x00). Closing slave connection due to inconsistent session state.
2018-06-22 20:20:37.960   warning: (45911) [readwritesplit] (discard_if_response_differs): Slave server 'DB03': response (0xff) differs from master's response(0x00). Closing slave connection due to inconsistent session state.
2018-06-22 20:20:37.960   warning: (45911) [readwritesplit] (discard_if_response_differs): Slave server 'DB02': response (0xff) differs from master's response(0x00). Closing slave connection due to inconsistent session state.
2018-06-22 20:20:37.960   warning: (45911) [readwritesplit] (discard_if_response_differs): Slave server 'DB05': response (0xff) differs from master's response(0x00). Closing slave connection due to inconsistent session state.
2018-06-22 20:20:37.960   warning: (45911) [readwritesplit] (discard_if_response_differs): Slave server 'DB04': response (0xff) differs from master's response(0x00). Closing slave connection due to inconsistent session state.

The output of the current list servers to register the backend's roles on GaleraMon right now:

root@LB03:~# maxadmin list servers
Servers.
-------------------+-----------------+-------+-------------+--------------------
Server             | Address         | Port  | Connections | Status
-------------------+-----------------+-------+-------------+--------------------
REP02     | 3.3.4.4   |  3306 |          20 | Slave, Running
REP03     | 1.1.2.2   |  3306 |          20 | Slave, Running
DB05      | 5.5.5.5    |  3306 |         348 | Slave, Synced, Running
DB01      |  1.1.1.1  |  3306 |         517 | Master, Synced, Running
DB02      | 2.2.2.2   |  3306 |         350 | Slave, Synced, Running
DB03      | 3.3.3.3    |  3306 |         350 | Slave, Synced, Running
DB04      | 4.4.4.4   |  3306 |         348 | Slave, Synced, Running
-------------------+-----------------+-------+-------------+--------------------

I found out the function which checks that inconsistency between the command executes on the master and the slave and then, fire the warning event:

static bool discard_if_response_differs(SRWBackend backend, uint8_t master_cmd, uint8_t slave_cmd)
{
    bool rval = false;
 
    if (master_cmd != slave_cmd)
    {
        MXS_WARNING("Slave server '%s': response (0x%02hhx) differs "
                    "from master's response(0x%02hhx). Closing slave "
                    "connection due to inconsistent session state.",
                    backend->name(), slave_cmd, master_cmd);
        backend->close(mxs::Backend::CLOSE_FATAL);
        rval = true;
    }
 
    return rval;
}

My request: is it possible to add to the log right after the first message the master_cmd and the slave_cmd? It's going to be very helpful to debug and fix it.

Thanks, folks.



 Comments   
Comment by markus makela [ 2018-06-22 ]

Should be possible to add it.

Generated at Thu Feb 08 04:10:30 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.