[MXS-4050] Out of the blue, we started getting leint_value errors Created: 2022-03-16  Updated: 2022-03-28  Resolved: 2022-03-28

Status: Closed
Project: MariaDB MaxScale
Component/s: readwritesplit
Affects Version/s: 2.4.17
Fix Version/s: 2.4.20

Type: Bug Priority: Major
Reporter: Jeff Smelser Assignee: markus makela
Resolution: Fixed Votes: 0
Labels: None
Environment:

Kubernetes using official maxscale docker.


 Description   

We have a kubernetes install of maxscale, its been rock solid for months and our of the blue, we are getting:

error : (8) (process_packets): Unexpected result state. cmd: 0x00, len: 120
error : (8) (process_packets): Unexpected result state. cmd: 0x00, len: 152
error : (8) (process_packets): Unexpected result state. cmd: 0x00, len: 40
error : (8) (leint_value): Unexpected length encoding 'ff' encountered when reading length-encoded integer.
error : (8) (leint_value): Unexpected length encoding 'ff' encountered when reading length-encoded integer.
warning: (8) (mxs_mysql_parse_ok_packet): recieved unexpecting session track type:213
warning: (8) (mxs_mysql_parse_ok_packet): recieved unexpecting session track type:43
warning: (8) (mxs_mysql_parse_ok_packet): recieved unexpecting session track type:99
warning: (8) (mxs_mysql_parse_ok_packet): recieved unexpecting session track type:106
warning: (8) (mxs_mysql_parse_ok_packet): recieved unexpecting session track type:68
error : (8) (leint_value): Unexpected length encoding 'ff' encountered when reading length-encoded integer.
warning: (8) (mxs_mysql_parse_ok_packet): recieved unexpecting session track type:97
warning: (8) (mxs_mysql_parse_ok_packet): recieved unexpecting session track type:55
warning: (8) (mxs_mysql_parse_ok_packet): recieved unexpecting session track type:100
warning: (8) (mxs_mysql_parse_ok_packet): recieved unexpecting session track type:116
warning: (8) (mxs_mysql_parse_ok_packet): recieved unexpecting session track type:115 (subsequent similar messages suppressed for 10000 milliseconds)
error : (8) (leint_value): Unexpected length encoding 'ff' encountered when reading length-encoded integer.
error : (8) (leint_value): Unexpected length encoding 'ff' encountered when reading length-encoded integer.
error : (8) (leint_value): Unexpected length encoding 'ff' encountered when reading length-encoded integer.
alert : (8) (sigfatal_handler): Fatal: MaxScale 2.4.17 received fatal signal 11. Commit ID: a8bfbbe254095d931aed351871fee72ce9ea645c System name: Linux Release string: CentOS Linux release 8.3.2011

I am trying to figure out where this error would even come from. The instance crashed and restarts, will run for a few minutes, and do it again.. Any help would be appreciated.

 Comments   
Comment by Johan Wikman [ 2022-03-17 ]

Please provide your config, but remove all sensitive data.

Comment by Jeff Smelser [ 2022-03-17 ]

We actually figured out the issue here. One of our teams decided to use an app to pull binary logs. We don't have that turned on and this is what happened. I feel like maxscale should handle this better than segfaulting. I understand why it obviously would not work, it just seems odd that maxscale acted like it was a completed foreign command too.

Is it possible to maybe have maxscale just complain in the logs when someone requests binary logs?

Comment by Johan Wikman [ 2022-03-18 ]

Ok, good that you found the cause.

Yes, MaxScale should obviously not crash in a situation like this, but deal with it and log something sensible.

Comment by markus makela [ 2022-03-23 ]

jsmelser_recharge did you have causal_reads enabled when this happened?

Comment by markus makela [ 2022-03-23 ]

I managed to reproduce this with causal_reads enabled. Turning it off or upgrading to 2.5 solved this.

Comment by Jeff Smelser [ 2022-03-23 ]

Yes and we are not really using it currently. We can turn it off. Thanks for finding that!

Generated at Thu Feb 08 04:25:50 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.