[MXS-3135] Maxscale fails when doing "prepare" with sysbench and ssl enabled Created: 2020-08-27  Updated: 2020-08-31  Resolved: 2020-08-28

Status: Closed
Project: MariaDB MaxScale
Component/s: Core
Affects Version/s: 2.5
Fix Version/s: 2.5.3

Type: Bug Priority: Major
Reporter: Petko Vasilev (Inactive) Assignee: markus makela
Resolution: Fixed Votes: 0
Labels: None


 Description   

Short summary: when using sysbench against maxscale 2.5 fails in the "prepare" stage when ssl is enabled. It used to work with 2.4

steps to reproduce:
1. start the example docker-compose from the maxscale repo, but with ssl enabled
the repo is here
https://github.com/mariadb-corporation/maxscale-docker/tree/maxscale-2.5
I've changed the images in the docker-compose.yaml to 10.5 instead of 10.3
there are also some changes to the example.cnf to enable ssl

[server1]
type=server
address=master
port=3306
protocol=MariaDBBackend
 
[server2]
type=server
address=slave1
port=3306
protocol=MariaDBBackend
 
[server3]
type=server
address=slave2
port=3306
protocol=MariaDBBackend
 
# Monitor for the servers
# This will keep MaxScale aware of the state of the servers.
# MySQL Monitor documentation:
# https://github.com/mariadb-corporation/MaxScale/blob/2.3/Documentation/Monitors/MariaDB-Monitor.md
 
[MariaDB-Monitor]
type=monitor
module=mariadbmon
servers=server1,server2,server3
user=maxuser
password=maxpwd
failcount=3
backend_connect_timeout=3
backend_write_timeout=3
backend_read_timeout=3
auto_failover=true
auto_rejoin=true
enforce_read_only_slaves=1
 
# Service definitions
# Service Definition for a read-only service and a read/write splitting service.
 
# ReadConnRoute documentation:
# https://github.com/mariadb-corporation/MaxScale/blob/2.3/Documentation/Routers/ReadConnRoute.md
 
[Read-Only-Service]
type=service
router=readconnroute
servers=server1,server2,server3
user=maxuser
password=maxpwd
router_options=slave
 
# ReadWriteSplit documentation:
# https://github.com/mariadb-corporation/MaxScale/blob/2.3/Documentation/Routers/ReadWriteSplit.md
 
[Read-Write-Service]
type=service
router=readwritesplit
servers=server1,server2,server3
user=maxuser
password=maxpwd
master_failure_mode=fail_on_write
master_accept_reads=true
 
# Listener definitions for the services
# Listeners represent the ports the services will listen on.
 
[Read-Only-Listener]
type=listener
service=Read-Only-Service
protocol=MySQLClient
port=4008
 
[Read-Write-Listener]
type=listener
service=Read-Write-Service
protocol=mariadbclient
port=4006
address=0.0.0.0
ssl=required
ssl_key=/etc/maxscale.cnf.d/cluster_ssl.key
ssl_cert=/etc/maxscale.cnf.d/cluster_ssl.crt
ssl_ca_cert=/etc/maxscale.cnf.d/cluster_ssl.ca
ssl_verify_peer_certificate=false

the ssl files are in the same folder as example.cnf

2. build sysbench (the one in the repos for my distro is old and does not have the "PREFERRED" ssl option).

    apt install git \
    && git clone https://github.com/akopytov/sysbench.git \
    && cd sysbench \
    && apt -y install make automake libtool pkg-config libaio-dev libmysqlclient-dev libssl-dev \
    && ./autogen.sh \
    && ./configure \
    && make -j \
    && make install \
    && mkdir /usr/share/sysbench/ \
    && cp -a src/lua/* /usr/share/sysbench/ \

3. run "docker-compose up -d" to start the cluster, use "docker ps -a" to ensure it is working

4. create the "sbtest" database on the server

mysql -h 127.0.0.1 -P 4006  -umaxuser -pmaxpwd --ssl --execute="create database sbtest;"

5. run sysbench

sysbench /usr/share/sysbench/oltp_read_write.lua --threads=4 --mysql-host=127.0.0.1 --mysql-user=maxuser --mysql-password=maxpwd --mysql-port=4006 --tables=2 --table_size=10000 --mysql_storage_engine=innodb --mysql-ssl=REQUIRED  prepare

6. here sysbench will stop on inserting into the database

sysbench 1.1.0-797e4c4 (using bundled LuaJIT 2.1.0-beta3)
 
Initializing worker threads...
 
Creating table 'sbtest1'...
Creating table 'sbtest2'...
Inserting 10000 records into 'sbtest2'
Inserting 10000 records into 'sbtest1'

Observations:
1. this only happens with ssl on
2. this works correctly with very small --table_size (<50), which looks like a timing/race condition
3. on the server side I see continuous recconects

  • on the server side I see the number of connections climbing, like it disconnects and reconnects every few seconds

        | Variable_name | Value |
        +---------------+-------+
        | Connections   | 688   |
    



 Comments   
Comment by markus makela [ 2020-08-27 ]

Seems to be caused by the different code used to calculate the number of bytes read when SSL is used.

Generated at Thu Feb 08 04:19:13 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.