Uploaded image for project: 'MariaDB MaxScale'
  1. MariaDB MaxScale
  2. MXS-3135

Maxscale fails when doing "prepare" with sysbench and ssl enabled

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • 2.5
    • 2.5.3
    • Core
    • None

    Description

      Short summary: when using sysbench against maxscale 2.5 fails in the "prepare" stage when ssl is enabled. It used to work with 2.4

      steps to reproduce:
      1. start the example docker-compose from the maxscale repo, but with ssl enabled
      the repo is here
      https://github.com/mariadb-corporation/maxscale-docker/tree/maxscale-2.5
      I've changed the images in the docker-compose.yaml to 10.5 instead of 10.3
      there are also some changes to the example.cnf to enable ssl

      [server1]
      type=server
      address=master
      port=3306
      protocol=MariaDBBackend
       
      [server2]
      type=server
      address=slave1
      port=3306
      protocol=MariaDBBackend
       
      [server3]
      type=server
      address=slave2
      port=3306
      protocol=MariaDBBackend
       
      # Monitor for the servers
      # This will keep MaxScale aware of the state of the servers.
      # MySQL Monitor documentation:
      # https://github.com/mariadb-corporation/MaxScale/blob/2.3/Documentation/Monitors/MariaDB-Monitor.md
       
      [MariaDB-Monitor]
      type=monitor
      module=mariadbmon
      servers=server1,server2,server3
      user=maxuser
      password=maxpwd
      failcount=3
      backend_connect_timeout=3
      backend_write_timeout=3
      backend_read_timeout=3
      auto_failover=true
      auto_rejoin=true
      enforce_read_only_slaves=1
       
      # Service definitions
      # Service Definition for a read-only service and a read/write splitting service.
       
      # ReadConnRoute documentation:
      # https://github.com/mariadb-corporation/MaxScale/blob/2.3/Documentation/Routers/ReadConnRoute.md
       
      [Read-Only-Service]
      type=service
      router=readconnroute
      servers=server1,server2,server3
      user=maxuser
      password=maxpwd
      router_options=slave
       
      # ReadWriteSplit documentation:
      # https://github.com/mariadb-corporation/MaxScale/blob/2.3/Documentation/Routers/ReadWriteSplit.md
       
      [Read-Write-Service]
      type=service
      router=readwritesplit
      servers=server1,server2,server3
      user=maxuser
      password=maxpwd
      master_failure_mode=fail_on_write
      master_accept_reads=true
       
      # Listener definitions for the services
      # Listeners represent the ports the services will listen on.
       
      [Read-Only-Listener]
      type=listener
      service=Read-Only-Service
      protocol=MySQLClient
      port=4008
       
      [Read-Write-Listener]
      type=listener
      service=Read-Write-Service
      protocol=mariadbclient
      port=4006
      address=0.0.0.0
      ssl=required
      ssl_key=/etc/maxscale.cnf.d/cluster_ssl.key
      ssl_cert=/etc/maxscale.cnf.d/cluster_ssl.crt
      ssl_ca_cert=/etc/maxscale.cnf.d/cluster_ssl.ca
      ssl_verify_peer_certificate=false
      

      the ssl files are in the same folder as example.cnf

      2. build sysbench (the one in the repos for my distro is old and does not have the "PREFERRED" ssl option).

          apt install git \
          && git clone https://github.com/akopytov/sysbench.git \
          && cd sysbench \
          && apt -y install make automake libtool pkg-config libaio-dev libmysqlclient-dev libssl-dev \
          && ./autogen.sh \
          && ./configure \
          && make -j \
          && make install \
          && mkdir /usr/share/sysbench/ \
          && cp -a src/lua/* /usr/share/sysbench/ \
      

      3. run "docker-compose up -d" to start the cluster, use "docker ps -a" to ensure it is working

      4. create the "sbtest" database on the server

      mysql -h 127.0.0.1 -P 4006  -umaxuser -pmaxpwd --ssl --execute="create database sbtest;"
      

      5. run sysbench

      sysbench /usr/share/sysbench/oltp_read_write.lua --threads=4 --mysql-host=127.0.0.1 --mysql-user=maxuser --mysql-password=maxpwd --mysql-port=4006 --tables=2 --table_size=10000 --mysql_storage_engine=innodb --mysql-ssl=REQUIRED  prepare
      

      6. here sysbench will stop on inserting into the database

      sysbench 1.1.0-797e4c4 (using bundled LuaJIT 2.1.0-beta3)
       
      Initializing worker threads...
       
      Creating table 'sbtest1'...
      Creating table 'sbtest2'...
      Inserting 10000 records into 'sbtest2'
      Inserting 10000 records into 'sbtest1'
      

      Observations:
      1. this only happens with ssl on
      2. this works correctly with very small --table_size (<50), which looks like a timing/race condition
      3. on the server side I see continuous recconects

      • on the server side I see the number of connections climbing, like it disconnects and reconnects every few seconds

            | Variable_name | Value |
            +---------------+-------+
            | Connections   | 688   |
        

      Attachments

        Activity

          People

            markus makela markus makela
            petko.vasilev Petko Vasilev (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.