Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-19315

Error 1236 after startup Master Slave with Maxscale monitor

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 10.3.13
    • Fix Version/s: 10.3
    • Component/s: Replication
    • Labels:
    • Environment:
      Docker version 18.09.2, build 6247962
      Kubernetes v1.13.2 - Local cluster - docker-for-desktop
      Maxscale 2.3.4
      MariaDB 10.3.13 (The most up-to-date for the official docker container)
      Master Slave Cluster

      Description

      Hello,

      I am trying to make a Master Slave cluster that is resilient to failover and rejoining.
      Whenever I start the cluster the replication starts and ends up on the wrong GTID position and fails with a 1236 error code.

      my.cnf for the MariaDB servers

      [mariadb]
      log-bin
      max_allowed_packet=128M
      gtid_strict_mode=1
      skip-name-resolve
      

      Here is the bash script to start the MariaDB servers

      [[ $(hostname) =~ -([0-9]+)$ ]] || exit 1
      export server_id=${BASH_REMATCH[1]}
       
      /usr/local/bin/docker-entrypoint.sh mysqld --log-bin=mariadb-bin --binlog-format=ROW --server-id=$((3000 + $server_id)) --log-slave-updates=1 --gtid-strict-mode=1 --innodb-flush-method=fsync --extra-port=3307 --extra_max_connections=1
      

      maxscale.cnf - the configuration file for the maxscale instance

      [maxscale]
      threads=auto
      admin_enabled=false
       
      [server0]
      type=server
      address=mariadb-0.mariadb.mariadb.svc.cluster.local
      port=3306
      protocol=MariaDBBackend
      persistpoolmax=10
      persistmaxtime=999999999999
       
      [server1]
      type=server
      address=mariadb-1.mariadb.mariadb.svc.cluster.local
      port=3306
      protocol=MariaDBBackend
      persistpoolmax=10
      persistmaxtime=999999999999
       
      [server2]
      type=server
      address=mariadb-2.mariadb.mariadb.svc.cluster.local
      port=3306
      protocol=MariaDBBackend
      persistpoolmax=10
      persistmaxtime=999999999999
       
      [Replication-Monitor]
      type=monitor
      module=mariadbmon
      servers=server0,server1,server2
      user=*****
      password=*****
      replication_user=*****
      replication_password=*****
      monitor_interval=2000
      auto_failover=true
      auto_rejoin=true
       
      [Splitter-Service]
      type=service
      router=readwritesplit
      servers=server0,server1,server2
      user=*****
      password=*****
       
      [Splitter-Listener]
      type=listener
      service=Splitter-Service
      protocol=MariaDBClient
      port=3306
       
      # MaxAdmin administrative interface
      [CLI]
      type=service
      router=cli
       
      [CLI-Listener]
      type=listener
      service=CLI
      protocol=maxscaled
      socket=default
      

      I checked the permissions for the user on the Maxscale monitor and I think the user has all the permissions it needs. Just in case, I tried this with the root user and an admin user with all of the possible permissions and I still get the same error.

      Here are the logs for the Master MariaDB server (serverid = 3000):

      Initializing database
      2019-04-23 13:01:03 6 [Warning] Failed to load slave replication state from table mysql.gtid_slave_pos: 1146: Table 'mysql.gtid_slave_pos' doesn't exist
       
       
      PLEASE REMEMBER TO SET A PASSWORD FOR THE MariaDB root USER !
      To do so, start the server, then issue the following commands:
       
      '/usr/bin/mysqladmin' -u root password 'new-password'
      '/usr/bin/mysqladmin' -u root -h  password 'new-password'
       
      Alternatively you can run:
      '/usr/bin/mysql_secure_installation'
       
      which will also give you the option of removing the test
      databases and anonymous user created by default.  This is
      strongly recommended for production servers.
       
      See the MariaDB Knowledgebase at http://mariadb.com/kb or the
      MySQL manual for more instructions.
       
      Please report any problems at http://mariadb.org/jira
       
      The latest information about MariaDB is available at http://mariadb.org/.
      You can find additional information about the MySQL part at:
      http://dev.mysql.com
      Consider joining MariaDB's strong and vibrant community:
      https://mariadb.org/get-involved/
       
      Database initialized
      MySQL init process in progress...
      2019-04-23 13:01:05 0 [Note] mysqld (mysqld 10.3.13-MariaDB-1:10.3.13+maria~bionic-log) starting as process 108 ...
      2019-04-23 13:01:05 0 [Note] InnoDB: Using Linux native AIO
      2019-04-23 13:01:05 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
      2019-04-23 13:01:05 0 [Note] InnoDB: Uses event mutexes
      2019-04-23 13:01:05 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
      2019-04-23 13:01:05 0 [Note] InnoDB: Number of pools: 1
      2019-04-23 13:01:05 0 [Note] InnoDB: Using SSE2 crc32 instructions
      2019-04-23 13:01:05 0 [Note] InnoDB: Initializing buffer pool, total size = 256M, instances = 1, chunk size = 128M
      2019-04-23 13:01:05 0 [Note] InnoDB: Completed initialization of buffer pool
      2019-04-23 13:01:05 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
      2019-04-23 13:01:05 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
      2019-04-23 13:01:05 0 [Note] InnoDB: Creating shared tablespace for temporary tables
      2019-04-23 13:01:05 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
      2019-04-23 13:01:05 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
      2019-04-23 13:01:05 0 [Note] InnoDB: Waiting for purge to start
      2019-04-23 13:01:06 0 [Note] InnoDB: 10.3.13 started; log sequence number 1630815; transaction id 21
      2019-04-23 13:01:06 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
      2019-04-23 13:01:06 0 [Note] Plugin 'FEEDBACK' is disabled.
      2019-04-23 13:01:06 0 [Note] InnoDB: Buffer pool(s) load completed at 190423 13:01:06
      2019-04-23 13:01:06 0 [Warning] 'user' entry 'root@mariadb-0' ignored in --skip-name-resolve mode.
      2019-04-23 13:01:06 0 [Warning] 'user' entry '@mariadb-0' ignored in --skip-name-resolve mode.
      2019-04-23 13:01:06 0 [Warning] 'proxies_priv' entry '@% root@mariadb-0' ignored in --skip-name-resolve mode.
      2019-04-23 13:01:06 0 [Note] Reading of all Master_info entries succeded
      2019-04-23 13:01:06 0 [Note] Added new Master_info '' to hash table
      2019-04-23 13:01:06 0 [Note] mysqld: ready for connections.
      Version: '10.3.13-MariaDB-1:10.3.13+maria~bionic-log'  socket: '/var/run/mysqld/mysqld.sock'  port: 0  mariadb.org binary distribution
      Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
      2019-04-23 13:01:09 11 [Warning] 'proxies_priv' entry '@% root@mariadb-0' ignored in --skip-name-resolve mode.
       
      /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/init.sql
       
       
      2019-04-23 13:01:09 0 [Note] mysqld (initiated by: unknown): Normal shutdown
      2019-04-23 13:01:09 0 [Note] Event Scheduler: Purging the queue. 0 events
      2019-04-23 13:01:09 0 [Note] InnoDB: FTS optimize thread exiting.
      2019-04-23 13:01:09 0 [Note] InnoDB: Starting shutdown...
      2019-04-23 13:01:09 0 [Note] InnoDB: Dumping buffer pool(s) to /var/lib/mysql/ib_buffer_pool
      2019-04-23 13:01:09 0 [Note] InnoDB: Buffer pool(s) dump completed at 190423 13:01:09
      2019-04-23 13:01:10 0 [Note] InnoDB: Shutdown completed; log sequence number 1630824; transaction id 24
      2019-04-23 13:01:10 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
      2019-04-23 13:01:10 0 [Note] mysqld: Shutdown complete
       
       
      MySQL init process done. Ready for start up.
       
      2019-04-23 13:01:10 0 [Note] mysqld (mysqld 10.3.13-MariaDB-1:10.3.13+maria~bionic-log) starting as process 7 ...
      2019-04-23 13:01:11 0 [Note] InnoDB: Using Linux native AIO
      2019-04-23 13:01:11 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
      2019-04-23 13:01:11 0 [Note] InnoDB: Uses event mutexes
      2019-04-23 13:01:11 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
      2019-04-23 13:01:11 0 [Note] InnoDB: Number of pools: 1
      2019-04-23 13:01:11 0 [Note] InnoDB: Using SSE2 crc32 instructions
      2019-04-23 13:01:11 0 [Note] InnoDB: Initializing buffer pool, total size = 256M, instances = 1, chunk size = 128M
      2019-04-23 13:01:11 0 [Note] InnoDB: Completed initialization of buffer pool
      2019-04-23 13:01:11 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
      2019-04-23 13:01:11 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
      2019-04-23 13:01:11 0 [Note] InnoDB: Creating shared tablespace for temporary tables
      2019-04-23 13:01:11 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
      2019-04-23 13:01:11 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
      2019-04-23 13:01:11 0 [Note] InnoDB: Waiting for purge to start
      2019-04-23 13:01:11 0 [Note] InnoDB: 10.3.13 started; log sequence number 1630824; transaction id 21
      2019-04-23 13:01:11 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
      2019-04-23 13:01:11 0 [Note] Plugin 'FEEDBACK' is disabled.
      2019-04-23 13:01:11 0 [Note] InnoDB: Buffer pool(s) load completed at 190423 13:01:11
      2019-04-23 13:01:11 0 [Note] Server socket created on IP: '::'.
      2019-04-23 13:01:11 0 [Note] Server socket created on IP: '::'.
      2019-04-23 13:01:11 0 [Warning] 'proxies_priv' entry '@% root@mariadb-0' ignored in --skip-name-resolve mode.
      2019-04-23 13:01:11 0 [Note] Reading of all Master_info entries succeded
      2019-04-23 13:01:11 0 [Note] Added new Master_info '' to hash table
      2019-04-23 13:01:11 0 [Note] mysqld: ready for connections.
      Version: '10.3.13-MariaDB-1:10.3.13+maria~bionic-log'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
      2019-04-23 13:01:13 17 [Note] Start binlog_dump to slave_server(3001), pos(, 4)
      2019-04-23 13:01:19 19 [Note] Start binlog_dump to slave_server(3002), pos(, 4)
      

      Here are the logs for one of the slave MariaDB servers (serverid = 3001):

      Initializing database
      2019-04-23 13:01:06 6 [Warning] Failed to load slave replication state from table mysql.gtid_slave_pos: 1146: Table 'mysql.gtid_slave_pos' doesn't exist
       
       
      PLEASE REMEMBER TO SET A PASSWORD FOR THE MariaDB root USER !
      To do so, start the server, then issue the following commands:
       
      '/usr/bin/mysqladmin' -u root password 'new-password'
      '/usr/bin/mysqladmin' -u root -h  password 'new-password'
       
      Alternatively you can run:
      '/usr/bin/mysql_secure_installation'
       
      which will also give you the option of removing the test
      databases and anonymous user created by default.  This is
      strongly recommended for production servers.
       
      See the MariaDB Knowledgebase at http://mariadb.com/kb or the
      MySQL manual for more instructions.
       
      Please report any problems at http://mariadb.org/jira
       
      The latest information about MariaDB is available at http://mariadb.org/.
      You can find additional information about the MySQL part at:
      http://dev.mysql.com
      Consider joining MariaDB's strong and vibrant community:
      https://mariadb.org/get-involved/
       
      Database initialized
      MySQL init process in progress...
      2019-04-23 13:01:08 0 [Note] mysqld (mysqld 10.3.13-MariaDB-1:10.3.13+maria~bionic-log) starting as process 111 ...
      2019-04-23 13:01:08 0 [Note] InnoDB: Using Linux native AIO
      2019-04-23 13:01:08 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
      2019-04-23 13:01:08 0 [Note] InnoDB: Uses event mutexes
      2019-04-23 13:01:08 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
      2019-04-23 13:01:08 0 [Note] InnoDB: Number of pools: 1
      2019-04-23 13:01:08 0 [Note] InnoDB: Using SSE2 crc32 instructions
      2019-04-23 13:01:08 0 [Note] InnoDB: Initializing buffer pool, total size = 256M, instances = 1, chunk size = 128M
      2019-04-23 13:01:08 0 [Note] InnoDB: Completed initialization of buffer pool
      2019-04-23 13:01:08 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
      2019-04-23 13:01:08 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
      2019-04-23 13:01:08 0 [Note] InnoDB: Creating shared tablespace for temporary tables
      2019-04-23 13:01:08 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
      2019-04-23 13:01:08 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
      2019-04-23 13:01:08 0 [Note] InnoDB: Waiting for purge to start
      2019-04-23 13:01:08 0 [Note] InnoDB: 10.3.13 started; log sequence number 1630815; transaction id 21
      2019-04-23 13:01:08 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
      2019-04-23 13:01:08 0 [Note] Plugin 'FEEDBACK' is disabled.
      2019-04-23 13:01:08 0 [Note] InnoDB: Buffer pool(s) load completed at 190423 13:01:08
      2019-04-23 13:01:08 0 [Warning] 'user' entry 'root@mariadb-1' ignored in --skip-name-resolve mode.
      2019-04-23 13:01:08 0 [Warning] 'user' entry '@mariadb-1' ignored in --skip-name-resolve mode.
      2019-04-23 13:01:08 0 [Warning] 'proxies_priv' entry '@% root@mariadb-1' ignored in --skip-name-resolve mode.
      2019-04-23 13:01:08 0 [Note] Reading of all Master_info entries succeded
      2019-04-23 13:01:08 0 [Note] Added new Master_info '' to hash table
      2019-04-23 13:01:08 0 [Note] mysqld: ready for connections.
      Version: '10.3.13-MariaDB-1:10.3.13+maria~bionic-log'  socket: '/var/run/mysqld/mysqld.sock'  port: 0  mariadb.org binary distribution
      Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
      2019-04-23 13:01:11 11 [Warning] 'proxies_priv' entry '@% root@mariadb-1' ignored in --skip-name-resolve mode.
       
      /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/init.sql
       
       
      2019-04-23 13:01:11 0 [Note] mysqld (initiated by: unknown): Normal shutdown
      2019-04-23 13:01:11 0 [Note] Event Scheduler: Purging the queue. 0 events
      2019-04-23 13:01:11 0 [Note] InnoDB: FTS optimize thread exiting.
      2019-04-23 13:01:11 0 [Note] InnoDB: Starting shutdown...
      2019-04-23 13:01:11 0 [Note] InnoDB: Dumping buffer pool(s) to /var/lib/mysql/ib_buffer_pool
      2019-04-23 13:01:11 0 [Note] InnoDB: Buffer pool(s) dump completed at 190423 13:01:11
      2019-04-23 13:01:12 0 [Note] InnoDB: Shutdown completed; log sequence number 1630824; transaction id 24
      2019-04-23 13:01:12 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
      2019-04-23 13:01:12 0 [Note] mysqld: Shutdown complete
       
       
      MySQL init process done. Ready for start up.
       
      2019-04-23 13:01:12 0 [Note] mysqld (mysqld 10.3.13-MariaDB-1:10.3.13+maria~bionic-log) starting as process 8 ...
      2019-04-23 13:01:12 0 [Note] InnoDB: Using Linux native AIO
      2019-04-23 13:01:12 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
      2019-04-23 13:01:12 0 [Note] InnoDB: Uses event mutexes
      2019-04-23 13:01:12 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
      2019-04-23 13:01:12 0 [Note] InnoDB: Number of pools: 1
      2019-04-23 13:01:12 0 [Note] InnoDB: Using SSE2 crc32 instructions
      2019-04-23 13:01:12 0 [Note] InnoDB: Initializing buffer pool, total size = 256M, instances = 1, chunk size = 128M
      2019-04-23 13:01:12 0 [Note] InnoDB: Completed initialization of buffer pool
      2019-04-23 13:01:12 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
      2019-04-23 13:01:12 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
      2019-04-23 13:01:12 0 [Note] InnoDB: Creating shared tablespace for temporary tables
      2019-04-23 13:01:12 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
      2019-04-23 13:01:12 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
      2019-04-23 13:01:12 0 [Note] InnoDB: Waiting for purge to start
      2019-04-23 13:01:12 0 [Note] InnoDB: 10.3.13 started; log sequence number 1630824; transaction id 21
      2019-04-23 13:01:12 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
      2019-04-23 13:01:12 0 [Note] Plugin 'FEEDBACK' is disabled.
      2019-04-23 13:01:12 0 [Note] InnoDB: Buffer pool(s) load completed at 190423 13:01:12
      2019-04-23 13:01:12 0 [Note] Server socket created on IP: '::'.
      2019-04-23 13:01:12 0 [Note] Server socket created on IP: '::'.
      2019-04-23 13:01:12 0 [Warning] 'proxies_priv' entry '@% root@mariadb-1' ignored in --skip-name-resolve mode.
      2019-04-23 13:01:12 0 [Note] Reading of all Master_info entries succeded
      2019-04-23 13:01:12 0 [Note] Added new Master_info '' to hash table
      2019-04-23 13:01:12 0 [Note] mysqld: ready for connections.
      Version: '10.3.13-MariaDB-1:10.3.13+maria~bionic-log'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
      2019-04-23 13:01:13 10 [Note] Master connection name: ''  Master_info_file: 'master.info'  Relay_info_file: 'relay-log.info'
      2019-04-23 13:01:13 10 [Warning] Neither --relay-log nor --relay-log-index were used; so replication may break when this MySQL server acts as a slave and has his hostname changed!! Please use '--log-basename=#' or '--relay-log=mysqld-relay-bin' to avoid this problem.
      2019-04-23 13:01:13 10 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='', master_port='3306', master_log_file='', master_log_pos='4'. New state master_host='mariadb-0.mariadb.mariadb.svc.cluster.local', master_port='3306', master_log_file='', master_log_pos='4'.
      2019-04-23 13:01:13 10 [Note] Previous Using_Gtid=No. New Using_Gtid=Current_Pos
      2019-04-23 13:01:13 11 [Note] Slave I/O thread: Start asynchronous replication to master 'root@mariadb-0.mariadb.mariadb.svc.cluster.local:3306' in log '' at position 4
      2019-04-23 13:01:13 12 [Note] Slave SQL thread initialized, starting replication in log 'FIRST' at position 4, relay log './mysqld-relay-bin.000001' position: 4; GTID position '0-3001-7235'
      2019-04-23 13:01:13 11 [Note] Slave I/O thread: connected to master 'root@mariadb-0.mariadb.mariadb.svc.cluster.local:3306',replication starts at GTID position '0-3001-7235'
      2019-04-23 13:01:13 11 [ERROR] Error reading packet from server: Error: connecting slave requested to start from GTID 0-3001-7235, which is not in the master's binlog. Since the master's binlog contains GTIDs with higher sequence numbers, it probably means that the slave has diverged due to executing extra erroneous transactions (server_errno=1236)
      2019-04-23 13:01:13 11 [ERROR] Slave I/O: Got fatal error 1236 from master when reading data from binary log: 'Error: connecting slave requested to start from GTID 0-3001-7235, which is not in the master's binlog. Since the master's binlog contains GTIDs with higher sequence numbers, it probably means that the slave has diverged due to executing extra erroneous transactions', Internal MariaDB error code: 1236
      2019-04-23 13:01:13 11 [Note] Slave I/O thread exiting, read up to log 'FIRST', position 4; GTID position 0-3001-7235
      

      Here are the logs from the Maxscale instance

      Info : MaxScale will be run in the terminal process.
      2019-04-23 13:01:13   notice : syslog logging is enabled.
      2019-04-23 13:01:13   notice : maxlog logging is enabled.
      2019-04-23 13:01:13   notice : Using up to 3.90GiB of memory for query classifier cache
      2019-04-23 13:01:13   notice : The collection of SQLite memory allocation statistics turned off.
      2019-04-23 13:01:13   notice : Threading mode of SQLite set to Multi-thread.
      2019-04-23 13:01:13   notice : MariaDB MaxScale 2.3.4 started (Commit: aea64aede280558ca6b55500dfa7eb049ec9c377)
      2019-04-23 13:01:13   notice : MaxScale is running in process 1
      Configuration file : /etc/maxscale.cnf
      Log directory      : /var/log/maxscale
      Data directory     : /var/lib/maxscale
      Module directory   : /usr/lib/x86_64-linux-gnu/maxscale
      Service cache      : /var/cache/maxscale
       
      2019-04-23 13:01:13   notice : Configuration file: /etc/maxscale.cnf
      2019-04-23 13:01:13   notice : Log directory: /var/log/maxscale
      2019-04-23 13:01:13   notice : Data directory: /var/lib/maxscale
      2019-04-23 13:01:13   notice : Module directory: /usr/lib/x86_64-linux-gnu/maxscale
      2019-04-23 13:01:13   notice : Service cache: /var/cache/maxscale
      2019-04-23 13:01:13   notice : No query classifier specified, using default 'qc_sqlite'.
      2019-04-23 13:01:13   notice : Loaded module qc_sqlite: V1.0.0 from /usr/lib/x86_64-linux-gnu/maxscale/libqc_sqlite.so
      2019-04-23 13:01:13   notice : Query classification results are cached and reused. Memory used per thread: 665.14MiB
      2019-04-23 13:01:13   notice : Loading /etc/maxscale.cnf.
      2019-04-23 13:01:13   notice : /etc/maxscale.cnf.d does not exist, not reading.
      2019-04-23 13:01:13   notice : Loaded module maxscaled: V2.0.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmaxscaled.so
      2019-04-23 13:01:13   warning: THE 'cli' MODULE AND 'maxadmin' ARE DEPRECATED: Use 'maxctrl' instead
      2019-04-23 13:01:13   notice : Loaded module cli: V1.0.0 from /usr/lib/x86_64-linux-gnu/maxscale/libcli.so
      2019-04-23 13:01:13   notice : Loaded module MariaDBClient: V1.1.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmariadbclient.so
      2019-04-23 13:01:13   notice : Initializing statement-based read/write split router module.
      2019-04-23 13:01:13   notice : Loaded module readwritesplit: V1.1.0 from /usr/lib/x86_64-linux-gnu/maxscale/libreadwritesplit.so
      2019-04-23 13:01:13   notice : Initialise the MariaDB Monitor module.
      2019-04-23 13:01:13   notice : Loaded module mariadbmon: V1.5.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmariadbmon.so
      2019-04-23 13:01:13   notice : Loaded module MariaDBBackend: V2.0.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmariadbbackend.so
      2019-04-23 13:01:13   notice : Loaded module MySQLBackendAuth: V1.0.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmysqlbackendauth.so
      2019-04-23 13:01:13   notice : Loaded module MaxAdminAuth: V2.1.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmaxadminauth.so
      2019-04-23 13:01:13   notice : Loaded module MySQLAuth: V1.1.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmysqlauth.so
      2019-04-23 13:01:13   notice : Housekeeper thread started.
      2019-04-23 13:01:13   warning: Removing stale journal file for monitor 'Replication-Monitor'.
      2019-04-23 13:01:13   notice : Encrypted password file /var/lib/maxscale/.secrets can't be accessed (No such file or directory). Password encryption is not used.
      2019-04-23 13:01:13   notice : Starting a total of 2 services...
      2019-04-23 13:01:13   notice : Listening for connections at [/var/run/maxscale/maxadmin.sock]:0 with protocol MaxScale Admin
      2019-04-23 13:01:13   notice : Service 'CLI' started (1/2)
      2019-04-23 13:01:13   error  : [Splitter-Service] Failed to connect to server 'server2' ([mariadb-2.mariadb.mariadb.svc.cluster.local]:3306) when checking authentication user credentials and permissions: 2002 Can't connect to MySQL server on 'mariadb-2.mariadb.mariadb.svc.cluster.local' (115)
      2019-04-23 13:01:13   notice : [Splitter-Service] Loaded 4 MySQL users for listener Splitter-Listener.
      2019-04-23 13:01:13   notice : Listening for connections at [::]:3306 with protocol MySQL
      2019-04-23 13:01:13   notice : Service 'Splitter-Service' started (2/2)
      2019-04-23 13:01:13   error  : Monitor was unable to connect to server server2[mariadb-2.mariadb.mariadb.svc.cluster.local:3306] : 'Can't connect to MySQL server on 'mariadb-2.mariadb.mariadb.svc.cluster.local' (115)'
      2019-04-23 13:01:13   notice : Selecting new master server.
      2019-04-23 13:01:13   warning: 'server2' is not a valid master candidate because it is down.
      2019-04-23 13:01:13   notice : Setting 'server0' as master.
      2019-04-23 13:01:13   notice : Server changed state: server0[mariadb-0.mariadb.mariadb.svc.cluster.local:3306]: new_master. [Running] -> [Master, Running]
      2019-04-23 13:01:13   notice : Server changed state: server2[mariadb-2.mariadb.mariadb.svc.cluster.local:3306]: server_down. [Running] -> [Down]
      2019-04-23 13:01:13   notice : Directing standalone server 'server1' to replicate from 'server0'.
      2019-04-23 13:01:13   notice : Slave connection from server1 to [mariadb-0.mariadb.mariadb.svc.cluster.local]:3306 created and started.
      2019-04-23 13:01:13   notice : 1 server(s) redirected or rejoined the cluster.
      2019-04-23 13:01:13   notice : MaxScale started with 6 worker threads, each with a stack size of 8388608 bytes.
      2019-04-23 13:01:19   notice : Server changed state: server2[mariadb-2.mariadb.mariadb.svc.cluster.local:3306]: server_up. [Down] -> [Running]
      2019-04-23 13:01:19   notice : Directing standalone server 'server2' to replicate from 'server0'.
      2019-04-23 13:01:19   notice : Slave connection from server2 to [mariadb-0.mariadb.mariadb.svc.cluster.local]:3306 created and started.
      2019-04-23 13:01:19   notice : 1 server(s) redirected or rejoined the cluster.
      

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              Elkin Andrei Elkin
              Reporter:
              ngustaf Nicholas Gustafson
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated: