Uploaded image for project: 'MariaDB MaxScale'
  1. MariaDB MaxScale
  2. MXS-4845

Maxscale: SSL certificates don't show up in SLAVE STATUS when Master rejoins after being Down

    XMLWordPrintable

Details

    • MXS-SPRINT-195

    Description

      SSL certificates don't show up in SLAVE STATUS when Master rejoins after being Down.

      MariaDB ver(Master & Slave): 10.6.7-3
      Maxscale ver: 6.4.11

      MariaDB Master config: master_server.cnf master_client.cnf
      MariaDB Slave config: slave_server.cnf slave_client.cnf
      Maxscale config: maxscale.cnf

      Steps to reproduce:
      1. Enable SSL replication and have Maxscale monitor the servers with SSL
      I followed this blog to create SSL certificates and enable SSL replication.

      [root@localhost ~]# maxctrl list servers
      ┌─────────┬───────────────┬──────┬─────────────┬─────────────────┬──────────┐
      │ Server  │ Address       │ Port │ Connections │ State           │ GTID     │
      ├─────────┼───────────────┼──────┼─────────────┼─────────────────┼──────────┤
      │ server1 │ 192.168.20.2233060           │ Master, Running │ 0-100-54
      ├─────────┼───────────────┼──────┼─────────────┼─────────────────┼──────────┤
      │ server2 │ 192.168.20.2833060           │ Slave, Running  │ 0-100-54
      └─────────┴───────────────┴──────┴─────────────┴─────────────────┴──────────┘
      

      2. SHOW SLAVE STATUS\G output from the slave server2

      MariaDB [(none)]> SHOW ALL SLAVES STATUS\G
      *************************** 1. row ***************************
                     Connection_name:
                     Slave_SQL_State: Slave has read all relay log; waiting for more updates
                      Slave_IO_State: Waiting for master to send event
                         Master_Host: 192.168.20.22
                         Master_User: repl_ssl
                         Master_Port: 3306
                       Connect_Retry: 60
                     Master_Log_File: mariadb-server2-bin.000016
                 Read_Master_Log_Pos: 405
                      Relay_Log_File: mariadb_slave3-relay-bin.000021
                       Relay_Log_Pos: 714
               Relay_Master_Log_File: mariadb-server2-bin.000016
                    Slave_IO_Running: Yes
                   Slave_SQL_Running: Yes
                     Replicate_Do_DB:
                 Replicate_Ignore_DB:
                  Replicate_Do_Table:
              Replicate_Ignore_Table:
             Replicate_Wild_Do_Table:
         Replicate_Wild_Ignore_Table:
                          Last_Errno: 0
                          Last_Error:
                        Skip_Counter: 0
                 Exec_Master_Log_Pos: 405
                     Relay_Log_Space: 1089
                     Until_Condition: None
                      Until_Log_File:
                       Until_Log_Pos: 0
                  Master_SSL_Allowed: Yes
                  Master_SSL_CA_File: /etc/my.cnf.d/certs/ca-cert.pem
                  Master_SSL_CA_Path:
                     Master_SSL_Cert: /etc/my.cnf.d/certs/client-cert.pem
                   Master_SSL_Cipher:
                      Master_SSL_Key: /etc/my.cnf.d/certs/client-key.pem
               Seconds_Behind_Master: 0
       Master_SSL_Verify_Server_Cert: No
                       Last_IO_Errno: 0
                       Last_IO_Error:
                      Last_SQL_Errno: 0
                      Last_SQL_Error:
         Replicate_Ignore_Server_Ids:
                    Master_Server_Id: 100
                      Master_SSL_Crl: /etc/my.cnf.d/certs/ca-cert.pem
                  Master_SSL_Crlpath:
                          Using_Gtid: Slave_Pos
                         Gtid_IO_Pos: 0-100-54
             Replicate_Do_Domain_Ids:
         Replicate_Ignore_Domain_Ids:
                       Parallel_Mode: optimistic
                           SQL_Delay: 0
                 SQL_Remaining_Delay: NULL
             Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
                    Slave_DDL_Groups: 8
      Slave_Non_Transactional_Groups: 1
          Slave_Transactional_Groups: 0
                Retried_transactions: 0
                  Max_relay_log_size: 1073741824
                Executed_log_entries: 128
           Slave_received_heartbeats: 0
              Slave_heartbeat_period: 30.000
                      Gtid_Slave_Pos: 0-100-54
      1 row in set (0.000 sec)
      

      3. stop mariadb service on Master and let the slave become the new Master

      systemctl stop mariadb
       
      maxctrl list servers
      ┌─────────┬───────────────┬──────┬─────────────┬────────────────┬──────────┐
      │ Server  │ Address       │ Port │ Connections │ State          │ GTID     │
      ├─────────┼───────────────┼──────┼─────────────┼────────────────┼──────────┤
      │ server1 │ 192.168.20.2233060           │ Down           │ 0-100-54
      ├─────────┼───────────────┼──────┼─────────────┼────────────────┼──────────┤
      │ server2 │ 192.168.20.2833060           │ Slave, Running │ 0-100-54
      └─────────┴───────────────┴──────┴─────────────┴────────────────┴──────────┘
       
      maxctrl list servers
      ┌─────────┬───────────────┬──────┬─────────────┬─────────────────┬──────────┐
      │ Server  │ Address       │ Port │ Connections │ State           │ GTID     │
      ├─────────┼───────────────┼──────┼─────────────┼─────────────────┼──────────┤
      │ server1 │ 192.168.20.2233060           │ Down            │ 0-100-54
      ├─────────┼───────────────┼──────┼─────────────┼─────────────────┼──────────┤
      │ server2 │ 192.168.20.2833060           │ Master, Running │ 0-100-54
      └─────────┴───────────────┴──────┴─────────────┴─────────────────┴──────────┘
      

      4. Now start the mariadb service

      systemctl start mariadb
       
      maxctrl list servers
      ┌─────────┬───────────────┬──────┬─────────────┬─────────────────┬──────────┐
      │ Server  │ Address       │ Port │ Connections │ State           │ GTID     │
      ├─────────┼───────────────┼──────┼─────────────┼─────────────────┼──────────┤
      │ server1 │ 192.168.20.2233060           │ Slave, Running  │ 0-100-54
      ├─────────┼───────────────┼──────┼─────────────┼─────────────────┼──────────┤
      │ server2 │ 192.168.20.2833060           │ Master, Running │ 0-100-54
      └─────────┴───────────────┴──────┴─────────────┴─────────────────┴──────────┘
      

      5. SLAVE STATUS output from server1 new Slave (previous Master)

      MariaDB [(none)]> SHOW ALL SLAVES STATUS\G
      *************************** 1. row ***************************
                     Connection_name:
                     Slave_SQL_State: Slave has read all relay log; waiting for more updates
                      Slave_IO_State: Waiting for master to send event
                         Master_Host: 192.168.20.28
                         Master_User: repl_ssl
                         Master_Port: 3306
                       Connect_Retry: 60
                     Master_Log_File: mariadb_slave3-bin.000002
                 Read_Master_Log_Pos: 2091
                      Relay_Log_File: mariadb-server2-relay-bin.000002
                       Relay_Log_Pos: 702
               Relay_Master_Log_File: mariadb_slave3-bin.000002
                    Slave_IO_Running: Yes
                   Slave_SQL_Running: Yes
                     Replicate_Do_DB:
                 Replicate_Ignore_DB:
                  Replicate_Do_Table:
              Replicate_Ignore_Table:
             Replicate_Wild_Do_Table:
         Replicate_Wild_Ignore_Table:
                          Last_Errno: 0
                          Last_Error:
                        Skip_Counter: 0
                 Exec_Master_Log_Pos: 2091
                     Relay_Log_Space: 1021
                     Until_Condition: None
                      Until_Log_File:
                       Until_Log_Pos: 0
                  Master_SSL_Allowed: Yes
                  Master_SSL_CA_File:
                  Master_SSL_CA_Path:
                     Master_SSL_Cert:
                   Master_SSL_Cipher:
                      Master_SSL_Key:
               Seconds_Behind_Master: 0
       Master_SSL_Verify_Server_Cert: No
                       Last_IO_Errno: 0
                       Last_IO_Error:
                      Last_SQL_Errno: 0
                      Last_SQL_Error:
         Replicate_Ignore_Server_Ids:
                    Master_Server_Id: 200
                      Master_SSL_Crl:
                  Master_SSL_Crlpath:
                          Using_Gtid: Current_Pos
                         Gtid_IO_Pos: 0-100-54
             Replicate_Do_Domain_Ids:
         Replicate_Ignore_Domain_Ids:
                       Parallel_Mode: optimistic
                           SQL_Delay: 0
                 SQL_Remaining_Delay: NULL
             Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
                    Slave_DDL_Groups: 0
      Slave_Non_Transactional_Groups: 0
          Slave_Transactional_Groups: 0
                Retried_transactions: 0
                  Max_relay_log_size: 1073741824
                Executed_log_entries: 8
           Slave_received_heartbeats: 0
              Slave_heartbeat_period: 30.000
                      Gtid_Slave_Pos: 0-100-54
      1 row in set (0.000 sec)
      

      Shows Master_SSL_Allowed: Yes but certificates are not displayed.

      Error log

      2023-11-01 11:34:06 0 [Note] /usr/sbin/mariadbd: ready for connections.
      Version: '10.6.7-3-MariaDB-enterprise-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MariaDB Enterprise Server
      2023-11-01 11:34:07 4 [Note] Master connection name: ''  Master_info_file: 'master.info'  Relay_info_file: 'relay-log.info'
      2023-11-01 11:34:07 4 [Warning] Neither --relay-log nor --relay-log-index were used; so replication may break when this MariaDB server acts as a replica and has its hostname changed. Please use '--log-basename=#' or '--relay-log=mariadb-server2-relay-bin' to avoid this problem.
      2023-11-01 11:34:07 4 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='', master_port='3306', master_log_file='', master_log_pos='4'. New state master_host='192.168.20.28', master_port='3306', master_log_file='', master_log_pos='4'.
      2023-11-01 11:34:07 4 [Note] Previous Using_Gtid=No. New Using_Gtid=Current_Pos
      2023-11-01 11:34:07 5 [Note] Slave I/O thread: Start asynchronous replication to master 'repl_ssl@192.168.20.28:3306' in log '' at position 4
      2023-11-01 11:34:07 6 [Note] Slave SQL thread initialized, starting replication in log 'FIRST' at position 4, relay log './mariadb-server2-relay-bin.000001' position: 4; GTID position '0-100-54'
      2023-11-01 11:34:07 5 [Note] Slave I/O thread: connected to master 'repl_ssl@192.168.20.28:3306',replication starts at GTID position '0-100-54'
      

      Attachments

        1. master_client.cnf
          0.1 kB
        2. master_server.cnf
          0.3 kB
        3. maxscale.cnf
          2 kB
        4. slave_client.cnf
          0.1 kB
        5. slave_server.cnf
          0.3 kB

        Issue Links

          Activity

            People

              esa.korhonen Esa Korhonen
              susmeet.khaire Susmeet Khaire
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.