Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-19669

proxy_protocol_networks does not evaluate/parse properly

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • 10.3.15
    • 10.3.21, 10.4.11, 10.5.1
    • Server
    • Ubuntu 18.04.2 LTS, MariaDB-1:10.3.15+maria~bionic

    Description

      Hi,

      proxy_protocol_networks does not seem to work properly in my setup. I have one working setup, and one non-working setup. The setups are equal, except hostnames and IP-addresses.

      Working setup;

      • db-demo-01 (172.20.20.61)
      • db-demo-02 (172.20.20.62)
      • db-demo-02 (172.20.20.62)
      • db-demo-ha01 (172.20.20.28)
      • db-demo-ha02 (172.20.20.29)

      Non-working setup:

      • db-prod-01 (172.20.20.71)
      • db-prod-02 (172.20.20.72)
      • db-prod-02 (172.20.20.72)
      • db-prod-ha01 (172.20.20.38)
      • db-prod-ha02 (172.20.20.39)

      All hosts are running Ubuntu 18.04.2 LTS. All within the same subnet (172.20.20.0/24). All ha*-nodes are running haproxy. The non-ha-nodes are running MariaDB Galera cluster.

      I have the following in my.cnf on all MariaDB nodes;

      proxy_protocol_networks=::1, 127.0.0.1, localhost, 172.20.20.0/24
      

      A snippet of the haproxy.cfg;

      server db-demo-01 db-demo-01.example.com:3306  check send-proxy-v2
      

      The Proxy Protocol works as expected on the 'demo' cluster. However, on the 'prod' cluster, having the exact same configuration (including the proxy_protocol_networks config above), haproxy cannot connect, complaining as following;

      Server mysql_cluster/db-prod-01 is DOWN, reason: Layer7 wrong status, code: 0, info: "#HY000Proxy header is not accepted from 172.20.20.38", check duration: 0ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.

      That IP is within the 172.20.20.0/24 range specified. If I explicitly list the IP (without CIDR notation) in the config, it works as expected;

      proxy_protocol_networks=::1, 127.0.0.1, localhost, 172.20.20.0/24, 172.20.20.38, 172.20.20.39
      

      The order of the 172.20.20.0/24 does not matter (i.e. it's not because it was bad parsing due to being the last entry in the list, or similar).

      Attachments

        Activity

          minorOffense Mathew Winstone added a comment - - edited

          I have the same problem.

          [root@dco-prod-db1 my.cnf.d]# yum list installed | grep -i maria
          MariaDB-backup.x86_64               10.3.17-1.el7.centos       @mariadb
          MariaDB-client.x86_64               10.3.17-1.el7.centos       @mariadb
          MariaDB-common.x86_64               10.3.17-1.el7.centos       @mariadb
          MariaDB-compat.x86_64               10.3.17-1.el7.centos       @mariadb
          MariaDB-devel.x86_64                10.3.17-1.el7.centos       @mariadb
          MariaDB-server.x86_64               10.3.17-1.el7.centos       @mariadb
          galera.x86_64                       25.3.26-1.rhel7.el7.centos @mariadb
          

          I have

          proxy-protocol-networks = 10.0.0.0/8,127.0.0.1/32,localhost
          

          In server.cnf on both clusters. And my haproxy config is

          # This file managed by Puppet
          global
            chroot  /var/lib/haproxy
            daemon
            group  haproxy
            log  68.183.200.115 local0
            maxconn  4000
            pidfile  /var/run/haproxy.pid
            stats  socket /var/lib/haproxy/stats
            user  haproxy
           
          defaults
            log  global
            maxconn  8000
            option  redispatch
            retries  3
            stats  enable
            timeout  http-request 10s
            timeout  queue 1m
            timeout  connect 10s
            timeout  client 1m
            timeout  server 1m
            timeout  check 10s
           
          listen mariadb00-536167aa-79a7-4ccc-9c39-48e8ad4ab481
            bind 0.0.0.0:3307
            balance roundrobin
            option tcpka
            option mysql-check user haproxy
            server dco-prod-db1 10.137.63.41:3306 check send-proxy weight 1
            server dco-prod-db2 10.137.95.31:3306 check send-proxy weight 1
            server dco-prod-db3 10.137.95.38:3306 check send-proxy weight 1
          

          The above cluster works but the following config does not

          # This file managed by Puppet
          global
            chroot  /var/lib/haproxy
            daemon
            group  haproxy
            log  165.22.224.7 local0
            maxconn  4000
            pidfile  /var/run/haproxy.pid
            stats  socket /var/lib/haproxy/stats
            user  haproxy
           
          defaults
            log  global
            maxconn  8000
            option  redispatch
            retries  3
            stats  enable
            timeout  http-request 10s
            timeout  queue 1m
            timeout  connect 10s
            timeout  client 1m
            timeout  server 1m
            timeout  check 10s
           
          listen mariadb00-5c7ca51f-bf0e-4ae9-84c8-96c08f58aeff
            bind 0.0.0.0:3307
            balance roundrobin
            option tcpka
            option mysql-check user haproxy
            server opl-prod-db4 10.137.159.153:3306 check send-proxy weight 1
            server opl-prod-db5 10.137.159.154:3306 check send-proxy weight 1
            server opl-prod-db6 10.137.159.168:3306 check send-proxy weight 1
          

          The only thing different is the IP addresses. Note that if I set the proxy_protocol_networks to "*" then it works fine in either case.

          minorOffense Mathew Winstone added a comment - - edited I have the same problem. [root @dco -prod-db1 my.cnf.d]# yum list installed | grep -i maria MariaDB-backup.x86_64 10.3 . 17 - 1 .el7.centos @mariadb MariaDB-client.x86_64 10.3 . 17 - 1 .el7.centos @mariadb MariaDB-common.x86_64 10.3 . 17 - 1 .el7.centos @mariadb MariaDB-compat.x86_64 10.3 . 17 - 1 .el7.centos @mariadb MariaDB-devel.x86_64 10.3 . 17 - 1 .el7.centos @mariadb MariaDB-server.x86_64 10.3 . 17 - 1 .el7.centos @mariadb galera.x86_64 25.3 . 26 - 1 .rhel7.el7.centos @mariadb I have proxy-protocol-networks = 10.0 . 0.0 / 8 , 127.0 . 0.1 / 32 ,localhost In server.cnf on both clusters. And my haproxy config is # This file managed by Puppet global chroot /var/lib/haproxy daemon group haproxy log 68.183 . 200.115 local0 maxconn 4000 pidfile /var/run/haproxy.pid stats socket /var/lib/haproxy/stats user haproxy   defaults log global maxconn 8000 option redispatch retries 3 stats enable timeout http-request 10s timeout queue 1m timeout connect 10s timeout client 1m timeout server 1m timeout check 10s   listen mariadb00-536167aa-79a7-4ccc-9c39-48e8ad4ab481 bind 0.0 . 0.0 : 3307 balance roundrobin option tcpka option mysql-check user haproxy server dco-prod-db1 10.137 . 63.41 : 3306 check send-proxy weight 1 server dco-prod-db2 10.137 . 95.31 : 3306 check send-proxy weight 1 server dco-prod-db3 10.137 . 95.38 : 3306 check send-proxy weight 1 The above cluster works but the following config does not # This file managed by Puppet global chroot /var/lib/haproxy daemon group haproxy log 165.22 . 224.7 local0 maxconn 4000 pidfile /var/run/haproxy.pid stats socket /var/lib/haproxy/stats user haproxy   defaults log global maxconn 8000 option redispatch retries 3 stats enable timeout http-request 10s timeout queue 1m timeout connect 10s timeout client 1m timeout server 1m timeout check 10s   listen mariadb00-5c7ca51f-bf0e-4ae9-84c8-96c08f58aeff bind 0.0 . 0.0 : 3307 balance roundrobin option tcpka option mysql-check user haproxy server opl-prod-db4 10.137 . 159.153 : 3306 check send-proxy weight 1 server opl-prod-db5 10.137 . 159.154 : 3306 check send-proxy weight 1 server opl-prod-db6 10.137 . 159.168 : 3306 check send-proxy weight 1 The only thing different is the IP addresses. Note that if I set the proxy_protocol_networks to "*" then it works fine in either case.

          Same problem on a MariaDB 10.3.20 server behind HAProxy 2.0.9 with Proxy Protocol setup (send-proxy-v2)

          HAProxy (10.66.7.54) => MariaDB (10.66.7.34)
          

          So the subnet mask is /24 (255.255.255.0)

          Setting exact IP is ok

          SET GLOBAL proxy_protocol_networks = '10.66.7.54/32';

          Setting /19 mask is ok

          SET GLOBAL proxy_protocol_networks = '10.66.0.0/19';

          Setting /24 netmask is NOT OK (although it should work)

          SET GLOBAL proxy_protocol_networks = '10.66.7.0/24';

          It seems that the network mask is miscalculated.
          Specifically, I noticed that setting netmask /24 with IP between 10.66.7.0 and 10.66.7.31 does not work for my setup, but works with an ip >= 10.66.7.32

          SET GLOBAL proxy_protocol_networks = '10.66.7.30/24';
          SET GLOBAL proxy_protocol_networks = '10.66.7.31/24';
          SET GLOBAL proxy_protocol_networks = '10.66.7.32/24';
          SET GLOBAL proxy_protocol_networks = '10.66.7.33/24';
          ...

          nico Nicolas Payart added a comment - Same problem on a MariaDB 10.3.20 server behind HAProxy 2.0.9 with Proxy Protocol setup (send-proxy-v2) HAProxy (10.66.7.54) => MariaDB (10.66.7.34) So the subnet mask is /24 (255.255.255.0) Setting exact IP is ok SET GLOBAL proxy_protocol_networks = '10.66.7.54/32'; Setting /19 mask is ok SET GLOBAL proxy_protocol_networks = '10.66.0.0/19'; Setting /24 netmask is NOT OK (although it should work) SET GLOBAL proxy_protocol_networks = '10.66.7.0/24'; It seems that the network mask is miscalculated. Specifically, I noticed that setting netmask /24 with IP between 10.66.7.0 and 10.66.7.31 does not work for my setup, but works with an ip >= 10.66.7.32 SET GLOBAL proxy_protocol_networks = '10.66.7.30/24'; SET GLOBAL proxy_protocol_networks = '10.66.7.31/24'; SET GLOBAL proxy_protocol_networks = '10.66.7.32/24'; SET GLOBAL proxy_protocol_networks = '10.66.7.33/24'; ...

          People

            wlad Vladislav Vaintroub
            jocke Joachim Tingvold
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.