[MDEV-19669] proxy_protocol_networks does not evaluate/parse properly Created: 2019-06-01  Updated: 2019-11-27  Resolved: 2019-11-27

Status: Closed
Project: MariaDB Server
Component/s: Server
Affects Version/s: 10.3.15
Fix Version/s: 10.3.21, 10.4.11, 10.5.1

Type: Bug Priority: Major
Reporter: Joachim Tingvold Assignee: Vladislav Vaintroub
Resolution: Fixed Votes: 1
Labels: proxy
Environment:

Ubuntu 18.04.2 LTS, MariaDB-1:10.3.15+maria~bionic



 Description   

Hi,

proxy_protocol_networks does not seem to work properly in my setup. I have one working setup, and one non-working setup. The setups are equal, except hostnames and IP-addresses.

Working setup;

  • db-demo-01 (172.20.20.61)
  • db-demo-02 (172.20.20.62)
  • db-demo-02 (172.20.20.62)
  • db-demo-ha01 (172.20.20.28)
  • db-demo-ha02 (172.20.20.29)

Non-working setup:

  • db-prod-01 (172.20.20.71)
  • db-prod-02 (172.20.20.72)
  • db-prod-02 (172.20.20.72)
  • db-prod-ha01 (172.20.20.38)
  • db-prod-ha02 (172.20.20.39)

All hosts are running Ubuntu 18.04.2 LTS. All within the same subnet (172.20.20.0/24). All ha*-nodes are running haproxy. The non-ha-nodes are running MariaDB Galera cluster.

I have the following in my.cnf on all MariaDB nodes;

proxy_protocol_networks=::1, 127.0.0.1, localhost, 172.20.20.0/24

A snippet of the haproxy.cfg;

server db-demo-01 db-demo-01.example.com:3306  check send-proxy-v2

The Proxy Protocol works as expected on the 'demo' cluster. However, on the 'prod' cluster, having the exact same configuration (including the proxy_protocol_networks config above), haproxy cannot connect, complaining as following;

Server mysql_cluster/db-prod-01 is DOWN, reason: Layer7 wrong status, code: 0, info: "#HY000Proxy header is not accepted from 172.20.20.38", check duration: 0ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.

That IP is within the 172.20.20.0/24 range specified. If I explicitly list the IP (without CIDR notation) in the config, it works as expected;

proxy_protocol_networks=::1, 127.0.0.1, localhost, 172.20.20.0/24, 172.20.20.38, 172.20.20.39

The order of the 172.20.20.0/24 does not matter (i.e. it's not because it was bad parsing due to being the last entry in the list, or similar).



 Comments   
Comment by Mathew Winstone [ 2019-08-26 ]

I have the same problem.

[root@dco-prod-db1 my.cnf.d]# yum list installed | grep -i maria
MariaDB-backup.x86_64               10.3.17-1.el7.centos       @mariadb
MariaDB-client.x86_64               10.3.17-1.el7.centos       @mariadb
MariaDB-common.x86_64               10.3.17-1.el7.centos       @mariadb
MariaDB-compat.x86_64               10.3.17-1.el7.centos       @mariadb
MariaDB-devel.x86_64                10.3.17-1.el7.centos       @mariadb
MariaDB-server.x86_64               10.3.17-1.el7.centos       @mariadb
galera.x86_64                       25.3.26-1.rhel7.el7.centos @mariadb

I have

proxy-protocol-networks = 10.0.0.0/8,127.0.0.1/32,localhost

In server.cnf on both clusters. And my haproxy config is

# This file managed by Puppet
global
  chroot  /var/lib/haproxy
  daemon
  group  haproxy
  log  68.183.200.115 local0
  maxconn  4000
  pidfile  /var/run/haproxy.pid
  stats  socket /var/lib/haproxy/stats
  user  haproxy
 
defaults
  log  global
  maxconn  8000
  option  redispatch
  retries  3
  stats  enable
  timeout  http-request 10s
  timeout  queue 1m
  timeout  connect 10s
  timeout  client 1m
  timeout  server 1m
  timeout  check 10s
 
listen mariadb00-536167aa-79a7-4ccc-9c39-48e8ad4ab481
  bind 0.0.0.0:3307
  balance roundrobin
  option tcpka
  option mysql-check user haproxy
  server dco-prod-db1 10.137.63.41:3306 check send-proxy weight 1
  server dco-prod-db2 10.137.95.31:3306 check send-proxy weight 1
  server dco-prod-db3 10.137.95.38:3306 check send-proxy weight 1

The above cluster works but the following config does not

# This file managed by Puppet
global
  chroot  /var/lib/haproxy
  daemon
  group  haproxy
  log  165.22.224.7 local0
  maxconn  4000
  pidfile  /var/run/haproxy.pid
  stats  socket /var/lib/haproxy/stats
  user  haproxy
 
defaults
  log  global
  maxconn  8000
  option  redispatch
  retries  3
  stats  enable
  timeout  http-request 10s
  timeout  queue 1m
  timeout  connect 10s
  timeout  client 1m
  timeout  server 1m
  timeout  check 10s
 
listen mariadb00-5c7ca51f-bf0e-4ae9-84c8-96c08f58aeff
  bind 0.0.0.0:3307
  balance roundrobin
  option tcpka
  option mysql-check user haproxy
  server opl-prod-db4 10.137.159.153:3306 check send-proxy weight 1
  server opl-prod-db5 10.137.159.154:3306 check send-proxy weight 1
  server opl-prod-db6 10.137.159.168:3306 check send-proxy weight 1

The only thing different is the IP addresses. Note that if I set the proxy_protocol_networks to "*" then it works fine in either case.

Comment by Nicolas Payart [ 2019-11-25 ]

Same problem on a MariaDB 10.3.20 server behind HAProxy 2.0.9 with Proxy Protocol setup (send-proxy-v2)

HAProxy (10.66.7.54) => MariaDB (10.66.7.34)

So the subnet mask is /24 (255.255.255.0)

Setting exact IP is ok

SET GLOBAL proxy_protocol_networks = '10.66.7.54/32';

Setting /19 mask is ok

SET GLOBAL proxy_protocol_networks = '10.66.0.0/19';

Setting /24 netmask is NOT OK (although it should work)

SET GLOBAL proxy_protocol_networks = '10.66.7.0/24';

It seems that the network mask is miscalculated.
Specifically, I noticed that setting netmask /24 with IP between 10.66.7.0 and 10.66.7.31 does not work for my setup, but works with an ip >= 10.66.7.32

SET GLOBAL proxy_protocol_networks = '10.66.7.30/24';
SET GLOBAL proxy_protocol_networks = '10.66.7.31/24';
SET GLOBAL proxy_protocol_networks = '10.66.7.32/24';
SET GLOBAL proxy_protocol_networks = '10.66.7.33/24';
...

Generated at Thu Feb 08 08:53:27 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.