[MDEV-29175] Connections are not released synchronously Created: 2022-07-26  Updated: 2023-11-28

Status: Confirmed
Project: MariaDB Server
Component/s: Server
Affects Version/s: 10.9.1, 10.3, 10.4, 10.5, 10.6, 10.7, 10.8, 10.9, 10.10
Fix Version/s: 10.4, 10.5, 10.6

Type: Bug Priority: Major
Reporter: Michael Assignee: Oleksandr Byelkin
Resolution: Unresolved Votes: 0
Labels: Compatibility, performance
Environment:

any



 Description   

In atk4/data php data framework we run unit tests with limited number of max. connections - https://github.com/atk4/data/blob/2c501c78f7e839ab2d55176d59ab258000ee1a76/.github/workflows/test-unit.yml#L156

When we limited it strictly to 1, we have observed random CI failures with MySQL and MariaDB databases.

Initially I thought this is issue in PHP mysqlnd and I opened https://github.com/php/php-src/issues/9151 where I described the issue exhaustively. Please see the php-src ticket for code to reproduce. I was able to reproduce it also using in lua.

Currently, it seems the MariaDB release the client connection too early and does not synchronously wait until the connection is fully released from the database and decremented from the active connections counters.

This behaviour can be seen with MySQL and MariaDB only. I tested also Microsoft SQL Server and PostgreSQL and these databases release the connection socket synchronously with the counters.



 Comments   
Comment by Elena Stepanova [ 2022-07-27 ]

Thanks for the report.

Reproducible easily enough on all of 10.3-10.10 (probably earlier ones too, I didn't check), as well as MySQL 5.7 and 8.0.

<?php
 
$port=3306;
 
$connroot = new mysqli('127.0.0.1:'.$port,'root','','');
$connroot->query("DROP USER IF EXISTS test_user@localhost");
$connroot->query("CREATE USER test_user@localhost WITH MAX_USER_CONNECTIONS 1");
$connroot->close();
 
foreach (range(0, 50_000) as $i) {
    $conn=new mysqli('127.0.0.1:'.$port,'test_user','','');
    if ($conn->connect_error) {
      die("Connection #$i failed: ".$conn->connect_error."\n\n");
    }
    $conn->close();
}
echo "All done\n";
exit;
?>

Disclaimer: I'm not a PHP writer, feel free to fix it as needed.

It usually fails in seconds with something like

10.3 bd935a41

PHP Warning:  mysqli::__construct(): (HY000/1226): User 'test_user' has exceeded the 'max_user_connections' resource (current value: 1) in /mnt8t/src/mariadb-toolbox/templates/test.php on line 11
Connection #29777 failed: User 'test_user' has exceeded the 'max_user_connections' resource (current value: 1)

In the error log, if enabled:

                143019 Connect  test_user@localhost as anonymous on 
                143019 Quit     
                143020 Connect  test_user@localhost as anonymous on 
                143020 Quit     
                143021 Connect  test_user@localhost as anonymous on 
                143022 Connect  test_user@localhost as anonymous on 
                143021 Quit     

Sometimes it misses the mark though, re-run then (increase the number of loops if necessary).

Comment by Sergei Golubchik [ 2022-08-01 ]

I don't think it's a bug as such. More like an implementation detail. Our own tests work around it. Supposedly, clients appreciate lower latency, and that's why a response packet as sent as soon as possible

Comment by Michael [ 2022-08-13 ]

With many fast reconnects the "active connection counter" in MariaDB can show value like 5 even there is a single active connection socket. This does not only causes problems with tests, but also in a real apps that connect and disconnect often.

The current behaviour requires the max. connection limit to be set much higher than needed or it causes random connect issues hard to debug.

> clients appreciate lower latency

How much the disconnect latency will increase when the "active connection counter" will be counted synchronously with client connect/reconnect? I doubt it will be more than 100us.

Generated at Thu Feb 08 10:06:29 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.