Semi-sync primary stops accepting NEW connections after replication port partition heals; established sessions unaffected; only restart recovers

*Affects (all tested, all affected):*

Official `mariadb` Docker images, Ubuntu 24.04 builds.

*Relation to MDEV-36934.* MDEV-36934 ("Semi sync makes the master unresponsive when a replica
is stopped") is listed as fixed in 10.6.23, 10.11.14, 11.4.8, *11.8.3* and 12.0.2. Every version
tested here already contains that fix and still reproduces. The behaviour is also *not identical*
to that ticket: there the primary is described as completely unresponsive. Here the server keeps
serving already-open sessions with no measurable degradation for as long as it is observed (a
session opened before the partition answered every 5 s for more than 5 minutes, never once delayed)
— only new connections fail to complete the handshake.

*Not a duplicate of MDEV-14131.* Similar error text to MDEV-14131 (reading initial communication packet), but distinct: our system error is 110 (not 22), `Threads_connected` stays far from the limit, and connection growth is a consequence of the stall, not its cause.

1. 1. Configuration

```
rpl_semi_sync_master_enabled = ON
rpl_semi_sync_master_wait_point = AFTER_SYNC
rpl_semi_sync_master_timeout = 1000
rpl_semi_sync_master_wait_no_slave = OFF
slave_net_timeout = 60 (replica)
max_connections = 151
thread_handling = one-thread-per-connection
```

One primary, one replica, GTID replication. The load required is minimal: one INSERT per second
and a handful of connections.

1. 1. Minimal reproducer

The attached script (`mdev-semisync-repro.sh`) builds the whole scenario from stock images and
prints a verdict. It refuses to proceed unless the scenario is actually in place (replica running,
semi-sync ON with one client, schema replicated) and unless the firewall rule is demonstrably
matching packets, so a "did not reproduce" result cannot come from a broken harness.

```
./mdev-semisync-repro.sh mariadb:11.8.6
```

1. 1. 1. Two ingredients are necessary

Both were established by elimination — four earlier harness designs failed to reproduce, each for
a different and identifiable reason:

1. *The cut must be a silent drop.* Tearing the connection down cleanly does not reproduce it.
With `docker network disconnect` the primary noticed immediately (`Stop semi-sync binlog_dump`
in the same second as the undo) and never wedged. With a silent `drop` the primary keeps a
half-open socket and only tears it down seconds later — that teardown is what precedes the wedge.
Note that `reject with tcp reset` *also* reproduces it, so an orphaned half-open connection is
not the whole story; see below.
2. *The dropped traffic must traverse the `forward` hook*, i.e. the replica must reach the
primary through a published/DNAT'd port rather than over a shared bridge. Two containers on the
same Docker bridge exchange frames at L2; with `br_netfilter` unloaded that traffic never reaches
netfilter at all and no `forward` rule can match it.

1. 1. Timeline (deterministic, identical across all three versions)

```
T+0 firewall rule applied (drop, slave -> primary replication port)
T+1 primary: "Timeout waiting for reply of binlog (...)"
"Semi-sync replication switched OFF"
new connections still work normally throughout the outage
T+90 rule removed
T+96 primary: "Stop semi-sync binlog_dump to slave (server_id: N)"
T+101 first new connection fails:
ERROR 2013 (HY000): Lost connection to server at
'handshake: reading initial communication packet', system error: 110
T+400 still failing; 57 of 110 probe samples failed and the condition never cleared
```

In a production system the same sequence was observed twice with a larger gap between the undo and
the teardown (33 s instead of 6 s), and the wedge began 1-2 s after the teardown line in both cases.

1. 1. Evidence from a session opened BEFORE the partition (fully functional throughout)

```
Threads_connected 18 -> 104 (monotonically increasing while wedged)
Threads_running 2 - 3
Aborted_connects 0
Connection_errors_max_connections 0
Connection_errors_accept 0
Connection_errors_internal 0
```

The server *accounts* the incoming connections — `Threads_connected` grows — but never completes
their handshake, and records *no connection error of any kind*. `max_connections` is nowhere near
being reached when the wedge starts; the growth is a consequence, not the cause (with retrying
clients it reached 104 of 151 in five minutes, so a long enough wedge would additionally exhaust
the connection limit).

`SHOW PROCESSLIST` during the wedge shows two things that look relevant:

• *several `Binlog Dump` threads for the same replica accumulating* (5 were present), and
• *threads in `Killed` state that never exit*, with `Time` values of 287-307 seconds.

The replica cannot re-establish replication, because its own reconnection is a new connection and
hits the same wall:

```
Last_IO_Errno: 2013
Last_IO_Error: error reconnecting to master ... - retry-time: 60 maximum-retries: 100000
message: Lost connection to server at 'waiting for initial communication packet',
system error: 110 "Connection timed out"
```

1. 1. Recovery

Only a server restart clears it. A normal shutdown request issued while in this state did not
complete either — the process stayed alive and had to be killed — so recovery required SIGKILL and
InnoDB crash recovery, which completed cleanly with one prepared XA transaction resolved.

1. 1. Why this is worth reporting even though semi-sync is a known problem area

The failure mode is silent from the outside. Existing connection pools keep working, so every
HTTP-level health check stays green while the server can no longer accept a single new connection
and, crucially, while the replica can no longer reconnect. In a real network partition the primary
survives the partition itself and then becomes unable to rebuild its standby, with no error
surfaced anywhere in the server log between the teardown line and the restart.