The idea is generally fine, but I think we can have it always enabled. That is DISABLE_TCP_LINGER and cmake changes should not be needed.

Also we should not need tcp_linger variable. tcp_linger_timeout alone should be enough.

I'd avoid define TCP_LINGER_TIMEOUT. Just put default timeout directly into sys_var.

https://www.nybek.com/blog/2015/03/05/cross-platform-testing-of-so_linger/ actually does not recommend setting SO_LINGER to {on,timeout > 0} cross-platform.
According to this article, only Windows would abort the connection (send RST) if acknowledgement for sent data was not received after linger timeout, which is, I guess, the reason for this patch.

However, according to the same article , call to shutdown() before closesocket() on Windows would make closesocket() to return immediately, and connection closed in background, as if setsockopt(SO_LINGER) was not called.

I checked our code, and we do shutdown() before closesocket() ( this does not make much sense, but we have this since long time already). So, the questions are :

• Is https://www.nybek.com/blog/2015/03/05/cross-platform-testing-of-so_linger/ correct wrt cross-platform SO_LINGER effects. If this is the case, perhaps we should make this setting avaialble only on Windows.

• with the current code, vio_close() where shutdown() followed by closesocket(), does SO_LINGER setting have any effect?

It's not only waiting for the sent data to be acknowledged, but also waiting for the connection's FIN being ACKed.

I don't know about how Windows handling shutdown(2) then close(2), but on all other systems, close(2) will obey the SO_LINGER settings, no matter whether the shutdown(2) was called or not.

As about shutdown(2) before close(2), I don't think the shutdown(2) immediately before the close(2) is necessary; removing it would save some syscall cost. I'd suggest to remove it, no matter whether this patch could be in or not.

Generally speaking, I think SO_LINGER option is fine (the compile time option is unnecessary; all systems have this option). But it will put the close(2) into sleep, so I'd suggest to default off for this option. And don't call setsockopt(2), if this option is off, which introduces unnecessary syscall cost.

Thanks,
sephe

Sepherosa Ziehau, thanks for commenting. Is this your patch?
As for cross-plattformness of SO_LINGER, all information I could find on the internet, is

• in case of {on, 0}

  setting, connection is aborted with RST immediately (from here https://www.nybek.com/blog/2015/03/05/cross-platform-testing-of-so_linger/)

• in case of {on, > 0} setting, on close(), given some unsent data, connection blocks (assuming blocking socket) for the specified l_linger setting, however only Windows would RST-abort connection after timeout, so the added effect cross-plattform is just waiting and blocking (from the same source)
  * non-blocking sockets behave strangely, and with {on, > 0}

  setting, again, only on Windows the actual LINGER effect of aborting connection
  after specified timeout can be seen https://www.nybek.com/blog/2015/04/29/so_linger-on-non-blocking-sockets/#so_linger_on_20

The nybeck.com seems to have done quite a lot of research on the option, which if I summarize here, would be : except Windows, where things work as expected, the SO_LINGER setting is really only useful either with timeout 0, or when sender wants to wait for specific time to ensure receiver gets the data, before closing connection.

timeout >= 0 on Windows and timeout = 0 on Unixes can be used to abort connection, so there are no connections in TIME_WAIT, so for that we can have the tcp_linger option, which I'd propose to make an "int" variable, in the {-1, 65535} range , -1 meaning default "off". With that, 0 linger is allowed and can actually be used on Unixes, where other timeout does not make much sense.

Speaking of our client-server protocol, the connection closure is normally initiated by the client, who sends COM_QUIT packet, closes the socket, and then server closes the socket as well.

In other cases, server is the first to close socket. I think there is at least one legitimate case, where server should abort the connection, rather than orderly close. That is, when server sends a large result set to the client, but the client is reading much slower than server is writing, so that socket buffers fill up, and server's send() runs into timeout. Do you think we should abort the connection via SO_LINGER=

in this case?

Can you give your thoughts on following:

• is this correct, that, outside of Windows, linger timeout > 0 does not make much sense, because it does not abort connection via RST?
• should the server abort connection if send() runs into timeout, disregarding tcp_linger settings.

> is this correct, that, outside of Windows, linger timeout > 0 does not make much sense, because it does not abort connection via RST?

This is correct.

> should the server abort connection if send() runs into timeout, disregarding tcp_linger settings.

This is correct too. However, the retransmission timeout could be quite lengthy.

I am not the author. I talked to the author about the background and the intention yesterday. We lean toward TCP keepalive based solution instead of this one at the moment. We will keep you updated on these.

>> should the server abort connection if send() runs into timeout, disregarding tcp_linger settings.

>This is correct too. However, the retransmission timeout could be quite lengthy.
Not sure about the retransmission. We use socket timeouts
there is a system variable net_write_timeout, defaulting to 60 seconds
https://mariadb.com/kb/en/library/server-system-variables/#net_write_timeout

If server cannot send a packet within net_write_timeout seconds, connection is closed (should really be aborted as we just discussed)

speaking if TCP keepalive, we have an upcoming patch for it as well
https://github.com/MariaDB/server/commit/44cfcce67f0c4a63989db406623bff2fe6faf40d

As for the userland send timeout. It's kinda different, an example:
1) The sending was not blocked in the last send(2).
2) The server close(2)'ed the socket after the last send(2) call. The data in the last send(2) was still in the TCP send buffer.
3) If the client's ACK for the last piece of data was excessively dropped, or the extreme case that the client itself was crashed.
4) Then the socket's (and inpcb in the BSD world) destruction would solely rely on the TCP retransmission timeout; as I said, it could be quite lengthy.

Thanks,
sephe

Sepherosa Ziehau
can you look at this patch to see if it will solve problems in most cases ?https://github.com/MariaDB/server/commit/8b09de139bf52777ba5e78821f3b64734bd22195

It would use abortive close, (no linger), if connection ran into timeout. I think this is going to the most frequent cases, where server close connection first (usually client does it). In this case, server does not care about TCP retransmission (as discussed, this would happen when writing large result sets and client application reads at slower pace than server writes)

There are some other cases where server would close the socket before client does it, e.g during connection on failed authentication. But then, server sends an error message, and it does somewhat care that the client receives it.

The patch also cleans up socket timeout handling, so that Windows is consistent with another platforms (using nonblocking socket IO + waiting in poll/select on single socket, rather than setsockopt). This makes identifying timeouts easier.

I also removed the superficial shutdown() before closesocket() in places where it was used, because it reportedly breaks SO_LINGER on Windows.

This is definitely a step forward to the right direction! The diff looks good. Thank you very much!

I pushed the mentioned patch into 10.3. Please let us know if there is anything else we should do WRT lingering.