[MDEV-27167] system variable deferred-networking to resolve early initialization Created: 2021-12-04  Updated: 2022-02-15  Resolved: 2022-02-15

Status: Closed
Project: MariaDB Server
Component/s: Upgrades
Fix Version/s: N/A

Type: Task Priority: Major
Reporter: Daniel Black Assignee: Sergei Golubchik
Resolution: Duplicate Votes: 0
Labels: None

Issue Links:
Relates
relates to MDEV-25670 mariadb container needs to run mysql_... Closed
relates to MDEV-27068 running mariadb-upgrade on mariadb se... Closed
relates to MDEV-27074 container start-up to take a long time Closed
relates to MDEV-27613 Fixing debian to only run the full my... Stalled

 Description   

The problem of early initialization of MariaDB manifesting in a few ways.
Currently we have multiple restarts of the mariadbd instance in containers because of multiple initialization stages causing slow starts like MDEV-27074 exhibits. Progress writing a mariadb-upgrade in a container (MDEV-25670) also hits this limitation.

MDEV-27068 showed the need for a skip-networking mariadbd instance to be run for mariadb-upgrade to be run before user traffic occurs.

Both these rely on the lack of TCP networking to prevent the user traffic occuring.

The problem of a background mariadbd instance is:

  • wasted CPU/IO for initialization of everything, just to get torn down after a few actions
  • Poor integration with systemd and other service managers
  • The actual start up instigates innodb recovery

The step between an initialization and a user accessible service is:

  • The accessibility of TCP ports
  • The systemd message initiating its ready to allow dependent services to start up.

This MDEV proposes the addition of a system variable deferred_networking to address this. Its behaviour is:

  • Defaults to 0, for backwards compatibility
  • When 1, TCP based ports are not listened to
  • When 1, No systemd message READY=1 is sent

These aspects allows a window for mariadb-upgrade and container based initialization (of user data, timezones) to occur. After this is finished setting of deferred_networking to 1 results in:

  • manipulation of the poll/fd structures to include TCP file descriptors
  • a non abort_loop=1 call (like termination) to move forward from the existing select/poll
  • A systemd message READY=1
  • No further manipulation of the deferred_networking system variable is permitted.

In the addition to the above:

  • mariadb-upgrade will add a --end-deferred-networking option to perform this system variable.

Hopefully the complexity of this is minimal and can make 10.7, and possibly earlier.

serg, what do you think of the concept? Any improvements? I'm happy to write it.

A couple of notes:

  • We want any startup, independent if mysql_upgrade is to be used or not, to run recovery of ALL used storage engines!
    If not, we cannot run a full upgrade!
  • We don't need to start with --skip-networking during the systemd init; There are no users that can or will connect during that period


 Comments   
Comment by Sergei Golubchik [ 2021-12-06 ]

MDEV-27068 suggests to start the server with skip-networking and a non-standart socket location, not accessible to users, and then to restart it after the upgrade. Deferred networking does not mean that the socket is reinitialized to a standard location, so this solution does not help MDEV-27068.

I don't understand how the deferred networking can help MDEV-27074

Neither I see how it can help MDEV-25670.


So, what is it for? What does it solve?

Comment by Daniel Black [ 2022-02-02 ]

Resolved container use case with sub-optimal script based version checking, a temporary instance in --skip-grant-tables, and mariadb-upgrade --upgrade system-tables (only).

monty does not want:

  • Two server starts always (would be required under systemd - MDEV-27613)
  • An offline mariadb-upgrade MDEV-27636 because potential errors in version checking in Debian package where users have multiple mariadb versions installed, even though the actually upgrade on a running instance would check the versions match.

So MDEV-27613 now has two options that reference this.

Comment by Michael Widenius [ 2022-02-15 ]

We don't need to start two servers for the normal case. We also don't need to skip networking on having another tempoary socket.
See MDEV-27613 for details

Comment by Michael Widenius [ 2022-02-15 ]

Feedback already needed. This task is superseded by MDEV-27613.

Comment by Michael Widenius [ 2022-02-15 ]

MDEV-27613 is the more proper way to solve it (as it has to be done in 10.2/10.3)

Generated at Thu Feb 08 09:50:51 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.