Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-27167

system variable deferred-networking to resolve early initialization

    XMLWordPrintable

Details

    • New Feature
    • Status: Stalled (View Workflow)
    • Major
    • Resolution: Unresolved
    • N/A
    • Upgrades
    • None

    Description

      Edit: reworded as focus on mariadb-upgrade lost track of the other requirements

      The need of exposed post initialization and pre-production (processing connections) ability to hook into MariaDB is exhibited by the functional requirements:

      • User accessible exposure of user data migration before the contain is accessible to applications - https://github.com/MariaDB/mariadb-docker/issues/515
      • Currently we have multiple restarts of the mariadbd instance in containers because of multiple initialization stages.
      • MariaDB-upgrade in a container is a temp server start with --skip-networking and there, no easy transition to a listening server, so we need to restart.

      The desired behaviour is for user data migration is:

      1. server start without TCP listening on standard ports
      1. user application instigates changes to user data and structures ready for the updated application
      1. some user application trigger of completed changes is signalled to the server, and the server starts listening on the standard TCP ports

      The desired behaviour of container initialization is:
      1. environment variables are used to create user, database, grants, replication etc (all under mariadbd --skip-networking)
      1. /docker-entrypoint-initdb.d user provided scripts are processed (under same running instance)
      1, the container entrypoint can signal to mariadb that all is complete and TCP listening can begin (without a restart).

      Both these rely on the lack of TCP networking to prevent the user traffic occurring from outside the initialization process's control.

      The problem of a multiple mariadbd starts is:

      • wasted CPU/IO for initialization of everything, just to get torn down after a few actions
      • Poor integration with systemd and other service managers
      • The actual start up instigates innodb recovery, which when incomplete, completely restarts.

      The step from a basic server start initialization, completed internal readiness, a and a general accessible mariadb service is:

      • The accessibility of TCP ports
      • The systemd message initiating its ready to allow dependent services to start up.

      This MDEV proposes the addition of a system variable deferred_networking to address this. Its behaviour is:

      • Defaults to 0, for backwards compatibility
      • When 1, standard TCP based ports are not listened to
      • When 1, No systemd message READY=1 is sent

      After this is finished setting of deferred_networking to 1 results in:

      • manipulation of the poll/fd structures to include TCP file descriptors
      • a non abort_loop=1 call (like termination) to move forward from the existing select/poll
      • A systemd message READY=1
      • No further manipulation of the deferred_networking system variable is permitted.

      serg, what do you think of the concept? Any improvements? I'm happy to write it.

      A couple of notes:

      • We want any startup, independent if mysql_upgrade is to be used or not, to run recovery of ALL used storage engines!
        If not, we cannot run a full upgrade!
      • We don't need to start with --skip-networking during the systemd init; There are no users that can or will connect during that period

      Attachments

        Activity

          People

            serg Sergei Golubchik
            danblack Daniel Black
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.