Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-9059

protocol: bundle first command with the authentication packet

Details

    • Task
    • Status: Stalled (View Workflow)
    • Major
    • Resolution: Unresolved
    • None
    • None
    • 10.2.0-10, 10.2.0-11, 10.2.1-1, 10.2.1-3, 10.2.1-4, 10.2.1-5, 10.2.3-2

    Description

      Optimistically assuming that the authentication was successful, the client doesn't need to wait for the ok packet from the server, but can send the first command right away, including it in the authentication packet.

      The idea is to store the command in separate buffer (or just make new network buffer) before authentication, then in case of success of the authentication execute it (auth itself can cause exchange of the packet that is why storing is requited).

      Attachments

        Issue Links

          blocks

          Task - A task that needs to be done. CONJ-463 faster connection with bundle first commands in authentication packet

          • Major - Major loss of function.
          • Closed
          is blocked by

          Task - A task that needs to be done. CONC-149 protocol: bundle first command with the authentication packet

          • Major - Major loss of function.
          • Closed
          relates to

          New Feature - A new feature of the product, which has yet to be developed. MXS-1574 Add support for bundled first command

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            serg Sergei Golubchik created issue -
            serg Sergei Golubchik made changes -
            Field Original Value New Value
            Fix Version/s 10.2 [ 14601 ]
            ratzpo Rasmus Johansson (Inactive) made changes -
            Sprint 10.2.0-10 [ 46 ]
            ratzpo Rasmus Johansson (Inactive) made changes -
            Assignee Oleksandr Byelkin [ sanja ]
            sanja Oleksandr Byelkin made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            sanja Oleksandr Byelkin made changes -
            Description Optimistically assuming that the authentication was successful, the client doesn't need to wait for the ok packet from the server, but can send the first command right away, including it in the authentication packet. Optimistically assuming that the authentication was successful, the client doesn't need to wait for the ok packet from the server, but can send the first command right away, including it in the authentication packet.

            The idea is to store the command in separate buffer (or just make new network buffer) before authentication, then in case of success of the authentication execute it (auth itself can cause exchange of the packet that is why storing is requited).
            ratzpo Rasmus Johansson (Inactive) made changes -
            Sprint 10.2.0-10 [ 46 ] 10.2.0-10, 10.2.0-11 [ 46, 47 ]
            ratzpo Rasmus Johansson (Inactive) made changes -
            Sprint 10.2.0-10, 10.2.0-11 [ 46, 47 ] 10.2.0-10, 10.2.0-11, 10.2.1-1 [ 46, 47, 56 ]
            ratzpo Rasmus Johansson (Inactive) made changes -
            Rank Ranked higher
            ratzpo Rasmus Johansson (Inactive) made changes -
            Sprint 10.2.0-10, 10.2.0-11, 10.2.1-1 [ 46, 47, 56 ] 10.2.0-10, 10.2.0-11, 10.2.1-1, 10.2.1-2 [ 46, 47, 56, 63 ]
            ratzpo Rasmus Johansson (Inactive) made changes -
            Sprint 10.2.0-10, 10.2.0-11, 10.2.1-1, 10.2.1-2 [ 46, 47, 56, 63 ] 10.2.0-10, 10.2.0-11, 10.2.1-1, 10.2.1-3 [ 46, 47, 56, 65 ]
            ratzpo Rasmus Johansson (Inactive) made changes -
            Rank Ranked lower
            ratzpo Rasmus Johansson (Inactive) made changes -
            Sprint 10.2.0-10, 10.2.0-11, 10.2.1-1, 10.2.1-3 [ 46, 47, 56, 65 ] 10.2.0-10, 10.2.0-11, 10.2.1-1, 10.2.1-3, 10.2.1-4 [ 46, 47, 56, 65, 66 ]
            ratzpo Rasmus Johansson (Inactive) made changes -
            Sprint 10.2.0-10, 10.2.0-11, 10.2.1-1, 10.2.1-3, 10.2.1-4 [ 46, 47, 56, 65, 66 ] 10.2.0-10, 10.2.0-11, 10.2.1-1, 10.2.1-3, 10.2.1-4, 10.2.1-5 [ 46, 47, 56, 65, 66, 68 ]
            diego dupin Diego Dupin made changes -
            sanja Oleksandr Byelkin added a comment -

            revision-id: e52317b01a50ee51d703913fb046f5419432dc2f (mariadb-10.2.2-35-ge52317b)
            parent(s): 553ca406cd44946d454d84c08a7247ef595378f5
            committer: Oleksandr Byelkin
            timestamp: 2016-10-11 15:56:54 +0200
            message:

            MDEV-9059: protocol: bundle first command with the authentication packet

            —

            sanja Oleksandr Byelkin added a comment - revision-id: e52317b01a50ee51d703913fb046f5419432dc2f (mariadb-10.2.2-35-ge52317b) parent(s): 553ca406cd44946d454d84c08a7247ef595378f5 committer: Oleksandr Byelkin timestamp: 2016-10-11 15:56:54 +0200 message: MDEV-9059 : protocol: bundle first command with the authentication packet —
            sanja Oleksandr Byelkin made changes -
            Assignee Oleksandr Byelkin [ sanja ] Sergei Golubchik [ serg ]
            Status In Progress [ 3 ] In Review [ 10002 ]
            ratzpo Rasmus Johansson (Inactive) made changes -
            Sprint 10.2.0-10, 10.2.0-11, 10.2.1-1, 10.2.1-3, 10.2.1-4, 10.2.1-5 [ 46, 47, 56, 65, 66, 68 ] 10.2.0-10, 10.2.0-11, 10.2.1-1, 10.2.1-3, 10.2.1-4, 10.2.1-5, 10.2.3-2 [ 46, 47, 56, 65, 66, 68, 105 ]
            ratzpo Rasmus Johansson (Inactive) made changes -
            Rank Ranked lower
            serg Sergei Golubchik added a comment -

            does Connector/C support it?

            serg Sergei Golubchik added a comment - does Connector/C support it?
            ratzpo Rasmus Johansson (Inactive) made changes -
            Sprint 10.2.0-10, 10.2.0-11, 10.2.1-1, 10.2.1-3, 10.2.1-4, 10.2.1-5, 10.2.3-2 [ 46, 47, 56, 65, 66, 68, 105 ] 10.2.0-10, 10.2.0-11, 10.2.1-1, 10.2.1-3, 10.2.1-4, 10.2.1-5, 10.2.3-2, 10.2.4-5 [ 46, 47, 56, 65, 66, 68, 105, 125 ]
            ratzpo Rasmus Johansson (Inactive) made changes -
            Rank Ranked lower
            serg Sergei Golubchik made changes -
            serg Sergei Golubchik added a comment -

            waiting for CONC-149

            serg Sergei Golubchik added a comment - waiting for CONC-149
            serg Sergei Golubchik made changes -
            Sprint 10.2.0-10, 10.2.0-11, 10.2.1-1, 10.2.1-3, 10.2.1-4, 10.2.1-5, 10.2.3-2, 10.2.4-5 [ 46, 47, 56, 65, 66, 68, 105, 125 ] 10.2.0-10, 10.2.0-11, 10.2.1-1, 10.2.1-3, 10.2.1-4, 10.2.1-5, 10.2.3-2 [ 46, 47, 56, 65, 66, 68, 105 ]
            diego dupin Diego Dupin made changes -
            diego dupin Diego Dupin made changes -
            serg Sergei Golubchik made changes -
            Fix Version/s 10.3 [ 22126 ]
            Fix Version/s 10.2 [ 14601 ]
            diego dupin Diego Dupin made changes -
            Comment [ not relevant anymore.
            This was develop mainly for java connector.

            After different discussion, comparaison was asked versus pipelining (send-send-send, and then read results one after the other).
            based on the assumption that authentication plugins are not often used.
             
            So, implementation with pipeline ask for all results, asuming authentication plugin is not used. If used, server close the connection immediatly and client recreate socket without using pipelining.

            Results are on CONJ-366 : connection take 349ms (±3ms) using pipelining, (= 40% faster than previous implementation)
            this take 354ms (± 4ms). Since this is slower and doesn't work on previous mariadb server / mysql servers. better to cancel this task.

            (for information, mysql java driver take 1183ms !)
            ]
            diego dupin Diego Dupin made changes -
            Comment [
            After different of tests, pipelining with assumption is not a viable solution.
            - pipelining break with maxscale
            - pipelining make socket close strange issue when using SSL.
            - if CONNECT_WITH_DB flag is not set (no db in url), then authentication plugin is asked, so authentication may have more used than imagined.
             
            Cancel previous comment. this task is still relevant. ]
            diego dupin Diego Dupin added a comment - - edited

            protocol change :

            new capability :

            (number has now to change since 34 has been used for MARIADB_CLIENT_STMT_BULK_OPERATIONS, but that's what is implemented in branch)
            MARIADB_CLIENT_COM_IN_AUTH = 1L << 34; /* support bundle first command with the authentication packet*/

            Handshake response packet

            • int<4> client capabilities
            • int<4> max packet size
            • int<1> client character collation
            • string<19> reserved
            • if not (server_capabilities & CLIENT_MYSQL)
              • int<4> extended client capabilities
            • else
              • string<4> reserved
            • string<NUL> username
            • if (server_capabilities & PLUGIN_AUTH_LENENC_CLIENT_DATA)
              • string<lenenc> authentication data
            • else if (server_capabilities & CLIENT_SECURE_CONNECTION)
              • int<1> length of authentication response
              • string<fix> authentication response (length is indicated by previous field)
            • else
              • int<1> 0x00
            • if (server_capabilities & CLIENT_CONNECT_WITH_DB)
              • string<NUL> default schema name
            • if (server_capabilities & CLIENT_PLUGIN_AUTH)
              • string<NUL> authentication plugin name
            • if (server_capabilities & CLIENT_CONNECT_ATTRS)
              • int<lenenc> size of connection attributes
              • loop {
                • string<lenenc> key
                • string<lenenc> value
            • if (server_capabilities & MARIADB_CLIENT_COM_IN_AUTH)
              • int<lenenc> COM_MULTI command length
              • byte<xxx> COM_MULTI command, with xxx equal to command length

            The result of this COM_MULTI command (if MARIADB_CLIENT_COM_IN_AUTH will be received after authentication is complete

            COM_MULTI command

            • int<1> 0xfe (COM_MULTI header)
            • loop
              • byte<1> command header
              • string<xxx> command
            diego dupin Diego Dupin added a comment - - edited protocol change : new capability : (number has now to change since 34 has been used for MARIADB_CLIENT_STMT_BULK_OPERATIONS, but that's what is implemented in branch) MARIADB_CLIENT_COM_IN_AUTH = 1L << 34; /* support bundle first command with the authentication packet*/ Handshake response packet int<4> client capabilities int<4> max packet size int<1> client character collation string<19> reserved if not (server_capabilities & CLIENT_MYSQL) int<4> extended client capabilities else string<4> reserved string<NUL> username if (server_capabilities & PLUGIN_AUTH_LENENC_CLIENT_DATA) string<lenenc> authentication data else if (server_capabilities & CLIENT_SECURE_CONNECTION) int<1> length of authentication response string<fix> authentication response (length is indicated by previous field) else int<1> 0x00 if (server_capabilities & CLIENT_CONNECT_WITH_DB) string<NUL> default schema name if (server_capabilities & CLIENT_PLUGIN_AUTH) string<NUL> authentication plugin name if (server_capabilities & CLIENT_CONNECT_ATTRS) int<lenenc> size of connection attributes loop { string<lenenc> key string<lenenc> value if (server_capabilities & MARIADB_CLIENT_COM_IN_AUTH) int<lenenc> COM_MULTI command length byte<xxx> COM_MULTI command, with xxx equal to command length The result of this COM_MULTI command (if MARIADB_CLIENT_COM_IN_AUTH will be received after authentication is complete COM_MULTI command int<1> 0xfe (COM_MULTI header) loop byte<1> command header string<xxx> command
            sanja Oleksandr Byelkin added a comment -

            github tree is bb-10.3-MDEV-9059

            sanja Oleksandr Byelkin added a comment - github tree is bb-10.3- MDEV-9059
            sanja Oleksandr Byelkin added a comment -

            revision-id: d62ea85e6f6c273a712f951da907ced06f01bc03 (mariadb-10.3.2-39-gd62ea85e6f6)
            parent(s): 2ba1616e5d0f7008d5f6bf2c6cbc439935f6138b
            author: Oleksandr Byelkin
            committer: Oleksandr Byelkin
            timestamp: 2017-11-07 17:39:29 +0100
            message:

            MDEV-9059: protocol: bundle first command with the authentication packet

            —

            sanja Oleksandr Byelkin added a comment - revision-id: d62ea85e6f6c273a712f951da907ced06f01bc03 (mariadb-10.3.2-39-gd62ea85e6f6) parent(s): 2ba1616e5d0f7008d5f6bf2c6cbc439935f6138b author: Oleksandr Byelkin committer: Oleksandr Byelkin timestamp: 2017-11-07 17:39:29 +0100 message: MDEV-9059 : protocol: bundle first command with the authentication packet —
            sanja Oleksandr Byelkin added a comment -

            Now Georg add tests to connector C

            sanja Oleksandr Byelkin added a comment - Now Georg add tests to connector C
            markus makela markus makela made changes -
            serg Sergei Golubchik made changes -
            Fix Version/s 10.4 [ 22408 ]
            Fix Version/s 10.3 [ 22126 ]
            julien.fritsch Julien Fritsch made changes -
            Epic Link PT-79 [ 68560 ]
            diego dupin Diego Dupin made changes -
            Assignee Sergei Golubchik [ serg ] Vladislav Vaintroub [ wlad ]
            ralf.gebhardt Ralf Gebhardt made changes -
            Rank Ranked higher
            ralf.gebhardt Ralf Gebhardt made changes -
            Rank Ranked higher
            wlad Vladislav Vaintroub added a comment -

            Review done : commented on https://github.com/mariadb/server/commit/d62ea85e6f6c273a712f951da907ced06f01bc03

            wlad Vladislav Vaintroub added a comment - Review done : commented on https://github.com/mariadb/server/commit/d62ea85e6f6c273a712f951da907ced06f01bc03
            wlad Vladislav Vaintroub made changes -
            Status In Review [ 10002 ] Stalled [ 10000 ]
            wlad Vladislav Vaintroub made changes -
            Assignee Vladislav Vaintroub [ wlad ] Oleksandr Byelkin [ sanja ]
            ralf.gebhardt Ralf Gebhardt made changes -
            Epic Link PT-79 [ 68560 ]
            ralf.gebhardt Ralf Gebhardt made changes -
            Fix Version/s 10.5 [ 23123 ]
            Fix Version/s 10.4 [ 22408 ]
            ralf.gebhardt Ralf Gebhardt made changes -
            Labels Connectors
            serg Sergei Golubchik made changes -
            Fix Version/s 10.5 [ 23123 ]
            diego dupin Diego Dupin made changes -
            diego dupin Diego Dupin made changes -
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 72409 ] MariaDB v4 [ 131653 ]

            People

              sanja Oleksandr Byelkin
              serg Sergei Golubchik
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.