Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-19237

Skip sending metadata when possible for binary protocol

    XMLWordPrintable

Details

    Description

      This task goal is to permit not sending metadata every time

      What MySQL has done in version 8

      MySQL has changed the protocol to permit skipping metadata for version 8.
      see https://dev.mysql.com/doc/refman/8.0/en/c-api-optional-metadata.html
      and https://dev.mysql.com/worklog/task/?id=8134

      In description MySQL task indicate some big improvement:

      • METADATA_FULL : 3.48w TPS, Net send 113M
      • METADATA_IGNORE: 13.8w TPS, Net send 30M

      MySQL implementation send metadata (or not) according to resultset_metadata variable.
      Associate capability is CLIENT_OPTIONAL_RESULTSET_METADATA (1UL << 25)

      When resultset_metadata = 0 (NONE) no metadata is send for COM_STMT_PREPARE/COM_STMT_EXECUTE/COM_QUERY

      Exchange format change

      Column count packet

      There is a "new metadata_follows flag"

      * int<lenenc> column count
      * int<1> metadata_follows flag (0x00: No metadata, 0x01: metadata follows)
      

      When metadata_follows = 0x00, protocol skip column informations packet (continuing by an EOF_packet if CLIENT_DEPRECATE_EOF capability is not set or rows if CLIENT_DEPRECATE_EOF is set).
      be careful not to rely on mysql documentation that indicate them in a wrong order

      COM_PREPARE response

      COM_STMT_PREPARE_OK:

      * int<1> 0x00 COM_STMT_PREPARE_OK header
      * int<4> statement id
      * int<2> number of columns in the returned result set (or 0 if statement does not return result set)
      * int<2> number of prepared statement parameters ('?' placeholders)
      * string<1> -not used-
      * int<2> number of warnings
      * if capabilities & CLIENT_OPTIONAL_RESULTSET_METADATA
      **  int<1> metadata_follows flag (0x00: No metadata, 0x01: metadata follows)
      }
      

      if metadata is set to 0x00, column definition packet that decribe each parameter and each column are skipped.

      Requirement:

      Connectors need metadata to parse resultset.
      Proposal is to offer this functionality, but in a more modular way.

      What do connectors need:

      For first execute of a prepared statement:

      For next execute:

      There are two issues with not sending metadata each time:

      • Metadata can change over time. Server already automatically reprepares internally, so it knows to resend the metadata even if the client asked not to send the metadata every time.
      • commands like "SELECT ?" will have metadata type changing according to parameter. Metadata must then be send each time.

      COM_STMT_PREPARE still need to have the possibility to ask for metadata (at least java need that possibility to have metadata (example) without any COM_STMT_EXECUTE execution)

      For production environment where DDL never change an option can be set on client level to use that same functionality for COM_QUERY.
      Client can then cache metadata and ask for COM_QUERY without metadata.

      MySQL doesn't permits modularity, so client request have to add this information.
      That can be either by:

      • create new commands COM_STMT_PREPARE_WITHOUT_META, COM_STMT_EXECUTE_WITHOUT_META and COM_QUERY_WITHOUT_META
      • changing existing COM_STMT_PREPARE, COM_STMT_EXECUTE and COM_QUERY format.

      Attachments

        Issue Links

          Activity

            People

              wlad Vladislav Vaintroub
              diego dupin Diego Dupin
              Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.