Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-19237

Skip sending metadata when possible for binary protocol

Details

    Description

      This task goal is to permit not sending metadata every time

      What MySQL has done in version 8

      MySQL has changed the protocol to permit skipping metadata for version 8.
      see https://dev.mysql.com/doc/refman/8.0/en/c-api-optional-metadata.html
      and https://dev.mysql.com/worklog/task/?id=8134

      In description MySQL task indicate some big improvement:

      • METADATA_FULL : 3.48w TPS, Net send 113M
      • METADATA_IGNORE: 13.8w TPS, Net send 30M

      MySQL implementation send metadata (or not) according to resultset_metadata variable.
      Associate capability is CLIENT_OPTIONAL_RESULTSET_METADATA (1UL << 25)

      When resultset_metadata = 0 (NONE) no metadata is send for COM_STMT_PREPARE/COM_STMT_EXECUTE/COM_QUERY

      Exchange format change

      Column count packet

      There is a "new metadata_follows flag"

      * int<lenenc> column count
      * int<1> metadata_follows flag (0x00: No metadata, 0x01: metadata follows)
      

      When metadata_follows = 0x00, protocol skip column informations packet (continuing by an EOF_packet if CLIENT_DEPRECATE_EOF capability is not set or rows if CLIENT_DEPRECATE_EOF is set).
      be careful not to rely on mysql documentation that indicate them in a wrong order

      COM_PREPARE response

      COM_STMT_PREPARE_OK:

      * int<1> 0x00 COM_STMT_PREPARE_OK header
      * int<4> statement id
      * int<2> number of columns in the returned result set (or 0 if statement does not return result set)
      * int<2> number of prepared statement parameters ('?' placeholders)
      * string<1> -not used-
      * int<2> number of warnings
      * if capabilities & CLIENT_OPTIONAL_RESULTSET_METADATA
      **  int<1> metadata_follows flag (0x00: No metadata, 0x01: metadata follows)
      }
      

      if metadata is set to 0x00, column definition packet that decribe each parameter and each column are skipped.

      Requirement:

      Connectors need metadata to parse resultset.
      Proposal is to offer this functionality, but in a more modular way.

      What do connectors need:

      For first execute of a prepared statement:

      For next execute:

      There are two issues with not sending metadata each time:

      • Metadata can change over time. Server already automatically reprepares internally, so it knows to resend the metadata even if the client asked not to send the metadata every time.
      • commands like "SELECT ?" will have metadata type changing according to parameter. Metadata must then be send each time.

      COM_STMT_PREPARE still need to have the possibility to ask for metadata (at least java need that possibility to have metadata (example) without any COM_STMT_EXECUTE execution)

      For production environment where DDL never change an option can be set on client level to use that same functionality for COM_QUERY.
      Client can then cache metadata and ask for COM_QUERY without metadata.

      MySQL doesn't permits modularity, so client request have to add this information.
      That can be either by:

      • create new commands COM_STMT_PREPARE_WITHOUT_META, COM_STMT_EXECUTE_WITHOUT_META and COM_QUERY_WITHOUT_META
      • changing existing COM_STMT_PREPARE, COM_STMT_EXECUTE and COM_QUERY format.

      Attachments

        Issue Links

          Activity

            diego dupin Diego Dupin created issue -
            diego dupin Diego Dupin made changes -
            Field Original Value New Value
            Key CONJ-697 MDEV-19237
            Project MariaDB Connector/J [ 10301 ] MariaDB Server [ 10000 ]
            diego dupin Diego Dupin made changes -
            Description This task goal is to permit not sending metadata every time for [COM_STMT_EXECUTE|https://mariadb.com/kb/en/library/com_stmt_execute/].
            [COM_STMT_PREPARE |https://mariadb.com/kb/en/library/com_stmt_prepare/] already send metadata.

            Benchmarks shows a 10% to 20% performance change avoiding sending metadata.
            Proposition is to avoid sending metadata if there is no change.

            There is 2 issues about not sending metadata each time :
            * Metadata can change in time. Server already automatically reprepare internally, so has the information to resend metadata next time.
            * commands like "SELECT ?" will have metadata type changing according to parameter. Metadata must then be send each time.

              
            2 possible solution concerning protocol :
            * reusing COM_STMT_EXECUTE
            * create a new dedicated commands

            h3. reusing COM_STMT_EXECUTE
            add [flag|https://mariadb.com/kb/en/library/com_stmt_execute/#flag] AVOID_SENDING_METADATA with value 128

            Current results protocol is :
            {noformat}
            Resultset metadata
              1 Column count packet
                  for each column (i.e column_count times)
                      Column Definition packet
            if not (CLIENT_DEPRECATE_EOF capability set) EOF_Packet
            n resultset row
            EOF packet
            {noformat}

            Without changing protocol,
            If metadata didn't change, [Column count packet|https://mariadb.com/kb/en/library/resultset/#column-count-packet]
            can be set to null value (0xFB) without column informations.

            Current results protocol would be :
            {noformat}
             Resultset metadata
                  1 Column count packet set to null
             if not (CLIENT_DEPRECATE_EOF capability set) EOF_Packet
             n resultset row
             EOF packet
            {noformat}

            and if metadata changed, the actual format is still valid

            This task goal is to permit not sending metadata every time for [COM_STMT_EXECUTE|https://mariadb.com/kb/en/library/com_stmt_execute/].
            [COM_STMT_PREPARE |https://mariadb.com/kb/en/library/com_stmt_prepare/] already send metadata.

            Benchmarks shows a 10% to 20% performance change avoiding sending metadata.
            Proposition is to avoid sending metadata if there is no change.

            There is 2 issues about not sending metadata each time :
            * Metadata can change in time. Server already automatically reprepare internally, so has the information to resend metadata next time.
            * commands like "SELECT ?" will have metadata type changing according to parameter. Metadata must then be send each time.

              
            2 possible solution concerning protocol :
            * reusing COM_STMT_EXECUTE
            * create a new dedicated commands

            h3. reusing COM_STMT_EXECUTE
            add [flag|https://mariadb.com/kb/en/library/com_stmt_execute/#flag] AVOID_SENDING_METADATA with value 128

            Current results protocol is :
            {noformat}
            Resultset metadata
              1 Column count packet
                  for each column (i.e column_count times)
                      Column Definition packet
            if not (CLIENT_DEPRECATE_EOF capability set) EOF_Packet
            n resultset row
            EOF packet
            {noformat}

            Without changing protocol,
            If metadata didn't change, [Column count packet|https://mariadb.com/kb/en/library/resultset/#column-count-packet]
            can be set to 0 encoded on 3 bytes (0xFC0000) without column informations.

            Current results protocol would be :
            {noformat}
             Resultset metadata
                  1 Column count packet set to null
             if not (CLIENT_DEPRECATE_EOF capability set) EOF_Packet
             n resultset row
             EOF packet
            {noformat}

            and if metadata changed, the actual format is still valid

            diego dupin Diego Dupin made changes -
            Description This task goal is to permit not sending metadata every time for [COM_STMT_EXECUTE|https://mariadb.com/kb/en/library/com_stmt_execute/].
            [COM_STMT_PREPARE |https://mariadb.com/kb/en/library/com_stmt_prepare/] already send metadata.

            Benchmarks shows a 10% to 20% performance change avoiding sending metadata.
            Proposition is to avoid sending metadata if there is no change.

            There is 2 issues about not sending metadata each time :
            * Metadata can change in time. Server already automatically reprepare internally, so has the information to resend metadata next time.
            * commands like "SELECT ?" will have metadata type changing according to parameter. Metadata must then be send each time.

              
            2 possible solution concerning protocol :
            * reusing COM_STMT_EXECUTE
            * create a new dedicated commands

            h3. reusing COM_STMT_EXECUTE
            add [flag|https://mariadb.com/kb/en/library/com_stmt_execute/#flag] AVOID_SENDING_METADATA with value 128

            Current results protocol is :
            {noformat}
            Resultset metadata
              1 Column count packet
                  for each column (i.e column_count times)
                      Column Definition packet
            if not (CLIENT_DEPRECATE_EOF capability set) EOF_Packet
            n resultset row
            EOF packet
            {noformat}

            Without changing protocol,
            If metadata didn't change, [Column count packet|https://mariadb.com/kb/en/library/resultset/#column-count-packet]
            can be set to 0 encoded on 3 bytes (0xFC0000) without column informations.

            Current results protocol would be :
            {noformat}
             Resultset metadata
                  1 Column count packet set to null
             if not (CLIENT_DEPRECATE_EOF capability set) EOF_Packet
             n resultset row
             EOF packet
            {noformat}

            and if metadata changed, the actual format is still valid

            This task goal is to permit not sending metadata every time for [COM_STMT_EXECUTE|https://mariadb.com/kb/en/library/com_stmt_execute/].
            [COM_STMT_PREPARE |https://mariadb.com/kb/en/library/com_stmt_prepare/] already send metadata.

            Benchmarks shows a 10% to 20% performance change avoiding sending metadata.
            Proposition is to avoid sending metadata if there is no change.

            There is 2 issues about not sending metadata each time :
            * Metadata can change in time. Server already automatically reprepare internally, so has the information to resend metadata next time.
            * commands like "SELECT ?" will have metadata type changing according to parameter. Metadata must then be send each time.

              
            2 possible solution concerning protocol :
            * reusing COM_STMT_EXECUTE
            * create a new dedicated commands

            h3. reusing COM_STMT_EXECUTE
            add [flag|https://mariadb.com/kb/en/library/com_stmt_execute/#flag] AVOID_SENDING_METADATA with value 128

            Current results protocol is :
            {noformat}
            Resultset metadata
              1 Column count packet
                  for each column (i.e column_count times)
                      Column Definition packet
            if not (CLIENT_DEPRECATE_EOF capability set) EOF_Packet
            n resultset row
            EOF packet
            {noformat}

            Without changing protocol,
            If metadata didn't change, [Column count packet|https://mariadb.com/kb/en/library/resultset/#column-count-packet]
            can be set to 0 encoded on 3 bytes (0xFC0000) without column informations.

            Current results protocol would be :
            {noformat}
             Resultset metadata
                  1 Column count packet set to 0xFC0000
             if not (CLIENT_DEPRECATE_EOF capability set) EOF_Packet
             n resultset row
             EOF packet
            {noformat}

            and if metadata changed, the actual format is still valid

            wlad Vladislav Vaintroub made changes -
            Assignee Diego Dupin [ diego dupin ] Vladislav Vaintroub [ wlad ]
            wlad Vladislav Vaintroub added a comment - - edited

            I think EOF packet is not needed after column count 0xFC0000. It is not easy to imagine any of the existing clients would be able to cope with 0 metadata packets

            wlad Vladislav Vaintroub added a comment - - edited I think EOF packet is not needed after column count 0xFC0000. It is not easy to imagine any of the existing clients would be able to cope with 0 metadata packets
            wlad Vladislav Vaintroub made changes -
            Fix Version/s 10.5 [ 23123 ]
            diego dupin Diego Dupin made changes -
            serg Sergei Golubchik made changes -
            Assignee Vladislav Vaintroub [ wlad ]
            serg Sergei Golubchik made changes -
            Fix Version/s 10.5 [ 23123 ]
            diego dupin Diego Dupin made changes -
            Description This task goal is to permit not sending metadata every time for [COM_STMT_EXECUTE|https://mariadb.com/kb/en/library/com_stmt_execute/].
            [COM_STMT_PREPARE |https://mariadb.com/kb/en/library/com_stmt_prepare/] already send metadata.

            Benchmarks shows a 10% to 20% performance change avoiding sending metadata.
            Proposition is to avoid sending metadata if there is no change.

            There is 2 issues about not sending metadata each time :
            * Metadata can change in time. Server already automatically reprepare internally, so has the information to resend metadata next time.
            * commands like "SELECT ?" will have metadata type changing according to parameter. Metadata must then be send each time.

              
            2 possible solution concerning protocol :
            * reusing COM_STMT_EXECUTE
            * create a new dedicated commands

            h3. reusing COM_STMT_EXECUTE
            add [flag|https://mariadb.com/kb/en/library/com_stmt_execute/#flag] AVOID_SENDING_METADATA with value 128

            Current results protocol is :
            {noformat}
            Resultset metadata
              1 Column count packet
                  for each column (i.e column_count times)
                      Column Definition packet
            if not (CLIENT_DEPRECATE_EOF capability set) EOF_Packet
            n resultset row
            EOF packet
            {noformat}

            Without changing protocol,
            If metadata didn't change, [Column count packet|https://mariadb.com/kb/en/library/resultset/#column-count-packet]
            can be set to 0 encoded on 3 bytes (0xFC0000) without column informations.

            Current results protocol would be :
            {noformat}
             Resultset metadata
                  1 Column count packet set to 0xFC0000
             if not (CLIENT_DEPRECATE_EOF capability set) EOF_Packet
             n resultset row
             EOF packet
            {noformat}

            and if metadata changed, the actual format is still valid

            This task goal is to permit not sending metadata every time

            h2. What MySQL has done in version 8

            MySQL has changed the protocol accordingly for version 8.
            see https://dev.mysql.com/doc/refman/8.0/en/c-api-optional-metadata.html
            and https://dev.mysql.com/worklog/task/?id=8134

            in description MySQL task indicate some big improvement:
            * METADATA_FULL : 3.48w TPS, Net send 113M
            * METADATA_IGNORE: 13.8w TPS, Net send 30M

            MySQL implementation send metadata or not according to [resultset_metadata|https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_resultset_metadata] variable.
            Associate capability is CLIENT_OPTIONAL_RESULTSET_METADATA (1UL << 25)

            When resultset_metadata = 0 (NONE) no metadata is send for COM_STMT_PREPARE/COM_STMT_EXECUTE/COM_QUERY

            But connectors needs metadata to parse resultset.
            Proposition here is to offer this functionality, but in a more modular way.


            h3. Exchange format change


            h4. [Column count packet|https://mariadb.com/kb/en/resultset/#resultset]

            There is a "new metadata_follows flag"

            {noformat}
            * int<lenenc> column count
            * int<1> metadata_follows flag (0x00: No metadata, 0x01: metadata follows)
            {noformat}


            When metadata_follows = 0x00, protocol skip column informations packet (continuing by an EOF_packet if CLIENT_DEPRECATE_EOF capability is not set or rows if CLIENT_DEPRECATE_EOF is set).
            be careful not to rely on [mysql documentation|https://dev.mysql.com/doc/dev/mysql-server/8.0.11/page_protocol_com_query_response_text_resultset.html] that indicate them in a wrong order

            h4. [COM_PREPARE response|https://mariadb.com/kb/en/com_stmt_prepare/#com_stmt_prepare-response]

            [COM_STMT_PREPARE_OK|https://mariadb.com/kb/en/com_stmt_prepare/#COM_STMT_PREPARE_OK]:


            {noformat}
            * int<1> 0x00 COM_STMT_PREPARE_OK header
            * int<4> statement id
            * int<2> number of columns in the returned result set (or 0 if statement does not return result set)
            * int<2> number of prepared statement parameters ('?' placeholders)
            * string<1> -not used-
            * int<2> number of warnings
            * if capabilities & CLIENT_OPTIONAL_RESULTSET_METADATA
            ** int<1> metadata_follows flag (0x00: No metadata, 0x01: metadata follows)
            }
            {noformat}


            if metadata is set to 0x00, column definition packet that decribe each parameter and each column are skipped.


            h2. Requirement:

            What does need connectors :
             - COM_STMT_PREPARE asked WITHOUT metadata.
             - COM_STMT_EXECUTE asked WITH METADATA. metadata is cached

            next query:
             - COM_STMT_EXECUTE WITHOUT metadata. use cached metadata

            There is 2 issues about not sending metadata each time :
            * Metadata can change in time. Server already automatically reprepare internally, so has the information to resend metadata next time even if asked for no metadata.
            * commands like "SELECT ?" will have metadata type changing according to parameter. Metadata must then be send each time.
             
              
            COM_STMT_PREPARE still need to have the possibility to asked metadata (at least java need that possibility to have metadata without any COM_STMT_EXECUTE for example for https://docs.oracle.com/javase/7/docs/api/java/sql/PreparedStatement.html#getParameterMetaData() without having executed COM_STMT_EXECUTE)


            for production environment where DDL never change an option can be set on client level to use that same functionality for COM_QUERY.
            Client can then cache metadata and ask for COM_QUERY without metadata.

            MySQL doesn't permits a modularity, so client request have to add this information.
            That can be either by:
            * create new commands COM_STMT_PREPARE_WITHOUT_META, COM_STMT_EXECUTE_WITHOUT_META and COM_QUERY_WITHOUT_META
            * changing existing COM_STMT_PREPARE, COM_STMT_EXECUTE and COM_QUERY format.

            diego dupin Diego Dupin made changes -
            Description This task goal is to permit not sending metadata every time

            h2. What MySQL has done in version 8

            MySQL has changed the protocol accordingly for version 8.
            see https://dev.mysql.com/doc/refman/8.0/en/c-api-optional-metadata.html
            and https://dev.mysql.com/worklog/task/?id=8134

            in description MySQL task indicate some big improvement:
            * METADATA_FULL : 3.48w TPS, Net send 113M
            * METADATA_IGNORE: 13.8w TPS, Net send 30M

            MySQL implementation send metadata or not according to [resultset_metadata|https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_resultset_metadata] variable.
            Associate capability is CLIENT_OPTIONAL_RESULTSET_METADATA (1UL << 25)

            When resultset_metadata = 0 (NONE) no metadata is send for COM_STMT_PREPARE/COM_STMT_EXECUTE/COM_QUERY

            But connectors needs metadata to parse resultset.
            Proposition here is to offer this functionality, but in a more modular way.


            h3. Exchange format change


            h4. [Column count packet|https://mariadb.com/kb/en/resultset/#resultset]

            There is a "new metadata_follows flag"

            {noformat}
            * int<lenenc> column count
            * int<1> metadata_follows flag (0x00: No metadata, 0x01: metadata follows)
            {noformat}


            When metadata_follows = 0x00, protocol skip column informations packet (continuing by an EOF_packet if CLIENT_DEPRECATE_EOF capability is not set or rows if CLIENT_DEPRECATE_EOF is set).
            be careful not to rely on [mysql documentation|https://dev.mysql.com/doc/dev/mysql-server/8.0.11/page_protocol_com_query_response_text_resultset.html] that indicate them in a wrong order

            h4. [COM_PREPARE response|https://mariadb.com/kb/en/com_stmt_prepare/#com_stmt_prepare-response]

            [COM_STMT_PREPARE_OK|https://mariadb.com/kb/en/com_stmt_prepare/#COM_STMT_PREPARE_OK]:


            {noformat}
            * int<1> 0x00 COM_STMT_PREPARE_OK header
            * int<4> statement id
            * int<2> number of columns in the returned result set (or 0 if statement does not return result set)
            * int<2> number of prepared statement parameters ('?' placeholders)
            * string<1> -not used-
            * int<2> number of warnings
            * if capabilities & CLIENT_OPTIONAL_RESULTSET_METADATA
            ** int<1> metadata_follows flag (0x00: No metadata, 0x01: metadata follows)
            }
            {noformat}


            if metadata is set to 0x00, column definition packet that decribe each parameter and each column are skipped.


            h2. Requirement:

            What does need connectors :
             - COM_STMT_PREPARE asked WITHOUT metadata.
             - COM_STMT_EXECUTE asked WITH METADATA. metadata is cached

            next query:
             - COM_STMT_EXECUTE WITHOUT metadata. use cached metadata

            There is 2 issues about not sending metadata each time :
            * Metadata can change in time. Server already automatically reprepare internally, so has the information to resend metadata next time even if asked for no metadata.
            * commands like "SELECT ?" will have metadata type changing according to parameter. Metadata must then be send each time.
             
              
            COM_STMT_PREPARE still need to have the possibility to asked metadata (at least java need that possibility to have metadata without any COM_STMT_EXECUTE for example for https://docs.oracle.com/javase/7/docs/api/java/sql/PreparedStatement.html#getParameterMetaData() without having executed COM_STMT_EXECUTE)


            for production environment where DDL never change an option can be set on client level to use that same functionality for COM_QUERY.
            Client can then cache metadata and ask for COM_QUERY without metadata.

            MySQL doesn't permits a modularity, so client request have to add this information.
            That can be either by:
            * create new commands COM_STMT_PREPARE_WITHOUT_META, COM_STMT_EXECUTE_WITHOUT_META and COM_QUERY_WITHOUT_META
            * changing existing COM_STMT_PREPARE, COM_STMT_EXECUTE and COM_QUERY format.

            This task goal is to permit not sending metadata every time

            h2. What MySQL has done in version 8

            MySQL has changed the protocol accordingly for version 8.
            see https://dev.mysql.com/doc/refman/8.0/en/c-api-optional-metadata.html
            and https://dev.mysql.com/worklog/task/?id=8134

            in description MySQL task indicate some big improvement:
            * METADATA_FULL : 3.48w TPS, Net send 113M
            * METADATA_IGNORE: 13.8w TPS, Net send 30M

            MySQL implementation send metadata or not according to [resultset_metadata|https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_resultset_metadata] variable.
            Associate capability is CLIENT_OPTIONAL_RESULTSET_METADATA (1UL << 25)

            When resultset_metadata = 0 (NONE) no metadata is send for COM_STMT_PREPARE/COM_STMT_EXECUTE/COM_QUERY

            But connectors needs metadata to parse resultset.
            Proposition here is to offer this functionality, but in a more modular way.


            h3. Exchange format change


            h4. [Column count packet|https://mariadb.com/kb/en/resultset/#resultset]

            There is a "new metadata_follows flag"

            {noformat}
            * int<lenenc> column count
            * int<1> metadata_follows flag (0x00: No metadata, 0x01: metadata follows)
            {noformat}


            When metadata_follows = 0x00, protocol skip column informations packet (continuing by an EOF_packet if CLIENT_DEPRECATE_EOF capability is not set or rows if CLIENT_DEPRECATE_EOF is set).
            be careful not to rely on [mysql documentation|https://dev.mysql.com/doc/dev/mysql-server/8.0.11/page_protocol_com_query_response_text_resultset.html] that indicate them in a wrong order

            h4. [COM_PREPARE response|https://mariadb.com/kb/en/com_stmt_prepare/#com_stmt_prepare-response]

            [COM_STMT_PREPARE_OK|https://mariadb.com/kb/en/com_stmt_prepare/#COM_STMT_PREPARE_OK]:


            {noformat}
            * int<1> 0x00 COM_STMT_PREPARE_OK header
            * int<4> statement id
            * int<2> number of columns in the returned result set (or 0 if statement does not return result set)
            * int<2> number of prepared statement parameters ('?' placeholders)
            * string<1> -not used-
            * int<2> number of warnings
            * if capabilities & CLIENT_OPTIONAL_RESULTSET_METADATA
            ** int<1> metadata_follows flag (0x00: No metadata, 0x01: metadata follows)
            }
            {noformat}


            if metadata is set to 0x00, column definition packet that decribe each parameter and each column are skipped.


            h2. Requirement:

            What does need connectors :
             - [COM_STMT_PREPARE|https://mariadb.com/kb/en/com_stmt_prepare/] asked WITHOUT metadata.
             - [COM_STMT_EXECUTE|https://mariadb.com/kb/en/com_stmt_execute/] asked WITH METADATA. metadata is cached

            next query:
             - [COM_STMT_EXECUTE|https://mariadb.com/kb/en/com_stmt_execute/] WITHOUT metadata. use cached metadata

            There is 2 issues about not sending metadata each time :
            * Metadata can change in time. Server already automatically reprepare internally, so has the information to resend metadata next time even if asked for no metadata.
            * commands like "SELECT ?" will have metadata type changing according to parameter. Metadata must then be send each time.
             
              
            [COM_STMT_PREPARE|https://mariadb.com/kb/en/com_stmt_prepare/] still need to have the possibility to asked metadata (at least java need that possibility to have metadata ([example|https://docs.oracle.com/javase/7/docs/api/java/sql/PreparedStatement.html#getParameterMetaData()]) without any COM_STMT_EXECUTE)


            For production environment where DDL never change an option can be set on client level to use that same functionality for COM_QUERY.
            Client can then cache metadata and ask for COM_QUERY without metadata.

            MySQL doesn't permits a modularity, so client request have to add this information.
            That can be either by:
            * create new commands COM_STMT_PREPARE_WITHOUT_META, COM_STMT_EXECUTE_WITHOUT_META and COM_QUERY_WITHOUT_META
            * changing existing COM_STMT_PREPARE, COM_STMT_EXECUTE and COM_QUERY format.

            diego dupin Diego Dupin made changes -
            Description This task goal is to permit not sending metadata every time

            h2. What MySQL has done in version 8

            MySQL has changed the protocol accordingly for version 8.
            see https://dev.mysql.com/doc/refman/8.0/en/c-api-optional-metadata.html
            and https://dev.mysql.com/worklog/task/?id=8134

            in description MySQL task indicate some big improvement:
            * METADATA_FULL : 3.48w TPS, Net send 113M
            * METADATA_IGNORE: 13.8w TPS, Net send 30M

            MySQL implementation send metadata or not according to [resultset_metadata|https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_resultset_metadata] variable.
            Associate capability is CLIENT_OPTIONAL_RESULTSET_METADATA (1UL << 25)

            When resultset_metadata = 0 (NONE) no metadata is send for COM_STMT_PREPARE/COM_STMT_EXECUTE/COM_QUERY

            But connectors needs metadata to parse resultset.
            Proposition here is to offer this functionality, but in a more modular way.


            h3. Exchange format change


            h4. [Column count packet|https://mariadb.com/kb/en/resultset/#resultset]

            There is a "new metadata_follows flag"

            {noformat}
            * int<lenenc> column count
            * int<1> metadata_follows flag (0x00: No metadata, 0x01: metadata follows)
            {noformat}


            When metadata_follows = 0x00, protocol skip column informations packet (continuing by an EOF_packet if CLIENT_DEPRECATE_EOF capability is not set or rows if CLIENT_DEPRECATE_EOF is set).
            be careful not to rely on [mysql documentation|https://dev.mysql.com/doc/dev/mysql-server/8.0.11/page_protocol_com_query_response_text_resultset.html] that indicate them in a wrong order

            h4. [COM_PREPARE response|https://mariadb.com/kb/en/com_stmt_prepare/#com_stmt_prepare-response]

            [COM_STMT_PREPARE_OK|https://mariadb.com/kb/en/com_stmt_prepare/#COM_STMT_PREPARE_OK]:


            {noformat}
            * int<1> 0x00 COM_STMT_PREPARE_OK header
            * int<4> statement id
            * int<2> number of columns in the returned result set (or 0 if statement does not return result set)
            * int<2> number of prepared statement parameters ('?' placeholders)
            * string<1> -not used-
            * int<2> number of warnings
            * if capabilities & CLIENT_OPTIONAL_RESULTSET_METADATA
            ** int<1> metadata_follows flag (0x00: No metadata, 0x01: metadata follows)
            }
            {noformat}


            if metadata is set to 0x00, column definition packet that decribe each parameter and each column are skipped.


            h2. Requirement:

            What does need connectors :
             - [COM_STMT_PREPARE|https://mariadb.com/kb/en/com_stmt_prepare/] asked WITHOUT metadata.
             - [COM_STMT_EXECUTE|https://mariadb.com/kb/en/com_stmt_execute/] asked WITH METADATA. metadata is cached

            next query:
             - [COM_STMT_EXECUTE|https://mariadb.com/kb/en/com_stmt_execute/] WITHOUT metadata. use cached metadata

            There is 2 issues about not sending metadata each time :
            * Metadata can change in time. Server already automatically reprepare internally, so has the information to resend metadata next time even if asked for no metadata.
            * commands like "SELECT ?" will have metadata type changing according to parameter. Metadata must then be send each time.
             
              
            [COM_STMT_PREPARE|https://mariadb.com/kb/en/com_stmt_prepare/] still need to have the possibility to asked metadata (at least java need that possibility to have metadata ([example|https://docs.oracle.com/javase/7/docs/api/java/sql/PreparedStatement.html#getParameterMetaData()]) without any COM_STMT_EXECUTE)


            For production environment where DDL never change an option can be set on client level to use that same functionality for COM_QUERY.
            Client can then cache metadata and ask for COM_QUERY without metadata.

            MySQL doesn't permits a modularity, so client request have to add this information.
            That can be either by:
            * create new commands COM_STMT_PREPARE_WITHOUT_META, COM_STMT_EXECUTE_WITHOUT_META and COM_QUERY_WITHOUT_META
            * changing existing COM_STMT_PREPARE, COM_STMT_EXECUTE and COM_QUERY format.

            This task goal is to permit not sending metadata every time

            h2. What MySQL has done in version 8

            MySQL has changed the protocol to permit skipping metadata for version 8.
            see https://dev.mysql.com/doc/refman/8.0/en/c-api-optional-metadata.html
            and https://dev.mysql.com/worklog/task/?id=8134

            In description MySQL task indicate some big improvement:
            * METADATA_FULL : 3.48w TPS, Net send 113M
            * METADATA_IGNORE: 13.8w TPS, Net send 30M

            MySQL implementation send metadata (or not) according to [resultset_metadata|https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_resultset_metadata] variable.
            Associate capability is CLIENT_OPTIONAL_RESULTSET_METADATA (1UL << 25)

            When resultset_metadata = 0 (NONE) no metadata is send for COM_STMT_PREPARE/COM_STMT_EXECUTE/COM_QUERY

            But connectors needs metadata to parse resultset.
            Proposition here is to offer this functionality, but in a more modular way.


            h3. Exchange format change


            h4. [Column count packet|https://mariadb.com/kb/en/resultset/#resultset]

            There is a "new metadata_follows flag"

            {noformat}
            * int<lenenc> column count
            * int<1> metadata_follows flag (0x00: No metadata, 0x01: metadata follows)
            {noformat}


            When metadata_follows = 0x00, protocol skip column informations packet (continuing by an EOF_packet if CLIENT_DEPRECATE_EOF capability is not set or rows if CLIENT_DEPRECATE_EOF is set).
            be careful not to rely on [mysql documentation|https://dev.mysql.com/doc/dev/mysql-server/8.0.11/page_protocol_com_query_response_text_resultset.html] that indicate them in a wrong order

            h4. [COM_PREPARE response|https://mariadb.com/kb/en/com_stmt_prepare/#com_stmt_prepare-response]

            [COM_STMT_PREPARE_OK|https://mariadb.com/kb/en/com_stmt_prepare/#COM_STMT_PREPARE_OK]:


            {noformat}
            * int<1> 0x00 COM_STMT_PREPARE_OK header
            * int<4> statement id
            * int<2> number of columns in the returned result set (or 0 if statement does not return result set)
            * int<2> number of prepared statement parameters ('?' placeholders)
            * string<1> -not used-
            * int<2> number of warnings
            * if capabilities & CLIENT_OPTIONAL_RESULTSET_METADATA
            ** int<1> metadata_follows flag (0x00: No metadata, 0x01: metadata follows)
            }
            {noformat}


            if metadata is set to 0x00, column definition packet that decribe each parameter and each column are skipped.


            h2. Requirement:

            What does need connectors :
             - [COM_STMT_PREPARE|https://mariadb.com/kb/en/com_stmt_prepare/] asked WITHOUT metadata.
             - [COM_STMT_EXECUTE|https://mariadb.com/kb/en/com_stmt_execute/] asked WITH METADATA. metadata is cached

            next query:
             - [COM_STMT_EXECUTE|https://mariadb.com/kb/en/com_stmt_execute/] WITHOUT metadata. use cached metadata

            There is 2 issues about not sending metadata each time :
            * Metadata can change in time. Server already automatically reprepare internally, so has the information to resend metadata next time even if asked for no metadata.
            * commands like "SELECT ?" will have metadata type changing according to parameter. Metadata must then be send each time.
             
              
            [COM_STMT_PREPARE|https://mariadb.com/kb/en/com_stmt_prepare/] still need to have the possibility to asked metadata (at least java need that possibility to have metadata ([example|https://docs.oracle.com/javase/7/docs/api/java/sql/PreparedStatement.html#getParameterMetaData()]) without any COM_STMT_EXECUTE)


            For production environment where DDL never change an option can be set on client level to use that same functionality for COM_QUERY.
            Client can then cache metadata and ask for COM_QUERY without metadata.

            MySQL doesn't permits a modularity, so client request have to add this information.
            That can be either by:
            * create new commands COM_STMT_PREPARE_WITHOUT_META, COM_STMT_EXECUTE_WITHOUT_META and COM_QUERY_WITHOUT_META
            * changing existing COM_STMT_PREPARE, COM_STMT_EXECUTE and COM_QUERY format.

            diego dupin Diego Dupin made changes -
            Description This task goal is to permit not sending metadata every time

            h2. What MySQL has done in version 8

            MySQL has changed the protocol to permit skipping metadata for version 8.
            see https://dev.mysql.com/doc/refman/8.0/en/c-api-optional-metadata.html
            and https://dev.mysql.com/worklog/task/?id=8134

            In description MySQL task indicate some big improvement:
            * METADATA_FULL : 3.48w TPS, Net send 113M
            * METADATA_IGNORE: 13.8w TPS, Net send 30M

            MySQL implementation send metadata (or not) according to [resultset_metadata|https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_resultset_metadata] variable.
            Associate capability is CLIENT_OPTIONAL_RESULTSET_METADATA (1UL << 25)

            When resultset_metadata = 0 (NONE) no metadata is send for COM_STMT_PREPARE/COM_STMT_EXECUTE/COM_QUERY

            But connectors needs metadata to parse resultset.
            Proposition here is to offer this functionality, but in a more modular way.


            h3. Exchange format change


            h4. [Column count packet|https://mariadb.com/kb/en/resultset/#resultset]

            There is a "new metadata_follows flag"

            {noformat}
            * int<lenenc> column count
            * int<1> metadata_follows flag (0x00: No metadata, 0x01: metadata follows)
            {noformat}


            When metadata_follows = 0x00, protocol skip column informations packet (continuing by an EOF_packet if CLIENT_DEPRECATE_EOF capability is not set or rows if CLIENT_DEPRECATE_EOF is set).
            be careful not to rely on [mysql documentation|https://dev.mysql.com/doc/dev/mysql-server/8.0.11/page_protocol_com_query_response_text_resultset.html] that indicate them in a wrong order

            h4. [COM_PREPARE response|https://mariadb.com/kb/en/com_stmt_prepare/#com_stmt_prepare-response]

            [COM_STMT_PREPARE_OK|https://mariadb.com/kb/en/com_stmt_prepare/#COM_STMT_PREPARE_OK]:


            {noformat}
            * int<1> 0x00 COM_STMT_PREPARE_OK header
            * int<4> statement id
            * int<2> number of columns in the returned result set (or 0 if statement does not return result set)
            * int<2> number of prepared statement parameters ('?' placeholders)
            * string<1> -not used-
            * int<2> number of warnings
            * if capabilities & CLIENT_OPTIONAL_RESULTSET_METADATA
            ** int<1> metadata_follows flag (0x00: No metadata, 0x01: metadata follows)
            }
            {noformat}


            if metadata is set to 0x00, column definition packet that decribe each parameter and each column are skipped.


            h2. Requirement:

            What does need connectors :
             - [COM_STMT_PREPARE|https://mariadb.com/kb/en/com_stmt_prepare/] asked WITHOUT metadata.
             - [COM_STMT_EXECUTE|https://mariadb.com/kb/en/com_stmt_execute/] asked WITH METADATA. metadata is cached

            next query:
             - [COM_STMT_EXECUTE|https://mariadb.com/kb/en/com_stmt_execute/] WITHOUT metadata. use cached metadata

            There is 2 issues about not sending metadata each time :
            * Metadata can change in time. Server already automatically reprepare internally, so has the information to resend metadata next time even if asked for no metadata.
            * commands like "SELECT ?" will have metadata type changing according to parameter. Metadata must then be send each time.
             
              
            [COM_STMT_PREPARE|https://mariadb.com/kb/en/com_stmt_prepare/] still need to have the possibility to asked metadata (at least java need that possibility to have metadata ([example|https://docs.oracle.com/javase/7/docs/api/java/sql/PreparedStatement.html#getParameterMetaData()]) without any COM_STMT_EXECUTE)


            For production environment where DDL never change an option can be set on client level to use that same functionality for COM_QUERY.
            Client can then cache metadata and ask for COM_QUERY without metadata.

            MySQL doesn't permits a modularity, so client request have to add this information.
            That can be either by:
            * create new commands COM_STMT_PREPARE_WITHOUT_META, COM_STMT_EXECUTE_WITHOUT_META and COM_QUERY_WITHOUT_META
            * changing existing COM_STMT_PREPARE, COM_STMT_EXECUTE and COM_QUERY format.

            This task goal is to permit not sending metadata every time

            h2. What MySQL has done in version 8

            MySQL has changed the protocol to permit skipping metadata for version 8.
            see https://dev.mysql.com/doc/refman/8.0/en/c-api-optional-metadata.html
            and https://dev.mysql.com/worklog/task/?id=8134

            In description MySQL task indicate some big improvement:
            * METADATA_FULL : 3.48w TPS, Net send 113M
            * METADATA_IGNORE: 13.8w TPS, Net send 30M

            MySQL implementation send metadata (or not) according to [resultset_metadata|https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_resultset_metadata] variable.
            Associate capability is CLIENT_OPTIONAL_RESULTSET_METADATA (1UL << 25)

            When resultset_metadata = 0 (NONE) no metadata is send for COM_STMT_PREPARE/COM_STMT_EXECUTE/COM_QUERY

            h3. Exchange format change


            h4. [Column count packet|https://mariadb.com/kb/en/resultset/#resultset]

            There is a "new metadata_follows flag"

            {noformat}
            * int<lenenc> column count
            * int<1> metadata_follows flag (0x00: No metadata, 0x01: metadata follows)
            {noformat}


            When metadata_follows = 0x00, protocol skip column informations packet (continuing by an EOF_packet if CLIENT_DEPRECATE_EOF capability is not set or rows if CLIENT_DEPRECATE_EOF is set).
            be careful not to rely on [mysql documentation|https://dev.mysql.com/doc/dev/mysql-server/8.0.11/page_protocol_com_query_response_text_resultset.html] that indicate them in a wrong order

            h4. [COM_PREPARE response|https://mariadb.com/kb/en/com_stmt_prepare/#com_stmt_prepare-response]

            [COM_STMT_PREPARE_OK|https://mariadb.com/kb/en/com_stmt_prepare/#COM_STMT_PREPARE_OK]:


            {noformat}
            * int<1> 0x00 COM_STMT_PREPARE_OK header
            * int<4> statement id
            * int<2> number of columns in the returned result set (or 0 if statement does not return result set)
            * int<2> number of prepared statement parameters ('?' placeholders)
            * string<1> -not used-
            * int<2> number of warnings
            * if capabilities & CLIENT_OPTIONAL_RESULTSET_METADATA
            ** int<1> metadata_follows flag (0x00: No metadata, 0x01: metadata follows)
            }
            {noformat}


            if metadata is set to 0x00, column definition packet that decribe each parameter and each column are skipped.


            h2. Requirement:

            Connectors needs metadata to parse resultset.
            Proposition here is to offer this functionality, but in a more modular way.

            What does need connectors :

            For first prepare statement:
             - [COM_STMT_PREPARE|https://mariadb.com/kb/en/com_stmt_prepare/] asked WITHOUT metadata.
             - [COM_STMT_EXECUTE|https://mariadb.com/kb/en/com_stmt_execute/] asked WITH METADATA. metadata is cached

            For next prepare statement:
             - [COM_STMT_EXECUTE|https://mariadb.com/kb/en/com_stmt_execute/] WITHOUT metadata. use cached metadata


            There is 2 issues about not sending metadata each time :
            * Metadata can change in time. Server already automatically reprepare internally, so has the information to resend metadata next time even if ask not to send metadata.
            * commands like "SELECT ?" will have metadata type changing according to parameter. Metadata must then be send each time.
             
              
            [COM_STMT_PREPARE|https://mariadb.com/kb/en/com_stmt_prepare/] still need to have the possibility to ask metadata (at least java need that possibility to have metadata ([example|https://docs.oracle.com/javase/7/docs/api/java/sql/PreparedStatement.html#getParameterMetaData()]) without any COM_STMT_EXECUTE execution)


            For production environment where DDL never change an option can be set on client level to use that same functionality for COM_QUERY.
            Client can then cache metadata and ask for COM_QUERY without metadata.

            MySQL doesn't permits a modularity, so client request have to add this information.
            That can be either by:
            * create new commands COM_STMT_PREPARE_WITHOUT_META, COM_STMT_EXECUTE_WITHOUT_META and COM_QUERY_WITHOUT_META
            * changing existing COM_STMT_PREPARE, COM_STMT_EXECUTE and COM_QUERY format.

            ratzpo Rasmus Johansson (Inactive) made changes -
            Fix Version/s 10.6 [ 24028 ]
            ratzpo Rasmus Johansson (Inactive) made changes -
            Assignee Oleksandr Byelkin [ sanja ]
            ratzpo Rasmus Johansson (Inactive) made changes -
            Priority Major [ 3 ] Critical [ 2 ]
            Lawrin Lawrin Novitsky made changes -
            wlad Vladislav Vaintroub made changes -
            Assignee Oleksandr Byelkin [ sanja ] Vladislav Vaintroub [ wlad ]
            serg Sergei Golubchik made changes -
            Description This task goal is to permit not sending metadata every time

            h2. What MySQL has done in version 8

            MySQL has changed the protocol to permit skipping metadata for version 8.
            see https://dev.mysql.com/doc/refman/8.0/en/c-api-optional-metadata.html
            and https://dev.mysql.com/worklog/task/?id=8134

            In description MySQL task indicate some big improvement:
            * METADATA_FULL : 3.48w TPS, Net send 113M
            * METADATA_IGNORE: 13.8w TPS, Net send 30M

            MySQL implementation send metadata (or not) according to [resultset_metadata|https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_resultset_metadata] variable.
            Associate capability is CLIENT_OPTIONAL_RESULTSET_METADATA (1UL << 25)

            When resultset_metadata = 0 (NONE) no metadata is send for COM_STMT_PREPARE/COM_STMT_EXECUTE/COM_QUERY

            h3. Exchange format change


            h4. [Column count packet|https://mariadb.com/kb/en/resultset/#resultset]

            There is a "new metadata_follows flag"

            {noformat}
            * int<lenenc> column count
            * int<1> metadata_follows flag (0x00: No metadata, 0x01: metadata follows)
            {noformat}


            When metadata_follows = 0x00, protocol skip column informations packet (continuing by an EOF_packet if CLIENT_DEPRECATE_EOF capability is not set or rows if CLIENT_DEPRECATE_EOF is set).
            be careful not to rely on [mysql documentation|https://dev.mysql.com/doc/dev/mysql-server/8.0.11/page_protocol_com_query_response_text_resultset.html] that indicate them in a wrong order

            h4. [COM_PREPARE response|https://mariadb.com/kb/en/com_stmt_prepare/#com_stmt_prepare-response]

            [COM_STMT_PREPARE_OK|https://mariadb.com/kb/en/com_stmt_prepare/#COM_STMT_PREPARE_OK]:


            {noformat}
            * int<1> 0x00 COM_STMT_PREPARE_OK header
            * int<4> statement id
            * int<2> number of columns in the returned result set (or 0 if statement does not return result set)
            * int<2> number of prepared statement parameters ('?' placeholders)
            * string<1> -not used-
            * int<2> number of warnings
            * if capabilities & CLIENT_OPTIONAL_RESULTSET_METADATA
            ** int<1> metadata_follows flag (0x00: No metadata, 0x01: metadata follows)
            }
            {noformat}


            if metadata is set to 0x00, column definition packet that decribe each parameter and each column are skipped.


            h2. Requirement:

            Connectors needs metadata to parse resultset.
            Proposition here is to offer this functionality, but in a more modular way.

            What does need connectors :

            For first prepare statement:
             - [COM_STMT_PREPARE|https://mariadb.com/kb/en/com_stmt_prepare/] asked WITHOUT metadata.
             - [COM_STMT_EXECUTE|https://mariadb.com/kb/en/com_stmt_execute/] asked WITH METADATA. metadata is cached

            For next prepare statement:
             - [COM_STMT_EXECUTE|https://mariadb.com/kb/en/com_stmt_execute/] WITHOUT metadata. use cached metadata


            There is 2 issues about not sending metadata each time :
            * Metadata can change in time. Server already automatically reprepare internally, so has the information to resend metadata next time even if ask not to send metadata.
            * commands like "SELECT ?" will have metadata type changing according to parameter. Metadata must then be send each time.
             
              
            [COM_STMT_PREPARE|https://mariadb.com/kb/en/com_stmt_prepare/] still need to have the possibility to ask metadata (at least java need that possibility to have metadata ([example|https://docs.oracle.com/javase/7/docs/api/java/sql/PreparedStatement.html#getParameterMetaData()]) without any COM_STMT_EXECUTE execution)


            For production environment where DDL never change an option can be set on client level to use that same functionality for COM_QUERY.
            Client can then cache metadata and ask for COM_QUERY without metadata.

            MySQL doesn't permits a modularity, so client request have to add this information.
            That can be either by:
            * create new commands COM_STMT_PREPARE_WITHOUT_META, COM_STMT_EXECUTE_WITHOUT_META and COM_QUERY_WITHOUT_META
            * changing existing COM_STMT_PREPARE, COM_STMT_EXECUTE and COM_QUERY format.

            This task goal is to permit not sending metadata every time

            h2. What MySQL has done in version 8

            MySQL has changed the protocol to permit skipping metadata for version 8.
            see https://dev.mysql.com/doc/refman/8.0/en/c-api-optional-metadata.html
            and https://dev.mysql.com/worklog/task/?id=8134

            In description MySQL task indicate some big improvement:
            * METADATA_FULL : 3.48w TPS, Net send 113M
            * METADATA_IGNORE: 13.8w TPS, Net send 30M

            MySQL implementation send metadata (or not) according to [resultset_metadata|https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_resultset_metadata] variable.
            Associate capability is CLIENT_OPTIONAL_RESULTSET_METADATA (1UL << 25)

            When resultset_metadata = 0 (NONE) no metadata is send for COM_STMT_PREPARE/COM_STMT_EXECUTE/COM_QUERY

            h3. Exchange format change

            h4. [Column count packet|https://mariadb.com/kb/en/resultset/#resultset]

            There is a "new metadata_follows flag"

            {noformat}
            * int<lenenc> column count
            * int<1> metadata_follows flag (0x00: No metadata, 0x01: metadata follows)
            {noformat}

            When metadata_follows = 0x00, protocol skip column informations packet (continuing by an EOF_packet if CLIENT_DEPRECATE_EOF capability is not set or rows if CLIENT_DEPRECATE_EOF is set).
            be careful not to rely on [mysql documentation|https://dev.mysql.com/doc/dev/mysql-server/8.0.11/page_protocol_com_query_response_text_resultset.html] that indicate them in a wrong order

            h4. [COM_PREPARE response|https://mariadb.com/kb/en/com_stmt_prepare/#com_stmt_prepare-response]

            [COM_STMT_PREPARE_OK|https://mariadb.com/kb/en/com_stmt_prepare/#COM_STMT_PREPARE_OK]:

            {noformat}
            * int<1> 0x00 COM_STMT_PREPARE_OK header
            * int<4> statement id
            * int<2> number of columns in the returned result set (or 0 if statement does not return result set)
            * int<2> number of prepared statement parameters ('?' placeholders)
            * string<1> -not used-
            * int<2> number of warnings
            * if capabilities & CLIENT_OPTIONAL_RESULTSET_METADATA
            ** int<1> metadata_follows flag (0x00: No metadata, 0x01: metadata follows)
            }
            {noformat}

            if metadata is set to 0x00, column definition packet that decribe each parameter and each column are skipped.

            h2. Requirement:

            Connectors need metadata to parse resultset.
            Proposal is to offer this functionality, but in a more modular way.

            What do connectors need:

            For first execute of a prepared statement:
             - [COM_STMT_PREPARE|https://mariadb.com/kb/en/com_stmt_prepare/] asked WITHOUT metadata.
             - [COM_STMT_EXECUTE|https://mariadb.com/kb/en/com_stmt_execute/] asked WITH METADATA. metadata is cached

            For next execute:
             - [COM_STMT_EXECUTE|https://mariadb.com/kb/en/com_stmt_execute/] WITHOUT metadata. use cached metadata

            There are two issues with not sending metadata each time:
            * Metadata can change over time. Server already automatically reprepares internally, so it knows to resend the metadata even if the client asked not to send the metadata every time.
            * commands like "SELECT ?" will have metadata type changing according to parameter. Metadata must then be send each time.

            [COM_STMT_PREPARE|https://mariadb.com/kb/en/com_stmt_prepare/] still need to have the possibility to ask for metadata (at least java need that possibility to have metadata ([example|https://docs.oracle.com/javase/7/docs/api/java/sql/PreparedStatement.html#getParameterMetaData()]) without any COM_STMT_EXECUTE execution)

            For production environment where DDL never change an option can be set on client level to use that same functionality for COM_QUERY.
            Client can then cache metadata and ask for COM_QUERY without metadata.

            MySQL doesn't permits modularity, so client request have to add this information.
            That can be either by:
            * create new commands COM_STMT_PREPARE_WITHOUT_META, COM_STMT_EXECUTE_WITHOUT_META and COM_QUERY_WITHOUT_META
            * changing existing COM_STMT_PREPARE, COM_STMT_EXECUTE and COM_QUERY format.

            serg Sergei Golubchik added a comment - - edited

            An approach that won't need need new COM commands and no application side changes could be:

            • send metadata on COM_STMT_PREPARE
            • send metadata on COM_STMT_EXECUTE iff it has changes since the last time metadata was sent for this prepared statement.

            Practically it'll mean that metadata will only be sent on COM_STMT_PREPARE.

            Connector will need to cache the metadata and update the cache every time metadata is received.

            Note: multiple result sets. A connector should be able to cache multiple metadata sets per one prepared statement.


            This approach will work automatically for any application as soon as the connector and the server are upgraded. One can reduce the transferred data even more with application-aware changes. Meaning, application tells the server "don't send any metadata, I know it already or don't care". I would suggest to move this application-aware optimization into a separate MDEV.

            serg Sergei Golubchik added a comment - - edited An approach that won't need need new COM commands and no application side changes could be: send metadata on COM_STMT_PREPARE send metadata on COM_STMT_EXECUTE iff it has changes since the last time metadata was sent for this prepared statement. Practically it'll mean that metadata will only be sent on COM_STMT_PREPARE. Connector will need to cache the metadata and update the cache every time metadata is received. Note: multiple result sets. A connector should be able to cache multiple metadata sets per one prepared statement. This approach will work automatically for any application as soon as the connector and the server are upgraded. One can reduce the transferred data even more with application-aware changes. Meaning, application tells the server "don't send any metadata, I know it already or don't care". I would suggest to move this application-aware optimization into a separate MDEV.
            ralf.gebhardt Ralf Gebhardt made changes -
            Component/s Binary Protocol [ 16120 ]
            wlad Vladislav Vaintroub added a comment - - edited

            I think with multiple result sets, it is rather peculiar.

            A prepared statement can produce multiple result sets. It happens (only) if it is a prepared CALL,
            which has SELECTS (SHOW, or CHECKSUM TABLE), either normal , or SQL-level PREPARED or EXECUTE IMMEDIATE. The user does not know how many result sets will come from such CALL. The number of result sets can change from one execution to another. The user does not get result set metadata of the internals SELECTs as response from COM_STMT_PREPARE, and I guess, these result sets metadata can't really be cached.

            However CALL can also have a single designated result set, which might be OK to cache on the client - output parameters set.

            With this, I think client should only be able to cache a single result set, and the server can have at most one "skip-metadata" result set per COM_STMT_EXECUTE.

            Note : semicolon-batch can't be prepared, and the protocol has no provision to report multiple results sets during COM_STMT_PREPARE.

            wlad Vladislav Vaintroub added a comment - - edited I think with multiple result sets, it is rather peculiar. A prepared statement can produce multiple result sets. It happens (only) if it is a prepared CALL, which has SELECTS (SHOW, or CHECKSUM TABLE), either normal , or SQL-level PREPARED or EXECUTE IMMEDIATE. The user does not know how many result sets will come from such CALL. The number of result sets can change from one execution to another. The user does not get result set metadata of the internals SELECTs as response from COM_STMT_PREPARE, and I guess, these result sets metadata can't really be cached. However CALL can also have a single designated result set, which might be OK to cache on the client - output parameters set. With this, I think client should only be able to cache a single result set, and the server can have at most one "skip-metadata" result set per COM_STMT_EXECUTE. Note : semicolon-batch can't be prepared, and the protocol has no provision to report multiple results sets during COM_STMT_PREPARE.
            wlad Vladislav Vaintroub made changes -
            wlad Vladislav Vaintroub made changes -
            Status Open [ 1 ] Confirmed [ 10101 ]
            wlad Vladislav Vaintroub made changes -
            Status Confirmed [ 10101 ] In Progress [ 3 ]
            wlad Vladislav Vaintroub made changes -
            Assignee Vladislav Vaintroub [ wlad ] Oleksandr Byelkin [ sanja ]
            Status In Progress [ 3 ] In Review [ 10002 ]
            wlad Vladislav Vaintroub added a comment - - edited

            sanja, could you please review ?
            You will notice constant MDEV_23913_FIXED which is currently set to 0, and this is about MDEV-23913, where metadata changes from prepare to execute, and that was not expected to change.

            Currently the detection of change is always based checksumming fields to be sent, but when MDEV-23913 is fixed, and MDEV_23913_FIXED is removed, it will be more efficient. Then change of the metadata will be almost always derived from whether reprepare ran, and fallbacks to checksumming will only be in degenerate cases, like SELECT ? or SELECT @user_var, or SELECT GREATEST(@a,@b) basically, when the type or the length/decimals etc of the result depends on user variables or a parameter

            wlad Vladislav Vaintroub added a comment - - edited sanja , could you please review ? You will notice constant MDEV_23913_FIXED which is currently set to 0, and this is about MDEV-23913 , where metadata changes from prepare to execute, and that was not expected to change. Currently the detection of change is always based checksumming fields to be sent, but when MDEV-23913 is fixed, and MDEV_23913_FIXED is removed, it will be more efficient. Then change of the metadata will be almost always derived from whether reprepare ran, and fallbacks to checksumming will only be in degenerate cases, like SELECT ? or SELECT @user_var, or SELECT GREATEST(@a,@b) basically, when the type or the length/decimals etc of the result depends on user variables or a parameter
            diego dupin Diego Dupin made changes -
            diego dupin Diego Dupin made changes -
            diego dupin Diego Dupin made changes -
            wlad Vladislav Vaintroub made changes -
            Assignee Oleksandr Byelkin [ sanja ] Vladislav Vaintroub [ wlad ]
            wlad Vladislav Vaintroub made changes -
            Status In Review [ 10002 ] Stalled [ 10000 ]
            wlad Vladislav Vaintroub made changes -
            Component/s Prepared Statements [ 10804 ]
            Component/s Binary Protocol [ 16120 ]

            Address Review comments, add vertical space and doxygenize function comments

            wlad Vladislav Vaintroub added a comment - Address Review comments, add vertical space and doxygenize function comments
            wlad Vladislav Vaintroub made changes -
            issue.field.resolutiondate 2020-11-23 18:45:09.0 2020-11-23 18:45:09.843
            wlad Vladislav Vaintroub made changes -
            Fix Version/s 10.6.0 [ 24431 ]
            Fix Version/s 10.6 [ 24028 ]
            Resolution Fixed [ 1 ]
            Status Stalled [ 10000 ] Closed [ 6 ]
            markus makela markus makela made changes -
            alice Alice Sherepa made changes -
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 95327 ] MariaDB v4 [ 133929 ]
            ycp Yuchen Pei made changes -

            People

              wlad Vladislav Vaintroub
              diego dupin Diego Dupin
              Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.