Details

    Description

      When using a proxy, such as MaxScale to do read/write split, It is not easy to keep session state consistent between different backends, especially for session user variables; When execute stored procedure, or do some select with user variable modification,It is hard for proxy to notice that a variable was changed, and get the lasted value;
      For example,

      select a, b from (select (@id:=@id+1) as id, a, b from t group by c order by d) tt where id < 10;
      select @a:=b, c from t
      ...
      

      Currently, Maxscale route these queries to every backends or route all to master, it is not efficient and sometimes not correct;

      So we suggest that server provide a session user variables tracker to inform proxy the what variable changed, and the lasted value, then propagate the results to other backends.

      Attachments

        Issue Links

          Activity

            The patch appered to be non working (no surprise because it has no tests).

            sanja Oleksandr Byelkin added a comment - The patch appered to be non working (no surprise because it has no tests).

            Current implementation forgot about:

            • NULL
            • variables types
            • charset for string and precision for numbers
            sanja Oleksandr Byelkin added a comment - Current implementation forgot about: NULL variables types charset for string and precision for numbers
            bar Alexander Barkov added a comment - - edited

            I'd propose to return variable tracking as:

            • the variable name
            • the variable value expression

            So the proxy will reconstruct a SET statement using this simple function:

            spnrintf(buff, "SET @%s=%s", name, expr);
            

            The server will put all necessary metadata into the expression itself.
            For example, in case of a string data, it would send the expression in this format:

            _latin1'str' COLLATE latin1_german2_ci
            

            Note, we will eventually support user defined types in user variables. An arbitrary data type has an arbitrary amount of data type attributes.
            So sending the basic metadata (e.g, length, precision, scale), like we do it in the result set in the client-server protocol, won't always be enough: this metadata is incomplete.

            bar Alexander Barkov added a comment - - edited I'd propose to return variable tracking as: the variable name the variable value expression So the proxy will reconstruct a SET statement using this simple function: spnrintf(buff, "SET @%s=%s" , name , expr); The server will put all necessary metadata into the expression itself. For example, in case of a string data, it would send the expression in this format: _latin1 'str' COLLATE latin1_german2_ci Note, we will eventually support user defined types in user variables. An arbitrary data type has an arbitrary amount of data type attributes. So sending the basic metadata (e.g, length, precision, scale), like we do it in the result set in the client-server protocol, won't always be enough: this metadata is incomplete.
            bar Alexander Barkov added a comment - - edited

            i suppose the proxy does not need to parse the expression. It only needs to get the variable name and its value from one backend and transparently transfer this exact value to another backend. Using the expression format looks the best for me.

            We already use the same approach in binary log by replacing user variables and SP variables with NAME_CONST() expressions:

            INSERT INTO t2 VALUES ( NAME_CONST('rec1.a',10),  NAME_CONST('rec1.b',_latin1'b10' COLLATE 'latin1_swedish_ci'))
            

            Notice, the expression contains all metadata:

            • in case of a number it's just 10 - an integer literal. It does not need any additional metatada.
            • In case of a string it's more complex: the expression, among the value itself, contains the character set and the collation

            This approach proved to be functional. Why not reuse it for variable tracking?

            Some more binlog examples for other data types:

            INSERT INTO t1 VALUES ( NAME_CONST('b8',_binary'a' COLLATE 'binary')=0x61)
            INSERT INTO t1 VALUES ( NAME_CONST('t0',TIME'01:01:01')=10101)
            INSERT INTO t1 VALUES ( NAME_CONST('dt0',TIMESTAMP'2001-01-01 01:01:01')=20010101010101)
            

            Notice, every data type encodes its values in its own way. In the way that the server will re-parse it back without any data loss.

            There is a good advantage in this approach:
            we can reuse a lot of code in the server for variable tracking.

            So a new data type will have to implement the functionality (to print a variable value as an expression) only one time, and it will be reused both for binary logging and variable tracking.

            bar Alexander Barkov added a comment - - edited i suppose the proxy does not need to parse the expression. It only needs to get the variable name and its value from one backend and transparently transfer this exact value to another backend. Using the expression format looks the best for me. We already use the same approach in binary log by replacing user variables and SP variables with NAME_CONST() expressions: INSERT INTO t2 VALUES ( NAME_CONST( 'rec1.a' ,10), NAME_CONST( 'rec1.b' ,_latin1 'b10' COLLATE 'latin1_swedish_ci' )) Notice, the expression contains all metadata: in case of a number it's just 10 - an integer literal. It does not need any additional metatada. In case of a string it's more complex: the expression, among the value itself, contains the character set and the collation This approach proved to be functional. Why not reuse it for variable tracking? Some more binlog examples for other data types: INSERT INTO t1 VALUES ( NAME_CONST( 'b8' ,_binary 'a' COLLATE 'binary' )=0x61) INSERT INTO t1 VALUES ( NAME_CONST( 't0' , TIME '01:01:01' )=10101) INSERT INTO t1 VALUES ( NAME_CONST( 'dt0' , TIMESTAMP '2001-01-01 01:01:01' )=20010101010101) Notice, every data type encodes its values in its own way. In the way that the server will re-parse it back without any data loss. There is a good advantage in this approach: we can reuse a lot of code in the server for variable tracking. So a new data type will have to implement the functionality (to print a variable value as an expression) only one time, and it will be reused both for binary logging and variable tracking.
            markus makela markus makela added a comment -

            I think Bar's proposal would be pretty good as it'd still allow the admin tool to see the value with a simple SELECT <insert constant here> AS <variable name>. This removes all client-side requirements for the feature to be used. it'd involve an extra round-trip but it's probably not super critical when it's used by interactive applications and I think doing things in SQL is more portable than implementing it in the connectors.

            markus makela markus makela added a comment - I think Bar's proposal would be pretty good as it'd still allow the admin tool to see the value with a simple SELECT <insert constant here> AS <variable name> . This removes all client-side requirements for the feature to be used. it'd involve an extra round-trip but it's probably not super critical when it's used by interactive applications and I think doing things in SQL is more portable than implementing it in the connectors.

            People

              sanja Oleksandr Byelkin
              dapeng dapeng huang
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.