Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-27161

Add option for SQL thread to limit maximum execution time per query replicated

Details

    Description

      Currently execution time for the replicated query is not limited in any clear way. On primary node we can use max_statement_time, but this value is neither passed to slave nor enforced for SQL thread when it's set locally. So, there is no way to "control" replication delay and just stop with error when it's too big already.

      I think option to stop with error after specific timeout (that may be different than local max_statement_time or primary's max_statement_time) would be useful for replication setups.

      Attachments

        Activity

          valerii Valerii Kravchuk created issue -
          serg Sergei Golubchik made changes -
          Field Original Value New Value
          Workflow MariaDB v3 [ 128099 ] MariaDB v4 [ 131610 ]
          Elkin Andrei Elkin added a comment - - edited

          Slave appliers indeed ignore global.max_statement_time which they must be initialized with (as the user connection threads do). Whether that's designed on purpose (doubts) or is actually buggy remains to be seen.
          Should there be no such effect the requested functionally could be provided without any new option.
          To add more, init-slave is around to fine tune slave appliers so @@SESSION level setting
          like this one could be technically done though that.

          Elkin Andrei Elkin added a comment - - edited Slave appliers indeed ignore global.max_statement_time which they must be initialized with (as the user connection threads do). Whether that's designed on purpose (doubts) or is actually buggy remains to be seen. Should there be no such effect the requested functionally could be provided without any new option. To add more, init-slave is around to fine tune slave appliers so @@SESSION level setting like this one could be technically done though that.

          What do you mean "slave appliers indeed ignore global.max_statement_time which they must be initialized with (as the user connection threads do)" ? Every THD gets max_statement_time, slave thread too. It's copied from global in plugin_thdvar_init() which is called from THD::init(). What slave threads don't do — they don't set the query timer to expire after max_statement_time seconds. In THD::set_query_timer() slave threads are explicitly excluded. This looks very much like on purpose, one can hardly write slave_thread || unintentionally.

          serg Sergei Golubchik added a comment - What do you mean "slave appliers indeed ignore global.max_statement_time which they must be initialized with (as the user connection threads do)" ? Every THD gets max_statement_time , slave thread too. It's copied from global in plugin_thdvar_init() which is called from THD::init() . What slave threads don't do — they don't set the query timer to expire after max_statement_time seconds. In THD::set_query_timer() slave threads are explicitly excluded. This looks very much like on purpose, one can hardly write slave_thread || unintentionally.
          serg Sergei Golubchik made changes -
          Assignee Sergei Golubchik [ serg ]
          serg Sergei Golubchik made changes -
          Assignee Sergei Golubchik [ serg ]

          I don't think it's a good idea. If the slave thread would always be using global.max_statement_time — that's one thing. But if you're adding an option anyway, better to have an option like --slave-max-statement-time and not --slave-uses-global-max-statement-time

          serg Sergei Golubchik added a comment - I don't think it's a good idea. If the slave thread would always be using global.max_statement_time — that's one thing. But if you're adding an option anyway, better to have an option like --slave-max-statement-time and not --slave-uses-global-max-statement-time
          ralf.gebhardt Ralf Gebhardt made changes -
          Fix Version/s 10.5 [ 23123 ]
          Fix Version/s 10.6 [ 24028 ]
          Fix Version/s 10.7 [ 24805 ]
          Fix Version/s 10.8 [ 26121 ]
          Fix Version/s 10.9 [ 26905 ]
          Fix Version/s 10.10 [ 27530 ]
          ralf.gebhardt Ralf Gebhardt made changes -
          Assignee Andrei Elkin [ elkin ]
          ralf.gebhardt Ralf Gebhardt made changes -
          Priority Major [ 3 ] Critical [ 2 ]
          serg Sergei Golubchik made changes -
          Fix Version/s 10.10 [ 27530 ]
          Elkin Andrei Elkin made changes -
          Assignee Andrei Elkin [ elkin ] Brandon Nesterenko [ JIRAUSER48702 ]
          bnestere Brandon Nesterenko made changes -
          Status Open [ 1 ] In Progress [ 3 ]

          Hi Andrei!

          This is ready for review.

          https://github.com/MariaDB/server/pull/2182

          bnestere Brandon Nesterenko added a comment - Hi Andrei! This is ready for review. https://github.com/MariaDB/server/pull/2182
          bnestere Brandon Nesterenko made changes -
          Assignee Brandon Nesterenko [ JIRAUSER48702 ] Andrei Elkin [ elkin ]
          Status In Progress [ 3 ] In Review [ 10002 ]
          ralf.gebhardt Ralf Gebhardt made changes -
          Fix Version/s 10.10 [ 27530 ]
          Fix Version/s 10.5 [ 23123 ]
          Fix Version/s 10.6 [ 24028 ]
          Fix Version/s 10.7 [ 24805 ]
          Fix Version/s 10.8 [ 26121 ]
          Fix Version/s 10.9 [ 26905 ]
          ralf.gebhardt Ralf Gebhardt made changes -
          Elkin Andrei Elkin added a comment -

          Completed the work. The final review was provided by Sergei (kudos for pretty constructive notes and heads-ups!).

          Elkin Andrei Elkin added a comment - Completed the work. The final review was provided by Sergei (kudos for pretty constructive notes and heads-ups!).
          Elkin Andrei Elkin made changes -
          Assignee Andrei Elkin [ elkin ] Brandon Nesterenko [ JIRAUSER48702 ]
          Status In Review [ 10002 ] Stalled [ 10000 ]
          Elkin Andrei Elkin made changes -
          Fix Version/s 10.10.0 [ 27912 ]
          Fix Version/s 10.10 [ 27530 ]
          Resolution Fixed [ 1 ]
          Status Stalled [ 10000 ] Closed [ 6 ]
          AirFocus AirFocus made changes -
          Description Currently execution time for the replicated query is not limited in any clear way. On primary node we can use max_statement_time, but this value is neither passed to slave nor enforced for SQL thread when it's set locally. So, there is no way to "control" replication delay and just stop with error when it's too big already.

          I think option to stop with error after specific timeout (that may be different than local max_statement_time or primary's max_statement_time) would be useful for replication setups.
          Currently execution time for the replicated query is not limited in any clear way. On primary node we can use max_statement_time, but this value is neither passed to slave nor enforced for SQL thread when it's set locally. So, there is no way to "control" replication delay and just stop with error when it's too big already.

          I think option to stop with error after specific timeout (that may be different than local max_statement_time or primary's max_statement_time) would be useful for replication setups.

          The adopted solution was to add a new variable, Slave_max_statement_time, which can be
          used to limit the time that slave is going to wait for a query to be executed.
          The slave will no honor max_statement_time.

          monty Michael Widenius added a comment - The adopted solution was to add a new variable, Slave_max_statement_time, which can be used to limit the time that slave is going to wait for a query to be executed. The slave will no honor max_statement_time.
          mariadb-jira-automation Jira Automation (IT) made changes -
          Zendesk Related Tickets 202115

          People

            bnestere Brandon Nesterenko
            valerii Valerii Kravchuk
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.