[MDEV-4427] query timeouts Created: 2013-04-23  Updated: 2014-11-02  Resolved: 2014-10-07

Status: Closed
Project: MariaDB Server
Component/s: OTHER
Fix Version/s: 10.1.1

Type: Task Priority: Critical
Reporter: roberto spadim Assignee: Michael Widenius
Resolution: Fixed Votes: 2
Labels: None

Issue Links:
Blocks
is blocked by MDEV-5231 Per query variables from Percona Serv... Closed
Relates
relates to MDEV-4623 SECURITY - LOCK CONNECTION <id>, UNLO... Open
relates to MDEV-6847 query timeout, executing kill only if... Open
relates to MDEV-6963 Testing for max_query_time Closed
relates to MDEV-7011 MAX_STATEMENT_TIME has no effect in a... Closed

 Description   

Hi guys, maybe this could be done easly (maybe in GSoC)
Some queries when executed in tables with high intensive SELECT/INSERT/UPDATE/DELETE, should be executed in a max time, or it start to lock the server

twitter/mysql have a similar feature, after some pre defined time, the query is canceled by server (KILL the query in a background thread)

example:
1)SELECT /* MAX_QUERY_TIME = 10 */ * FROM SOME_BIG_TABLE
2)UPDATE SOME_BIG_TABLE SET VALUE='SOME_VALUE' WHERE PRIMARY_KEY='SOME VALUE'

the UPDATE will be blocked by SELECT while it's executing... the point is, kill (1) after 10 seconds if there's a blocked query trying to use this table, in this case the query (2)

this help a lot myisam tables (with table lock only method, no page and no row lock levels) with poor performace of select+update+delete+insert

here some information about twitter mysql implementation:
https://github.com/twitter/mysql/wiki/Statement-Timeout

i prefer a float value based in seconds (instead milliseconds), i didn't found information to set it off, maybe set max_statement_time=0


after MDEV-4623 done, we could instead of KILL query, 'PAUSE' it

Reading mariadb docs, i found this, but it's not implemented ?!

https://kb.askmonty.org/en/how-to-limittimeout-queries/

MAX_QUERY_TIME variable

You can limit the time any query is executed by setting the MAX_QUERY_TIME variable. The variable takes the number of seconds (with microsecond precision) the query should be executed. If the query is run more than the given timeout, it will be killed as by a KILL SOFT command.
SET @@SESSION.MAX_QUERY_TIME=0.5; # Abort if query takes more than 1/2 second
SELECT SLEEP(100);
max_query_time can also be given as an argument to mysqld at startup. This sets the GLOBAL variable that is used as default for all new connections.
You can check how many times timeouts has expired by examining the status variable max_query_timeouts:
show global status like 'max_query_timeouts';
This feature is scheduled for MariaDB 5.5.29



 Comments   
Comment by roberto spadim [ 2013-04-23 ]

it's complementary to MDEV-250

Comment by roberto spadim [ 2013-06-05 ]

twiteer commits:
https://github.com/twitter/mysql/commit/d6fbb29baca84e91be9c9b7822ae19a9e786319f
https://github.com/twitter/mysql/commit/2bb4adb1e920e87c35700bbd400ab6083c155f63#sql/sql_parse.cc
https://github.com/twitter/mysql/commit/8a29e0b65cf008f82338efc3cbafc628997c718b#sql/sql_parse.cc
https://github.com/twitter/mysql/commit/d6fbb29baca84e91be9c9b7822ae19a9e786319f#sql/sql_parse.cc

should be nice change it a bit...
kill is only usefull if table is being locked for a long time, and not only being executed by a long time....
example... only one user running a select that need 1 minute, and kill time set to 30seconds, it will never execute right? but if table is only being used to select, why not execute it? example:

where we need kill:
1) select very slow.... kill_time=30s
2) update table with very slow select.... this will lock for only 30seconds
3) lock table xxx, this will wait 30s and update to lock

where we don't need kill
1) select very slow.... kill_time=30s
2) anothers selects...

Comment by Sergei Golubchik [ 2014-04-17 ]

mysql-5.7.4 has it

Comment by Sergei Golubchik [ 2014-04-17 ]

possible implementation:

  • session level variables @@max_query_time.
  • can be set per statement too (with SET STATEMENT
  • now
    • on query start we remember when it must end
    • in thd_kill_level() we check that current time is not beyond the end
  • or
    • separate alarm thread
    • sets thd->killed appropriately
    • this approach significantly reduces the number of get_time() calls
  • or
    • check what 5.7 does
Comment by Michael Widenius [ 2014-04-22 ]

We already have one implementation of this task. We are just waiting for the sponsor to pay for it so that we can push it.

Comment by roberto spadim [ 2014-05-26 ]

hi guys! any news?

Comment by roberto spadim [ 2014-05-26 ]


hi monty! could you tell some details about this implementation?
we will have a separate alarm thread, get_time() calls, or do what 5.7 does (what 5.7 do?) ?


to everybody:
any idea if the second part of this MDEV could be done? only kill the query that is blocking another query
for example if i have a
select * from a big_table, executing for a long time (> max statment time)
but mariadb don't have another query waiting for a table lock, update, delete or anything else, just leave the query running and don't kill?

thanks guys

Comment by Michael Widenius [ 2014-10-07 ]

This is now implemented in 10.1 and I plan to push it into 10.1

Implementation details:

  • Not same code as in MySQL 5.7
  • Separate thread who handle signaling of timeouts.
  • One code for all platforms: Based on pthread_cond_timedwait()
  • Works by sending the internal KILL_QUERY signal to the thread. Exactly same as doing 'KILL QUERY ...'.
  • One can kill anything except stored procedures (Different than in 5.7).
  • Uses set_timespec_nsec() to specify time (uses my_hrtime() internally).
  • Code in mysys/thr_timer.cc

Second part could be relatively easily be done by adding some test in 'THD::awake()' (at least for table_locks).
However for this case, there is a need to add some new syntax to mark queries that should only be killed if someone else needs the resource.

Comment by Michael Widenius [ 2014-10-07 ]

Code pushed to 10.1 tree.
Will now update documentation.

Comment by roberto spadim [ 2014-10-07 ]

nice news i'm cloning and testing right now

Comment by roberto spadim [ 2014-10-07 ]

should i open another MDEV just to explain the "PAUSE" and track?

Comment by roberto spadim [ 2014-10-07 ]

i'm reading patch code , there's a timeout_hard, is it being used or any plan about it? or it's something about killing mysqld and the thread have a kill_timeout condition and change it to timeout_hard? didn't found where it is used

+ KILL_TIMEOUT= 8,
+ KILL_TIMEOUT_HARD= 9,

Comment by roberto spadim [ 2014-10-07 ]

other doubt, i didn't understood reading the source code

how timeout thread know if we are timedout or not
for example...
1) we have a global thread that check each connection and send signals about timeout or not
2) each connection with max timeout variable create a second thread to check timeout (not a global thread, but a per query thread)

Comment by Michael Widenius [ 2014-10-08 ]

Yes, please create another mdev for new features.

The HARD option is an addition to MariaDB. You can use HARD as an option for the KILL command. It means that the query should be killed as soon as possible, even if it would leave the table in an inconsistent state.
See https://mariadb.com/kb/en/data-manipulation-kill-connection-query/ for details.

It's 1)
There is one thread created to handle timeout for all other threads. This works by doing a pthread_cond_timedwait() for the next timeout.
See mysys/thr_timer.c for details of how this is implemented.

Comment by roberto spadim [ 2014-10-09 ]

nice =] created

about the hard, it's not being used yet by the timeout right? i'm considering that the current design could be the 'hard timeout' and we could create a soft timeout, the soft when the process receive the kill_timeout it will check if the current thread is or isn't blocking others process, i don't know the best method to do this but i think at the thread running is easier than a external process, since we alread have a lock about the current process and we just need to check mdl (at a first version) + storage level (partition/page/row) locks (in a second version), just to think about an roadmap about how to implmeemnt the "soft" timeout

Generated at Thu Feb 08 06:56:20 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.