Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-4623

SECURITY - LOCK CONNECTION <id>, UNLOCK CONNECTION SQL command

    XMLWordPrintable

Details

    • Task
    • Status: Open (View Workflow)
    • Minor
    • Resolution: Unresolved
    • None
    • None
    • None

    Description

      Sometimes a program start a query batch with many many queries and we want to "pause" the program, we have (5) problems to solve in this MDEV
      but the program don't have a "pause" button, and we as dba must lock it to allow others users to use database and after some time unlock it and continue program batch, well that's the first (1) problem of this MDEV
      The same happen for shared hosts where many applications execute queries with high i/o or high cpu use, and we need to throttle / pause this queries, here is just to pause, throttle is another feature but it will not be included here

      well the MDEV here is a LOCK / UNLOCK CONNECTION, SQL command.
      Ideas:

      LOCK CONNECTION <connection id> [TIMEOUT <timeout>] [WITHOUT/WITH DEADLOCK] [WITHOUT/WITH QUERY CACHE] [PAUSE QUERY]
            must have: <connection id>
            optional: [TIMEOUT timeout] [WITHOUT/WITH DEADLOCK] [WITHOUT/WITH QUERY CACHE] [PAUSE QUERY]
            here default: 
                  timeout=0 (forever)  
                  WITH DEADLOCK (if connection that owned the LOCK CONNECTION disconect, all connections locked won't be unlocked)
                  WITH QUERY CACHE (if query is in query cache return, just block queries that aren't in query cache, deletes, handler, update, alter, etc...)
                  without PAUSE QUERY - in the end of this MDEV i explain what it do, it's near KILL but don't drop connection
      UNLOCK CONNECTION <connection id>
            must have: <connection id>

      the second (2) problem is deadlock, if we LOCK CONNECTION and we disconnect from server (internet connections are very bad with cellphones and <3g bands) the connection will unlock, well that's why i put the "WITHOUT DEAD LOCK" in LOCK CONNECTION

      now the third (3) and last one problem, is know if a connection is or isn't locked, for this we should show this information in PROCESS LIST
      a new column with "CONNECT_LOCKED", ENUM ('Y','N') can solve the problem (maybe the connection id that locked this query)

      well that's all guys
      if anyone could help me how to add "LOCK CONNECTION" words in parser i could do/help with this patch without big problems

      resuming.. with this we will have 4 commands to admin connections:

      1)SHOW PROCESSLIST; (problem 3)
      2)KILL [USER | QUERY] xxxx;  (done)
      3)LOCK CONNECTION xxx [TIMEOUT yyy]; (problem 1)
      3)LOCK CONNECTION xxx [TIMEOUT yyy] WITHOUT DEADLOCK; (problem 2)
      4)UNLOCK CONNECTION (problems (1) and (2) )

      i think it will help many DBAs (including me =) )

      ---- implementation

      "LOCK CONNECTION <id>"
      1)search the process (id),
      2)lock it

          i think it's a THD variable, and a mutex lock it to prevent THD being changed or deleted from memory

      3) write that the connection is locked

            a new variable "lock_connection_from" in THD
            0 = unlocked
            -1 = locked with a connection that don't exists (deadlock)
            >0 = locked with a connection id that exists

      4) write the flags

            lock_connection_timeout in seconds
            lock_connection_lock_time the time we set the lock (for timeout)
            lock_connection_with_query_cache  (boolean, 0=without query cache, 1 = with query cache [default])
            lock_connection_with_deadlock (boolean, 0=without deadlock [default], 1 = with deadlock)

      5) unlock THD
      6) return
      -------------
      UNLOCK:

      1)lock THD
      2) set lock_connection_from=0
      3)unlock THD

      -------------
      at sql_parse (if i'm not wrong, that's where query_cache_send_result() is called) :

      0) wait packages <-- if connection was locked here don't lock until (2)
      1) receive sql <-- if connection was locked here don't lock until (2)
      2) wait until connection is locked <--- here check if it's locked or not

      wait() = sleep(0,001 second)
      "check log" procedure:
       
            if lock_connection_from=0, { unlocked go to (3) }
            if (now() - lock_connection_lock_time >lock_connection_timeout AND lock_connection_timeout>0), { set lock_connection_from=0, timedout and unlocked, go to (3) }
            if (query in query cache and lock_connection_with_query_cache =1), {"in query cache", go to (3) }
            if lock_connection_from=-1, { wait() and go back to "check lock" again }
            if lock_connection_from>0 {
                check if connection exists in processlist (i don't know how to do this in source today, but i know it can be done)
                if exist{ wait() and go back to "check lock" again }
                if not, {
                          check flag lock_connection_with_deadlock
                          if lock_connection_with_deadlock=0,{ set lock_connection_from=0, go to (3)}
                          if lock_connection_with_deadlock=1,{ set lock_connection_from=-1, wait() and go back to "check lock" again }
                }
            }
            go back to "check log"
       
      (3)
       |
       v

      3) parse <-- if the connection was locked here, continue with parse, optimize, execute, return
      4) optimize <-- if the connection was locked here, continue with optimize, execute, return
      5) execute <-- if the connection was locked here, continue with execute, return
      6) return <-- if the connection was locked here, continue with return and go back to (0)

      it will work like a query waiting a table with "LOCK TABLE xxx" but no table will be locked
      query will wait connection lock be removed before continue to parse the query (or check query cache)

      ------------
      at PROCESS_LIST
      add "CONNECTION LOCKED" with "lock id + lock timeout + lock type (dead lock and query cache)", example:

      unlocked:

      "UNLOCKED" or NULL or "" or "123456 TIMEDOUT"  (when processlist can check that it's timedout) or 
      "DEADLOCKED TIMEDOUT"  (when processlist can check that it's timedout)

      locked:

      "123456 TIMEOUT 1234 WITH DEADLOCK WITH QUERY CACHE, TIMEOUT IN xxx SECONDS"
      "123456 WITH DEADLOCK WITH QUERY CACHE" (without timeout)

      locked but deadlocked (the locker connection was killed and lock persist)

      "DEADLOCKED TIMEOUT 1234 WITH QUERY CACHE"
      "DEADLOCKED WITH QUERY CACHE"
      "DEADLOCKED"

      i think a string is easier to understand, since it's only for DBAs
      we don't need how many time it was locked, since we can get it at TIME_MS column

      ----------------------
      PROBLEM (4) AND (5)
      PAUSE QUERY option

      maybe we could do 'online LOCK', in this case i will call it like a 'PAUSE' query, since we will not drop connection, just stop the execution and we unlock it we will restart or continue the job

      instead of wait query do the job, and after lock connection we will stop query and lock connection

      in this case:
      when lock (LOCK CONNECTION <id> PAUSE QUERY) (problem (4)):
      remove locks, put thread in a 'pause' status and save all 'done' job (selected rows, query plan, etc)
      when unlock (UNLOCK CONNECTION <id>) (problem (5)):
      put thread in a 'resuming' status, and
      if tables changed => clean 'done' job and restart select
      if tables don't changed => restore 'done' job and continue select where it was

      for UPDATE/DELETE/ALTER i think we can't do this since some engines are not transactional or some commands are not 'ONLINE', but for transactional engines maybe we could do something like:

      BEGIN TRANSACTION
      selects
      updates
      inserts
      ....

      LOCKED CONNECTION HERE
      save all SQL commands in this transaction (selects, updates, inserts)
      execute a "ROLLBACK"

      ....
      UNLOCKED HERE
      retrieve all SQL commands in the last transaction, execute all commands again and continue transaction

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              rspadim roberto spadim
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.