Details

    • Task
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • 11.1(EOL)
    • None
    • None

    Description

      Hi guys, I have some queries like this:

      SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%" 
      UNION 
      SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%" 
      UNION 
      SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%" 
      UNION 
      SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%" 
      UNION 
      SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%" 
      UNION 
      SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
      LIMIT 1

      It's a search about a document number inside all documents (ok i could use others index but that's not the case), the problem is....
      The first SELECT query (with % at end of the query) return really fast (since it can use left side of % to search inside index), but executing others queries are very slow, the question is

      could we optimize this kind of query, since we don't have ORDER BY and we have a LIMIT?

      for example, (i don't know how is the code today and if this could break a shard strategy)
      1) execute the first query,
      2) if this return more than LIMIT + OFFSET rows, stop all unions,
      3) if not go to next query
      3.1) check that SELECT execution order (1,2,3,4,5,...) is necessary since we need deterministic results in UNION order or we will need a ORDER BY and loose this logic
      3.2) maybe if we want something faster we could do an: estimate cost of each query and execute queries using estimate cost order (i think this could be an optimizer option, instead of an SQL_ORDER_BLABLA, this avoid sql standard problems and we could use this in some old systems, if it fail we get back with the standard query order), or extend the first UNION clausule with UNION NO_ORDER, or something like it, i'm considering here two problems... in this example the first query is the faster... and return the right data, if i change the first with the last, maybe i could get the wrong result, in this case the (3.2) optimization couldn't be used, but think about a server that the optimizer stay on and we forget to set it off (like query cache = on and we don't use SQL_NO_CACHE) an NO_ORDER or something like it could be nice to ensure that union will be executed in the right order even with optimizer option turned on just an idea... maybe we could use other better idea, for example of SQL_CACHE, only the first SELECT allow the SQL_CACHE others SELECT don't allow it, something like it is nice
      (select sql_cache sql_no_union_order * form blabal union select * from blabla2 ....)
      with this we could mix in a single complex query both union order and non union order
      select (sql_union order) , (sql_no_union_order) from (any other query)

      the explain of this query today is (count( * )=1.826.326 rows):

      id select_type table type possible_keys key key_len ref rows Extra execution time without flush cache (seconds)
      1 PRIMARY mov_documentos range PRIMARY PRIMARY 77   1 Using where; Using index 0.031 <- fast
      2 UNION mov_documentos range PRIMARY PRIMARY 77   628434 Using where; Using index 6.755 <- slow (maybe a outlier result)
      3 UNION mov_documentos range PRIMARY PRIMARY 77   628434 Using where; Using index 0.671 <- relative fast
      4 UNION mov_documentos index   PRIMARY 77   1826326 Using where; Using index 1.467 <- slow
      5 UNION mov_documentos index   PRIMARY 77   1826326 Using where; Using index 1.373 <- slow
      6 UNION mov_documentos index   PRIMARY 77   1826326 Using where; Using index 1.528 <- slow
        UNION RESULT <union1,2,3,4,5,6> or <union optimized 1,4,2,5,3,6> ALL           (Union limit optimized) Total query time: 5.258

      if we execute a diferent union since we can optimize based in query cost, we could show at last row (7): something like "<union 1,4,2,5,3,6>" and at final Extra column could put "<Union order optimizer>"

      in this case we have a optimization from 5.258 seconds to 0.031 seconds ~= 170x faster

      well that's all =]

      Attachments

        Activity

          rspadim roberto spadim created issue -
          rspadim roberto spadim made changes -
          Field Original Value New Value
          Description Hi guys, I have some queries like this:
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1


          It's a search about a document number, but the problem is....
          The first query at first line (with % at end of the query) return really fast, but executing others queries are very slow, the question is
          could we optimize this kind of query, since limit=1 row?

          for example, at each "UNION" select, check if the total return rows is inside LIMIT clausule, in this case we CAN'T have an order by at outer 'select' since we need a sort before LIMIT, example (select union select union select order by ... limit xxx) or ((select) union (select) order by ... limit xxx)

          the explain of this query today is:

          id select_type table type possible_keys key key_len ref rows Extra
          1 PRIMARY mov_documentos range PRIMARY PRIMARY 77 1 Using where; Using index
          2 UNION mov_documentos range PRIMARY PRIMARY 77 625612 Using where; Using index
          3 UNION mov_documentos range PRIMARY PRIMARY 77 625612 Using where; Using index
          4 UNION mov_documentos index PRIMARY 77 1798490 Using where; Using index
          5 UNION mov_documentos index PRIMARY 77 1798490 Using where; Using index
          6 UNION mov_documentos index PRIMARY 77 1798490 Using where; Using index
          UNION RESULT <union1,2,3,4,5,6> ALL

          and execution time of the first one is near or less than 1 second and others are more than 1 second (maybe less too since if the first query return the others probably will return the same row too but in a slower search algorithm since it have other WHERE clause)

          well that's all =]
          Hi guys, I have some queries like this:
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1


          It's a search about a document number, but the problem is....
          The first query at first line (with % at end of the query) return really fast, but executing others queries are very slow, the question is
          could we optimize this kind of query, since limit=1 row?

          for example, at each "UNION" select, check if the total return rows is inside LIMIT clausule, in this case we CAN'T have an order by at outer 'select' since we need a sort before LIMIT, example (select union select union select order by ... limit xxx) or ((select) union (select) order by ... limit xxx)

          the explain of this query today is:

          ||id|select_type |table|type|possible_keys|key|key_len|ref|rows|Extra||
          |1 |PRIMARY |mov_documentos |range |PRIMARY |PRIMARY |77 |1 |Using where; Using index|
          |2 |UNION |mov_documentos |range |PRIMARY |PRIMARY |77 |625612 |Using where; Using index|
          |3 |UNION |mov_documentos |range |PRIMARY |PRIMARY |77 |625612 |Using where; Using index|
          |4 |UNION |mov_documentos |index |PRIMARY |77 |1798490 |Using where; Using index|
          |5 |UNION |mov_documentos |index |PRIMARY |77 |1798490 |Using where; Using index|
          |6 |UNION |mov_documentos |index |PRIMARY |77 |1798490 |Using where; Using index|
          ||UNION |RESULT <union1,2,3,4,5,6> ALL |

          and execution time of the first one is near or less than 1 second and others are more than 1 second (maybe less too since if the first query return the others probably will return the same row too but in a slower search algorithm since it have other WHERE clause)

          well that's all =]
          rspadim roberto spadim made changes -
          Description Hi guys, I have some queries like this:
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1


          It's a search about a document number, but the problem is....
          The first query at first line (with % at end of the query) return really fast, but executing others queries are very slow, the question is
          could we optimize this kind of query, since limit=1 row?

          for example, at each "UNION" select, check if the total return rows is inside LIMIT clausule, in this case we CAN'T have an order by at outer 'select' since we need a sort before LIMIT, example (select union select union select order by ... limit xxx) or ((select) union (select) order by ... limit xxx)

          the explain of this query today is:

          ||id|select_type |table|type|possible_keys|key|key_len|ref|rows|Extra||
          |1 |PRIMARY |mov_documentos |range |PRIMARY |PRIMARY |77 |1 |Using where; Using index|
          |2 |UNION |mov_documentos |range |PRIMARY |PRIMARY |77 |625612 |Using where; Using index|
          |3 |UNION |mov_documentos |range |PRIMARY |PRIMARY |77 |625612 |Using where; Using index|
          |4 |UNION |mov_documentos |index |PRIMARY |77 |1798490 |Using where; Using index|
          |5 |UNION |mov_documentos |index |PRIMARY |77 |1798490 |Using where; Using index|
          |6 |UNION |mov_documentos |index |PRIMARY |77 |1798490 |Using where; Using index|
          ||UNION |RESULT <union1,2,3,4,5,6> ALL |

          and execution time of the first one is near or less than 1 second and others are more than 1 second (maybe less too since if the first query return the others probably will return the same row too but in a slower search algorithm since it have other WHERE clause)

          well that's all =]
          Hi guys, I have some queries like this:
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1


          It's a search about a document number, but the problem is....
          The first query at first line (with % at end of the query) return really fast, but executing others queries are very slow, the question is
          could we optimize this kind of query, since limit=1 row?

          for example, at each "UNION" select, check if the total return rows is inside LIMIT clausule, in this case we CAN'T have an order by at outer 'select' since we need a sort before LIMIT, example (select union select union select order by ... limit xxx) or ((select) union (select) order by ... limit xxx)

          the explain of this query today is:

          ||id||select_type ||table||type||possible_keys||key||key_len||ref||rows||Extra||
          |1 |PRIMARY |mov_documentos |range |PRIMARY |PRIMARY |77 |1 |Using where; Using index|
          |2 |UNION |mov_documentos |range |PRIMARY |PRIMARY |77 |625612 |Using where; Using index|
          |3 |UNION |mov_documentos |range |PRIMARY |PRIMARY |77 |625612 |Using where; Using index|
          |4 |UNION |mov_documentos |index |PRIMARY |77 |1798490 |Using where; Using index|
          |5 |UNION |mov_documentos |index |PRIMARY |77 |1798490 |Using where; Using index|
          |6 |UNION |mov_documentos |index |PRIMARY |77 |1798490 |Using where; Using index|
          | |UNION |RESULT <union1,2,3,4,5,6> ALL |

          and execution time of the first one is near or less than 1 second and others are more than 1 second (maybe less too since if the first query return the others probably will return the same row too but in a slower search algorithm since it have other WHERE clause)

          well that's all =]
          rspadim roberto spadim made changes -
          Description Hi guys, I have some queries like this:
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1


          It's a search about a document number, but the problem is....
          The first query at first line (with % at end of the query) return really fast, but executing others queries are very slow, the question is
          could we optimize this kind of query, since limit=1 row?

          for example, at each "UNION" select, check if the total return rows is inside LIMIT clausule, in this case we CAN'T have an order by at outer 'select' since we need a sort before LIMIT, example (select union select union select order by ... limit xxx) or ((select) union (select) order by ... limit xxx)

          the explain of this query today is:

          ||id||select_type ||table||type||possible_keys||key||key_len||ref||rows||Extra||
          |1 |PRIMARY |mov_documentos |range |PRIMARY |PRIMARY |77 |1 |Using where; Using index|
          |2 |UNION |mov_documentos |range |PRIMARY |PRIMARY |77 |625612 |Using where; Using index|
          |3 |UNION |mov_documentos |range |PRIMARY |PRIMARY |77 |625612 |Using where; Using index|
          |4 |UNION |mov_documentos |index |PRIMARY |77 |1798490 |Using where; Using index|
          |5 |UNION |mov_documentos |index |PRIMARY |77 |1798490 |Using where; Using index|
          |6 |UNION |mov_documentos |index |PRIMARY |77 |1798490 |Using where; Using index|
          | |UNION |RESULT <union1,2,3,4,5,6> ALL |

          and execution time of the first one is near or less than 1 second and others are more than 1 second (maybe less too since if the first query return the others probably will return the same row too but in a slower search algorithm since it have other WHERE clause)

          well that's all =]
          Hi guys, I have some queries like this:
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1


          It's a search about a document number, but the problem is....
          The first query at first line (with % at end of the query) return really fast, but executing others queries are very slow, the question is
          could we optimize this kind of query, since limit=1 row?

          for example, at each "UNION" select, check if the total return rows is inside LIMIT clausule, in this case we CAN'T have an order by at outer 'select' since we need a sort before LIMIT, example (select union select union select order by ... limit xxx) or ((select) union (select) order by ... limit xxx)

          the explain of this query today is:


          || *id* || *select_type* || *table* || *type* || *possible_keys* || *key* || *key_len* || *ref* || *rows* || *Extra* ||
          || 1 || PRIMARY || mov_documentos || range || PRIMARY || PRIMARY || 77 || || 1 || Using where; Using index ||
          || 2 || UNION || mov_documentos || range || PRIMARY || PRIMARY || 77 || || 628434 || Using where; Using index ||
          || 3 || UNION || mov_documentos || range || PRIMARY || PRIMARY || 77 || || 628434 || Using where; Using index ||
          || 4 || UNION || mov_documentos || index || || PRIMARY || 77 || || 1826326 || Using where; Using index ||
          || 5 || UNION || mov_documentos || index || || PRIMARY || 77 || || 1826326 || Using where; Using index ||
          || 6 || UNION || mov_documentos || index || || PRIMARY || 77 || || 1826326 || Using where; Using index ||
          || || UNION RESULT || <union1,2,3,4,5,6> || ALL || || || || || || ||


          and execution time of the first one is near or less than 1 second and others are more than 1 second (maybe less too since if the first query return the others probably will return the same row too but in a slower search algorithm since it have other WHERE clause)

          well that's all =]
          rspadim roberto spadim made changes -
          Description Hi guys, I have some queries like this:
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1


          It's a search about a document number, but the problem is....
          The first query at first line (with % at end of the query) return really fast, but executing others queries are very slow, the question is
          could we optimize this kind of query, since limit=1 row?

          for example, at each "UNION" select, check if the total return rows is inside LIMIT clausule, in this case we CAN'T have an order by at outer 'select' since we need a sort before LIMIT, example (select union select union select order by ... limit xxx) or ((select) union (select) order by ... limit xxx)

          the explain of this query today is:


          || *id* || *select_type* || *table* || *type* || *possible_keys* || *key* || *key_len* || *ref* || *rows* || *Extra* ||
          || 1 || PRIMARY || mov_documentos || range || PRIMARY || PRIMARY || 77 || || 1 || Using where; Using index ||
          || 2 || UNION || mov_documentos || range || PRIMARY || PRIMARY || 77 || || 628434 || Using where; Using index ||
          || 3 || UNION || mov_documentos || range || PRIMARY || PRIMARY || 77 || || 628434 || Using where; Using index ||
          || 4 || UNION || mov_documentos || index || || PRIMARY || 77 || || 1826326 || Using where; Using index ||
          || 5 || UNION || mov_documentos || index || || PRIMARY || 77 || || 1826326 || Using where; Using index ||
          || 6 || UNION || mov_documentos || index || || PRIMARY || 77 || || 1826326 || Using where; Using index ||
          || || UNION RESULT || <union1,2,3,4,5,6> || ALL || || || || || || ||


          and execution time of the first one is near or less than 1 second and others are more than 1 second (maybe less too since if the first query return the others probably will return the same row too but in a slower search algorithm since it have other WHERE clause)

          well that's all =]
          Hi guys, I have some queries like this:
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1


          It's a search about a document number, but the problem is....
          The first query at first line (with % at end of the query) return really fast, but executing others queries are very slow, the question is
          could we optimize this kind of query, since limit=1 row?

          for example, at each "UNION" select, check if the total return rows is inside LIMIT clausule, in this case we CAN'T have an order by at outer 'select' since we need a sort before LIMIT, example (select union select union select order by ... limit xxx) or ((select) union (select) order by ... limit xxx)

          the explain of this query today is:


          || *id* || *select_type* || *table* || *type* || *possible_keys* || *key* || *key_len* || *ref* || *rows* || *Extra* ||
          | 1 | PRIMARY | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 1 | Using where; Using index |
          | 2 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index |
          | 3 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index |
          | 4 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index |
          | 5 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index |
          | 6 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index |
          | | UNION RESULT | <union1,2,3,4,5,6> | ALL | | | | | | |


          and execution time of the first one is near or less than 1 second and others are more than 1 second (maybe less too since if the first query return the others probably will return the same row too but in a slower search algorithm since it have other WHERE clause)

          well that's all =]
          rspadim roberto spadim made changes -
          Description Hi guys, I have some queries like this:
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1


          It's a search about a document number, but the problem is....
          The first query at first line (with % at end of the query) return really fast, but executing others queries are very slow, the question is
          could we optimize this kind of query, since limit=1 row?

          for example, at each "UNION" select, check if the total return rows is inside LIMIT clausule, in this case we CAN'T have an order by at outer 'select' since we need a sort before LIMIT, example (select union select union select order by ... limit xxx) or ((select) union (select) order by ... limit xxx)

          the explain of this query today is:


          || *id* || *select_type* || *table* || *type* || *possible_keys* || *key* || *key_len* || *ref* || *rows* || *Extra* ||
          | 1 | PRIMARY | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 1 | Using where; Using index |
          | 2 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index |
          | 3 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index |
          | 4 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index |
          | 5 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index |
          | 6 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index |
          | | UNION RESULT | <union1,2,3,4,5,6> | ALL | | | | | | |


          and execution time of the first one is near or less than 1 second and others are more than 1 second (maybe less too since if the first query return the others probably will return the same row too but in a slower search algorithm since it have other WHERE clause)

          well that's all =]
          Hi guys, I have some queries like this:
          [code]
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1


          It's a search about a document number, but the problem is....
          The first query at first line (with % at end of the query) return really fast, but executing others queries are very slow, the question is
          could we optimize this kind of query, since limit=1 row?

          for example, at each "UNION" select, check if the total return rows is inside LIMIT clausule, in this case we CAN'T have an order by at outer 'select' since we need a sort before LIMIT, example (select union select union select order by ... limit xxx) or ((select) union (select) order by ... limit xxx)

          the explain of this query today is (count(*)=1.826.326):


          || *id* || *select_type* || *table* || *type* || *possible_keys* || *key* || *key_len* || *ref* || *rows* || *Extra* || execution time without flush cache (seconds) ||
          | 1 | PRIMARY | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 1 | Using where; Using index | 0.031 |
          | 2 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 6.755 |
          | 3 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 0.671 |
          | 4 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.467 |
          | 5 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.373 |
          | 6 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.528 |
          | | UNION RESULT | <union1,2,3,4,5,6> | ALL | | | | | | |


          and execution time of the first one is near or less than 1 second and others are more than 1 second (maybe less too since if the first query return the others probably will return the same row too but in a slower search algorithm since it have other WHERE clause)



          well that's all =]
          rspadim roberto spadim made changes -
          Description Hi guys, I have some queries like this:
          [code]
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1


          It's a search about a document number, but the problem is....
          The first query at first line (with % at end of the query) return really fast, but executing others queries are very slow, the question is
          could we optimize this kind of query, since limit=1 row?

          for example, at each "UNION" select, check if the total return rows is inside LIMIT clausule, in this case we CAN'T have an order by at outer 'select' since we need a sort before LIMIT, example (select union select union select order by ... limit xxx) or ((select) union (select) order by ... limit xxx)

          the explain of this query today is (count(*)=1.826.326):


          || *id* || *select_type* || *table* || *type* || *possible_keys* || *key* || *key_len* || *ref* || *rows* || *Extra* || execution time without flush cache (seconds) ||
          | 1 | PRIMARY | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 1 | Using where; Using index | 0.031 |
          | 2 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 6.755 |
          | 3 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 0.671 |
          | 4 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.467 |
          | 5 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.373 |
          | 6 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.528 |
          | | UNION RESULT | <union1,2,3,4,5,6> | ALL | | | | | | |


          and execution time of the first one is near or less than 1 second and others are more than 1 second (maybe less too since if the first query return the others probably will return the same row too but in a slower search algorithm since it have other WHERE clause)



          well that's all =]
          Hi guys, I have some queries like this:
          {code:sql}
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1
          {code}

          It's a search about a document number, but the problem is....
          The first query at first line (with % at end of the query) return really fast, but executing others queries are very slow, the question is
          could we optimize this kind of query, since limit=1 row?

          for example, at each "UNION" select, check if the total return rows is inside LIMIT clausule, in this case we CAN'T have an order by at outer 'select' since we need a sort before LIMIT, example (select union select union select order by ... limit xxx) or ((select) union (select) order by ... limit xxx)

          the explain of this query today is (count(*)=1.826.326):


          || *id* || *select_type* || *table* || *type* || *possible_keys* || *key* || *key_len* || *ref* || *rows* || *Extra* || execution time without flush cache (seconds) ||
          | 1 | PRIMARY | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 1 | Using where; Using index | 0.031 |
          | 2 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 6.755 |
          | 3 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 0.671 |
          | 4 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.467 |
          | 5 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.373 |
          | 6 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.528 |
          | | UNION RESULT | <union1,2,3,4,5,6> | ALL | | | | | | |


          and execution time of the first one is near or less than 1 second and others are more than 1 second (maybe less too since if the first query return the others probably will return the same row too but in a slower search algorithm since it have other WHERE clause)



          well that's all =]
          rspadim roberto spadim made changes -
          Description Hi guys, I have some queries like this:
          {code:sql}
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1
          {code}

          It's a search about a document number, but the problem is....
          The first query at first line (with % at end of the query) return really fast, but executing others queries are very slow, the question is
          could we optimize this kind of query, since limit=1 row?

          for example, at each "UNION" select, check if the total return rows is inside LIMIT clausule, in this case we CAN'T have an order by at outer 'select' since we need a sort before LIMIT, example (select union select union select order by ... limit xxx) or ((select) union (select) order by ... limit xxx)

          the explain of this query today is (count(*)=1.826.326):


          || *id* || *select_type* || *table* || *type* || *possible_keys* || *key* || *key_len* || *ref* || *rows* || *Extra* || execution time without flush cache (seconds) ||
          | 1 | PRIMARY | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 1 | Using where; Using index | 0.031 |
          | 2 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 6.755 |
          | 3 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 0.671 |
          | 4 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.467 |
          | 5 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.373 |
          | 6 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.528 |
          | | UNION RESULT | <union1,2,3,4,5,6> | ALL | | | | | | |


          and execution time of the first one is near or less than 1 second and others are more than 1 second (maybe less too since if the first query return the others probably will return the same row too but in a slower search algorithm since it have other WHERE clause)



          well that's all =]
          Hi guys, I have some queries like this:
          {code:sql}
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1
          {code}

          It's a search about a document number inside all documents (ok i could use others index but that's not the case), the problem is....
          The first SELECT query (with % at end of the query) return really fast (since it can use left side of % to search inside index), but executing others queries are very slow, the question is

          could we optimize this kind of query, since we don't have ORDER BY and we have a LIMIT?

          for example, (i don't know how is the code today and if this could break a shard strategy)
          1) execute the first query,
          2) if this return more than LIMIT + OFFSET rows, stop all unions,
          3) if not go to next query
          3.1) check that SELECT execution order (1,2,3,4,5,...) is necessary since we need deterministic results in UNION order or we will need a ORDER BY and loose this logic
          3.2) maybe if we want something faster we could do an: estimate cost of each query and execute queries using estimate cost order (i think this could be an optimizer option, instead of an SQL_ORDER_BLABLA, this avoid sql standard problems and we could use this in some old systems, if it fail we get back with the standard query order)
           

          the explain of this query today is (count(*)=1.826.326):

          || *id* || *select_type* || *table* || *type* || *possible_keys* || *key* || *key_len* || *ref* || *rows* || *Extra* || execution time without flush cache (seconds) ||
          | 1 | PRIMARY | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 1 | Using where; Using index | 0.031 <- fast |
          | 2 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 6.755 <- slow |
          | 3 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 0.671 <- relative fast |
          | 4 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.467 <- slow |
          | 5 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.373 <- slow |
          | 6 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.528 <- slow |
          | | UNION RESULT | <union1,2,3,4,5,6> | ALL | | | | | | |


          well that's all =]
          rspadim roberto spadim made changes -
          Description Hi guys, I have some queries like this:
          {code:sql}
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1
          {code}

          It's a search about a document number inside all documents (ok i could use others index but that's not the case), the problem is....
          The first SELECT query (with % at end of the query) return really fast (since it can use left side of % to search inside index), but executing others queries are very slow, the question is

          could we optimize this kind of query, since we don't have ORDER BY and we have a LIMIT?

          for example, (i don't know how is the code today and if this could break a shard strategy)
          1) execute the first query,
          2) if this return more than LIMIT + OFFSET rows, stop all unions,
          3) if not go to next query
          3.1) check that SELECT execution order (1,2,3,4,5,...) is necessary since we need deterministic results in UNION order or we will need a ORDER BY and loose this logic
          3.2) maybe if we want something faster we could do an: estimate cost of each query and execute queries using estimate cost order (i think this could be an optimizer option, instead of an SQL_ORDER_BLABLA, this avoid sql standard problems and we could use this in some old systems, if it fail we get back with the standard query order)
           

          the explain of this query today is (count(*)=1.826.326):

          || *id* || *select_type* || *table* || *type* || *possible_keys* || *key* || *key_len* || *ref* || *rows* || *Extra* || execution time without flush cache (seconds) ||
          | 1 | PRIMARY | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 1 | Using where; Using index | 0.031 <- fast |
          | 2 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 6.755 <- slow |
          | 3 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 0.671 <- relative fast |
          | 4 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.467 <- slow |
          | 5 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.373 <- slow |
          | 6 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.528 <- slow |
          | | UNION RESULT | <union1,2,3,4,5,6> | ALL | | | | | | |


          well that's all =]
          Hi guys, I have some queries like this:
          {code:sql}
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1
          {code}

          It's a search about a document number inside all documents (ok i could use others index but that's not the case), the problem is....
          The first SELECT query (with % at end of the query) return really fast (since it can use left side of % to search inside index), but executing others queries are very slow, the question is

          could we optimize this kind of query, since we don't have ORDER BY and we have a LIMIT?

          for example, (i don't know how is the code today and if this could break a shard strategy)
          1) execute the first query,
          2) if this return more than LIMIT + OFFSET rows, stop all unions,
          3) if not go to next query
          3.1) check that SELECT execution order (1,2,3,4,5,...) is necessary since we need deterministic results in UNION order or we will need a ORDER BY and loose this logic
          3.2) maybe if we want something faster we could do an: estimate cost of each query and execute queries using estimate cost order (i think this could be an optimizer option, instead of an SQL_ORDER_BLABLA, this avoid sql standard problems and we could use this in some old systems, if it fail we get back with the standard query order)
           

          the explain of this query today is (count(*)=1.826.326):

          || *id* || *select_type* || *table* || *type* || *possible_keys* || *key* || *key_len* || *ref* || *rows* || *Extra* || execution time without flush cache (seconds) ||
          | 1 | PRIMARY | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 1 | Using where; Using index | 0.031 <- fast |
          | 2 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 6.755 <- slow |
          | 3 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 0.671 <- relative fast |
          | 4 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.467 <- slow |
          | 5 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.373 <- slow |
          | 6 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.528 <- slow |
          | | UNION RESULT | <union1,2,3,4,5,6> (if we execute a diferent union since we can optimize based in query cost, we could show <union 1,4,2,5,3,6> and at final Extra column could put <Union order optimizer>) | ALL | | | | | | | (Union order optimizer) |


          well that's all =]
          rspadim roberto spadim made changes -
          Description Hi guys, I have some queries like this:
          {code:sql}
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1
          {code}

          It's a search about a document number inside all documents (ok i could use others index but that's not the case), the problem is....
          The first SELECT query (with % at end of the query) return really fast (since it can use left side of % to search inside index), but executing others queries are very slow, the question is

          could we optimize this kind of query, since we don't have ORDER BY and we have a LIMIT?

          for example, (i don't know how is the code today and if this could break a shard strategy)
          1) execute the first query,
          2) if this return more than LIMIT + OFFSET rows, stop all unions,
          3) if not go to next query
          3.1) check that SELECT execution order (1,2,3,4,5,...) is necessary since we need deterministic results in UNION order or we will need a ORDER BY and loose this logic
          3.2) maybe if we want something faster we could do an: estimate cost of each query and execute queries using estimate cost order (i think this could be an optimizer option, instead of an SQL_ORDER_BLABLA, this avoid sql standard problems and we could use this in some old systems, if it fail we get back with the standard query order)
           

          the explain of this query today is (count(*)=1.826.326):

          || *id* || *select_type* || *table* || *type* || *possible_keys* || *key* || *key_len* || *ref* || *rows* || *Extra* || execution time without flush cache (seconds) ||
          | 1 | PRIMARY | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 1 | Using where; Using index | 0.031 <- fast |
          | 2 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 6.755 <- slow |
          | 3 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 0.671 <- relative fast |
          | 4 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.467 <- slow |
          | 5 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.373 <- slow |
          | 6 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.528 <- slow |
          | | UNION RESULT | <union1,2,3,4,5,6> (if we execute a diferent union since we can optimize based in query cost, we could show <union 1,4,2,5,3,6> and at final Extra column could put <Union order optimizer>) | ALL | | | | | | | (Union order optimizer) |


          well that's all =]
          Hi guys, I have some queries like this:
          {code:sql}
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1
          {code}

          It's a search about a document number inside all documents (ok i could use others index but that's not the case), the problem is....
          The first SELECT query (with % at end of the query) return really fast (since it can use left side of % to search inside index), but executing others queries are very slow, the question is

          could we optimize this kind of query, since we don't have ORDER BY and we have a LIMIT?

          for example, (i don't know how is the code today and if this could break a shard strategy)
          1) execute the first query,
          2) if this return more than LIMIT + OFFSET rows, stop all unions,
          3) if not go to next query
          3.1) check that SELECT execution order (1,2,3,4,5,...) is necessary since we need deterministic results in UNION order or we will need a ORDER BY and loose this logic
          3.2) maybe if we want something faster we could do an: estimate cost of each query and execute queries using estimate cost order (i think this could be an optimizer option, instead of an SQL_ORDER_BLABLA, this avoid sql standard problems and we could use this in some old systems, if it fail we get back with the standard query order)
           

          the explain of this query today is (count(*)=1.826.326):

          || *id* || *select_type* || *table* || *type* || *possible_keys* || *key* || *key_len* || *ref* || *rows* || *Extra* || execution time without flush cache (seconds) ||
          | 1 | PRIMARY | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 1 | Using where; Using index | 0.031 <- fast |
          | 2 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 6.755 <- slow |
          | 3 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 0.671 <- relative fast |
          | 4 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.467 <- slow |
          | 5 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.373 <- slow |
          | 6 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.528 <- slow |
          | | UNION RESULT | <union1,2,3,4,5,6> \n (if we execute a diferent union since we can optimize based in query cost, we could show <union 1,4,2,5,3,6> and at final Extra column could put <Union order optimizer>) | ALL | | | | | | (Union order optimizer) | |


          well that's all =]
          rspadim roberto spadim made changes -
          Description Hi guys, I have some queries like this:
          {code:sql}
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1
          {code}

          It's a search about a document number inside all documents (ok i could use others index but that's not the case), the problem is....
          The first SELECT query (with % at end of the query) return really fast (since it can use left side of % to search inside index), but executing others queries are very slow, the question is

          could we optimize this kind of query, since we don't have ORDER BY and we have a LIMIT?

          for example, (i don't know how is the code today and if this could break a shard strategy)
          1) execute the first query,
          2) if this return more than LIMIT + OFFSET rows, stop all unions,
          3) if not go to next query
          3.1) check that SELECT execution order (1,2,3,4,5,...) is necessary since we need deterministic results in UNION order or we will need a ORDER BY and loose this logic
          3.2) maybe if we want something faster we could do an: estimate cost of each query and execute queries using estimate cost order (i think this could be an optimizer option, instead of an SQL_ORDER_BLABLA, this avoid sql standard problems and we could use this in some old systems, if it fail we get back with the standard query order)
           

          the explain of this query today is (count(*)=1.826.326):

          || *id* || *select_type* || *table* || *type* || *possible_keys* || *key* || *key_len* || *ref* || *rows* || *Extra* || execution time without flush cache (seconds) ||
          | 1 | PRIMARY | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 1 | Using where; Using index | 0.031 <- fast |
          | 2 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 6.755 <- slow |
          | 3 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 0.671 <- relative fast |
          | 4 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.467 <- slow |
          | 5 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.373 <- slow |
          | 6 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.528 <- slow |
          | | UNION RESULT | <union1,2,3,4,5,6> \n (if we execute a diferent union since we can optimize based in query cost, we could show <union 1,4,2,5,3,6> and at final Extra column could put <Union order optimizer>) | ALL | | | | | | (Union order optimizer) | |


          well that's all =]
          Hi guys, I have some queries like this:
          {code:sql}
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1
          {code}

          It's a search about a document number inside all documents (ok i could use others index but that's not the case), the problem is....
          The first SELECT query (with % at end of the query) return really fast (since it can use left side of % to search inside index), but executing others queries are very slow, the question is

          could we optimize this kind of query, since we don't have ORDER BY and we have a LIMIT?

          for example, (i don't know how is the code today and if this could break a shard strategy)
          1) execute the first query,
          2) if this return more than LIMIT + OFFSET rows, stop all unions,
          3) if not go to next query
          3.1) check that SELECT execution order (1,2,3,4,5,...) is necessary since we need deterministic results in UNION order or we will need a ORDER BY and loose this logic
          3.2) maybe if we want something faster we could do an: estimate cost of each query and execute queries using estimate cost order (i think this could be an optimizer option, instead of an SQL_ORDER_BLABLA, this avoid sql standard problems and we could use this in some old systems, if it fail we get back with the standard query order)
           

          the explain of this query today is (count(*)=1.826.326):

          || *id* || *select_type* || *table* || *type* || *possible_keys* || *key* || *key_len* || *ref* || *rows* || *Extra* || execution time without flush cache (seconds) ||
          | 1 | PRIMARY | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 1 | Using where; Using index | 0.031 <- fast |
          | 2 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 6.755 <- slow |
          | 3 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 0.671 <- relative fast |
          | 4 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.467 <- slow |
          | 5 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.373 <- slow |
          | 6 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.528 <- slow |
          | | UNION RESULT | <union1,2,3,4,5,6> | ALL | | | | | | (Union order optimizer) | |

          if we execute a diferent union since we can optimize based in query cost, we could show at last row (7): something like "<union 1,4,2,5,3,6>" and at final Extra column could put "<Union order optimizer>"

          well that's all =]
          rspadim roberto spadim made changes -
          Description Hi guys, I have some queries like this:
          {code:sql}
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1
          {code}

          It's a search about a document number inside all documents (ok i could use others index but that's not the case), the problem is....
          The first SELECT query (with % at end of the query) return really fast (since it can use left side of % to search inside index), but executing others queries are very slow, the question is

          could we optimize this kind of query, since we don't have ORDER BY and we have a LIMIT?

          for example, (i don't know how is the code today and if this could break a shard strategy)
          1) execute the first query,
          2) if this return more than LIMIT + OFFSET rows, stop all unions,
          3) if not go to next query
          3.1) check that SELECT execution order (1,2,3,4,5,...) is necessary since we need deterministic results in UNION order or we will need a ORDER BY and loose this logic
          3.2) maybe if we want something faster we could do an: estimate cost of each query and execute queries using estimate cost order (i think this could be an optimizer option, instead of an SQL_ORDER_BLABLA, this avoid sql standard problems and we could use this in some old systems, if it fail we get back with the standard query order)
           

          the explain of this query today is (count(*)=1.826.326):

          || *id* || *select_type* || *table* || *type* || *possible_keys* || *key* || *key_len* || *ref* || *rows* || *Extra* || execution time without flush cache (seconds) ||
          | 1 | PRIMARY | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 1 | Using where; Using index | 0.031 <- fast |
          | 2 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 6.755 <- slow |
          | 3 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 0.671 <- relative fast |
          | 4 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.467 <- slow |
          | 5 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.373 <- slow |
          | 6 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.528 <- slow |
          | | UNION RESULT | <union1,2,3,4,5,6> | ALL | | | | | | (Union order optimizer) | |

          if we execute a diferent union since we can optimize based in query cost, we could show at last row (7): something like "<union 1,4,2,5,3,6>" and at final Extra column could put "<Union order optimizer>"

          well that's all =]
          Hi guys, I have some queries like this:
          {code:sql}
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1
          {code}

          It's a search about a document number inside all documents (ok i could use others index but that's not the case), the problem is....
          The first SELECT query (with % at end of the query) return really fast (since it can use left side of % to search inside index), but executing others queries are very slow, the question is

          could we optimize this kind of query, since we don't have ORDER BY and we have a LIMIT?

          for example, (i don't know how is the code today and if this could break a shard strategy)
          1) execute the first query,
          2) if this return more than LIMIT + OFFSET rows, stop all unions,
          3) if not go to next query
          3.1) check that SELECT execution order (1,2,3,4,5,...) is necessary since we need deterministic results in UNION order or we will need a ORDER BY and loose this logic
          3.2) maybe if we want something faster we could do an: estimate cost of each query and execute queries using estimate cost order (i think this could be an optimizer option, instead of an SQL_ORDER_BLABLA, this avoid sql standard problems and we could use this in some old systems, if it fail we get back with the standard query order)
           

          the explain of this query today is (count(*)=1.826.326 rows):

          || *id* || *select_type* || *table* || *type* || *possible_keys* || *key* || *key_len* || *ref* || *rows* || *Extra* || execution time without flush cache (seconds) ||
          | 1 | PRIMARY | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 1 | Using where; Using index | 0.031 <- fast |
          | 2 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 6.755 <- slow |
          | 3 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 0.671 <- relative fast |
          | 4 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.467 <- slow |
          | 5 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.373 <- slow |
          | 6 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.528 <- slow |
          | | UNION RESULT | <union1,2,3,4,5,6> | ALL | | | | | | (Union order optimizer) | |

          if we execute a diferent union since we can optimize based in query cost, we could show at last row (7): something like "<union 1,4,2,5,3,6>" and at final Extra column could put "<Union order optimizer>"

          well that's all =]
          rspadim roberto spadim made changes -
          Description Hi guys, I have some queries like this:
          {code:sql}
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1
          {code}

          It's a search about a document number inside all documents (ok i could use others index but that's not the case), the problem is....
          The first SELECT query (with % at end of the query) return really fast (since it can use left side of % to search inside index), but executing others queries are very slow, the question is

          could we optimize this kind of query, since we don't have ORDER BY and we have a LIMIT?

          for example, (i don't know how is the code today and if this could break a shard strategy)
          1) execute the first query,
          2) if this return more than LIMIT + OFFSET rows, stop all unions,
          3) if not go to next query
          3.1) check that SELECT execution order (1,2,3,4,5,...) is necessary since we need deterministic results in UNION order or we will need a ORDER BY and loose this logic
          3.2) maybe if we want something faster we could do an: estimate cost of each query and execute queries using estimate cost order (i think this could be an optimizer option, instead of an SQL_ORDER_BLABLA, this avoid sql standard problems and we could use this in some old systems, if it fail we get back with the standard query order)
           

          the explain of this query today is (count(*)=1.826.326 rows):

          || *id* || *select_type* || *table* || *type* || *possible_keys* || *key* || *key_len* || *ref* || *rows* || *Extra* || execution time without flush cache (seconds) ||
          | 1 | PRIMARY | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 1 | Using where; Using index | 0.031 <- fast |
          | 2 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 6.755 <- slow |
          | 3 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 0.671 <- relative fast |
          | 4 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.467 <- slow |
          | 5 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.373 <- slow |
          | 6 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.528 <- slow |
          | | UNION RESULT | <union1,2,3,4,5,6> | ALL | | | | | | (Union order optimizer) | |

          if we execute a diferent union since we can optimize based in query cost, we could show at last row (7): something like "<union 1,4,2,5,3,6>" and at final Extra column could put "<Union order optimizer>"

          well that's all =]
          Hi guys, I have some queries like this:
          {code:sql}
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1
          {code}

          It's a search about a document number inside all documents (ok i could use others index but that's not the case), the problem is....
          The first SELECT query (with % at end of the query) return really fast (since it can use left side of % to search inside index), but executing others queries are very slow, the question is

          could we optimize this kind of query, since we don't have ORDER BY and we have a LIMIT?

          for example, (i don't know how is the code today and if this could break a shard strategy)
          1) execute the first query,
          2) if this return more than LIMIT + OFFSET rows, stop all unions,
          3) if not go to next query
          3.1) check that SELECT execution order (1,2,3,4,5,...) is necessary since we need deterministic results in UNION order or we will need a ORDER BY and loose this logic
          3.2) maybe if we want something faster we could do an: estimate cost of each query and execute queries using estimate cost order (i think this could be an optimizer option, instead of an SQL_ORDER_BLABLA, this avoid sql standard problems and we could use this in some old systems, if it fail we get back with the standard query order)
           

          the explain of this query today is (count(*)=1.826.326 rows):

          || *id* || *select_type* || *table* || *type* || *possible_keys* || *key* || *key_len* || *ref* || *rows* || *Extra* || execution time without flush cache (seconds) ||
          | 1 | PRIMARY | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 1 | Using where; Using index | 0.031 <- fast |
          | 2 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 6.755 <- slow (maybe a outlier result) |
          | 3 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 0.671 <- relative fast |
          | 4 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.467 <- slow |
          | 5 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.373 <- slow |
          | 6 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.528 <- slow |
          | | UNION RESULT | <union1,2,3,4,5,6> | ALL | | | | | | (Union order optimizer) | Total query time: 5.258 |

          if we execute a diferent union since we can optimize based in query cost, we could show at last row (7): something like "<union 1,4,2,5,3,6>" and at final Extra column could put "<Union order optimizer>"

          well that's all =]
          serg Sergei Golubchik made changes -
          Affects Version/s 10.1.0 [ 12200 ]
          Affects Version/s 10.1 [ 16100 ]
          elenst Elena Stepanova made changes -
          Affects Version/s 10.1.0 [ 12200 ]
          Issue Type Bug [ 1 ] Task [ 3 ]
          elenst Elena Stepanova made changes -
          Fix Version/s 10.1 [ 16100 ]

          good idea!

          serg Sergei Golubchik added a comment - good idea!
          rspadim roberto spadim made changes -
          Description Hi guys, I have some queries like this:
          {code:sql}
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1
          {code}

          It's a search about a document number inside all documents (ok i could use others index but that's not the case), the problem is....
          The first SELECT query (with % at end of the query) return really fast (since it can use left side of % to search inside index), but executing others queries are very slow, the question is

          could we optimize this kind of query, since we don't have ORDER BY and we have a LIMIT?

          for example, (i don't know how is the code today and if this could break a shard strategy)
          1) execute the first query,
          2) if this return more than LIMIT + OFFSET rows, stop all unions,
          3) if not go to next query
          3.1) check that SELECT execution order (1,2,3,4,5,...) is necessary since we need deterministic results in UNION order or we will need a ORDER BY and loose this logic
          3.2) maybe if we want something faster we could do an: estimate cost of each query and execute queries using estimate cost order (i think this could be an optimizer option, instead of an SQL_ORDER_BLABLA, this avoid sql standard problems and we could use this in some old systems, if it fail we get back with the standard query order)
           

          the explain of this query today is (count(*)=1.826.326 rows):

          || *id* || *select_type* || *table* || *type* || *possible_keys* || *key* || *key_len* || *ref* || *rows* || *Extra* || execution time without flush cache (seconds) ||
          | 1 | PRIMARY | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 1 | Using where; Using index | 0.031 <- fast |
          | 2 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 6.755 <- slow (maybe a outlier result) |
          | 3 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 0.671 <- relative fast |
          | 4 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.467 <- slow |
          | 5 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.373 <- slow |
          | 6 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.528 <- slow |
          | | UNION RESULT | <union1,2,3,4,5,6> | ALL | | | | | | (Union order optimizer) | Total query time: 5.258 |

          if we execute a diferent union since we can optimize based in query cost, we could show at last row (7): something like "<union 1,4,2,5,3,6>" and at final Extra column could put "<Union order optimizer>"

          well that's all =]
          Hi guys, I have some queries like this:
          {code:sql}
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1
          {code}

          It's a search about a document number inside all documents (ok i could use others index but that's not the case), the problem is....
          The first SELECT query (with % at end of the query) return really fast (since it can use left side of % to search inside index), but executing others queries are very slow, the question is

          could we optimize this kind of query, since we don't have ORDER BY and we have a LIMIT?

          for example, (i don't know how is the code today and if this could break a shard strategy)
          1) execute the first query,
          2) if this return more than LIMIT + OFFSET rows, stop all unions,
          3) if not go to next query
          3.1) check that SELECT execution order (1,2,3,4,5,...) is necessary since we need deterministic results in UNION order or we will need a ORDER BY and loose this logic
          3.2) maybe if we want something faster we could do an: estimate cost of each query and execute queries using estimate cost order (i think this could be an optimizer option, instead of an SQL_ORDER_BLABLA, this avoid sql standard problems and we could use this in some old systems, if it fail we get back with the standard query order), or extend the first UNION clausule with UNION NO_ORDER, or something like it, i'm considering here two problems... in this example the first query is the faster... and return the right data, if i change the first with the last, maybe i could get the wrong result, in this case the (3.2) optimization couldn't be used, but think about a server that the optimizer stay on and we forget to set it off (like query cache = on and we don't use SQL_NO_CACHE) an NO_ORDER or something like it could be nice to ensure that union will be executed in the right order even with optimizer option turned on just an idea... maybe we could use other better idea, for example of SQL_CACHE, only the first SELECT allow the SQL_CACHE others SELECT don't allow it, something like it is nice
          (select sql_cache sql_no_union_order * form blabal union select * from blabla2 ....)
          with this we could mix in a single complex query both union order and non union order
          select (sql_union order) , (sql_no_union_order) from (any other query)
           

          the explain of this query today is (count( * )=1.826.326 rows):

          || *id* || *select_type* || *table* || *type* || *possible_keys* || *key* || *key_len* || *ref* || *rows* || *Extra* || execution time without flush cache (seconds) ||
          | 1 | PRIMARY | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 1 | Using where; Using index | 0.031 <- fast |
          | 2 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 6.755 <- slow (maybe a outlier result) |
          | 3 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 0.671 <- relative fast |
          | 4 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.467 <- slow |
          | 5 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.373 <- slow |
          | 6 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.528 <- slow |
          | | UNION RESULT | <union1,2,3,4,5,6> | ALL | | | | | | (Union order optimizer) | Total query time: 5.258 |

          if we execute a diferent union since we can optimize based in query cost, we could show at last row (7): something like "<union 1,4,2,5,3,6>" and at final Extra column could put "<Union order optimizer>"

          well that's all =]
          rspadim roberto spadim made changes -
          Description Hi guys, I have some queries like this:
          {code:sql}
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1
          {code}

          It's a search about a document number inside all documents (ok i could use others index but that's not the case), the problem is....
          The first SELECT query (with % at end of the query) return really fast (since it can use left side of % to search inside index), but executing others queries are very slow, the question is

          could we optimize this kind of query, since we don't have ORDER BY and we have a LIMIT?

          for example, (i don't know how is the code today and if this could break a shard strategy)
          1) execute the first query,
          2) if this return more than LIMIT + OFFSET rows, stop all unions,
          3) if not go to next query
          3.1) check that SELECT execution order (1,2,3,4,5,...) is necessary since we need deterministic results in UNION order or we will need a ORDER BY and loose this logic
          3.2) maybe if we want something faster we could do an: estimate cost of each query and execute queries using estimate cost order (i think this could be an optimizer option, instead of an SQL_ORDER_BLABLA, this avoid sql standard problems and we could use this in some old systems, if it fail we get back with the standard query order), or extend the first UNION clausule with UNION NO_ORDER, or something like it, i'm considering here two problems... in this example the first query is the faster... and return the right data, if i change the first with the last, maybe i could get the wrong result, in this case the (3.2) optimization couldn't be used, but think about a server that the optimizer stay on and we forget to set it off (like query cache = on and we don't use SQL_NO_CACHE) an NO_ORDER or something like it could be nice to ensure that union will be executed in the right order even with optimizer option turned on just an idea... maybe we could use other better idea, for example of SQL_CACHE, only the first SELECT allow the SQL_CACHE others SELECT don't allow it, something like it is nice
          (select sql_cache sql_no_union_order * form blabal union select * from blabla2 ....)
          with this we could mix in a single complex query both union order and non union order
          select (sql_union order) , (sql_no_union_order) from (any other query)
           

          the explain of this query today is (count( * )=1.826.326 rows):

          || *id* || *select_type* || *table* || *type* || *possible_keys* || *key* || *key_len* || *ref* || *rows* || *Extra* || execution time without flush cache (seconds) ||
          | 1 | PRIMARY | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 1 | Using where; Using index | 0.031 <- fast |
          | 2 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 6.755 <- slow (maybe a outlier result) |
          | 3 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 0.671 <- relative fast |
          | 4 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.467 <- slow |
          | 5 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.373 <- slow |
          | 6 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.528 <- slow |
          | | UNION RESULT | <union1,2,3,4,5,6> | ALL | | | | | | (Union order optimizer) | Total query time: 5.258 |

          if we execute a diferent union since we can optimize based in query cost, we could show at last row (7): something like "<union 1,4,2,5,3,6>" and at final Extra column could put "<Union order optimizer>"

          well that's all =]
          Hi guys, I have some queries like this:
          {code:sql}
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1
          {code}

          It's a search about a document number inside all documents (ok i could use others index but that's not the case), the problem is....
          The first SELECT query (with % at end of the query) return really fast (since it can use left side of % to search inside index), but executing others queries are very slow, the question is

          could we optimize this kind of query, since we don't have ORDER BY and we have a LIMIT?

          for example, (i don't know how is the code today and if this could break a shard strategy)
          1) execute the first query,
          2) if this return more than LIMIT + OFFSET rows, stop all unions,
          3) if not go to next query
          3.1) check that SELECT execution order (1,2,3,4,5,...) is necessary since we need deterministic results in UNION order or we will need a ORDER BY and loose this logic
          3.2) maybe if we want something faster we could do an: estimate cost of each query and execute queries using estimate cost order (i think this could be an optimizer option, instead of an SQL_ORDER_BLABLA, this avoid sql standard problems and we could use this in some old systems, if it fail we get back with the standard query order), or extend the first UNION clausule with UNION NO_ORDER, or something like it, i'm considering here two problems... in this example the first query is the faster... and return the right data, if i change the first with the last, maybe i could get the wrong result, in this case the (3.2) optimization couldn't be used, but think about a server that the optimizer stay on and we forget to set it off (like query cache = on and we don't use SQL_NO_CACHE) an NO_ORDER or something like it could be nice to ensure that union will be executed in the right order even with optimizer option turned on just an idea... maybe we could use other better idea, for example of SQL_CACHE, only the first SELECT allow the SQL_CACHE others SELECT don't allow it, something like it is nice
          (select sql_cache sql_no_union_order * form blabal union select * from blabla2 ....)
          with this we could mix in a single complex query both union order and non union order
          select (sql_union order) , (sql_no_union_order) from (any other query)
           

          the explain of this query today is (count( * )=1.826.326 rows):

          || *id* || *select_type* || *table* || *type* || *possible_keys* || *key* || *key_len* || *ref* || *rows* || *Extra* || execution time without flush cache (seconds) ||
          | 1 | PRIMARY | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 1 | Using where; Using index | 0.031 <- fast |
          | 2 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 6.755 <- slow (maybe a outlier result) |
          | 3 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 0.671 <- relative fast |
          | 4 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.467 <- slow |
          | 5 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.373 <- slow |
          | 6 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.528 <- slow |
          | | UNION RESULT | <union1,2,3,4,5,6> | ALL | | | | | | (Union order optimizer) | Total query time: 5.258 |

          if we execute a diferent union since we can optimize based in query cost, we could show at last row (7): something like "<union 1,4,2,5,3,6>" and at final Extra column could put "<Union order optimizer>"

          in this case we have a optimization from 5.258 seconds to 0.031 seconds ~= 170x faster


          well that's all =]
          rspadim roberto spadim made changes -
          Description Hi guys, I have some queries like this:
          {code:sql}
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1
          {code}

          It's a search about a document number inside all documents (ok i could use others index but that's not the case), the problem is....
          The first SELECT query (with % at end of the query) return really fast (since it can use left side of % to search inside index), but executing others queries are very slow, the question is

          could we optimize this kind of query, since we don't have ORDER BY and we have a LIMIT?

          for example, (i don't know how is the code today and if this could break a shard strategy)
          1) execute the first query,
          2) if this return more than LIMIT + OFFSET rows, stop all unions,
          3) if not go to next query
          3.1) check that SELECT execution order (1,2,3,4,5,...) is necessary since we need deterministic results in UNION order or we will need a ORDER BY and loose this logic
          3.2) maybe if we want something faster we could do an: estimate cost of each query and execute queries using estimate cost order (i think this could be an optimizer option, instead of an SQL_ORDER_BLABLA, this avoid sql standard problems and we could use this in some old systems, if it fail we get back with the standard query order), or extend the first UNION clausule with UNION NO_ORDER, or something like it, i'm considering here two problems... in this example the first query is the faster... and return the right data, if i change the first with the last, maybe i could get the wrong result, in this case the (3.2) optimization couldn't be used, but think about a server that the optimizer stay on and we forget to set it off (like query cache = on and we don't use SQL_NO_CACHE) an NO_ORDER or something like it could be nice to ensure that union will be executed in the right order even with optimizer option turned on just an idea... maybe we could use other better idea, for example of SQL_CACHE, only the first SELECT allow the SQL_CACHE others SELECT don't allow it, something like it is nice
          (select sql_cache sql_no_union_order * form blabal union select * from blabla2 ....)
          with this we could mix in a single complex query both union order and non union order
          select (sql_union order) , (sql_no_union_order) from (any other query)
           

          the explain of this query today is (count( * )=1.826.326 rows):

          || *id* || *select_type* || *table* || *type* || *possible_keys* || *key* || *key_len* || *ref* || *rows* || *Extra* || execution time without flush cache (seconds) ||
          | 1 | PRIMARY | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 1 | Using where; Using index | 0.031 <- fast |
          | 2 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 6.755 <- slow (maybe a outlier result) |
          | 3 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 0.671 <- relative fast |
          | 4 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.467 <- slow |
          | 5 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.373 <- slow |
          | 6 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.528 <- slow |
          | | UNION RESULT | <union1,2,3,4,5,6> | ALL | | | | | | (Union order optimizer) | Total query time: 5.258 |

          if we execute a diferent union since we can optimize based in query cost, we could show at last row (7): something like "<union 1,4,2,5,3,6>" and at final Extra column could put "<Union order optimizer>"

          in this case we have a optimization from 5.258 seconds to 0.031 seconds ~= 170x faster


          well that's all =]
          Hi guys, I have some queries like this:
          {code:sql}
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;$;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "j;1;%;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;$;%294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;294007/3nfe-1%"
          UNION
          SELECT doc_hash_key FROM mov_documentos WHERE doc_hash_key LIKE "%;%;%;%294007/3nfe-1%"
          LIMIT 1
          {code}

          It's a search about a document number inside all documents (ok i could use others index but that's not the case), the problem is....
          The first SELECT query (with % at end of the query) return really fast (since it can use left side of % to search inside index), but executing others queries are very slow, the question is

          could we optimize this kind of query, since we don't have ORDER BY and we have a LIMIT?

          for example, (i don't know how is the code today and if this could break a shard strategy)
          1) execute the first query,
          2) if this return more than LIMIT + OFFSET rows, stop all unions,
          3) if not go to next query
          3.1) check that SELECT execution order (1,2,3,4,5,...) is necessary since we need deterministic results in UNION order or we will need a ORDER BY and loose this logic
          3.2) maybe if we want something faster we could do an: estimate cost of each query and execute queries using estimate cost order (i think this could be an optimizer option, instead of an SQL_ORDER_BLABLA, this avoid sql standard problems and we could use this in some old systems, if it fail we get back with the standard query order), or extend the first UNION clausule with UNION NO_ORDER, or something like it, i'm considering here two problems... in this example the first query is the faster... and return the right data, if i change the first with the last, maybe i could get the wrong result, in this case the (3.2) optimization couldn't be used, but think about a server that the optimizer stay on and we forget to set it off (like query cache = on and we don't use SQL_NO_CACHE) an NO_ORDER or something like it could be nice to ensure that union will be executed in the right order even with optimizer option turned on just an idea... maybe we could use other better idea, for example of SQL_CACHE, only the first SELECT allow the SQL_CACHE others SELECT don't allow it, something like it is nice
          (select sql_cache sql_no_union_order * form blabal union select * from blabla2 ....)
          with this we could mix in a single complex query both union order and non union order
          select (sql_union order) , (sql_no_union_order) from (any other query)
           

          the explain of this query today is (count( * )=1.826.326 rows):

          || *id* || *select_type* || *table* || *type* || *possible_keys* || *key* || *key_len* || *ref* || *rows* || *Extra* || execution time without flush cache (seconds) ||
          | 1 | PRIMARY | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 1 | Using where; Using index | 0.031 <- fast |
          | 2 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 6.755 <- slow (maybe a outlier result) |
          | 3 | UNION | mov_documentos | range | PRIMARY | PRIMARY | 77 | | 628434 | Using where; Using index | 0.671 <- relative fast |
          | 4 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.467 <- slow |
          | 5 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.373 <- slow |
          | 6 | UNION | mov_documentos | index | | PRIMARY | 77 | | 1826326 | Using where; Using index | 1.528 <- slow |
          | | UNION RESULT | <union1,2,3,4,5,6> or <union optimized 1,4,2,5,3,6> | ALL | | | | | | (Union limit optimized) | Total query time: 5.258 |

          if we execute a diferent union since we can optimize based in query cost, we could show at last row (7): something like "<union 1,4,2,5,3,6>" and at final Extra column could put "<Union order optimizer>"

          in this case we have a optimization from 5.258 seconds to 0.031 seconds ~= 170x faster


          well that's all =]
          ratzpo Rasmus Johansson (Inactive) made changes -
          Workflow MariaDB v2 [ 53728 ] MariaDB v3 [ 65025 ]
          serg Sergei Golubchik made changes -
          Fix Version/s 10.1 [ 16100 ]
          rspadim roberto spadim made changes -
          Priority Minor [ 4 ] Major [ 3 ]
          serg Sergei Golubchik made changes -
          Workflow MariaDB v3 [ 65025 ] MariaDB v4 [ 130258 ]
          alice Alice Sherepa made changes -
          Assignee Sergei Petrunia [ psergey ]
          alice Alice Sherepa made changes -
          Fix Version/s 10.13 [ 28501 ]
          julien.fritsch Julien Fritsch made changes -
          Fix Version/s 11.1 [ 28549 ]
          julien.fritsch Julien Fritsch made changes -
          Fix Version/s 10.13 [ 28501 ]

          People

            psergei Sergei Petrunia
            rspadim roberto spadim
            Votes:
            2 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.