Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-20871

Queries with entity_id IN ('1', '2', ..... , '70000') run much slower in MariaDB 10.3.18 than on MariaDB 10.1

Details

    • Bug
    • Status: Closed (View Workflow)
    • Critical
    • Resolution: Fixed
    • 10.3.18
    • 10.3.22, 10.4.12, 10.5.1
    • Optimizer
    • None
    • Magento 2.3 Shop, MariaDB 10.3.18

    Description

      We discovered, that queries with a long IN-list for an INT column take much longer than on 10.1, when the list entries are supplied as strings.

      Is that intended or a bug?

      References: https://github.com/magento/magento2/issues/25199
      https://magento.stackexchange.com/questions/292867/which-server-settings-can-cause-magento-reindex-to-run-really-slow/293345#293345

      Attachments

        Issue Links

          Activity

            10.3.20 is running now for more than 25 mins that query, after I set global in_predicate_conversion_threshold = 0;

            alexm Alexander Menk added a comment - 10.3.20 is running now for more than 25 mins that query, after I set global in_predicate_conversion_threshold = 0;

            on 10.3.20 with global in_predicate_conversion_threshold = 0;

            real 27m21.634s
            user 0m0.043s
            sys 0m0.141s
            id select_type table type possible_keys key key_len ref rows Extra
            1 PRIMARY <derived3> ALL NULL NULL NULL NULL 73584 Start temporary; Using temporary; Using filesort
            1 PRIMARY cp eq_ref PRIMARY PRIMARY 4 tvc_0._col_1 1 Using where; Using index; End temporary
            1 PRIMARY cpw range PRIMARY,CATALOG_PRODUCT_WEBSITE_WEBSITE_ID CATALOG_PRODUCT_WEBSITE_WEBSITE_ID 2 NULL 40195 Using where; Using index; Using join buffer (flat, BNL join)
            1 PRIMARY cpsd eq_ref CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID,CATALOG_PRODUCT_ENTITY_INT_ATTRIBUTE_ID,CATALOG_PRODUCT_ENTITY_INT_STORE_ID CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID 8 perftest.cp.entity_id,const,const 1
            1 PRIMARY cpss eq_ref CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID,CATALOG_PRODUCT_ENTITY_INT_ATTRIBUTE_ID,CATALOG_PRODUCT_ENTITY_INT_STORE_ID CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID 8 perftest.cp.entity_id,const,const 1 Using where
            1 PRIMARY cpvd eq_ref CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID,CATALOG_PRODUCT_ENTITY_INT_ATTRIBUTE_ID,CATALOG_PRODUCT_ENTITY_INT_STORE_ID CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID 8 perftest.cp.entity_id,const,const 1
            1 PRIMARY cpvs eq_ref CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID,CATALOG_PRODUCT_ENTITY_INT_ATTRIBUTE_ID,CATALOG_PRODUCT_ENTITY_INT_STORE_ID CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID 8 perftest.cp.entity_id,const,const 1 Using where
            1 PRIMARY ccp ref CATALOG_CATEGORY_PRODUCT_PRODUCT_ID CATALOG_CATEGORY_PRODUCT_PRODUCT_ID 4 perftest.cp.entity_id 2
            3 DERIVED NULL NULL NULL NULL NULL NULL NULL No tables used

            alexm Alexander Menk added a comment - on 10.3.20 with global in_predicate_conversion_threshold = 0; real 27m21.634s user 0m0.043s sys 0m0.141s id select_type table type possible_keys key key_len ref rows Extra 1 PRIMARY <derived3> ALL NULL NULL NULL NULL 73584 Start temporary; Using temporary; Using filesort 1 PRIMARY cp eq_ref PRIMARY PRIMARY 4 tvc_0._col_1 1 Using where; Using index; End temporary 1 PRIMARY cpw range PRIMARY,CATALOG_PRODUCT_WEBSITE_WEBSITE_ID CATALOG_PRODUCT_WEBSITE_WEBSITE_ID 2 NULL 40195 Using where; Using index; Using join buffer (flat, BNL join) 1 PRIMARY cpsd eq_ref CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID,CATALOG_PRODUCT_ENTITY_INT_ATTRIBUTE_ID,CATALOG_PRODUCT_ENTITY_INT_STORE_ID CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID 8 perftest.cp.entity_id,const,const 1 1 PRIMARY cpss eq_ref CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID,CATALOG_PRODUCT_ENTITY_INT_ATTRIBUTE_ID,CATALOG_PRODUCT_ENTITY_INT_STORE_ID CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID 8 perftest.cp.entity_id,const,const 1 Using where 1 PRIMARY cpvd eq_ref CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID,CATALOG_PRODUCT_ENTITY_INT_ATTRIBUTE_ID,CATALOG_PRODUCT_ENTITY_INT_STORE_ID CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID 8 perftest.cp.entity_id,const,const 1 1 PRIMARY cpvs eq_ref CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID,CATALOG_PRODUCT_ENTITY_INT_ATTRIBUTE_ID,CATALOG_PRODUCT_ENTITY_INT_STORE_ID CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID 8 perftest.cp.entity_id,const,const 1 Using where 1 PRIMARY ccp ref CATALOG_CATEGORY_PRODUCT_PRODUCT_ID CATALOG_CATEGORY_PRODUCT_PRODUCT_ID 4 perftest.cp.entity_id 2 3 DERIVED NULL NULL NULL NULL NULL NULL NULL No tables used

            This might be related: https://jira.mariadb.org/browse/MDEV-12176?focusedCommentId=119660&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-119660
            I am trying
            set in_predicate_conversion_threshold = 4294967295;
            set global in_predicate_conversion_threshold = 4294967295;

            And yeah, it works faster on 10.3.20

            real 0m1.819s
            user 0m0.061s
            sys 0m0.110s
            id select_type table type possible_keys key key_len ref rows Extra
            1 SIMPLE cpw ref PRIMARY,CATALOG_PRODUCT_WEBSITE_WEBSITE_ID CATALOG_PRODUCT_WEBSITE_WEBSITE_ID 2 const 1 Using where; Using index
            1 SIMPLE cpsd eq_ref CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID,CATALOG_PRODUCT_ENTITY_INT_ATTRIBUTE_ID,CATALOG_PRODUCT_ENTITY_INT_STORE_ID CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID 8 perftest.cpw.product_id,const,const 1
            1 SIMPLE cp eq_ref PRIMARY PRIMARY 4 perftest.cpw.product_id 1 Using where; Using index
            1 SIMPLE cpss eq_ref CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID,CATALOG_PRODUCT_ENTITY_INT_ATTRIBUTE_ID,CATALOG_PRODUCT_ENTITY_INT_STORE_ID CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID 8 perftest.cpw.product_id,const,const 1 Using where
            1 SIMPLE cpvd eq_ref CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID,CATALOG_PRODUCT_ENTITY_INT_ATTRIBUTE_ID,CATALOG_PRODUCT_ENTITY_INT_STORE_ID CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID 8 perftest.cpw.product_id,const,const 1
            1 SIMPLE cpvs eq_ref CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID,CATALOG_PRODUCT_ENTITY_INT_ATTRIBUTE_ID,CATALOG_PRODUCT_ENTITY_INT_STORE_ID CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID 8 perftest.cpw.product_id,const,const 1 Using where
            1 SIMPLE ccp ref CATALOG_CATEGORY_PRODUCT_PRODUCT_ID CATALOG_CATEGORY_PRODUCT_PRODUCT_ID 4 perftest.cpw.product_id 2

            alexm Alexander Menk added a comment - This might be related: https://jira.mariadb.org/browse/MDEV-12176?focusedCommentId=119660&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-119660 I am trying set in_predicate_conversion_threshold = 4294967295; set global in_predicate_conversion_threshold = 4294967295; And yeah, it works faster on 10.3.20 real 0m1.819s user 0m0.061s sys 0m0.110s id select_type table type possible_keys key key_len ref rows Extra 1 SIMPLE cpw ref PRIMARY,CATALOG_PRODUCT_WEBSITE_WEBSITE_ID CATALOG_PRODUCT_WEBSITE_WEBSITE_ID 2 const 1 Using where; Using index 1 SIMPLE cpsd eq_ref CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID,CATALOG_PRODUCT_ENTITY_INT_ATTRIBUTE_ID,CATALOG_PRODUCT_ENTITY_INT_STORE_ID CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID 8 perftest.cpw.product_id,const,const 1 1 SIMPLE cp eq_ref PRIMARY PRIMARY 4 perftest.cpw.product_id 1 Using where; Using index 1 SIMPLE cpss eq_ref CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID,CATALOG_PRODUCT_ENTITY_INT_ATTRIBUTE_ID,CATALOG_PRODUCT_ENTITY_INT_STORE_ID CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID 8 perftest.cpw.product_id,const,const 1 Using where 1 SIMPLE cpvd eq_ref CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID,CATALOG_PRODUCT_ENTITY_INT_ATTRIBUTE_ID,CATALOG_PRODUCT_ENTITY_INT_STORE_ID CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID 8 perftest.cpw.product_id,const,const 1 1 SIMPLE cpvs eq_ref CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID,CATALOG_PRODUCT_ENTITY_INT_ATTRIBUTE_ID,CATALOG_PRODUCT_ENTITY_INT_STORE_ID CATALOG_PRODUCT_ENTITY_INT_ENTITY_ID_ATTRIBUTE_ID_STORE_ID 8 perftest.cpw.product_id,const,const 1 Using where 1 SIMPLE ccp ref CATALOG_CATEGORY_PRODUCT_PRODUCT_ID CATALOG_CATEGORY_PRODUCT_PRODUCT_ID 4 perftest.cpw.product_id 2

            Alexander,
            Now the conversion of the IN Predicate into IN subquery is not done if the arguments of the left and right parts of the predicate are of the different types.
            Wait for 10.3.21.

            igor Igor Babaev (Inactive) added a comment - Alexander, Now the conversion of the IN Predicate into IN subquery is not done if the arguments of the left and right parts of the predicate are of the different types. Wait for 10.3.21.
            varun Varun Gupta (Inactive) added a comment - - edited

            This is related to MDEV-20900 and should be fixed by that. The fix would be available in 10.3.22

            varun Varun Gupta (Inactive) added a comment - - edited This is related to MDEV-20900 and should be fixed by that. The fix would be available in 10.3.22

            People

              varun Varun Gupta (Inactive)
              alexm Alexander Menk
              Votes:
              2 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.