otto, I noted a failure elsewhere in main.innodb_ext_key in MDEV-20169 (which is about main.partition_innodb failures, while this test does not use partitioning). The MDEV-20169 problem does not look like a new regression in 10.5, but its probability has significantly increased on 10.5.

In the buildbot.askmonty.org cross-reference, I was unable to find anything similar to exactly that type of failure. I only saw a couple 10.5 failures that look similar to the one I linked in MDEV-20169.

That said, this failure could still share a root cause with MDEV-20169.

Temporarily disabled in https://github.com/MariaDB/server/commit/6f0b621caf76cf636771cab2e78b6e1846b31c78 - we can't have this keeping the CI red permanently, that halts all other quality assurance / testing work. Please fix and re-enable when you have a chance.

On my first phase of investigation I could not reproduce this failure on the current 10.5 and also on the commit which disabled this test.

vladbogo Can you help and provide instructions on how to reproduce the build https://buildbot.mariadb.org/#/builders/27/builds/4241 ? Since the CI systems is consistently (or at least was) producing this error, and the CI environment is a fully standardized and codified environment, reproducing this error should be trivial.

otto The dockerfile that sets up the environment can be found here https://github.com/MariaDB/mariadb.org-tools/blob/master/buildbot.mariadb.org/dockerfiles/ubuntu-1604.dockerfile. The steps to reproduce would be: 1) build the image associated with the docker file (e.g. docker build -t ubuntu-1604 --file ubuntu-1604.dockerfile .); 2) then start a container (e.g. docker run --user root -it ubuntu-1604 bash). 3) Finally, just fetch the tarball that the builder fetches (see the "fetch_tarball" step). Unfortunately, only one month of history is kept due to space limitations so anything older than that I assume can only be reproduced by reverting git.

Well i checked today on buildbot
https://buildbot.mariadb.org/#/builders/27/builds/5587/steps/8/logs/stdio
and the test main.innodb_ext_key passes there, so looks like it does not fail reliably.

So i saw the cost of the plan in the optimizer_trace for table t2

              {

                "table": "t2",

                "range_analysis": {

                  "table_scan": {

                    "rows": 48,

                    "cost": 12.6

                  },

"analyzing_range_alternatives": {

                    "range_scan_alternatives": [

                      {

                        "index": "page_timestamp",

                        "ranges": ["(38928081) <= (rev_page) <= (38928081)"],

                        "rowid_ordered": false,

                        "using_mrr": false,

                        "index_only": false,

                        "rows": 10,

                        "cost": 12.14841776,

                        "chosen": true

                      }

                    ],

So the cost are 12.145 and 12.6 which is very close and due to the statistics from INNODB we may be getting different plans, so increasing the rows would be the best option here.

After increasing the rows in table t2 (the ones that would not satify the range), I get

 "table": "t2",

                "range_analysis": {

                  "table_scan": {

                    "rows": 56,

                    "cost": 14.2

                  },

and

                  "analyzing_range_alternatives": {

                    "range_scan_alternatives": [

                      {

                        "index": "page_timestamp",

                        "ranges": ["(38928081) <= (rev_page) <= (38928081)"],

                        "rowid_ordered": false,

                        "using_mrr": false,

                        "index_only": false,

                        "rows": 10,

                        "cost": 12.14841776,

                        "chosen": true

                      }

                    ],

Patch
https://github.com/MariaDB/server/commit/1f43c18735c809e9108bb48d61841c4419510134

Btw, the test is currently not listed in mysql-test/unstable-tests. Not sure what has removed Otto's commit (a merge?)

Ok to push.