This is a copy of this MySQL bug: http://bugs.mysql.com/bug.php?id=69359
I am filing it here in hopes it will get more attention than from the MySQL folks.
Note, this bug occurs in BOTH MySQL and MariaDB.
Restatement of bug follows...
See
http://stackoverflow.com/questions/16848190/mysql-why-isnt-foo-is-null-optimized-away
for a description. Quoting that here:
I have two tables Person and Message and the latter has a foreign key to the former. Each table has id as the primary key column, and the Person table also has a column personId which is (uniquely) indexed.
The query below should take advantage of the personId key index, but instead MySQL requires scanning the entire Message table for some reason:
mysql> EXPLAIN SELECT `m`.*
|
-> FROM
|
-> `Message` AS `m`
|
-> LEFT JOIN
|
-> `Person` AS `p` ON (`m`.`person` = `p`.`id`)
|
-> WHERE
|
-> 'M002649397' IS NULL OR
|
-> `p`.`personId` = 'M002649397';
|
+----+-------------+-------+--------+---------------+---------+---------+----------------+--------+-------------+
|
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
|
+----+-------------+-------+--------+---------------+---------+---------+----------------+--------+-------------+
|
| 1 | SIMPLE | m | ALL | NULL | NULL | NULL | NULL | 273220 | |
|
| 1 | SIMPLE | p | eq_ref | PRIMARY | PRIMARY | 8 | pcom.m.person | 1 | Using where |
|
+----+-------------+-------+--------+---------------+---------+---------+----------------+--------+-------------+
|
2 rows in set (0.00 sec)
|
But when I comment out the 'M002649397' IS NULL OR clause (which has no effect on the result), the query suddenly gets more efficient:
mysql> EXPLAIN SELECT `m`.*
|
-> FROM
|
-> `Message` AS `m`
|
-> LEFT JOIN
|
-> `Person` AS `p` ON (`m`.`person` = `p`.`id`)
|
-> WHERE
|
-> -- 'M002649397' IS NULL OR
|
-> `p`.`personId` = 'M002649397';
|
+----+-------------+-------+-------+--------------------+--------------------+---------+-------+------+-------------+
|
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
|
+----+-------------+-------+-------+--------------------+--------------------+---------+-------+------+-------------+
|
| 1 | SIMPLE | p | const | PRIMARY,personId | personId | 767 | const | 1 | Using index |
|
| 1 | SIMPLE | m | ref | FK9C2397E7A0F6ED11 | FK9C2397E7A0F6ED11 | 9 | const | 3 | Using where |
|
+----+-------------+-------+-------+--------------------+--------------------+---------+-------+------+-------------+
|
2 rows in set (0.01 sec)
|
The bug is that the 'M002649397' IS NULL expression, which is always false, is not being optimized away.
Here is a schema to test with:
create table Message (
|
type char(1) not null,
|
id bigint not null auto_increment,
|
createTime datetime not null,
|
updateTime datetime not null,
|
person bigint,
|
primary key (id)
|
) ENGINE=InnoDB default charset=utf8 collate=utf8_bin;
|
|
create table Person (
|
id bigint not null auto_increment,
|
createTime datetime not null,
|
updateTime datetime not null,
|
firstName varchar(255),
|
lastName varchar(255),
|
middleName varchar(255),
|
personId varchar(255) not null unique,
|
primary key (id)
|
) ENGINE=InnoDB default charset=utf8 collate=utf8_bin;
|
|
create index idx_Message_createTime on Message (createTime);
|
|
alter table Message
|
add index FK9C2397E7A0F6ED11 (person),
|
add constraint FK9C2397E7A0F6ED11
|
foreign key (person)
|
references Person (id);
|
|
create index idx_Person_lastName on Person (lastName);
|
{"report":{"fcp":857.8000000715256,"ttfb":228.10000002384186,"pageVisibility":"visible","entityId":24914,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":1,"journeyId":"eb006470-8124-4751-90d4-a96718bf1492","navigationType":0,"readyForUser":941.3999999761581,"redirectCount":0,"resourceLoadedEnd":963.3999999761581,"resourceLoadedStart":234.5,"resourceTiming":[{"duration":128.5,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2bu7/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":234.5,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":234.5,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":363,"responseStart":0,"secureConnectionStart":0},{"duration":128.60000002384186,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2bu7/820016/12ta74/8679b4946efa1a0bb029a3a22206fb5d/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":234.70000004768372,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":234.70000004768372,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":363.3000000715256,"responseStart":0,"secureConnectionStart":0},{"duration":138.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/fbf975c0cce4b1abf04784eeae9ba1f4-CDN/lu2bu7/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":235,"connectEnd":235,"connectStart":235,"domainLookupEnd":235,"domainLookupStart":235,"fetchStart":235,"redirectEnd":0,"redirectStart":0,"requestStart":235,"responseEnd":373.5,"responseStart":373.5,"secureConnectionStart":235},{"duration":207.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/099b33461394b8015fc36c0a4b96e19f-CDN/lu2bu7/820016/12ta74/8679b4946efa1a0bb029a3a22206fb5d/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true","startTime":235.10000002384186,"connectEnd":235.10000002384186,"connectStart":235.10000002384186,"domainLookupEnd":235.10000002384186,"domainLookupStart":235.10000002384186,"fetchStart":235.10000002384186,"redirectEnd":0,"redirectStart":0,"requestStart":235.10000002384186,"responseEnd":442.60000002384186,"responseStart":442.60000002384186,"secureConnectionStart":235.10000002384186},{"duration":210.59999990463257,"initiatorType":"script","name":"https://jira.mariadb.org/s/94c15bff32baef80f4096a08aceae8bc-CDN/lu2bu7/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":235.30000007152557,"connectEnd":235.30000007152557,"connectStart":235.30000007152557,"domainLookupEnd":235.30000007152557,"domainLookupStart":235.30000007152557,"fetchStart":235.30000007152557,"redirectEnd":0,"redirectStart":0,"requestStart":235.30000007152557,"responseEnd":445.89999997615814,"responseStart":445.89999997615814,"secureConnectionStart":235.30000007152557},{"duration":211.39999997615814,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":235.5,"connectEnd":235.5,"connectStart":235.5,"domainLookupEnd":235.5,"domainLookupStart":235.5,"fetchStart":235.5,"redirectEnd":0,"redirectStart":0,"requestStart":235.5,"responseEnd":446.89999997615814,"responseStart":446.89999997615814,"secureConnectionStart":235.5},{"duration":212.20000004768372,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":235.60000002384186,"connectEnd":235.60000002384186,"connectStart":235.60000002384186,"domainLookupEnd":235.60000002384186,"domainLookupStart":235.60000002384186,"fetchStart":235.60000002384186,"redirectEnd":0,"redirectStart":0,"requestStart":235.60000002384186,"responseEnd":447.8000000715256,"responseStart":447.8000000715256,"secureConnectionStart":235.60000002384186},{"duration":278.3000000715256,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2bu7/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":235.89999997615814,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":235.89999997615814,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":514.2000000476837,"responseStart":0,"secureConnectionStart":0},{"duration":212.39999997615814,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":236,"connectEnd":236,"connectStart":236,"domainLookupEnd":236,"domainLookupStart":236,"fetchStart":236,"redirectEnd":0,"redirectStart":0,"requestStart":236,"responseEnd":448.39999997615814,"responseStart":448.39999997615814,"secureConnectionStart":236},{"duration":278.10000002384186,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2bu7/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":236.20000004768372,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":236.20000004768372,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":514.3000000715256,"responseStart":0,"secureConnectionStart":0},{"duration":212.79999995231628,"initiatorType":"script","name":"https://jira.mariadb.org/s/3339d87fa2538a859872f2df449bf8d0-CDN/lu2bu7/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":236.30000007152557,"connectEnd":236.30000007152557,"connectStart":236.30000007152557,"domainLookupEnd":236.30000007152557,"domainLookupStart":236.30000007152557,"fetchStart":236.30000007152557,"redirectEnd":0,"redirectStart":0,"requestStart":236.30000007152557,"responseEnd":449.10000002384186,"responseStart":449.10000002384186,"secureConnectionStart":236.30000007152557},{"duration":280.59999990463257,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":237.30000007152557,"connectEnd":237.30000007152557,"connectStart":237.30000007152557,"domainLookupEnd":237.30000007152557,"domainLookupStart":237.30000007152557,"fetchStart":237.30000007152557,"redirectEnd":0,"redirectStart":0,"requestStart":237.30000007152557,"responseEnd":517.8999999761581,"responseStart":517.8999999761581,"secureConnectionStart":237.30000007152557},{"duration":279.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":241.60000002384186,"connectEnd":241.60000002384186,"connectStart":241.60000002384186,"domainLookupEnd":241.60000002384186,"domainLookupStart":241.60000002384186,"fetchStart":241.60000002384186,"redirectEnd":0,"redirectStart":0,"requestStart":241.60000002384186,"responseEnd":521.1000000238419,"responseStart":521.1000000238419,"secureConnectionStart":241.60000002384186},{"duration":21.5,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":540.3999999761581,"connectEnd":540.3999999761581,"connectStart":540.3999999761581,"domainLookupEnd":540.3999999761581,"domainLookupStart":540.3999999761581,"fetchStart":540.3999999761581,"redirectEnd":0,"redirectStart":0,"requestStart":540.3999999761581,"responseEnd":561.8999999761581,"responseStart":561.8999999761581,"secureConnectionStart":540.3999999761581},{"duration":213.5,"initiatorType":"link","name":"https://jira.mariadb.org/s/50bc9be5bfead1a25e72c1a9338c94f6-CDN/lu2bu7/820016/12ta74/e108c7645258ccb43280ed3404e3e949/_/download/contextbatch/css/com.atlassian.jira.plugins.jira-development-integration-plugin:0,-_super,-jira.view.issue,-jira.global,-jira.general,-jira.browse.project,-project.issue.navigator,-atl.general/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":565.8000000715256,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":565.8000000715256,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":779.3000000715256,"responseStart":0,"secureConnectionStart":0},{"duration":213.20000004768372,"initiatorType":"link","name":"https://jira.mariadb.org/s/d5715adaadd168a9002b108b2b039b50-CDN/lu2bu7/820016/12ta74/be4b45e9cec53099498fa61c8b7acba4/_/download/contextbatch/css/jira.project.sidebar,-_super,-project.issue.navigator,-jira.general,-jira.browse.project,-jira.view.issue,-jira.global,-atl.general,-com.atlassian.jira.projects.sidebar.init/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":566.3999999761581,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":566.3999999761581,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":779.6000000238419,"responseStart":0,"secureConnectionStart":0},{"duration":350.2999999523163,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/e65b778d185daf5aee24936755b43da6/_/download/contextbatch/js/browser-metrics-plugin.contrib,-_super,-project.issue.navigator,-jira.view.issue,-atl.general/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":567.2000000476837,"connectEnd":567.2000000476837,"connectStart":567.2000000476837,"domainLookupEnd":567.2000000476837,"domainLookupStart":567.2000000476837,"fetchStart":567.2000000476837,"redirectEnd":0,"redirectStart":0,"requestStart":567.2000000476837,"responseEnd":917.5,"responseStart":917.5,"secureConnectionStart":567.2000000476837},{"duration":392.3000000715256,"initiatorType":"script","name":"https://jira.mariadb.org/s/86ee9bbc76cd1bcd8556fcdcf46241c9-CDN/lu2bu7/820016/12ta74/e108c7645258ccb43280ed3404e3e949/_/download/contextbatch/js/com.atlassian.jira.plugins.jira-development-integration-plugin:0,-_super,-jira.view.issue,-jira.global,-jira.general,-jira.browse.project,-project.issue.navigator,-atl.general/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true","startTime":567.5,"connectEnd":567.5,"connectStart":567.5,"domainLookupEnd":567.5,"domainLookupStart":567.5,"fetchStart":567.5,"redirectEnd":0,"redirectStart":0,"requestStart":567.5,"responseEnd":959.8000000715256,"responseStart":959.8000000715256,"secureConnectionStart":567.5},{"duration":395.59999990463257,"initiatorType":"script","name":"https://jira.mariadb.org/s/f51ef5507eea4c158f257c66c93b2a3f-CDN/lu2bu7/820016/12ta74/be4b45e9cec53099498fa61c8b7acba4/_/download/contextbatch/js/jira.project.sidebar,-_super,-project.issue.navigator,-jira.general,-jira.browse.project,-jira.view.issue,-jira.global,-atl.general,-com.atlassian.jira.projects.sidebar.init/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true","startTime":567.8000000715256,"connectEnd":567.8000000715256,"connectStart":567.8000000715256,"domainLookupEnd":567.8000000715256,"domainLookupStart":567.8000000715256,"fetchStart":567.8000000715256,"redirectEnd":0,"redirectStart":0,"requestStart":567.8000000715256,"responseEnd":963.3999999761581,"responseStart":963.3999999761581,"secureConnectionStart":567.8000000715256},{"duration":212.39999997615814,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":858.1000000238419,"connectEnd":858.1000000238419,"connectStart":858.1000000238419,"domainLookupEnd":858.1000000238419,"domainLookupStart":858.1000000238419,"fetchStart":858.1000000238419,"redirectEnd":0,"redirectStart":0,"requestStart":858.1000000238419,"responseEnd":1070.5,"responseStart":1070.5,"secureConnectionStart":858.1000000238419}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":16,"responseStart":228,"responseEnd":234,"domLoading":231,"domInteractive":1034,"domContentLoadedEventStart":1034,"domContentLoadedEventEnd":1089,"domComplete":1252,"loadEventStart":1252,"loadEventEnd":1253,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":984.2000000476837},{"name":"bigPipe.sidebar-id.end","time":984.8999999761581},{"name":"bigPipe.activity-panel-pipe-id.start","time":985.1000000238419},{"name":"bigPipe.activity-panel-pipe-id.end","time":988.1000000238419},{"name":"activityTabFullyLoaded","time":1110.7000000476837}],"measures":[],"correlationId":"5c1372cccea6d2","effectiveType":"4g","downlink":9.2,"rtt":0,"serverDuration":122,"dbReadsTimeInMs":12,"dbConnsTimeInMs":21,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}
The query has form
m LEFT JOIN p ON ...
LEFT JOIN allows only one join order:
m, p.
(This is because LEFT JOIN needs to find all records in `m`, regardless
of whether they have a match in `p`).
When we use that join order, the ON expression allows to access table `p`
using eq_ref access method (that is, a lookup on a primary key).
However, the condition `p`.`personId` = 'M002649397' is not useful.
If we take a careful look at the WHERE clause, we can see that is only
satisfied when
`p`.`personId` = 'M002649397'
This means that we're not interested in results of LEFT JOIN that have a
NULL-complemented row for table `p`. In other words, we can replace the
LEFT JOIN with INNER JOIN.
Once we have INNER JOIN, we can use both join orders:
When we use the join order "p, m" (like the second EXPLAIN does), we can
to limit the number of rows we get from table `p`.
use condition
The problem here seems to be that the optimizer is unable to convert left
join into inner join.
When the WHERE clause is just
`p`.`personId` = 'M002649397'
then conversion works (check out EXPLAIN EXTENDED - it shows "JOIN")
When the WHERE clause is
'M002649397' IS NULL OR `p`.`personId` = 'M002649397';
then conversion doesn't work.