There is a big table in a statement-based parallel replication setup, like this:
CREATE TABLE `t` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`action` enum('update','create','delete','query') NOT NULL,
`status` enum('unclaimed','claimed','error','completed','paused','cancelled','exemptable','violation','blocked') NOT NULL,
... more columns here ...
`last_change` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`error_message` text CHARACTER SET utf8,
`partition_key` int(11) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
... more multiple-column indexes here ...
) ENGINE=InnoDB AUTO_INCREMENT=3184963639 DEFAULT CHARSET=latin1
select * from information_schema.tables where TABLE_SCHEMA='test' and TABLE_NAME = 't'\G
*************************** 1. row ***************************
TABLE_CATALOG: def
TABLE_SCHEMA: test
TABLE_NAME: t
TABLE_TYPE: BASE TABLE
ENGINE: InnoDB
VERSION: 10
ROW_FORMAT: Compact
TABLE_ROWS: 271226130
AVG_ROW_LENGTH: 227
DATA_LENGTH: 61576822784
MAX_DATA_LENGTH: 0
INDEX_LENGTH: 93808164864
DATA_FREE: 27502051328
AUTO_INCREMENT: 3184963639
CREATE_TIME: 2016-08-05 13:08:55
UPDATE_TIME: NULL
CHECK_TIME: NULL
TABLE_COLLATION: latin1_swedish_ci
CHECKSUM: NULL
CREATE_OPTIONS:
TABLE_COMMENT:
1 row in set (0.00 sec)
Usually status is updated for several ids in a single UPDATE like this:
mysql> EXPLAIN UPDATE test.t SET status = 'claimed' WHERE id IN (3191531520, 3191531548, 3191675779, 3191675794, 3191849444, 3191879324, 3192120630, 3192120649, 3192120666, 3192120681, 3192120694, 3192120710, 3192120722, 3192120732, 3192120742, 3192120754, 3192120765, 3192152806, 3192177807, 3192252972, 3192284688, 3192284434, 3192284713, 3192284461, 3192284485, 3192284748, 3192284506, 3192284763, 3192284777, 3192284528, 3192284790, 3192284556, 3192284574, 3192284590, 3192284615, 3192284368, 3192284634, 3192284383, 3192284651, 3192284400, 3192284415, 3192284671, 3192440124, 3192570421, 3192570439, 3192570456, 3192570471, 3192570487)\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: t
type: range
possible_keys: PRIMARY
key: PRIMARY
key_len: 8
ref: NULL
rows: 48
Extra: Using where
1 row in set (0.00 sec)
So, we have range access and reasonable estimation of rows in range. We may get this plan hundreds of times in a row, but sometimes we end up with a totally different plan while this same query is executed by slave:
2339259 system user test Connect 2594 updating UPDATE t SET status = 'claimed' WHERE id IN (3191531520, 3191531548, 3191675779, 3191675794, 3191849444, 3191879324, 3192120630, 3192120649, 3192120666, 3192120681, 3192120694, 3192120710, 3192120722, 3192120732, 3192120742, 3192120754, 3192120765, 3192152806, 3192177807, 3192252972, 3192284688, 3192284434, 3192284713, 3192284461, 3192284485, 3192284748, 3192284506, 3192284763, 3192284777, 3192284528, 3192284790, 3192284556, 3192284574, 3192284590, 3192284615, 3192284368, 3192284634, 3192284383, 3192284651, 3192284400, 3192284415, 3192284671, 3192440124, 3192570421, 3192570439, 3192570456, 3192570471, 3192570487) 0.000
mysql> SHOW EXPLAIN FOR 2339259\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: t
type: index
possible_keys: PRIMARY
key: PRIMARY
key_len: 8
ref: NULL
rows: 1
Extra: Using where
1 row in set, 1 warning (0.01 sec)
Optimizer used full index scan for the PRIMARY key (and estimated rows as 1), and this caused a very slow execution of the query and millions of locks set in the process that stalled all other parallel replication threads eventually in a "deadlock".
I suspect there is a bug/corner case somewhere in the optimizer when it decides about indexes available for range access. Maybe this is caused by bad InnoDB statistics etc, but in any case I think we should prevent such an execution path.
relates to
MDEV-10790
InnoDB statistics update may temporarily cause wrong index cardinalities
Open
links to
Elena Stepanova
made changes -
2016-08-26 18:16
Field
Original Value
New Value
Fix Version/s
10.1
[ 16100
]
Assignee
Sergei Petrunia
[ psergey
]
Sergei Petrunia
made changes -
2016-09-11 14:59
Component/s
Storage Engine - InnoDB
[ 10129
]
Component/s
Optimizer
[ 10200
]
Sergei Petrunia
made changes -
2016-09-13 13:08
Remote Link
This issue links to "MySQL Bug #82968 (Web Link)"
[ 27502
]
Sergei Golubchik
made changes -
2016-09-27 13:48
Priority
Critical
[ 2
]
Blocker
[ 1
]
Sergei Petrunia
made changes -
2016-09-28 13:20
Fix Version/s
10.1.18
[ 22110
]
Fix Version/s
10.0.28
[ 22107
]
Fix Version/s
10.1
[ 16100
]
Resolution
Fixed
[ 1
]
Status
Open
[ 1
]
Closed
[ 6
]
Sergei Golubchik
made changes -
2021-12-06 21:43
Workflow
MariaDB v3
[ 76768
]
MariaDB v4
[ 150809
]
{"report":{"fcp":1375.4000000953674,"ttfb":534,"pageVisibility":"visible","entityId":57763,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":0.5,"journeyId":"40318eb1-5f69-4be4-98eb-819223181635","navigationType":0,"readyForUser":1474.5,"redirectCount":0,"resourceLoadedEnd":1451.6000003814697,"resourceLoadedStart":543.7000002861023,"resourceTiming":[{"duration":297.69999980926514,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":543.7000002861023,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":543.7000002861023,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":841.4000000953674,"responseStart":0,"secureConnectionStart":0},{"duration":298.1000003814697,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true&whisper-enabled=true","startTime":544,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":544,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":842.1000003814697,"responseStart":0,"secureConnectionStart":0},{"duration":354.69999980926514,"initiatorType":"script","name":"https://jira.mariadb.org/s/0917945aaa57108d00c5076fea35e069-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":544.3000001907349,"connectEnd":544.3000001907349,"connectStart":544.3000001907349,"domainLookupEnd":544.3000001907349,"domainLookupStart":544.3000001907349,"fetchStart":544.3000001907349,"redirectEnd":0,"redirectStart":0,"requestStart":544.3000001907349,"responseEnd":899,"responseStart":899,"secureConnectionStart":544.3000001907349},{"duration":409.80000019073486,"initiatorType":"script","name":"https://jira.mariadb.org/s/2d8175ec2fa4c816e8023260bd8c1786-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true&whisper-enabled=true","startTime":544.8000001907349,"connectEnd":544.8000001907349,"connectStart":544.8000001907349,"domainLookupEnd":544.8000001907349,"domainLookupStart":544.8000001907349,"fetchStart":544.8000001907349,"redirectEnd":0,"redirectStart":0,"requestStart":544.8000001907349,"responseEnd":954.6000003814697,"responseStart":954.6000003814697,"secureConnectionStart":544.8000001907349},{"duration":413.7000002861023,"initiatorType":"script","name":"https://jira.mariadb.org/s/a9324d6758d385eb45c462685ad88f1d-CDN/lu2cib/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":545,"connectEnd":545,"connectStart":545,"domainLookupEnd":545,"domainLookupStart":545,"fetchStart":545,"redirectEnd":0,"redirectStart":0,"requestStart":545,"responseEnd":958.7000002861023,"responseStart":958.7000002861023,"secureConnectionStart":545},{"duration":413.69999980926514,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":545.3000001907349,"connectEnd":545.3000001907349,"connectStart":545.3000001907349,"domainLookupEnd":545.3000001907349,"domainLookupStart":545.3000001907349,"fetchStart":545.3000001907349,"redirectEnd":0,"redirectStart":0,"requestStart":545.3000001907349,"responseEnd":959,"responseStart":959,"secureConnectionStart":545.3000001907349},{"duration":414,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":545.5,"connectEnd":545.5,"connectStart":545.5,"domainLookupEnd":545.5,"domainLookupStart":545.5,"fetchStart":545.5,"redirectEnd":0,"redirectStart":0,"requestStart":545.5,"responseEnd":959.5,"responseStart":959.5,"secureConnectionStart":545.5},{"duration":497.40000009536743,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2cib/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":545.7000002861023,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":545.7000002861023,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1043.1000003814697,"responseStart":0,"secureConnectionStart":0},{"duration":414.5,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":545.8000001907349,"connectEnd":545.8000001907349,"connectStart":545.8000001907349,"domainLookupEnd":545.8000001907349,"domainLookupStart":545.8000001907349,"fetchStart":545.8000001907349,"redirectEnd":0,"redirectStart":0,"requestStart":545.8000001907349,"responseEnd":960.3000001907349,"responseStart":960.3000001907349,"secureConnectionStart":545.8000001907349},{"duration":497.30000019073486,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":546,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":546,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1043.3000001907349,"responseStart":0,"secureConnectionStart":0},{"duration":414.8999996185303,"initiatorType":"script","name":"https://jira.mariadb.org/s/5d5e8fe91fbc506585e83ea3b62ccc4b-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":546.1000003814697,"connectEnd":546.1000003814697,"connectStart":546.1000003814697,"domainLookupEnd":546.1000003814697,"domainLookupStart":546.1000003814697,"fetchStart":546.1000003814697,"redirectEnd":0,"redirectStart":0,"requestStart":546.1000003814697,"responseEnd":961,"responseStart":961,"secureConnectionStart":546.1000003814697},{"duration":674.6999998092651,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":558.3000001907349,"connectEnd":558.3000001907349,"connectStart":558.3000001907349,"domainLookupEnd":558.3000001907349,"domainLookupStart":558.3000001907349,"fetchStart":558.3000001907349,"redirectEnd":0,"redirectStart":0,"requestStart":558.3000001907349,"responseEnd":1233,"responseStart":1233,"secureConnectionStart":558.3000001907349},{"duration":847.5999999046326,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":562.3000001907349,"connectEnd":562.3000001907349,"connectStart":562.3000001907349,"domainLookupEnd":562.3000001907349,"domainLookupStart":562.3000001907349,"fetchStart":562.3000001907349,"redirectEnd":0,"redirectStart":0,"requestStart":562.3000001907349,"responseEnd":1409.9000000953674,"responseStart":1409.9000000953674,"secureConnectionStart":562.3000001907349},{"duration":178.7999997138977,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":1055.1000003814697,"connectEnd":1055.1000003814697,"connectStart":1055.1000003814697,"domainLookupEnd":1055.1000003814697,"domainLookupStart":1055.1000003814697,"fetchStart":1055.1000003814697,"redirectEnd":0,"redirectStart":0,"requestStart":1055.1000003814697,"responseEnd":1233.9000000953674,"responseStart":1233.9000000953674,"secureConnectionStart":1055.1000003814697},{"duration":137.90000009536743,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/e65b778d185daf5aee24936755b43da6/_/download/contextbatch/js/browser-metrics-plugin.contrib,-_super,-atl.general/batch.js?agile_global_admin_condition=true&jag=true&slack-enabled=true&whisper-enabled=true","startTime":1313.7000002861023,"connectEnd":1313.7000002861023,"connectStart":1313.7000002861023,"domainLookupEnd":1313.7000002861023,"domainLookupStart":1313.7000002861023,"fetchStart":1313.7000002861023,"redirectEnd":0,"redirectStart":0,"requestStart":1313.7000002861023,"responseEnd":1451.6000003814697,"responseStart":1451.6000003814697,"secureConnectionStart":1313.7000002861023},{"duration":193.59999990463257,"initiatorType":"script","name":"https://www.google-analytics.com/analytics.js","startTime":1368.6000003814697,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":1368.6000003814697,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1562.2000002861023,"responseStart":0,"secureConnectionStart":0}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":301,"responseStart":534,"responseEnd":562,"domLoading":538,"domInteractive":1600,"domContentLoadedEventStart":1600,"domContentLoadedEventEnd":1657,"domComplete":1898,"loadEventStart":1898,"loadEventEnd":1899,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":1568.2000002861023},{"name":"bigPipe.sidebar-id.end","time":1569},{"name":"bigPipe.activity-panel-pipe-id.start","time":1569.1000003814697},{"name":"bigPipe.activity-panel-pipe-id.end","time":1575.1000003814697},{"name":"activityTabFullyLoaded","time":1690.5}],"measures":[],"correlationId":"6037fa3b9f85d6","effectiveType":"4g","downlink":9.4,"rtt":0,"serverDuration":160,"dbReadsTimeInMs":33,"dbConnsTimeInMs":44,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}
Attached files
10.0-debug-printouts.diff includes
mdev10649.test