Type:
Bug
Priority:
Major
Resolution:
Fixed
Affects Version/s:
5.5(EOL) , (16) 10.1(EOL) , 10.2(EOL) , 10.3(EOL) , 10.4(EOL) , 10.5 , 10.6 , 10.7(EOL) , 10.8(EOL) , 10.9(EOL) , 10.10(EOL) , 10.11 , 11.0(EOL) , 11.1(EOL) , 11.2(EOL) , 11.3(EOL) , 11.4
In an InnoDB record cursor, there is a counter row_prebuilt_t::n_rows_fetched that counts records that have been fetched in a key range scan. The main purpose of this counter is to control whether to trigger a prefetch of a cache that is slated for removal in MDEV-16232 .
On 64-bit systems, this counter cannot overflow. If we overestimate that InnoDB could read 10¹² or 2⁴⁰ records per second in a range scan, it would take more than half a year of such a range scan before the counter would wrap around. An even more rigid limitation would be that InnoDB only supports 2³² pages per tablespace, and you could fit at most a few thousand or 2¹² records per page, that is, there can't be more than some 2⁴⁴ records per index.
I would propose to revise an overflow check as follows, disabling it on 64-bit systems:
diff --git a/storage/innobase/row/row0sel.cc b/storage/innobase/row/row0sel.cc
index 4504fabd4e0..b217f6d20fe 100644
--- a/storage/innobase/row/row0sel.cc
+++ b/storage/innobase/row/row0sel.cc
@@ -4406,13 +4406,11 @@ row_search_mvcc(
goto func_exit;
}
+# if SIZEOF_SIZE_T < 8
+ if (UNIV_LIKELY(~prebuilt->n_rows_fetched))
+#endif
prebuilt->n_rows_fetched++;
- if (prebuilt->n_rows_fetched > 1000000000) {
- /* Prevent wrap-over */
- prebuilt->n_rows_fetched = 500000000;
- }
-
mode = pcur->search_mode;
}
relates to
MDEV-16232
Use fewer mini-transactions
Stalled
Marko Mäkelä
made changes -
2024-01-16 07:05
Description
In an InnoDB record cursor, there is a counter {{row_prebuilt_t::n_rows_fetched}} that counts records that have been fetched in a key range scan. The main purpose of this counter is to control whether to trigger a prefetch of a cache that is slated for removal in MDEV-16232 .
On 64-bit systems, this counter cannot overflow. If we overestimate that InnoDB could read 10¹² or 2⁴⁰ records per second in a range scan, it would take more than half a year of such a range scan before the counter would wrap around. An even more rigid limitation would be that InnoDB only supports 2³² pages per tablespace, and you could fit at most a few thousand records per page, that is, there can't be more than some 2³⁶ records per index.
I would propose to revise an overflow check as follows, disabling it on 64-bit systems:
{code:c++}
diff --git a/storage/innobase/row/row0sel.cc b/storage/innobase/row/row0sel.cc
index 4504fabd4e0..b217f6d20fe 100644
--- a/storage/innobase/row/row0sel.cc
+++ b/storage/innobase/row/row0sel.cc
@@ -4406,13 +4406,11 @@ row_search_mvcc(
goto func_exit;
}
+#if SIZEOF_SIZE_T < 8
+ if (UNIV_LIKELY(~prebuilt->n_rows_fetched))
+#endif
prebuilt->n_rows_fetched++;
- if (prebuilt->n_rows_fetched > 1000000000) {
- /* Prevent wrap-over */
- prebuilt->n_rows_fetched = 500000000;
- }
-
mode = pcur->search_mode;
}
{code}
In an InnoDB record cursor, there is a counter {{row_prebuilt_t::n_rows_fetched}} that counts records that have been fetched in a key range scan. The main purpose of this counter is to control whether to trigger a prefetch of a cache that is slated for removal in MDEV-16232 .
On 64-bit systems, this counter cannot overflow. If we overestimate that InnoDB could read 10¹² or 2⁴⁰ records per second in a range scan, it would take more than half a year of such a range scan before the counter would wrap around. An even more rigid limitation would be that InnoDB only supports 2³² pages per tablespace, and you could fit at most a few thousand or 2¹² records per page, that is, there can't be more than some 2⁴⁴ records per index.
I would propose to revise an overflow check as follows, disabling it on 64-bit systems:
{code:c++}
diff --git a/storage/innobase/row/row0sel.cc b/storage/innobase/row/row0sel.cc
index 4504fabd4e0..b217f6d20fe 100644
--- a/storage/innobase/row/row0sel.cc
+++ b/storage/innobase/row/row0sel.cc
@@ -4406,13 +4406,11 @@ row_search_mvcc(
goto func_exit;
}
+#if SIZEOF_SIZE_T < 8
+ if (UNIV_LIKELY(~prebuilt->n_rows_fetched))
+#endif
prebuilt->n_rows_fetched++;
- if (prebuilt->n_rows_fetched > 1000000000) {
- /* Prevent wrap-over */
- prebuilt->n_rows_fetched = 500000000;
- }
-
mode = pcur->search_mode;
}
{code}
Marko Mäkelä
made changes -
2024-01-16 08:02
Status
Open
[ 1
]
In Progress
[ 3
]
Marko Mäkelä
made changes -
2024-01-16 08:02
Assignee
Marko Mäkelä
[ marko
]
Vladislav Lesin
[ vlad.lesin
]
Status
In Progress
[ 3
]
In Review
[ 10002
]
Vladislav Lesin
made changes -
2024-01-25 14:59
Assignee
Vladislav Lesin
[ vlad.lesin
]
Marko Mäkelä
[ marko
]
Status
In Review
[ 10002
]
Stalled
[ 10000
]
Marko Mäkelä
made changes -
2024-01-30 12:30
issue.field.resolutiondate
2024-01-30 12:30:25.0
2024-01-30 12:30:24.768
Marko Mäkelä
made changes -
2024-01-30 12:30
Fix Version/s
10.4.34
[ 29625
]
Fix Version/s
10.5.25
[ 29626
]
Fix Version/s
10.6.18
[ 29627
]
Fix Version/s
10.11.8
[ 29630
]
Fix Version/s
11.0.6
[ 29628
]
Fix Version/s
11.1.5
[ 29629
]
Fix Version/s
11.2.4
[ 29631
]
Fix Version/s
11.3.3
[ 29632
]
Fix Version/s
11.4.2
[ 29633
]
Fix Version/s
10.4
[ 22408
]
Fix Version/s
10.5
[ 23123
]
Fix Version/s
10.6
[ 24028
]
Fix Version/s
10.11
[ 27614
]
Fix Version/s
11.0
[ 28320
]
Fix Version/s
11.1
[ 28549
]
Fix Version/s
11.3
[ 28565
]
Fix Version/s
11.2
[ 28603
]
Resolution
Fixed
[ 1
]
Status
Stalled
[ 10000
]
Closed
[ 6
]
Julien Fritsch
made changes -
2024-02-20 09:32
Fix Version/s
11.3.3
[ 29632
]
{"report":{"fcp":1291.7999997138977,"ttfb":300.5,"pageVisibility":"visible","entityId":127280,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":0.5,"journeyId":"9e288e9e-d44c-458f-b110-57b172505721","navigationType":0,"readyForUser":1420.6999998092651,"redirectCount":0,"resourceLoadedEnd":1459.6999998092651,"resourceLoadedStart":306.2999997138977,"resourceTiming":[{"duration":480.90000009536743,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":306.2999997138977,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":306.2999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":787.1999998092651,"responseStart":0,"secureConnectionStart":0},{"duration":481.09999990463257,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true&whisper-enabled=true","startTime":306.5,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":306.5,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":787.5999999046326,"responseStart":0,"secureConnectionStart":0},{"duration":489.80000019073486,"initiatorType":"script","name":"https://jira.mariadb.org/s/0917945aaa57108d00c5076fea35e069-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":306.7999997138977,"connectEnd":306.7999997138977,"connectStart":306.7999997138977,"domainLookupEnd":306.7999997138977,"domainLookupStart":306.7999997138977,"fetchStart":306.7999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":306.7999997138977,"responseEnd":796.5999999046326,"responseStart":796.5999999046326,"secureConnectionStart":306.7999997138977},{"duration":551.2999997138977,"initiatorType":"script","name":"https://jira.mariadb.org/s/2d8175ec2fa4c816e8023260bd8c1786-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true&whisper-enabled=true","startTime":307,"connectEnd":307,"connectStart":307,"domainLookupEnd":307,"domainLookupStart":307,"fetchStart":307,"redirectEnd":0,"redirectStart":0,"requestStart":307,"responseEnd":858.2999997138977,"responseStart":858.2999997138977,"secureConnectionStart":307},{"duration":555,"initiatorType":"script","name":"https://jira.mariadb.org/s/a9324d6758d385eb45c462685ad88f1d-CDN/lu2cib/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":307.2999997138977,"connectEnd":307.2999997138977,"connectStart":307.2999997138977,"domainLookupEnd":307.2999997138977,"domainLookupStart":307.2999997138977,"fetchStart":307.2999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":307.2999997138977,"responseEnd":862.2999997138977,"responseStart":862.2999997138977,"secureConnectionStart":307.2999997138977},{"duration":555.6999998092651,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":307.40000009536743,"connectEnd":307.40000009536743,"connectStart":307.40000009536743,"domainLookupEnd":307.40000009536743,"domainLookupStart":307.40000009536743,"fetchStart":307.40000009536743,"redirectEnd":0,"redirectStart":0,"requestStart":307.40000009536743,"responseEnd":863.0999999046326,"responseStart":863.0999999046326,"secureConnectionStart":307.40000009536743},{"duration":555.9000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":307.59999990463257,"connectEnd":307.59999990463257,"connectStart":307.59999990463257,"domainLookupEnd":307.59999990463257,"domainLookupStart":307.59999990463257,"fetchStart":307.59999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":307.59999990463257,"responseEnd":863.5,"responseStart":863.5,"secureConnectionStart":307.59999990463257},{"duration":639.7000002861023,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2cib/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":307.69999980926514,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":307.69999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":947.4000000953674,"responseStart":0,"secureConnectionStart":0},{"duration":556,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":307.90000009536743,"connectEnd":307.90000009536743,"connectStart":307.90000009536743,"domainLookupEnd":307.90000009536743,"domainLookupStart":307.90000009536743,"fetchStart":307.90000009536743,"redirectEnd":0,"redirectStart":0,"requestStart":307.90000009536743,"responseEnd":863.9000000953674,"responseStart":863.9000000953674,"secureConnectionStart":307.90000009536743},{"duration":639.5,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":308.09999990463257,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":308.09999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":947.5999999046326,"responseStart":0,"secureConnectionStart":0},{"duration":556.3000001907349,"initiatorType":"script","name":"https://jira.mariadb.org/s/5d5e8fe91fbc506585e83ea3b62ccc4b-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":308.19999980926514,"connectEnd":308.19999980926514,"connectStart":308.19999980926514,"domainLookupEnd":308.19999980926514,"domainLookupStart":308.19999980926514,"fetchStart":308.19999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":308.19999980926514,"responseEnd":864.5,"responseStart":864.5,"secureConnectionStart":308.19999980926514},{"duration":658.9000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":309.09999990463257,"connectEnd":309.09999990463257,"connectStart":309.09999990463257,"domainLookupEnd":309.09999990463257,"domainLookupStart":309.09999990463257,"fetchStart":309.09999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":309.09999990463257,"responseEnd":968,"responseStart":968,"secureConnectionStart":309.09999990463257},{"duration":742.4000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":314,"connectEnd":314,"connectStart":314,"domainLookupEnd":314,"domainLookupStart":314,"fetchStart":314,"redirectEnd":0,"redirectStart":0,"requestStart":314,"responseEnd":1056.4000000953674,"responseStart":1056.4000000953674,"secureConnectionStart":314},{"duration":69.2000002861023,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":989.6999998092651,"connectEnd":989.6999998092651,"connectStart":989.6999998092651,"domainLookupEnd":989.6999998092651,"domainLookupStart":989.6999998092651,"fetchStart":989.6999998092651,"redirectEnd":0,"redirectStart":0,"requestStart":989.6999998092651,"responseEnd":1058.9000000953674,"responseStart":1058.9000000953674,"secureConnectionStart":989.6999998092651},{"duration":211.2000002861023,"initiatorType":"link","name":"https://jira.mariadb.org/s/d5715adaadd168a9002b108b2b039b50-CDN/lu2cib/820016/12ta74/be4b45e9cec53099498fa61c8b7acba4/_/download/contextbatch/css/jira.project.sidebar,-_super,-project.issue.navigator,-jira.general,-jira.browse.project,-jira.view.issue,-jira.global,-atl.general,-com.atlassian.jira.projects.sidebar.init/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true&whisper-enabled=true","startTime":1240.2999997138977,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":1240.2999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1451.5,"responseStart":0,"secureConnectionStart":0},{"duration":210.90000009536743,"initiatorType":"link","name":"https://jira.mariadb.org/s/50bc9be5bfead1a25e72c1a9338c94f6-CDN/lu2cib/820016/12ta74/e108c7645258ccb43280ed3404e3e949/_/download/contextbatch/css/com.atlassian.jira.plugins.jira-development-integration-plugin:0,-_super,-jira.view.issue,-jira.global,-jira.general,-jira.browse.project,-project.issue.navigator,-atl.general/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true&whisper-enabled=true","startTime":1240.7999997138977,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":1240.7999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1451.6999998092651,"responseStart":0,"secureConnectionStart":0},{"duration":210.2000002861023,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/e65b778d185daf5aee24936755b43da6/_/download/contextbatch/js/browser-metrics-plugin.contrib,-_super,-project.issue.navigator,-jira.view.issue,-atl.general/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true&whisper-enabled=true","startTime":1241.7999997138977,"connectEnd":1241.7999997138977,"connectStart":1241.7999997138977,"domainLookupEnd":1241.7999997138977,"domainLookupStart":1241.7999997138977,"fetchStart":1241.7999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":1241.7999997138977,"responseEnd":1452,"responseStart":1452,"secureConnectionStart":1241.7999997138977},{"duration":216.09999990463257,"initiatorType":"script","name":"https://jira.mariadb.org/s/097ae97cb8fbec7d6ea4bbb1f26955b9-CDN/lu2cib/820016/12ta74/be4b45e9cec53099498fa61c8b7acba4/_/download/contextbatch/js/jira.project.sidebar,-_super,-project.issue.navigator,-jira.general,-jira.browse.project,-jira.view.issue,-jira.global,-atl.general,-com.atlassian.jira.projects.sidebar.init/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true&whisper-enabled=true","startTime":1242.1999998092651,"connectEnd":1242.1999998092651,"connectStart":1242.1999998092651,"domainLookupEnd":1242.1999998092651,"domainLookupStart":1242.1999998092651,"fetchStart":1242.1999998092651,"redirectEnd":0,"redirectStart":0,"requestStart":1242.1999998092651,"responseEnd":1458.2999997138977,"responseStart":1458.1999998092651,"secureConnectionStart":1242.1999998092651},{"duration":217.19999980926514,"initiatorType":"script","name":"https://jira.mariadb.org/s/e0bf5781d46ea69fb123572974cf39de-CDN/lu2cib/820016/12ta74/e108c7645258ccb43280ed3404e3e949/_/download/contextbatch/js/com.atlassian.jira.plugins.jira-development-integration-plugin:0,-_super,-jira.view.issue,-jira.global,-jira.general,-jira.browse.project,-project.issue.navigator,-atl.general/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true&whisper-enabled=true","startTime":1242.5,"connectEnd":1242.5,"connectStart":1242.5,"domainLookupEnd":1242.5,"domainLookupStart":1242.5,"fetchStart":1242.5,"redirectEnd":0,"redirectStart":0,"requestStart":1242.5,"responseEnd":1459.6999998092651,"responseStart":1459.6999998092651,"secureConnectionStart":1242.5}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":103,"responseStart":300,"responseEnd":306,"domLoading":304,"domInteractive":1501,"domContentLoadedEventStart":1501,"domContentLoadedEventEnd":1559,"domComplete":1652,"loadEventStart":1652,"loadEventEnd":1653,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":1460.4000000953674},{"name":"bigPipe.sidebar-id.end","time":1461.2999997138977},{"name":"bigPipe.activity-panel-pipe-id.start","time":1461.5},{"name":"bigPipe.activity-panel-pipe-id.end","time":1465.6999998092651},{"name":"activityTabFullyLoaded","time":1593.1999998092651}],"measures":[],"correlationId":"b85ef7c1493a70","effectiveType":"4g","downlink":9.6,"rtt":0,"serverDuration":136,"dbReadsTimeInMs":23,"dbConnsTimeInMs":32,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}
LGTM