Type:
Bug
Priority:
Major
Resolution:
Fixed
Affects Version/s:
5.5(EOL) , (16) 10.1(EOL) , 10.2(EOL) , 10.3(EOL) , 10.4(EOL) , 10.5 , 10.6 , 10.7(EOL) , 10.8(EOL) , 10.9(EOL) , 10.10(EOL) , 10.11 , 11.0(EOL) , 11.1(EOL) , 11.2(EOL) , 11.3(EOL) , 11.4
In an InnoDB record cursor, there is a counter row_prebuilt_t::n_rows_fetched that counts records that have been fetched in a key range scan. The main purpose of this counter is to control whether to trigger a prefetch of a cache that is slated for removal in MDEV-16232 .
On 64-bit systems, this counter cannot overflow. If we overestimate that InnoDB could read 10¹² or 2⁴⁰ records per second in a range scan, it would take more than half a year of such a range scan before the counter would wrap around. An even more rigid limitation would be that InnoDB only supports 2³² pages per tablespace, and you could fit at most a few thousand or 2¹² records per page, that is, there can't be more than some 2⁴⁴ records per index.
I would propose to revise an overflow check as follows, disabling it on 64-bit systems:
diff --git a/storage/innobase/row/row0sel.cc b/storage/innobase/row/row0sel.cc
index 4504fabd4e0..b217f6d20fe 100644
--- a/storage/innobase/row/row0sel.cc
+++ b/storage/innobase/row/row0sel.cc
@@ -4406,13 +4406,11 @@ row_search_mvcc(
goto func_exit;
}
+# if SIZEOF_SIZE_T < 8
+ if (UNIV_LIKELY(~prebuilt->n_rows_fetched))
+#endif
prebuilt->n_rows_fetched++;
- if (prebuilt->n_rows_fetched > 1000000000) {
- /* Prevent wrap-over */
- prebuilt->n_rows_fetched = 500000000;
- }
-
mode = pcur->search_mode;
}
relates to
MDEV-16232
Use fewer mini-transactions
Stalled
Marko Mäkelä
made changes -
2024-01-16 07:05
Description
In an InnoDB record cursor, there is a counter {{row_prebuilt_t::n_rows_fetched}} that counts records that have been fetched in a key range scan. The main purpose of this counter is to control whether to trigger a prefetch of a cache that is slated for removal in MDEV-16232 .
On 64-bit systems, this counter cannot overflow. If we overestimate that InnoDB could read 10¹² or 2⁴⁰ records per second in a range scan, it would take more than half a year of such a range scan before the counter would wrap around. An even more rigid limitation would be that InnoDB only supports 2³² pages per tablespace, and you could fit at most a few thousand records per page, that is, there can't be more than some 2³⁶ records per index.
I would propose to revise an overflow check as follows, disabling it on 64-bit systems:
{code:c++}
diff --git a/storage/innobase/row/row0sel.cc b/storage/innobase/row/row0sel.cc
index 4504fabd4e0..b217f6d20fe 100644
--- a/storage/innobase/row/row0sel.cc
+++ b/storage/innobase/row/row0sel.cc
@@ -4406,13 +4406,11 @@ row_search_mvcc(
goto func_exit;
}
+#if SIZEOF_SIZE_T < 8
+ if (UNIV_LIKELY(~prebuilt->n_rows_fetched))
+#endif
prebuilt->n_rows_fetched++;
- if (prebuilt->n_rows_fetched > 1000000000) {
- /* Prevent wrap-over */
- prebuilt->n_rows_fetched = 500000000;
- }
-
mode = pcur->search_mode;
}
{code}
In an InnoDB record cursor, there is a counter {{row_prebuilt_t::n_rows_fetched}} that counts records that have been fetched in a key range scan. The main purpose of this counter is to control whether to trigger a prefetch of a cache that is slated for removal in MDEV-16232 .
On 64-bit systems, this counter cannot overflow. If we overestimate that InnoDB could read 10¹² or 2⁴⁰ records per second in a range scan, it would take more than half a year of such a range scan before the counter would wrap around. An even more rigid limitation would be that InnoDB only supports 2³² pages per tablespace, and you could fit at most a few thousand or 2¹² records per page, that is, there can't be more than some 2⁴⁴ records per index.
I would propose to revise an overflow check as follows, disabling it on 64-bit systems:
{code:c++}
diff --git a/storage/innobase/row/row0sel.cc b/storage/innobase/row/row0sel.cc
index 4504fabd4e0..b217f6d20fe 100644
--- a/storage/innobase/row/row0sel.cc
+++ b/storage/innobase/row/row0sel.cc
@@ -4406,13 +4406,11 @@ row_search_mvcc(
goto func_exit;
}
+#if SIZEOF_SIZE_T < 8
+ if (UNIV_LIKELY(~prebuilt->n_rows_fetched))
+#endif
prebuilt->n_rows_fetched++;
- if (prebuilt->n_rows_fetched > 1000000000) {
- /* Prevent wrap-over */
- prebuilt->n_rows_fetched = 500000000;
- }
-
mode = pcur->search_mode;
}
{code}
Marko Mäkelä
made changes -
2024-01-16 08:02
Status
Open
[ 1
]
In Progress
[ 3
]
Marko Mäkelä
made changes -
2024-01-16 08:02
Assignee
Marko Mäkelä
[ marko
]
Vladislav Lesin
[ vlad.lesin
]
Status
In Progress
[ 3
]
In Review
[ 10002
]
Vladislav Lesin
made changes -
2024-01-25 14:59
Assignee
Vladislav Lesin
[ vlad.lesin
]
Marko Mäkelä
[ marko
]
Status
In Review
[ 10002
]
Stalled
[ 10000
]
Marko Mäkelä
made changes -
2024-01-30 12:30
issue.field.resolutiondate
2024-01-30 12:30:25.0
2024-01-30 12:30:24.768
Marko Mäkelä
made changes -
2024-01-30 12:30
Fix Version/s
10.4.34
[ 29625
]
Fix Version/s
10.5.25
[ 29626
]
Fix Version/s
10.6.18
[ 29627
]
Fix Version/s
10.11.8
[ 29630
]
Fix Version/s
11.0.6
[ 29628
]
Fix Version/s
11.1.5
[ 29629
]
Fix Version/s
11.2.4
[ 29631
]
Fix Version/s
11.3.3
[ 29632
]
Fix Version/s
11.4.2
[ 29633
]
Fix Version/s
10.4
[ 22408
]
Fix Version/s
10.5
[ 23123
]
Fix Version/s
10.6
[ 24028
]
Fix Version/s
10.11
[ 27614
]
Fix Version/s
11.0
[ 28320
]
Fix Version/s
11.1
[ 28549
]
Fix Version/s
11.3
[ 28565
]
Fix Version/s
11.2
[ 28603
]
Resolution
Fixed
[ 1
]
Status
Stalled
[ 10000
]
Closed
[ 6
]
Julien Fritsch
made changes -
2024-02-20 09:32
Fix Version/s
11.3.3
[ 29632
]
{"report":{"fcp":1601.0999999046326,"ttfb":363.69999980926514,"pageVisibility":"visible","entityId":127280,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":32,"apdex":0.5,"journeyId":"5c842be1-9a0e-4c31-a649-1eff5db0e767","navigationType":0,"readyForUser":1729.5,"redirectCount":0,"resourceLoadedEnd":1102.7999997138977,"resourceLoadedStart":388.69999980926514,"resourceTiming":[{"duration":28.59999990463257,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":388.69999980926514,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":388.69999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":417.2999997138977,"responseStart":0,"secureConnectionStart":0},{"duration":37.69999980926514,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true&whisper-enabled=true","startTime":388.90000009536743,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":388.90000009536743,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":426.59999990463257,"responseStart":0,"secureConnectionStart":0},{"duration":405,"initiatorType":"script","name":"https://jira.mariadb.org/s/0917945aaa57108d00c5076fea35e069-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":389.19999980926514,"connectEnd":389.19999980926514,"connectStart":389.19999980926514,"domainLookupEnd":389.19999980926514,"domainLookupStart":389.19999980926514,"fetchStart":389.19999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":430.69999980926514,"responseEnd":794.1999998092651,"responseStart":451.90000009536743,"secureConnectionStart":389.19999980926514},{"duration":672.2999997138977,"initiatorType":"script","name":"https://jira.mariadb.org/s/2d8175ec2fa4c816e8023260bd8c1786-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true&whisper-enabled=true","startTime":389.40000009536743,"connectEnd":389.40000009536743,"connectStart":389.40000009536743,"domainLookupEnd":389.40000009536743,"domainLookupStart":389.40000009536743,"fetchStart":389.40000009536743,"redirectEnd":0,"redirectStart":0,"requestStart":430.40000009536743,"responseEnd":1061.6999998092651,"responseStart":446.40000009536743,"secureConnectionStart":389.40000009536743},{"duration":36.40000009536743,"initiatorType":"script","name":"https://jira.mariadb.org/s/a9324d6758d385eb45c462685ad88f1d-CDN/lu2cib/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":389.59999990463257,"connectEnd":389.59999990463257,"connectStart":389.59999990463257,"domainLookupEnd":389.59999990463257,"domainLookupStart":389.59999990463257,"fetchStart":389.59999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":389.59999990463257,"responseEnd":426,"responseStart":426,"secureConnectionStart":389.59999990463257},{"duration":36.90000009536743,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":389.69999980926514,"connectEnd":389.69999980926514,"connectStart":389.69999980926514,"domainLookupEnd":389.69999980926514,"domainLookupStart":389.69999980926514,"fetchStart":389.69999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":389.69999980926514,"responseEnd":426.59999990463257,"responseStart":426.59999990463257,"secureConnectionStart":389.69999980926514},{"duration":37.10000038146973,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":389.7999997138977,"connectEnd":389.7999997138977,"connectStart":389.7999997138977,"domainLookupEnd":389.7999997138977,"domainLookupStart":389.7999997138977,"fetchStart":389.7999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":389.7999997138977,"responseEnd":426.90000009536743,"responseStart":426.90000009536743,"secureConnectionStart":389.7999997138977},{"duration":38.69999980926514,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2cib/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":390.09999990463257,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":390.09999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":428.7999997138977,"responseStart":0,"secureConnectionStart":0},{"duration":37.200000286102295,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":390.2999997138977,"connectEnd":390.2999997138977,"connectStart":390.2999997138977,"domainLookupEnd":390.2999997138977,"domainLookupStart":390.2999997138977,"fetchStart":390.2999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":390.2999997138977,"responseEnd":427.5,"responseStart":427.5,"secureConnectionStart":390.2999997138977},{"duration":38.5,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":390.40000009536743,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":390.40000009536743,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":428.90000009536743,"responseStart":0,"secureConnectionStart":0},{"duration":39.19999980926514,"initiatorType":"script","name":"https://jira.mariadb.org/s/5d5e8fe91fbc506585e83ea3b62ccc4b-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":390.5,"connectEnd":390.5,"connectStart":390.5,"domainLookupEnd":390.5,"domainLookupStart":390.5,"fetchStart":390.5,"redirectEnd":0,"redirectStart":0,"requestStart":390.5,"responseEnd":429.69999980926514,"responseStart":429.69999980926514,"secureConnectionStart":390.5},{"duration":692.2000002861023,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":408.69999980926514,"connectEnd":408.69999980926514,"connectStart":408.69999980926514,"domainLookupEnd":408.69999980926514,"domainLookupStart":408.69999980926514,"fetchStart":408.69999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":457.90000009536743,"responseEnd":1100.9000000953674,"responseStart":1075.1999998092651,"secureConnectionStart":408.69999980926514},{"duration":694,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":408.7999997138977,"connectEnd":408.7999997138977,"connectStart":408.7999997138977,"domainLookupEnd":408.7999997138977,"domainLookupStart":408.7999997138977,"fetchStart":408.7999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":528.5999999046326,"responseEnd":1102.7999997138977,"responseStart":1093.5,"secureConnectionStart":408.7999997138977},{"duration":182.2000002861023,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":956.6999998092651,"connectEnd":956.6999998092651,"connectStart":956.6999998092651,"domainLookupEnd":956.6999998092651,"domainLookupStart":956.6999998092651,"fetchStart":956.6999998092651,"redirectEnd":0,"redirectStart":0,"requestStart":1106.6999998092651,"responseEnd":1138.9000000953674,"responseStart":1137.0999999046326,"secureConnectionStart":956.6999998092651}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":67,"responseStart":364,"responseEnd":412,"domLoading":368,"domInteractive":1863,"domContentLoadedEventStart":1863,"domContentLoadedEventEnd":1922,"domComplete":2301,"loadEventStart":2301,"loadEventEnd":2301,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":1821.9000000953674},{"name":"bigPipe.sidebar-id.end","time":1822.6999998092651},{"name":"bigPipe.activity-panel-pipe-id.start","time":1822.7999997138977},{"name":"bigPipe.activity-panel-pipe-id.end","time":1828.0999999046326},{"name":"activityTabFullyLoaded","time":1941.1999998092651}],"measures":[],"correlationId":"ffcc91b1a10d6","effectiveType":"4g","downlink":10,"rtt":0,"serverDuration":154,"dbReadsTimeInMs":21,"dbConnsTimeInMs":31,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}