Type:
Priority:
Resolution:
Fixed
Affects Version/s:
10.6.9 , 10.7.5 , 10.8.4 , 10.9.2 , 10.10.1
As part of MDEV-13542 CHECK TABLE code was refactored so that it would avoid crashes due to corrupted data. mleich produced a core dump where something had caused corruption, and subsequently shutdown crashed:
10.6 92032499874259bae7455130958ea7f38c4d53a3
# 2022-08-31T09:36:38 [1748421] | Version: '10.6.10-MariaDB-debug-log' socket: '/dev/shm/rqg/1661956911/39/1_clone/mysql.sock' port: 25364 Source distribution
# 2022-08-31T09:36:38 [1748421] | 2022-08-31 9:34:54 5 [Warning] InnoDB: Cannot save statistics for table test.t8 because file ./test/t8.ibd cannot be decrypted.
# 2022-08-31T09:36:38 [1748421] | 2022-08-31 9:34:55 5 [ERROR] InnoDB: In page 14 of index PRIMARY of table test.t8
# 2022-08-31T09:36:38 [1748421] | InnoDB: broken FIL_PAGE_NEXT link
# 2022-08-31T09:36:38 [1748421] | 2022-08-31 9:34:55 5 [ERROR] InnoDB: In page 42 of index k of table test.t8
# 2022-08-31T09:36:38 [1748421] | InnoDB: broken FIL_PAGE_NEXT link
# 2022-08-31T09:36:38 [1748421] | 2022-08-31 9:35:04 0 [Note] /data/Server_bin/bb-10.6-MDEV-29374_asan/bin/mysqld (initiated by: root[root] @ localhost [127.0.0.1]): Normal shutdown
# 2022-08-31T09:36:38 [1748421] | 2022-08-31 9:35:04 0 [Note] InnoDB: FTS optimize thread exiting.
# 2022-08-31T09:36:38 [1748421] | 2022-08-31 9:35:04 0 [Note] InnoDB: Starting shutdown...
# 2022-08-31T09:36:38 [1748421] | 2022-08-31 9:35:04 0 [Note] InnoDB: Dumping buffer pool(s) to /dev/shm/rqg/1661956911/39/1_clone/data/ib_buffer_pool
# 2022-08-31T09:36:38 [1748421] | 2022-08-31 9:35:04 0 [Note] InnoDB: Restricted to 95 pages due to innodb_buf_pool_dump_pct=25
# 2022-08-31T09:36:38 [1748421] | 2022-08-31 9:35:04 0 [Note] InnoDB: Buffer pool(s) dump completed at 220831 9:35:04
# 2022-08-31T09:36:38 [1748421] | mysqld: /data/Server/bb-10.6-MDEV-29374/storage/innobase/include/sux_lock.h:79: void sux_lock<ssux>::free() [with ssux = ssux_lock_impl<false>]: Assertion `!writer.load(std::memory_order_relaxed)' failed.
The broken FIL_PAGE_NEXT link messages were reported by btr_validate_level() , which is executed as part of non-QUICK CHECK TABLE . The crash appears to occur because we forgot to release the index latch after reporting the corruption:
diff --git a/storage/innobase/btr/btr0btr.cc b/storage/innobase/btr/btr0btr.cc
index 772ac99a5d5..3e48955e85a 100644
--- a/storage/innobase/btr/btr0btr.cc
+++ b/storage/innobase/btr/btr0btr.cc
@@ -4879,6 +4879,7 @@ btr_validate_level(
loop:
if (!block) {
invalid_page:
+ mtr.commit();
func_exit:
mem_heap_free(heap);
return err;
The messages about failing to save persistent statistics seem to be unrelated to this, because that code is not accessing the dict_index_t::lock at all.
When it comes to the cause of the corruption itself, I think that an rr replay trace will be needed.
is caused by
MDEV-13542
Crashing on a corrupted page is unhelpful
Closed
{"report":{"fcp":878,"ttfb":206.59999990463257,"pageVisibility":"visible","entityId":114371,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":1,"journeyId":"a709b0b8-f34d-4c10-8d13-da3e07c3298d","navigationType":0,"readyForUser":961.0999999046326,"redirectCount":0,"resourceLoadedEnd":1085.7999997138977,"resourceLoadedStart":219.09999990463257,"resourceTiming":[{"duration":74.5,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":219.09999990463257,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":219.09999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":293.59999990463257,"responseStart":0,"secureConnectionStart":0},{"duration":74.7999997138977,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true&whisper-enabled=true","startTime":219.5,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":219.5,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":294.2999997138977,"responseStart":0,"secureConnectionStart":0},{"duration":143.40000009536743,"initiatorType":"script","name":"https://jira.mariadb.org/s/0917945aaa57108d00c5076fea35e069-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":219.69999980926514,"connectEnd":219.69999980926514,"connectStart":219.69999980926514,"domainLookupEnd":219.69999980926514,"domainLookupStart":219.69999980926514,"fetchStart":219.69999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":219.69999980926514,"responseEnd":363.09999990463257,"responseStart":363.09999990463257,"secureConnectionStart":219.69999980926514},{"duration":245.7000002861023,"initiatorType":"script","name":"https://jira.mariadb.org/s/2d8175ec2fa4c816e8023260bd8c1786-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true&whisper-enabled=true","startTime":219.89999961853027,"connectEnd":219.89999961853027,"connectStart":219.89999961853027,"domainLookupEnd":219.89999961853027,"domainLookupStart":219.89999961853027,"fetchStart":219.89999961853027,"redirectEnd":0,"redirectStart":0,"requestStart":219.89999961853027,"responseEnd":465.59999990463257,"responseStart":465.59999990463257,"secureConnectionStart":219.89999961853027},{"duration":250.09999990463257,"initiatorType":"script","name":"https://jira.mariadb.org/s/a9324d6758d385eb45c462685ad88f1d-CDN/lu2cib/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":220.09999990463257,"connectEnd":220.09999990463257,"connectStart":220.09999990463257,"domainLookupEnd":220.09999990463257,"domainLookupStart":220.09999990463257,"fetchStart":220.09999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":220.09999990463257,"responseEnd":470.19999980926514,"responseStart":470.19999980926514,"secureConnectionStart":220.09999990463257},{"duration":250.59999990463257,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":220.2999997138977,"connectEnd":220.2999997138977,"connectStart":220.2999997138977,"domainLookupEnd":220.2999997138977,"domainLookupStart":220.2999997138977,"fetchStart":220.2999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":220.2999997138977,"responseEnd":470.8999996185303,"responseStart":470.8999996185303,"secureConnectionStart":220.2999997138977},{"duration":250.89999961853027,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":220.5,"connectEnd":220.5,"connectStart":220.5,"domainLookupEnd":220.5,"domainLookupStart":220.5,"fetchStart":220.5,"redirectEnd":0,"redirectStart":0,"requestStart":220.5,"responseEnd":471.3999996185303,"responseStart":471.2999997138977,"secureConnectionStart":220.5},{"duration":298.09999990463257,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2cib/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":220.69999980926514,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":220.69999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":518.7999997138977,"responseStart":0,"secureConnectionStart":0},{"duration":251,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":220.89999961853027,"connectEnd":220.89999961853027,"connectStart":220.89999961853027,"domainLookupEnd":220.89999961853027,"domainLookupStart":220.89999961853027,"fetchStart":220.89999961853027,"redirectEnd":0,"redirectStart":0,"requestStart":220.89999961853027,"responseEnd":471.8999996185303,"responseStart":471.8999996185303,"secureConnectionStart":220.89999961853027},{"duration":297.7999997138977,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":221.09999990463257,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":221.09999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":518.8999996185303,"responseStart":0,"secureConnectionStart":0},{"duration":251.40000009536743,"initiatorType":"script","name":"https://jira.mariadb.org/s/5d5e8fe91fbc506585e83ea3b62ccc4b-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":221.19999980926514,"connectEnd":221.19999980926514,"connectStart":221.19999980926514,"domainLookupEnd":221.19999980926514,"domainLookupStart":221.19999980926514,"fetchStart":221.19999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":221.19999980926514,"responseEnd":472.59999990463257,"responseStart":472.59999990463257,"secureConnectionStart":221.19999980926514},{"duration":578.2999997138977,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":222.09999990463257,"connectEnd":222.09999990463257,"connectStart":222.09999990463257,"domainLookupEnd":222.09999990463257,"domainLookupStart":222.09999990463257,"fetchStart":222.09999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":222.09999990463257,"responseEnd":800.3999996185303,"responseStart":800.3999996185303,"secureConnectionStart":222.09999990463257},{"duration":859.1999998092651,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":226.59999990463257,"connectEnd":226.59999990463257,"connectStart":226.59999990463257,"domainLookupEnd":226.59999990463257,"domainLookupStart":226.59999990463257,"fetchStart":226.59999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":226.59999990463257,"responseEnd":1085.7999997138977,"responseStart":1085.7999997138977,"secureConnectionStart":226.59999990463257},{"duration":403.09999990463257,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":564.7999997138977,"connectEnd":564.7999997138977,"connectStart":564.7999997138977,"domainLookupEnd":564.7999997138977,"domainLookupStart":564.7999997138977,"fetchStart":564.7999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":564.7999997138977,"responseEnd":967.8999996185303,"responseStart":967.8999996185303,"secureConnectionStart":564.7999997138977},{"duration":215.5,"initiatorType":"script","name":"https://www.google-analytics.com/analytics.js","startTime":871.6999998092651,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":871.6999998092651,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1087.1999998092651,"responseStart":0,"secureConnectionStart":0}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":41,"responseStart":206,"responseEnd":217,"domLoading":216,"domInteractive":1143,"domContentLoadedEventStart":1143,"domContentLoadedEventEnd":1185,"domComplete":2025,"loadEventStart":2025,"loadEventEnd":2027,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":1122.7999997138977},{"name":"bigPipe.sidebar-id.end","time":1123.6999998092651},{"name":"bigPipe.activity-panel-pipe-id.start","time":1123.7999997138977},{"name":"bigPipe.activity-panel-pipe-id.end","time":1126.1999998092651},{"name":"activityTabFullyLoaded","time":1192}],"measures":[],"correlationId":"7a7e0e2adcd7cc","effectiveType":"4g","downlink":10,"rtt":0,"serverDuration":102,"dbReadsTimeInMs":17,"dbConnsTimeInMs":27,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}
Bug
Blocker
