Type:
Priority:
Resolution:
Fixed
Affects Version/s:
10.5 , 10.6 , 10.7(EOL) , 10.8(EOL) , 10.9(EOL) , 10.10(EOL) , 10.11 , 11.0(EOL) , 11.1(EOL)
mleich produced rr replay traces of something that led to a corrupted database after mariadb-backup --prepare claimed to have completed successfully.
To reproduce the corruption with the attached copy of the backup data.tar.xz
tar xJf data.tar.xz
mariabackup --prepare --target-dir "$(pwd)"/data
mariadbd --innodb-page-size=8k --datadir $(pwd)/data
The database will fail to start up as follows:
10.6 a6c0a2769663850c055e41a83b0cca9dd9ab89a3
2023-05-31 14:11:49 0 [Note] InnoDB: Opened 3 undo tablespaces
2023-05-31 14:11:49 0 [Warning] InnoDB: innodb_undo_tablespaces=0 disables dedicated undo log tablespaces
2023-05-31 14:11:49 0 [ERROR] InnoDB: Trying to read 8192 bytes at 6545408 outside the bounds of the file: .//undo001
2023-05-31 14:11:49 0 [ERROR] InnoDB: File './/undo001' is corrupted
2023-05-31 14:11:49 0 [Note] InnoDB: Retry with innodb_force_recovery=5
2023-05-31 14:11:49 0 [ERROR] InnoDB: Plugin initialization aborted at srv0start.cc[1484] with error Data structure corruption
2023-05-31 14:11:49 0 [Note] InnoDB: Starting shutdown...
2023-05-31 14:11:49 0 [ERROR] Plugin 'InnoDB' init function returned error.
2023-05-31 14:11:49 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
With the following fix, the recovery will succeed:
diff --git a/storage/innobase/log/log0recv.cc b/storage/innobase/log/log0recv.cc
index 2764cc79829..91e87643ab1 100644
--- a/storage/innobase/log/log0recv.cc
+++ b/storage/innobase/log/log0recv.cc
@@ -3645,10 +3645,12 @@ void recv_sys_t::apply(bool last_batch)
if (fil_space_t *space = fil_space_get(id + srv_undo_space_id_start))
{
ut_ad(UT_LIST_GET_LEN(space->chain) == 1);
+ ut_ad(space->recv_size >= t.pages);
fil_node_t *file= UT_LIST_GET_FIRST(space->chain);
ut_ad(file->is_open());
os_file_truncate(file->name, file->handle,
- os_offset_t{t.pages} << srv_page_size_shift, true);
+ os_offset_t{space->recv_size} <<
+ srv_page_size_shift, true);
}
}
}
An analysis of the rr replay trace of mariadb-backup --prepare showed the following:
space->size will be recovered as 1536 by reading the first page of undo001 , at FIL_PAGE_LSN=579965 , in fil_node_t::read_page0() space->recv_size will be recovered as 640, 768, …, 1536 by recv_sys_t::parse() The file undo001 will be incorrectly truncated to t.pages (640) pages by the above code (without the patch).
There will be subsequent calls to write pages after the truncated size. InnoDB is always supposed to extend data files before any writes take place. The server error message is referring to page 799, on which buf_page_t::write_complete() was invoked in the trace.
{"report":{"fcp":1492.3000001907349,"ttfb":486,"pageVisibility":"visible","entityId":122387,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":0.5,"journeyId":"84d4aaf1-20de-44c0-8f2c-841f70e0c4e3","navigationType":0,"readyForUser":1559.7000002861023,"redirectCount":0,"resourceLoadedEnd":1825.2000002861023,"resourceLoadedStart":491.7000002861023,"resourceTiming":[{"duration":263.69999980926514,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":491.7000002861023,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":491.7000002861023,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":755.4000000953674,"responseStart":0,"secureConnectionStart":0},{"duration":263.5,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true&whisper-enabled=true","startTime":492.09999990463257,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":492.09999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":755.5999999046326,"responseStart":0,"secureConnectionStart":0},{"duration":293.7999997138977,"initiatorType":"script","name":"https://jira.mariadb.org/s/0917945aaa57108d00c5076fea35e069-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":492.2000002861023,"connectEnd":492.2000002861023,"connectStart":492.2000002861023,"domainLookupEnd":492.2000002861023,"domainLookupStart":492.2000002861023,"fetchStart":492.2000002861023,"redirectEnd":0,"redirectStart":0,"requestStart":492.2000002861023,"responseEnd":786,"responseStart":786,"secureConnectionStart":492.2000002861023},{"duration":388.30000019073486,"initiatorType":"script","name":"https://jira.mariadb.org/s/2d8175ec2fa4c816e8023260bd8c1786-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true&whisper-enabled=true","startTime":492.5,"connectEnd":492.5,"connectStart":492.5,"domainLookupEnd":492.5,"domainLookupStart":492.5,"fetchStart":492.5,"redirectEnd":0,"redirectStart":0,"requestStart":492.5,"responseEnd":880.8000001907349,"responseStart":880.7000002861023,"secureConnectionStart":492.5},{"duration":408.2999997138977,"initiatorType":"script","name":"https://jira.mariadb.org/s/a9324d6758d385eb45c462685ad88f1d-CDN/lu2cib/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":492.7000002861023,"connectEnd":492.7000002861023,"connectStart":492.7000002861023,"domainLookupEnd":492.7000002861023,"domainLookupStart":492.7000002861023,"fetchStart":492.7000002861023,"redirectEnd":0,"redirectStart":0,"requestStart":492.7000002861023,"responseEnd":901,"responseStart":901,"secureConnectionStart":492.7000002861023},{"duration":418.90000009536743,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":492.90000009536743,"connectEnd":492.90000009536743,"connectStart":492.90000009536743,"domainLookupEnd":492.90000009536743,"domainLookupStart":492.90000009536743,"fetchStart":492.90000009536743,"redirectEnd":0,"redirectStart":0,"requestStart":492.90000009536743,"responseEnd":911.8000001907349,"responseStart":911.8000001907349,"secureConnectionStart":492.90000009536743},{"duration":421.09999990463257,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":493,"connectEnd":493,"connectStart":493,"domainLookupEnd":493,"domainLookupStart":493,"fetchStart":493,"redirectEnd":0,"redirectStart":0,"requestStart":493,"responseEnd":914.0999999046326,"responseStart":914,"secureConnectionStart":493},{"duration":521.8999996185303,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2cib/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":493.2000002861023,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":493.2000002861023,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1015.0999999046326,"responseStart":0,"secureConnectionStart":0},{"duration":422.69999980926514,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":493.30000019073486,"connectEnd":493.30000019073486,"connectStart":493.30000019073486,"domainLookupEnd":493.30000019073486,"domainLookupStart":493.30000019073486,"fetchStart":493.30000019073486,"redirectEnd":0,"redirectStart":0,"requestStart":493.30000019073486,"responseEnd":916,"responseStart":916,"secureConnectionStart":493.30000019073486},{"duration":521.9000000953674,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":493.5,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":493.5,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1015.4000000953674,"responseStart":0,"secureConnectionStart":0},{"duration":427.3999996185303,"initiatorType":"script","name":"https://jira.mariadb.org/s/5d5e8fe91fbc506585e83ea3b62ccc4b-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":493.7000002861023,"connectEnd":493.7000002861023,"connectStart":493.7000002861023,"domainLookupEnd":493.7000002861023,"domainLookupStart":493.7000002861023,"fetchStart":493.7000002861023,"redirectEnd":0,"redirectStart":0,"requestStart":493.7000002861023,"responseEnd":921.0999999046326,"responseStart":921.0999999046326,"secureConnectionStart":493.7000002861023},{"duration":873.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":494.90000009536743,"connectEnd":494.90000009536743,"connectStart":494.90000009536743,"domainLookupEnd":494.90000009536743,"domainLookupStart":494.90000009536743,"fetchStart":494.90000009536743,"redirectEnd":0,"redirectStart":0,"requestStart":494.90000009536743,"responseEnd":1368.4000000953674,"responseStart":1368.4000000953674,"secureConnectionStart":494.90000009536743},{"duration":1330.3000001907349,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":494.90000009536743,"connectEnd":494.90000009536743,"connectStart":494.90000009536743,"domainLookupEnd":494.90000009536743,"domainLookupStart":494.90000009536743,"fetchStart":494.90000009536743,"redirectEnd":0,"redirectStart":0,"requestStart":494.90000009536743,"responseEnd":1825.2000002861023,"responseStart":1825.2000002861023,"secureConnectionStart":494.90000009536743},{"duration":340.59999990463257,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":1028.3000001907349,"connectEnd":1028.3000001907349,"connectStart":1028.3000001907349,"domainLookupEnd":1028.3000001907349,"domainLookupStart":1028.3000001907349,"fetchStart":1028.3000001907349,"redirectEnd":0,"redirectStart":0,"requestStart":1028.3000001907349,"responseEnd":1368.9000000953674,"responseStart":1368.9000000953674,"secureConnectionStart":1028.3000001907349},{"duration":488.90000009536743,"initiatorType":"script","name":"https://www.google-analytics.com/analytics.js","startTime":1417.0999999046326,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":1417.0999999046326,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1906,"responseStart":0,"secureConnectionStart":0}],"fetchStart":1,"domainLookupStart":1,"domainLookupEnd":1,"connectStart":1,"connectEnd":1,"requestStart":260,"responseStart":486,"responseEnd":488,"domLoading":490,"domInteractive":1852,"domContentLoadedEventStart":1852,"domContentLoadedEventEnd":1902,"domComplete":3319,"loadEventStart":3319,"loadEventEnd":3320,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":1827.2000002861023},{"name":"bigPipe.sidebar-id.end","time":1828.0999999046326},{"name":"bigPipe.activity-panel-pipe-id.start","time":1828.2000002861023},{"name":"bigPipe.activity-panel-pipe-id.end","time":1830.5},{"name":"activityTabFullyLoaded","time":1910.8000001907349}],"measures":[],"correlationId":"d6bb878481be53","effectiveType":"4g","downlink":9.9,"rtt":0,"serverDuration":107,"dbReadsTimeInMs":14,"dbConnsTimeInMs":24,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}
Bug
Critical
:MDEV-14795 InnoDB system tablespace cannot be shrunk
