Type:
Bug
Priority:
Critical
Resolution:
Fixed
Affects Version/s:
10.6
mleich provided a copy of a data directory as well as rr replay traces that leads to a recovery failure with an assertion failure.
Unfortunately, the data directory enables encryption and includes a 96 MiB ib_logfile0 that wrapped around once. Because encrypted data does not compress well, a compressed copy of the data directory would be too large to attach here.
The rr replay trace from before the crash is of limited use, because rr replay would report a replay divergence near the end of the trace. Right before the 10.6 version of MDEV-29911 , recovery will appear to succeed:
10.6 1fe830b56a2bd9b12b643d7b39417255215ae5da
2023-05-29 16:45:43 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=142804091,143054310
2023-05-29 16:45:43 0 [Note] InnoDB: Starting final batch to recover 174 pages from redo log.
2023-05-29 16:45:43 0 [Note] InnoDB: Last binlog file './mysql-bin.000001', position 2691230
2023-05-29 16:45:43 0 [Note] InnoDB: Removing encryption and resizing redo log from 100663296 to 4294967296 bytes; LSN=151202977
If I attempt recovery with the fix of MDEV-29911 , I will hit a debug assertion instead:
10.6 f2c17cc9d9bcd634887846d3064bcb71243f9cc0
2023-05-29 16:47:48 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=142804091,143054310
mariadbd: /mariadb/10.6/storage/innobase/log/log0recv.cc:3629: void recv_sys_t::apply(bool): Assertion `!last_batch || recovered_lsn == scanned_lsn' failed.
During the time of the assertion failure, we have recv_sys.recovered_lsn==151202728 and recv_sys.scanned_lsn==151209984 . The largest observed *contiguous_lsn in recv_scan_log_recs() is 150516736 .
In the non-crashing run, recovery only proceeded up to 151202977 . This would seem to suggest that the "successful" run without MDEV-29911 may be incorrect.
MariaDB Server 10.8 and later versions could be unaffected by this exact bug, because the separate log block and log record parsers were unified when MDEV-14425 replaced the 512-byte log blocks with mini-transaction-sized log blocks. Other recovery bugs are possible; in MDEV-31353 there is a recent example.
relates to
MDEV-29911
InnoDB recovery and mariadb-backup --prepare fail to report detailed progress
Closed
{"report":{"fcp":1262.1000003814697,"ttfb":216.5,"pageVisibility":"visible","entityId":122345,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":0.5,"journeyId":"1c536ccc-b6b7-47ae-a478-d1fc56a9002e","navigationType":0,"readyForUser":1337.1999998092651,"redirectCount":0,"resourceLoadedEnd":1774.4000005722046,"resourceLoadedStart":237.19999980926514,"resourceTiming":[{"duration":543,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":237.19999980926514,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":237.19999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":780.1999998092651,"responseStart":0,"secureConnectionStart":0},{"duration":543,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2cib/820016/12ta74/2bf333562ca6724060a9d5f1535471f6/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":237.5,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":237.5,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":780.5,"responseStart":0,"secureConnectionStart":0},{"duration":552,"initiatorType":"script","name":"https://jira.mariadb.org/s/0917945aaa57108d00c5076fea35e069-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":237.69999980926514,"connectEnd":237.69999980926514,"connectStart":237.69999980926514,"domainLookupEnd":237.69999980926514,"domainLookupStart":237.69999980926514,"fetchStart":237.69999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":237.69999980926514,"responseEnd":789.6999998092651,"responseStart":789.6999998092651,"secureConnectionStart":237.69999980926514},{"duration":594.5999994277954,"initiatorType":"script","name":"https://jira.mariadb.org/s/2d8175ec2fa4c816e8023260bd8c1786-CDN/lu2cib/820016/12ta74/2bf333562ca6724060a9d5f1535471f6/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true","startTime":237.9000005722046,"connectEnd":237.9000005722046,"connectStart":237.9000005722046,"domainLookupEnd":237.9000005722046,"domainLookupStart":237.9000005722046,"fetchStart":237.9000005722046,"redirectEnd":0,"redirectStart":0,"requestStart":237.9000005722046,"responseEnd":832.5,"responseStart":832.5,"secureConnectionStart":237.9000005722046},{"duration":598.3999996185303,"initiatorType":"script","name":"https://jira.mariadb.org/s/a9324d6758d385eb45c462685ad88f1d-CDN/lu2cib/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":238.10000038146973,"connectEnd":238.10000038146973,"connectStart":238.10000038146973,"domainLookupEnd":238.10000038146973,"domainLookupStart":238.10000038146973,"fetchStart":238.10000038146973,"redirectEnd":0,"redirectStart":0,"requestStart":238.10000038146973,"responseEnd":836.5,"responseStart":836.5,"secureConnectionStart":238.10000038146973},{"duration":598.6000003814697,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":238.30000019073486,"connectEnd":238.30000019073486,"connectStart":238.30000019073486,"domainLookupEnd":238.30000019073486,"domainLookupStart":238.30000019073486,"fetchStart":238.30000019073486,"redirectEnd":0,"redirectStart":0,"requestStart":238.30000019073486,"responseEnd":836.9000005722046,"responseStart":836.9000005722046,"secureConnectionStart":238.30000019073486},{"duration":598.6999998092651,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":238.60000038146973,"connectEnd":238.60000038146973,"connectStart":238.60000038146973,"domainLookupEnd":238.60000038146973,"domainLookupStart":238.60000038146973,"fetchStart":238.60000038146973,"redirectEnd":0,"redirectStart":0,"requestStart":238.60000038146973,"responseEnd":837.3000001907349,"responseStart":837.3000001907349,"secureConnectionStart":238.60000038146973},{"duration":695.9000005722046,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2cib/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":238.69999980926514,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":238.69999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":934.6000003814697,"responseStart":0,"secureConnectionStart":0},{"duration":598.8999996185303,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":238.9000005722046,"connectEnd":238.9000005722046,"connectStart":238.9000005722046,"domainLookupEnd":238.9000005722046,"domainLookupStart":238.9000005722046,"fetchStart":238.9000005722046,"redirectEnd":0,"redirectStart":0,"requestStart":238.9000005722046,"responseEnd":837.8000001907349,"responseStart":837.8000001907349,"secureConnectionStart":238.9000005722046},{"duration":695.8000001907349,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":239.10000038146973,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":239.10000038146973,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":934.9000005722046,"responseStart":0,"secureConnectionStart":0},{"duration":599.2000007629395,"initiatorType":"script","name":"https://jira.mariadb.org/s/5d5e8fe91fbc506585e83ea3b62ccc4b-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":239.19999980926514,"connectEnd":239.19999980926514,"connectStart":239.19999980926514,"domainLookupEnd":239.19999980926514,"domainLookupStart":239.19999980926514,"fetchStart":239.19999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":239.19999980926514,"responseEnd":838.4000005722046,"responseStart":838.4000005722046,"secureConnectionStart":239.19999980926514},{"duration":1015.5999994277954,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":240.10000038146973,"connectEnd":240.10000038146973,"connectStart":240.10000038146973,"domainLookupEnd":240.10000038146973,"domainLookupStart":240.10000038146973,"fetchStart":240.10000038146973,"redirectEnd":0,"redirectStart":0,"requestStart":240.10000038146973,"responseEnd":1255.6999998092651,"responseStart":1255.6999998092651,"secureConnectionStart":240.10000038146973},{"duration":1534.3000001907349,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":240.10000038146973,"connectEnd":240.10000038146973,"connectStart":240.10000038146973,"domainLookupEnd":240.10000038146973,"domainLookupStart":240.10000038146973,"fetchStart":240.10000038146973,"redirectEnd":0,"redirectStart":0,"requestStart":240.10000038146973,"responseEnd":1774.4000005722046,"responseStart":1774.4000005722046,"secureConnectionStart":240.10000038146973},{"duration":311.5,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":946.5,"connectEnd":946.5,"connectStart":946.5,"domainLookupEnd":946.5,"domainLookupStart":946.5,"fetchStart":946.5,"redirectEnd":0,"redirectStart":0,"requestStart":946.5,"responseEnd":1258,"responseStart":1258,"secureConnectionStart":946.5}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":34,"responseStart":217,"responseEnd":221,"domLoading":235,"domInteractive":1796,"domContentLoadedEventStart":1796,"domContentLoadedEventEnd":1835,"domComplete":2567,"loadEventStart":2567,"loadEventEnd":2569,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":1778.8000001907349},{"name":"bigPipe.sidebar-id.end","time":1779.6999998092651},{"name":"bigPipe.activity-panel-pipe-id.start","time":1779.9000005722046},{"name":"bigPipe.activity-panel-pipe-id.end","time":1782.4000005722046},{"name":"activityTabFullyLoaded","time":1841}],"measures":[],"correlationId":"e236de494d3b77","effectiveType":"4g","downlink":10,"rtt":0,"serverDuration":119,"dbReadsTimeInMs":17,"dbConnsTimeInMs":27,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}