Work flow:
|
1. Start server
|
2. 33 sessions run concurrend DDL and DML
|
3. At some point of time SIGKILL gets sent to the server during 2. is ongoing
|
4. Attempt to restart the server
|
|
# 2020-07-17T19:57:26 [99933] | [rr 15697 73975]2020-07-17 19:56:49 0 [Note] Recovering after a crash using mysql-bin
|
# 2020-07-17T19:57:26 [99933] | [rr 15697 82508]2020-07-17 19:56:53 0 [ERROR] InnoDB: n recs wrong 2817 2816
|
# 2020-07-17T19:57:26 [99933] | [rr 15697 82512]2020-07-17 19:56:53 0 [Note] InnoDB: Uncompressed page, stored checksum in field1 673628625, calculated checksums for field1: crc32 673628625, innodb 1039194117, page type 17855 == INDEX.none 3735928559, stored checksum in field2 673628625, calculated checksums for field2: crc32 673628625, innodb 500201189, none 3735928559, page LSN 0 33227406, low 4 bytes of LSN at page end 33227406, page number (if stored to page already) 42, space id (if created with >= MySQL-4.1.1 and stored already) 12
|
# 2020-07-17T19:57:26 [99933] | [rr 15697 82516]2020-07-17 19:56:53 0 [Note] InnoDB: Page may be an index page where index id is 40
|
# 2020-07-17T19:57:26 [99933] | [rr 15697 82520]2020-07-17 19:56:53 0 [Note] InnoDB: Index 40 is `k` in table `test`.`t8`
|
# 2020-07-17T19:57:26 [99933] | [rr 15697 82524]2020-07-17 19:56:53 0 [ERROR] [FATAL] InnoDB: Apparent corruption of an index page [page id: space=12, page number=42] to be written to data file. We intentionally crash the server to prevent corrupt data from ending up in data files.
|
# 2020-07-17T19:57:26 [99933] | [rr 15697 82543]200717 19:56:53 [rr 15697 82546][ERROR] mysqld got signal 6 ;
|
|
(rr) bt
|
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
|
#1 0x0000640d09c208b1 in __GI_abort () at abort.c:79
|
#2 0x000055754a97a0ae in ib::fatal::~fatal (this=<optimized out>, __in_chrg=<optimized out>) at /home/mleich/bb-10.4-thiru/storage/innobase/ut/ut0ut.cc:601
|
#3 0x000055754aa15ecf in buf_dblwr_assert_on_corrupt_block (block=block@entry=0x757b0cbf4a60) at /home/mleich/bb-10.4-thiru/storage/innobase/buf/buf0dblwr.cc:843
|
#4 0x000055754aa16008 in buf_dblwr_check_block (block=block@entry=0x757b0cbf4a60) at /home/mleich/bb-10.4-thiru/storage/innobase/buf/buf0dblwr.cc:898
|
#5 0x000055754aa1751d in buf_dblwr_flush_buffered_writes () at /home/mleich/bb-10.4-thiru/storage/innobase/buf/buf0dblwr.cc:1036
|
#6 0x000055754aa18654 in buf_dblwr_add_to_batch (bpage=bpage@entry=0x757b0cc11b30) at /home/mleich/bb-10.4-thiru/storage/innobase/buf/buf0dblwr.cc:1170
|
#7 0x000055754aa27cac in buf_flush_write_block_low (bpage=bpage@entry=0x757b0cc11b30, flush_type=flush_type@entry=BUF_FLUSH_LRU, sync=sync@entry=false) at /home/mleich/bb-10.4-thiru/storage/innobase/buf/buf0flu.cc:1049
|
#8 0x000055754aa2aace in buf_flush_page (buf_pool=buf_pool@entry=0x55754d4415d0, bpage=bpage@entry=0x757b0cc11b30, flush_type=flush_type@entry=BUF_FLUSH_LRU, sync=sync@entry=false) at /home/mleich/bb-10.4-thiru/storage/innobase/buf/buf0flu.cc:1197
|
#9 0x000055754aa2c877 in buf_flush_try_neighbors (page_id=page_id@entry=..., flush_type=flush_type@entry=BUF_FLUSH_LRU, n_flushed=119, n_to_flush=n_to_flush@entry=548) at /home/mleich/bb-10.4-thiru/storage/innobase/buf/buf0flu.cc:1423
|
#10 0x000055754aa2cb2b in buf_flush_page_and_try_neighbors (bpage=bpage@entry=0x757b0cc0b570, flush_type=flush_type@entry=BUF_FLUSH_LRU, n_to_flush=n_to_flush@entry=548, count=count@entry=0x3dff6cc42c60)
|
at /home/mleich/bb-10.4-thiru/storage/innobase/buf/buf0flu.cc:1497
|
#11 0x000055754aa2d30d in buf_flush_LRU_list_batch (buf_pool=buf_pool@entry=0x55754d4415d0, max=548, n=n@entry=0x3dff6cc42c60) at /home/mleich/bb-10.4-thiru/storage/innobase/buf/buf0flu.cc:1631
|
#12 0x000055754aa2d93a in buf_do_LRU_batch (buf_pool=buf_pool@entry=0x55754d4415d0, max=max@entry=548, n=n@entry=0x3dff6cc42c60) at /home/mleich/bb-10.4-thiru/storage/innobase/buf/buf0flu.cc:1694
|
#13 0x000055754aa2db52 in buf_flush_batch (buf_pool=buf_pool@entry=0x55754d4415d0, flush_type=flush_type@entry=BUF_FLUSH_LRU, min_n=min_n@entry=548, lsn_limit=lsn_limit@entry=0, n=n@entry=0x3dff6cc42c60)
|
at /home/mleich/bb-10.4-thiru/storage/innobase/buf/buf0flu.cc:1823
|
#14 0x000055754aa2dd44 in buf_flush_do_batch (buf_pool=buf_pool@entry=0x55754d4415d0, type=type@entry=BUF_FLUSH_LRU, min_n=548, lsn_limit=lsn_limit@entry=0, n=n@entry=0x3dff6cc42c60) at /home/mleich/bb-10.4-thiru/storage/innobase/buf/buf0flu.cc:1984
|
#15 0x000055754aa2e12b in buf_flush_LRU_list (buf_pool=buf_pool@entry=0x55754d4415d0) at /home/mleich/bb-10.4-thiru/storage/innobase/buf/buf0flu.cc:2246
|
#16 0x000055754aa2e2a4 in pc_flush_slot () at /home/mleich/bb-10.4-thiru/storage/innobase/buf/buf0flu.cc:2767
|
#17 0x000055754aa2f9e3 in buf_flush_page_cleaner_coordinator () at /home/mleich/bb-10.4-thiru/storage/innobase/buf/buf0flu.cc:3196
|
#18 0x00006a93691756db in start_thread (arg=0x3dff6cc43700) at pthread_create.c:463
|
#19 0x0000640d09d01a3f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
(rr)
|
|
Observed on
|
- origin/bb-10.4-thiru a13662dac1df224c607983815b2fe85ee72749c2 2020-07-17T22:49:03+05:30
|
- some actual 10.5.5 ~ 2020-07-13
|
|
RQG
|
git clone https://github.com/mleich1/rqg --branch experimental RQG
|
origin/experimental f3d72ce51b150ee1f52a93d18f2024af293d925e 2020-07-13T20:46:53+02:00
|
|
perl rqg.pl \
|
--grammar=conf/mariadb/innodb_compression_encryption.yy \
|
--gendata=conf/mariadb/innodb_compression_encryption.zz \
|
--mysqld=--plugin-load-add=file_key_management.so \
|
--mysqld=--loose-file-key-management-filename=/home/mleich/RQG_mleich/conf/mariadb/encryption_keys.txt \
|
--mysqld=--innodb-encrypt-log \
|
--mysqld=--innodb-encrypt-tables \
|
--mysqld=--innodb_use_native_aio=0 \
|
--mysqld=--innodb_lock_schedule_algorithm=fcfs \
|
--mysqld=--loose-idle_write_transaction_timeout=0 \
|
--mysqld=--loose-idle_transaction_timeout=0 \
|
--mysqld=--loose-idle_readonly_transaction_timeout=0 \
|
--mysqld=--connect_timeout=60 \
|
--mysqld=--interactive_timeout=28800 \
|
--mysqld=--slave_net_timeout=60 \
|
--mysqld=--net_read_timeout=30 \
|
--mysqld=--net_write_timeout=60 \
|
--mysqld=--loose-table_lock_wait_timeout=50 \
|
--mysqld=--wait_timeout=28800 \
|
--mysqld=--lock-wait-timeout=86400 \
|
--mysqld=--innodb-lock-wait-timeout=50 \
|
--no-mask \
|
--queries=10000000 \
|
--seed=random \
|
--reporters=Backtrace \
|
--reporters=ErrorLog \
|
--reporters=Deadlock1 \
|
--reporters=CrashRecovery1 \
|
--validators=None \
|
--mysqld=--log_output=none \
|
--mysqld=--log-bin \
|
--mysqld=--log_bin_trust_function_creators=1 \
|
--mysqld=--loose-max-statement-time=30 \
|
--mysqld=--loose-debug_assert_on_not_freed_memory=0 \
|
--engine=InnoDB \
|
--restart_timeout=120 \
|
--duration=300 \
|
--mysqld=--loose-innodb_fatal_semaphore_wait_threshold=300 \
|
--threads=33 \
|
--mysqld=--innodb_page_size=64K \
|
--mysqld=--innodb-buffer-pool-size=256M \
|
--duration=300 \
|
--no_mask \
|
--workdir=<local settings> \
|
--vardir=<local settings> \
|
--mtr-build-thread=<local settings> \
|
--basedir1=<local settings> \
|
--basedir2=<local settings> \
|
--script_debug=_nix_ \
|
--rr=Server \
|
--rr_options=--chaos
|
|
- is part of
-
MDEV-11799
InnoDB can abort if the doublewrite buffer contains a bad and a good copy
-
-
Closed
{"report":{"fcp":1133.6999999284744,"ttfb":504.39999997615814,"pageVisibility":"visible","entityId":89715,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":0.5,"journeyId":"4add401e-2d5e-4859-a5ad-0cad7bd19eb8","navigationType":0,"readyForUser":1230.7999999523163,"redirectCount":0,"resourceLoadedEnd":1397.8999999761581,"resourceLoadedStart":554.6999999284744,"resourceTiming":[{"duration":45.200000047683716,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":554.6999999284744,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":554.6999999284744,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":599.8999999761581,"responseStart":0,"secureConnectionStart":0},{"duration":45.299999952316284,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2cib/820016/12ta74/2bf333562ca6724060a9d5f1535471f6/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":555,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":555,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":600.2999999523163,"responseStart":0,"secureConnectionStart":0},{"duration":93.60000002384186,"initiatorType":"script","name":"https://jira.mariadb.org/s/0917945aaa57108d00c5076fea35e069-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":555.1999999284744,"connectEnd":555.1999999284744,"connectStart":555.1999999284744,"domainLookupEnd":555.1999999284744,"domainLookupStart":555.1999999284744,"fetchStart":555.1999999284744,"redirectEnd":0,"redirectStart":0,"requestStart":555.1999999284744,"responseEnd":648.7999999523163,"responseStart":648.7999999523163,"secureConnectionStart":555.1999999284744},{"duration":141.30000007152557,"initiatorType":"script","name":"https://jira.mariadb.org/s/2d8175ec2fa4c816e8023260bd8c1786-CDN/lu2cib/820016/12ta74/2bf333562ca6724060a9d5f1535471f6/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true","startTime":555.2999999523163,"connectEnd":555.2999999523163,"connectStart":555.2999999523163,"domainLookupEnd":555.2999999523163,"domainLookupStart":555.2999999523163,"fetchStart":555.2999999523163,"redirectEnd":0,"redirectStart":0,"requestStart":555.2999999523163,"responseEnd":696.6000000238419,"responseStart":696.6000000238419,"secureConnectionStart":555.2999999523163},{"duration":145.09999990463257,"initiatorType":"script","name":"https://jira.mariadb.org/s/a9324d6758d385eb45c462685ad88f1d-CDN/lu2cib/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":555.6000000238419,"connectEnd":555.6000000238419,"connectStart":555.6000000238419,"domainLookupEnd":555.6000000238419,"domainLookupStart":555.6000000238419,"fetchStart":555.6000000238419,"redirectEnd":0,"redirectStart":0,"requestStart":555.6000000238419,"responseEnd":700.6999999284744,"responseStart":700.6000000238419,"secureConnectionStart":555.6000000238419},{"duration":145.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":555.6999999284744,"connectEnd":555.6999999284744,"connectStart":555.6999999284744,"domainLookupEnd":555.6999999284744,"domainLookupStart":555.6999999284744,"fetchStart":555.6999999284744,"redirectEnd":0,"redirectStart":0,"requestStart":555.6999999284744,"responseEnd":701.1999999284744,"responseStart":701.1999999284744,"secureConnectionStart":555.6999999284744},{"duration":146.70000004768372,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":555.8999999761581,"connectEnd":555.8999999761581,"connectStart":555.8999999761581,"domainLookupEnd":555.8999999761581,"domainLookupStart":555.8999999761581,"fetchStart":555.8999999761581,"redirectEnd":0,"redirectStart":0,"requestStart":555.8999999761581,"responseEnd":702.6000000238419,"responseStart":702.6000000238419,"secureConnectionStart":555.8999999761581},{"duration":239.09999990463257,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2cib/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":556.1000000238419,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":556.1000000238419,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":795.1999999284744,"responseStart":0,"secureConnectionStart":0},{"duration":148,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":556.2999999523163,"connectEnd":556.2999999523163,"connectStart":556.2999999523163,"domainLookupEnd":556.2999999523163,"domainLookupStart":556.2999999523163,"fetchStart":556.2999999523163,"redirectEnd":0,"redirectStart":0,"requestStart":556.2999999523163,"responseEnd":704.2999999523163,"responseStart":704.2999999523163,"secureConnectionStart":556.2999999523163},{"duration":238.89999997615814,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":556.3999999761581,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":556.3999999761581,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":795.2999999523163,"responseStart":0,"secureConnectionStart":0},{"duration":148.59999990463257,"initiatorType":"script","name":"https://jira.mariadb.org/s/5d5e8fe91fbc506585e83ea3b62ccc4b-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":556.6000000238419,"connectEnd":556.6000000238419,"connectStart":556.6000000238419,"domainLookupEnd":556.6000000238419,"domainLookupStart":556.6000000238419,"fetchStart":556.6000000238419,"redirectEnd":0,"redirectStart":0,"requestStart":556.6000000238419,"responseEnd":705.1999999284744,"responseStart":705.1999999284744,"secureConnectionStart":556.6000000238419},{"duration":838.8999999761581,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":557.5,"connectEnd":557.5,"connectStart":557.5,"domainLookupEnd":557.5,"domainLookupStart":557.5,"fetchStart":557.5,"redirectEnd":0,"redirectStart":0,"requestStart":557.5,"responseEnd":1396.3999999761581,"responseStart":1396.3999999761581,"secureConnectionStart":557.5},{"duration":840.2999999523163,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":557.6000000238419,"connectEnd":557.6000000238419,"connectStart":557.6000000238419,"domainLookupEnd":557.6000000238419,"domainLookupStart":557.6000000238419,"fetchStart":557.6000000238419,"redirectEnd":0,"redirectStart":0,"requestStart":557.6000000238419,"responseEnd":1397.8999999761581,"responseStart":1397.8999999761581,"secureConnectionStart":557.6000000238419},{"duration":75.5,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":810.2999999523163,"connectEnd":810.2999999523163,"connectStart":810.2999999523163,"domainLookupEnd":810.2999999523163,"domainLookupStart":810.2999999523163,"fetchStart":810.2999999523163,"redirectEnd":0,"redirectStart":0,"requestStart":810.2999999523163,"responseEnd":885.7999999523163,"responseStart":885.7999999523163,"secureConnectionStart":810.2999999523163},{"duration":461.7000000476837,"initiatorType":"link","name":"https://jira.mariadb.org/s/d5715adaadd168a9002b108b2b039b50-CDN/lu2cib/820016/12ta74/be4b45e9cec53099498fa61c8b7acba4/_/download/contextbatch/css/jira.project.sidebar,-_super,-project.issue.navigator,-jira.general,-jira.browse.project,-jira.view.issue,-jira.global,-atl.general,-com.atlassian.jira.projects.sidebar.init/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":1069.2999999523163,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":1069.2999999523163,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1531,"responseStart":0,"secureConnectionStart":0}],"fetchStart":0,"domainLookupStart":31,"domainLookupEnd":41,"connectStart":41,"connectEnd":60,"secureConnectionStart":49,"requestStart":61,"responseStart":504,"responseEnd":515,"domLoading":551,"domInteractive":1469,"domContentLoadedEventStart":1469,"domContentLoadedEventEnd":1527,"domComplete":2001,"loadEventStart":2001,"loadEventEnd":2002,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":1430.2999999523163},{"name":"bigPipe.sidebar-id.end","time":1431.1999999284744},{"name":"bigPipe.activity-panel-pipe-id.start","time":1431.1999999284744},{"name":"bigPipe.activity-panel-pipe-id.end","time":1435.6999999284744},{"name":"activityTabFullyLoaded","time":1544.3999999761581}],"measures":[],"correlationId":"507fbae9e7f8cf","effectiveType":"4g","downlink":9.3,"rtt":0,"serverDuration":118,"dbReadsTimeInMs":14,"dbConnsTimeInMs":24,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}
I did some quick initial analysis of the rr replay trace. The file is encrypted, and the decrypted value of PAGE_N_RECS before applying any redo log is 0xa92 and redo log apply is growing it to 0xaff (2815), which appears to be off by one:
10.4 956f21c3b0beba89835e8d24fde4b67b589ab142 with MDEV-23199 fix
Thread 4 hit Hardware watchpoint 1: -location block->frame[0x36]@2
Old value = "\n\222"
New value = <incomplete sequence \371>
__memmove_avx_unaligned_erms ()
at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:249
249 in ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S
(rr) when
Current event: 9067
…
Thread 4 hit Hardware watchpoint 1: -location block->frame[0x36]@2
Old value = "\n\377"
New value = "\n\376"
page_header_set_field (page=page@entry=0x757b0df20000 "\317\371:\376",
page_zip=page_zip@entry=0x0, field=field@entry=16, val=2814)
at /home/mleich/bb-10.4-thiru/storage/innobase/include/page0page.ic:172
172 if (page_zip) {
Current event: 9067
Thread 4 hit Hardware watchpoint 1: -location block->frame[0x36]@2
Old value = "\n\376"
New value = "\n\377"
page_header_set_field (page=page@entry=0x757b0df20000 "\317\371:\376", page_zip=page_zip@entry=0x0, field=field@entry=16, val=2815) at /home/mleich/bb-10.4-thiru/storage/innobase/include/page0page.ic:172
172 if (page_zip) {
Current event: 9067
…
[rr 15697 71203]2020-07-17 19:56:48 0 [Note] InnoDB: Buffer pool(s) load completed at 200717 19:56:48
[rr 15697 73269]2020-07-17 19:56:49 0 [Warning] /home/mleich/Server_bin/bb-10.4-thiru_debug_Og/bin/mysqld: unknown variable 'loose-table_lock_wait_timeout=50'
[rr 15697 73975]2020-07-17 19:56:49 0 [Note] Recovering after a crash using mysql-bin
[rr 15697 82508]2020-07-17 19:56:53 0 [ERROR] InnoDB: n recs wrong 2817 2816
I had set up a hardware watchpoint on the PAGE_N_RECS: watch
l block>frame[0x36]@2.mleich, what else should we copy than ../1/data_orig to be able to start up the recovery on a separate system? What are the encryption parameters?
My preference would be to repeat the recovery failure locally, possibly with --debug=d,ib_log or some instrumentation added to the executable (such as added calls to page_simple_validate_new() after applying each record to the page), and then (if needed) use this information to find out what the server did wrong before being killed.