10.3 server crashes with a non-debug assertion failure when it starts on the attached datadir.
The datadir was created on the current 10.0 server as a part of the undo-upgrade scenario:
- start the current 10.0 server;
- create some tables and run some DML on them;
- kill the server during operation (with SIGKILL);
- restart the server with innodb-force-recovery=3, no client activity;
- shutdown the server normally;
Then the current 10.3 server is started on the same datadir. It starts, but crashes immediately afterwards as below.
10.3 b52bb6eb82db8
|
2018-06-12 18:15:28 0x7fb1809ea700 InnoDB: Assertion failure in file /data/src/10.3/storage/innobase/trx/trx0purge.cc line 121
|
InnoDB: Failing assertion: purge_sys.tail.commit <= purge_sys.rseg->last_commit
|
|
#5 0x00007fb1a1634448 in __GI_abort () at abort.c:89
|
#6 0x00007fb1a44f44fb in ut_dbg_assertion_failed (expr=0x7fb1a4b6b0c8 "purge_sys.tail.commit <= purge_sys.rseg->last_commit", file=0x7fb1a4b6af80 "/data/src/10.3/storage/innobase/trx/trx0purge.cc", line=121) at /data/src/10.3/storage/innobase/ut/ut0dbg.cc:61
|
#7 0x00007fb1a44c2dfb in TrxUndoRsegsIterator::set_next (this=0x7fb1a530c0a0 <purge_sys+416>) at /data/src/10.3/storage/innobase/trx/trx0purge.cc:121
|
#8 0x00007fb1a44c0d4b in trx_purge_choose_next_log () at /data/src/10.3/storage/innobase/trx/trx0purge.cc:1213
|
#9 0x00007fb1a44c0fa4 in trx_purge_get_next_rec (n_pages_handled=0x7fb1809e9dd0, heap=0x7fb1a85d2780) at /data/src/10.3/storage/innobase/trx/trx0purge.cc:1286
|
#10 0x00007fb1a44c11f5 in trx_purge_fetch_next_rec (roll_ptr=0x7fb17400d8f0, n_pages_handled=0x7fb1809e9dd0, heap=0x7fb1a85d2780) at /data/src/10.3/storage/innobase/trx/trx0purge.cc:1356
|
#11 0x00007fb1a44c151c in trx_purge_attach_undo_recs (n_purge_threads=4) at /data/src/10.3/storage/innobase/trx/trx0purge.cc:1429
|
#12 0x00007fb1a44c19ff in trx_purge (n_purge_threads=4, truncate=false) at /data/src/10.3/storage/innobase/trx/trx0purge.cc:1559
|
#13 0x00007fb1a4492d44 in srv_do_purge (n_total_purged=0x7fb1809e9ed0) at /data/src/10.3/storage/innobase/srv/srv0srv.cc:2583
|
#14 0x00007fb1a4493172 in srv_purge_coordinator_thread (arg=0x0) at /data/src/10.3/storage/innobase/srv/srv0srv.cc:2714
|
#15 0x00007fb1a3293064 in start_thread (arg=0x7fb1809ea700) at pthread_create.c:309
|
#16 0x00007fb1a16e662d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
|
This current test was run with --innodb-page-size=8K --loose-innodb_log_compressed_pages=on --loose-innodb-change-buffering=none, I'm not sure whether any of them important. Naturally, to reproduce the crash on the attached datadir, the server needs to be also started with --innodb-page-size=8K, other two options don't make a difference; otherwise all defaults.
ib_logfile-s are compressed and attached separately just to overcome the 10M limitation in JIRA. I don't know if they are needed, the crash happens with and without them.
Similar-looking crashes upon upgrade from 10.1 have also been observed before.
10.2 doesn't crash on the same datadir.
{"report":{"fcp":1618.7000000476837,"ttfb":431.09999990463257,"pageVisibility":"visible","entityId":66915,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":0.5,"journeyId":"47a4bfd2-c6cd-454d-9faa-42024285032e","navigationType":0,"readyForUser":1723.7000000476837,"redirectCount":0,"resourceLoadedEnd":1303.5,"resourceLoadedStart":449.7999999523163,"resourceTiming":[{"duration":204.40000009536743,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":449.7999999523163,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":449.7999999523163,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":654.2000000476837,"responseStart":0,"secureConnectionStart":0},{"duration":204.5,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true&whisper-enabled=true","startTime":450.09999990463257,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":450.09999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":654.5999999046326,"responseStart":0,"secureConnectionStart":0},{"duration":633.4000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/s/0917945aaa57108d00c5076fea35e069-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":450.39999985694885,"connectEnd":450.39999985694885,"connectStart":450.39999985694885,"domainLookupEnd":450.39999985694885,"domainLookupStart":450.39999985694885,"fetchStart":450.39999985694885,"redirectEnd":0,"redirectStart":0,"requestStart":682,"responseEnd":1083.7999999523163,"responseStart":734.7000000476837,"secureConnectionStart":450.39999985694885},{"duration":853,"initiatorType":"script","name":"https://jira.mariadb.org/s/2d8175ec2fa4c816e8023260bd8c1786-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true&whisper-enabled=true","startTime":450.5,"connectEnd":450.5,"connectStart":450.5,"domainLookupEnd":450.5,"domainLookupStart":450.5,"fetchStart":450.5,"redirectEnd":0,"redirectStart":0,"requestStart":681.7999999523163,"responseEnd":1303.5,"responseStart":733.8999998569489,"secureConnectionStart":450.5},{"duration":298.90000009536743,"initiatorType":"script","name":"https://jira.mariadb.org/s/a9324d6758d385eb45c462685ad88f1d-CDN/lu2cib/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":450.7999999523163,"connectEnd":450.7999999523163,"connectStart":450.7999999523163,"domainLookupEnd":450.7999999523163,"domainLookupStart":450.7999999523163,"fetchStart":450.7999999523163,"redirectEnd":0,"redirectStart":0,"requestStart":682.0999999046326,"responseEnd":749.7000000476837,"responseStart":747.5,"secureConnectionStart":450.7999999523163},{"duration":298.60000014305115,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":450.89999985694885,"connectEnd":450.89999985694885,"connectStart":450.89999985694885,"domainLookupEnd":450.89999985694885,"domainLookupStart":450.89999985694885,"fetchStart":450.89999985694885,"redirectEnd":0,"redirectStart":0,"requestStart":682.2000000476837,"responseEnd":749.5,"responseStart":745,"secureConnectionStart":450.89999985694885},{"duration":298.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":451.09999990463257,"connectEnd":451.09999990463257,"connectStart":451.09999990463257,"domainLookupEnd":451.09999990463257,"domainLookupStart":451.09999990463257,"fetchStart":451.09999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":682.2999999523163,"responseEnd":749.5999999046326,"responseStart":746.2000000476837,"secureConnectionStart":451.09999990463257},{"duration":224.09999990463257,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2cib/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":451.2999999523163,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":451.2999999523163,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":675.3999998569489,"responseStart":0,"secureConnectionStart":0},{"duration":299.2000000476837,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":451.59999990463257,"connectEnd":451.59999990463257,"connectStart":451.59999990463257,"domainLookupEnd":451.59999990463257,"domainLookupStart":451.59999990463257,"fetchStart":451.59999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":682.7000000476837,"responseEnd":750.7999999523163,"responseStart":748.2999999523163,"secureConnectionStart":451.59999990463257},{"duration":224.5,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":451.7000000476837,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":451.7000000476837,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":676.2000000476837,"responseStart":0,"secureConnectionStart":0},{"duration":300.09999990463257,"initiatorType":"script","name":"https://jira.mariadb.org/s/5d5e8fe91fbc506585e83ea3b62ccc4b-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":451.7999999523163,"connectEnd":451.7999999523163,"connectStart":451.7999999523163,"domainLookupEnd":451.7999999523163,"domainLookupStart":451.7999999523163,"fetchStart":451.7999999523163,"redirectEnd":0,"redirectStart":0,"requestStart":695.3999998569489,"responseEnd":751.8999998569489,"responseStart":750.8999998569489,"secureConnectionStart":451.7999999523163},{"duration":839.2999999523163,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":453.59999990463257,"connectEnd":453.59999990463257,"connectStart":453.59999990463257,"domainLookupEnd":453.59999990463257,"domainLookupStart":453.59999990463257,"fetchStart":453.59999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":862.8999998569489,"responseEnd":1292.8999998569489,"responseStart":1288.0999999046326,"secureConnectionStart":453.59999990463257},{"duration":845.6000001430511,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":453.59999990463257,"connectEnd":453.59999990463257,"connectStart":453.59999990463257,"domainLookupEnd":453.59999990463257,"domainLookupStart":453.59999990463257,"fetchStart":453.59999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":1097,"responseEnd":1299.2000000476837,"responseStart":1294.2000000476837,"secureConnectionStart":453.59999990463257},{"duration":292,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":1246.5,"connectEnd":1246.5,"connectStart":1246.5,"domainLookupEnd":1246.5,"domainLookupStart":1246.5,"fetchStart":1246.5,"redirectEnd":0,"redirectStart":0,"requestStart":1505.0999999046326,"responseEnd":1538.5,"responseStart":1537.8999998569489,"secureConnectionStart":1246.5},{"duration":328.09999990463257,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":1555.5,"connectEnd":1555.5,"connectStart":1555.5,"domainLookupEnd":1555.5,"domainLookupStart":1555.5,"fetchStart":1555.5,"redirectEnd":0,"redirectStart":0,"requestStart":1850.7999999523163,"responseEnd":1883.5999999046326,"responseStart":1883,"secureConnectionStart":1555.5}],"fetchStart":0,"domainLookupStart":194,"domainLookupEnd":200,"connectStart":200,"connectEnd":219,"secureConnectionStart":208,"requestStart":220,"responseStart":431,"responseEnd":433,"domLoading":447,"domInteractive":1804,"domContentLoadedEventStart":1804,"domContentLoadedEventEnd":1865,"domComplete":2368,"loadEventStart":2368,"loadEventEnd":2369,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":1778.5},{"name":"bigPipe.sidebar-id.end","time":1779.2999999523163},{"name":"bigPipe.activity-panel-pipe-id.start","time":1779.5},{"name":"bigPipe.activity-panel-pipe-id.end","time":1780.5999999046326},{"name":"activityTabFullyLoaded","time":1884}],"measures":[],"correlationId":"a493a47083dcc7","effectiveType":"4g","downlink":10,"rtt":0,"serverDuration":140,"dbReadsTimeInMs":42,"dbConnsTimeInMs":54,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}