Type:
Priority:
Resolution:
Fixed
The InnoDB clustered index record system columns DB_TRX_ID,DB_ROLL_PTR are used by multi-versioning and for determining if a record is implicitly locked. After the history is no longer needed, these columns can safely be reset to 0 and 1<<55 (to indicate a fresh insert).
When a reader sees 0 in the DB_TRX_ID column, it can instantly determine that the record is present the read view. There is no need to acquire the transaction system mutex to check if the transaction exists, because writes can never be conducted by a transaction whose ID is 0.
The persistent InnoDB undo log is split into two parts: insert_undo and update_undo. The insert_undo log is discarded at transaction commit or rollback, and the update_undo log is processed by the purge subsystem. As part of this change, we must merge the two types of undo logs into one, and the purge subsystem will reset the DB_TRX_ID whenever a clustered index record is ‘touched’.
This will change the persistent InnoDB file formats, not only in the undo log and redo log, but also in the data files. There are some debug assertions that would not allow any record to contain DB_TRX_ID=0.
A new redo log format tag must be introduced so that the writes of the system columns can be properly redo-logged. (See MDEV-11432 MDEV-11782
The undo log format will be changed as well. To be able to get rid of legacy code, InnoDB startup should detect if any old-format undo logs are present. If yes, startup will be refused, and the user must perform a slow shutdown (SET GLOBAL innodb_fast_shutdown=0) with the old server in order to empty the undo logs.
A proof-of-concept implementation for 10.2 consists of 4 consecutive commits. It is missing any of the above-mentioned upgrade logic.
made changes -
2017-07-06 00:10
Status
Open
[ 1
]
In Progress
[ 3
]
made changes -
2017-07-06 00:15
Assignee
Marko Mäkelä
[ marko
]
Jan Lindström
[ jplindst
]
Status
In Progress
[ 3
]
In Review
[ 10002
]
made changes -
2017-07-06 09:50
Assignee
Jan Lindström
[ jplindst
]
Marko Mäkelä
[ marko
]
Status
In Review
[ 10002
]
Stalled
[ 10000
]
made changes -
2017-07-07 14:51
Fix Version/s
10.3.1
[ 22532
]
Fix Version/s
10.3
[ 22126
]
Resolution
Fixed
[ 1
]
Status
Stalled
[ 10000
]
Closed
[ 6
]
made changes -
2021-12-06 21:23
Workflow
MariaDB v3
[ 80015
]
MariaDB v4
[ 133182
]
made changes -
2022-01-14 13:44
Link
This issue relates to TODO-3302
[ TODO-3302
]
made changes -
2022-09-27 20:26
Remote Link
This issue links to "Page (MariaDB Confluence)"
[ 34500
]
{"report":{"fcp":2449.199999809265,"ttfb":575.5999999046326,"pageVisibility":"visible","entityId":60609,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":0.5,"journeyId":"3b5323c7-2940-4fb8-a2e7-7bc439afea64","navigationType":0,"readyForUser":2616.4000000953674,"redirectCount":0,"resourceLoadedEnd":1978.1999998092651,"resourceLoadedStart":583.1999998092651,"resourceTiming":[{"duration":566.8000001907349,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":583.1999998092651,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":583.1999998092651,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1150,"responseStart":0,"secureConnectionStart":0},{"duration":566.7999997138977,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true&whisper-enabled=true","startTime":583.5,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":583.5,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1150.2999997138977,"responseStart":0,"secureConnectionStart":0},{"duration":959.3000001907349,"initiatorType":"script","name":"https://jira.mariadb.org/s/0917945aaa57108d00c5076fea35e069-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":583.6999998092651,"connectEnd":583.6999998092651,"connectStart":583.6999998092651,"domainLookupEnd":583.6999998092651,"domainLookupStart":583.6999998092651,"fetchStart":583.6999998092651,"redirectEnd":0,"redirectStart":0,"requestStart":1152.0999999046326,"responseEnd":1543,"responseStart":1193.5,"secureConnectionStart":583.6999998092651},{"duration":1302.4000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/s/2d8175ec2fa4c816e8023260bd8c1786-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true&whisper-enabled=true","startTime":583.7999997138977,"connectEnd":583.7999997138977,"connectStart":583.7999997138977,"domainLookupEnd":583.7999997138977,"domainLookupStart":583.7999997138977,"fetchStart":583.7999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":1152.1999998092651,"responseEnd":1886.1999998092651,"responseStart":1195.5,"secureConnectionStart":583.7999997138977},{"duration":623.9000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/s/a9324d6758d385eb45c462685ad88f1d-CDN/lu2cib/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":584,"connectEnd":584,"connectStart":584,"domainLookupEnd":584,"domainLookupStart":584,"fetchStart":584,"redirectEnd":0,"redirectStart":0,"requestStart":1152.5,"responseEnd":1207.9000000953674,"responseStart":1203,"secureConnectionStart":584},{"duration":624,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":584.1999998092651,"connectEnd":584.1999998092651,"connectStart":584.1999998092651,"domainLookupEnd":584.1999998092651,"domainLookupStart":584.1999998092651,"fetchStart":584.1999998092651,"redirectEnd":0,"redirectStart":0,"requestStart":1152.5999999046326,"responseEnd":1208.1999998092651,"responseStart":1205.4000000953674,"secureConnectionStart":584.1999998092651},{"duration":623.9000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":584.5,"connectEnd":584.5,"connectStart":584.5,"domainLookupEnd":584.5,"domainLookupStart":584.5,"fetchStart":584.5,"redirectEnd":0,"redirectStart":0,"requestStart":1152.6999998092651,"responseEnd":1208.4000000953674,"responseStart":1206.4000000953674,"secureConnectionStart":584.5},{"duration":567.5,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2cib/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":584.6999998092651,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":584.6999998092651,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1152.1999998092651,"responseStart":0,"secureConnectionStart":0},{"duration":654.7999997138977,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":584.9000000953674,"connectEnd":584.9000000953674,"connectStart":584.9000000953674,"domainLookupEnd":584.9000000953674,"domainLookupStart":584.9000000953674,"fetchStart":584.9000000953674,"redirectEnd":0,"redirectStart":0,"requestStart":1182.0999999046326,"responseEnd":1239.6999998092651,"responseStart":1222.5,"secureConnectionStart":584.9000000953674},{"duration":577.5,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":585,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":585,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1162.5,"responseStart":0,"secureConnectionStart":0},{"duration":679.4000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/s/5d5e8fe91fbc506585e83ea3b62ccc4b-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":585.1999998092651,"connectEnd":585.1999998092651,"connectStart":585.1999998092651,"domainLookupEnd":585.1999998092651,"domainLookupStart":585.1999998092651,"fetchStart":585.1999998092651,"redirectEnd":0,"redirectStart":0,"requestStart":1182.4000000953674,"responseEnd":1264.5999999046326,"responseStart":1224.5999999046326,"secureConnectionStart":585.1999998092651},{"duration":1271.8000001907349,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":593.6999998092651,"connectEnd":593.6999998092651,"connectStart":593.6999998092651,"domainLookupEnd":593.6999998092651,"domainLookupStart":593.6999998092651,"fetchStart":593.6999998092651,"redirectEnd":0,"redirectStart":0,"requestStart":1593.2999997138977,"responseEnd":1865.5,"responseStart":1859.0999999046326,"secureConnectionStart":593.6999998092651},{"duration":1380.2999997138977,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":597.9000000953674,"connectEnd":597.9000000953674,"connectStart":597.9000000953674,"domainLookupEnd":597.9000000953674,"domainLookupStart":597.9000000953674,"fetchStart":597.9000000953674,"redirectEnd":0,"redirectStart":0,"requestStart":1953.0999999046326,"responseEnd":1978.1999998092651,"responseStart":1966.0999999046326,"secureConnectionStart":597.9000000953674},{"duration":293.5,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":1696.5,"connectEnd":1696.5,"connectStart":1696.5,"domainLookupEnd":1696.5,"domainLookupStart":1696.5,"fetchStart":1696.5,"redirectEnd":0,"redirectStart":0,"requestStart":1954.4000000953674,"responseEnd":1990,"responseStart":1989,"secureConnectionStart":1696.5}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":224,"responseStart":576,"responseEnd":597,"domLoading":580,"domInteractive":2720,"domContentLoadedEventStart":2720,"domContentLoadedEventEnd":2797,"domComplete":3939,"loadEventStart":3939,"loadEventEnd":3940,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":2655.5999999046326},{"name":"bigPipe.sidebar-id.end","time":2656.5999999046326},{"name":"bigPipe.activity-panel-pipe-id.start","time":2656.699999809265},{"name":"bigPipe.activity-panel-pipe-id.end","time":2669.5999999046326},{"name":"activityTabFullyLoaded","time":2831.7999997138977}],"measures":[],"correlationId":"43e292a0142fcd","effectiveType":"4g","downlink":10,"rtt":0,"serverDuration":283,"dbReadsTimeInMs":93,"dbConnsTimeInMs":112,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}
Task
Major
MDEV-13697 DB_TRX_ID is not always reset when the history is removed
MDEV-11658 Simpler, faster IMPORT of InnoDB tables
Page Loading...