InnoDB enables an ‘adaptive’ flushing mode by default, which attempts to ensure a steady flow of page flushing and avoid write bursts on log checkpoint.
There is a non-default option innodb_adaptive_flushing=OFF that is still based on waking up the page cleaner thread at least once per second. I think that we should offer a purely event-driven flushing option that would ignore innodb_io_capacity altogether and perform best effort for the buffer pool to remain between innodb_max_dirty_pages_pct_lwm and innodb_max_dirty_pages_pct.
I implemented a crude prototype of this.
Attachments
Issue Links
is blocked by
MDEV-2339910.5 performance regression with IO-bound tpcc
Closed
MDEV-23855InnoDB log checkpointing causes regression for write-heavy OLTP
In the purely event-driven innodb_adaptive_flushing=OFF mode, it might even make sense to skip the buf_flush_wait_flushed() altogether, and simply perform a ‘best effort’ checkpoint based on MIN(oldest_modification).
Marko Mäkelä
added a comment - In the purely event-driven innodb_adaptive_flushing=OFF mode, it might even make sense to skip the buf_flush_wait_flushed() altogether, and simply perform a ‘best effort’ checkpoint based on MIN(oldest_modification) .
Perhaps the periodic log checkpoints should be moved from srv_master_callback() to the page cleaner thread (affecting both modes of flushing), possibly as part of fixing MDEV-23855.
Marko Mäkelä
added a comment - Perhaps the periodic log checkpoints should be moved from srv_master_callback() to the page cleaner thread (affecting both modes of flushing), possibly as part of fixing MDEV-23855 .
People
Vladislav Vaintroub
Marko Mäkelä
Votes:
0Vote for this issue
Watchers:
5Start watching this issue
Dates
Created:
Updated:
Git Integration
Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.
{"report":{"fcp":978.6000000238419,"ttfb":210.5,"pageVisibility":"visible","entityId":91848,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":0.5,"journeyId":"a89defa8-5991-4dfa-a38d-0276f964cd8a","navigationType":0,"readyForUser":1048.1000000238419,"redirectCount":0,"resourceLoadedEnd":894.7000000476837,"resourceLoadedStart":215.10000002384186,"resourceTiming":[{"duration":209.10000002384186,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2bu7/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":215.10000002384186,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":215.10000002384186,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":424.2000000476837,"responseStart":0,"secureConnectionStart":0},{"duration":209.30000001192093,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2bu7/820016/12ta74/8679b4946efa1a0bb029a3a22206fb5d/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":215.30000001192093,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":215.30000001192093,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":424.60000002384186,"responseStart":0,"secureConnectionStart":0},{"duration":364.80000001192093,"initiatorType":"script","name":"https://jira.mariadb.org/s/fbf975c0cce4b1abf04784eeae9ba1f4-CDN/lu2bu7/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":215.5,"connectEnd":429.80000001192093,"connectStart":429.80000001192093,"domainLookupEnd":429.80000001192093,"domainLookupStart":429.80000001192093,"fetchStart":215.5,"redirectEnd":0,"redirectStart":0,"requestStart":430.5,"responseEnd":580.3000000119209,"responseStart":449.80000001192093,"secureConnectionStart":429.80000001192093},{"duration":498.5999999642372,"initiatorType":"script","name":"https://jira.mariadb.org/s/099b33461394b8015fc36c0a4b96e19f-CDN/lu2bu7/820016/12ta74/8679b4946efa1a0bb029a3a22206fb5d/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true","startTime":215.70000004768372,"connectEnd":215.70000004768372,"connectStart":215.70000004768372,"domainLookupEnd":215.70000004768372,"domainLookupStart":215.70000004768372,"fetchStart":215.70000004768372,"redirectEnd":0,"redirectStart":0,"requestStart":430.80000001192093,"responseEnd":714.3000000119209,"responseStart":452.7000000476837,"secureConnectionStart":215.70000004768372},{"duration":243,"initiatorType":"script","name":"https://jira.mariadb.org/s/94c15bff32baef80f4096a08aceae8bc-CDN/lu2bu7/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":215.80000001192093,"connectEnd":215.80000001192093,"connectStart":215.80000001192093,"domainLookupEnd":215.80000001192093,"domainLookupStart":215.80000001192093,"fetchStart":215.80000001192093,"redirectEnd":0,"redirectStart":0,"requestStart":431.2000000476837,"responseEnd":458.80000001192093,"responseStart":457.7000000476837,"secureConnectionStart":215.80000001192093},{"duration":241.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":216,"connectEnd":216,"connectStart":216,"domainLookupEnd":216,"domainLookupStart":216,"fetchStart":216,"redirectEnd":0,"redirectStart":0,"requestStart":432.5,"responseEnd":457.5,"responseStart":456.30000001192093,"secureConnectionStart":216},{"duration":244.80000001192093,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":216.10000002384186,"connectEnd":216.10000002384186,"connectStart":216.10000002384186,"domainLookupEnd":216.10000002384186,"domainLookupStart":216.10000002384186,"fetchStart":216.10000002384186,"redirectEnd":0,"redirectStart":0,"requestStart":432,"responseEnd":460.9000000357628,"responseStart":459.10000002384186,"secureConnectionStart":216.10000002384186},{"duration":214.80000001192093,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2bu7/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":216.30000001192093,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":216.30000001192093,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":431.10000002384186,"responseStart":0,"secureConnectionStart":0},{"duration":245.80000001192093,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":216.5,"connectEnd":216.5,"connectStart":216.5,"domainLookupEnd":216.5,"domainLookupStart":216.5,"fetchStart":216.5,"redirectEnd":0,"redirectStart":0,"requestStart":435.2000000476837,"responseEnd":462.30000001192093,"responseStart":460,"secureConnectionStart":216.5},{"duration":214.79999995231628,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2bu7/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":220.70000004768372,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":220.70000004768372,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":435.5,"responseStart":0,"secureConnectionStart":0},{"duration":246.4000000357628,"initiatorType":"script","name":"https://jira.mariadb.org/s/3339d87fa2538a859872f2df449bf8d0-CDN/lu2bu7/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":221,"connectEnd":221,"connectStart":221,"domainLookupEnd":221,"domainLookupStart":221,"fetchStart":221,"redirectEnd":0,"redirectStart":0,"requestStart":437.80000001192093,"responseEnd":467.4000000357628,"responseStart":461.10000002384186,"secureConnectionStart":221},{"duration":670.6999999880791,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":223.80000001192093,"connectEnd":223.80000001192093,"connectStart":223.80000001192093,"domainLookupEnd":223.80000001192093,"domainLookupStart":223.80000001192093,"fetchStart":223.80000001192093,"redirectEnd":0,"redirectStart":0,"requestStart":883,"responseEnd":894.5,"responseStart":893.1000000238419,"secureConnectionStart":223.80000001192093},{"duration":670.9000000357628,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":223.80000001192093,"connectEnd":223.80000001192093,"connectStart":223.80000001192093,"domainLookupEnd":223.80000001192093,"domainLookupStart":223.80000001192093,"fetchStart":223.80000001192093,"redirectEnd":0,"redirectStart":0,"requestStart":882.8000000119209,"responseEnd":894.7000000476837,"responseStart":893.7000000476837,"secureConnectionStart":223.80000001192093},{"duration":223,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":706.2000000476837,"connectEnd":706.2000000476837,"connectStart":706.2000000476837,"domainLookupEnd":706.2000000476837,"domainLookupStart":706.2000000476837,"fetchStart":706.2000000476837,"redirectEnd":0,"redirectStart":0,"requestStart":895.9000000357628,"responseEnd":929.2000000476837,"responseStart":928.5,"secureConnectionStart":706.2000000476837}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":52,"responseStart":210,"responseEnd":223,"domLoading":213,"domInteractive":1095,"domContentLoadedEventStart":1095,"domContentLoadedEventEnd":1137,"domComplete":2062,"loadEventStart":2062,"loadEventEnd":2063,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":1077},{"name":"bigPipe.sidebar-id.end","time":1077.7000000476837},{"name":"bigPipe.activity-panel-pipe-id.start","time":1077.9000000357628},{"name":"bigPipe.activity-panel-pipe-id.end","time":1079.2000000476837},{"name":"activityTabFullyLoaded","time":1155.4000000357628}],"measures":[],"correlationId":"5e699bc487f685","effectiveType":"4g","downlink":9,"rtt":0,"serverDuration":95,"dbReadsTimeInMs":12,"dbConnsTimeInMs":20,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}
Task
Major
MDEV-23399 10.5 performance regression with IO-bound tpcc
In the purely event-driven innodb_adaptive_flushing=OFF mode, it might even make sense to skip the buf_flush_wait_flushed() altogether, and simply perform a ‘best effort’ checkpoint based on MIN(oldest_modification).