Type:
Priority:
Resolution:
Fixed
Affects Version/s:
10.2.41 , 10.3.32 , 10.4.22 , 10.5.13 , 10.6.5
Environment:
CentOS 7
Original issue and repro-https://bugs.mysql.com/bug.php?id=68079
Current repro and data provided in developer comment via link.
Flamegraphs from Intel machine attached (two variants per- one uses perf polling at 99Hz, other uses mariadb-stacktrace collecting thread dumps every 5 seconds for 60 iterations). Stack trace from AMD machine attached.
Three sets of flamegraphs-
Regular 10.6.5 using utf8mb3 (not set as default) charset for tables with AHI enabled
Regular 10.6.5 using latin1 (set as default) charset for tables with AHI disabled (default)
Custom 10.6.5 using utf8mb3 (not set as default) charset for tables with AHI disabled (default) where line 867 of mysys/charset.c has been removed prior to compiling MariaDB 10.6.5 CS
Base concern is despite this being a read workload, adding cores to the workload has a diminishing impact (very noticeable going from 4 to 8 cores on our Intel setup where 8 cores should have been ideal for scaling)-
Benchmark
Average number of seconds to run all queries: 11.982 seconds
Minimum number of seconds to run all queries: 11.878 seconds
Maximum number of seconds to run all queries: 12.231 seconds
Number of clients running queries: 1
Average number of queries per client: 1000
Benchmark
Average number of seconds to run all queries: 8.712 seconds
Minimum number of seconds to run all queries: 8.347 seconds
Maximum number of seconds to run all queries: 9.358 seconds
Number of clients running queries: 4
Average number of queries per client: 250
Benchmark
Average number of seconds to run all queries: 7.104 seconds
Minimum number of seconds to run all queries: 6.999 seconds
Maximum number of seconds to run all queries: 7.216 seconds
Number of clients running queries: 8
Average number of queries per client: 125
Benchmark
Average number of seconds to run all queries: 5.572 seconds
Minimum number of seconds to run all queries: 5.464 seconds
Maximum number of seconds to run all queries: 5.695 seconds
Number of clients running queries: 16
Average number of queries per client: 62
Flamegraphs seem to demonstrate counter from MDEV-6274
Benchmark
Average number of seconds to run all queries: 11.891 seconds
Minimum number of seconds to run all queries: 11.738 seconds
Maximum number of seconds to run all queries: 12.050 seconds
Number of clients running queries: 1
Average number of queries per client: 1000
Benchmark
Average number of seconds to run all queries: 5.934 seconds
Minimum number of seconds to run all queries: 5.067 seconds
Maximum number of seconds to run all queries: 6.386 seconds
Number of clients running queries: 4
Average number of queries per client: 250
Benchmark
Average number of seconds to run all queries: 3.806 seconds
Minimum number of seconds to run all queries: 3.673 seconds
Maximum number of seconds to run all queries: 3.922 seconds
Number of clients running queries: 8
Average number of queries per client: 125
Benchmark
Average number of seconds to run all queries: 2.762 seconds
Minimum number of seconds to run all queries: 2.729 seconds
Maximum number of seconds to run all queries: 2.824 seconds
Number of clients running queries: 16
Average number of queries per client: 62
Flamegraphs from modified binary still suggest additional collation/charset optimization may stand in the way of even better performance, and likewise for what appear to be some low-level InnoDB locks.
Would like to see MariaDB achieve scaling closer to 200% performance gain for doubling core count, which on this same workload Postgres comes very close to achieving.
{"report":{"fcp":840.9000000953674,"ttfb":253.59999990463257,"pageVisibility":"visible","entityId":107667,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":0.5,"journeyId":"638296cd-a7c3-4095-949a-a40441ab8752","navigationType":0,"readyForUser":1071.4000000953674,"redirectCount":0,"resourceLoadedEnd":620.5999999046326,"resourceLoadedStart":259.19999980926514,"resourceTiming":[{"duration":9.099999904632568,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":259.19999980926514,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":259.19999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":268.2999997138977,"responseStart":0,"secureConnectionStart":0},{"duration":9.099999904632568,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true&whisper-enabled=true","startTime":259.5,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":259.5,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":268.59999990463257,"responseStart":0,"secureConnectionStart":0},{"duration":94.90000009536743,"initiatorType":"script","name":"https://jira.mariadb.org/s/0917945aaa57108d00c5076fea35e069-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":259.69999980926514,"connectEnd":259.69999980926514,"connectStart":259.69999980926514,"domainLookupEnd":259.69999980926514,"domainLookupStart":259.69999980926514,"fetchStart":259.69999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":282.40000009536743,"responseEnd":354.59999990463257,"responseStart":312.5,"secureConnectionStart":259.69999980926514},{"duration":112.7999997138977,"initiatorType":"script","name":"https://jira.mariadb.org/s/2d8175ec2fa4c816e8023260bd8c1786-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true&whisper-enabled=true","startTime":259.90000009536743,"connectEnd":259.90000009536743,"connectStart":259.90000009536743,"domainLookupEnd":259.90000009536743,"domainLookupStart":259.90000009536743,"fetchStart":259.90000009536743,"redirectEnd":0,"redirectStart":0,"requestStart":282.5,"responseEnd":372.69999980926514,"responseStart":321.09999990463257,"secureConnectionStart":259.90000009536743},{"duration":57.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/a9324d6758d385eb45c462685ad88f1d-CDN/lu2cib/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":260.19999980926514,"connectEnd":260.19999980926514,"connectStart":260.19999980926514,"domainLookupEnd":260.19999980926514,"domainLookupStart":260.19999980926514,"fetchStart":260.19999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":282.59999990463257,"responseEnd":317.69999980926514,"responseStart":317.19999980926514,"secureConnectionStart":260.19999980926514},{"duration":55.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":260.2999997138977,"connectEnd":260.2999997138977,"connectStart":260.2999997138977,"domainLookupEnd":260.2999997138977,"domainLookupStart":260.2999997138977,"fetchStart":260.2999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":282.69999980926514,"responseEnd":315.7999997138977,"responseStart":315.09999990463257,"secureConnectionStart":260.2999997138977},{"duration":56.59999990463257,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":260.5,"connectEnd":260.5,"connectStart":260.5,"domainLookupEnd":260.5,"domainLookupStart":260.5,"fetchStart":260.5,"redirectEnd":0,"redirectStart":0,"requestStart":282.69999980926514,"responseEnd":317.09999990463257,"responseStart":316.59999990463257,"secureConnectionStart":260.5},{"duration":20.200000286102295,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2cib/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":260.69999980926514,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":260.69999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":280.90000009536743,"responseStart":0,"secureConnectionStart":0},{"duration":55.700000286102295,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":260.7999997138977,"connectEnd":260.7999997138977,"connectStart":260.7999997138977,"domainLookupEnd":260.7999997138977,"domainLookupStart":260.7999997138977,"fetchStart":260.7999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":282.7999997138977,"responseEnd":316.5,"responseStart":316,"secureConnectionStart":260.7999997138977},{"duration":21,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":261,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":261,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":282,"responseStart":0,"secureConnectionStart":0},{"duration":63.59999990463257,"initiatorType":"script","name":"https://jira.mariadb.org/s/5d5e8fe91fbc506585e83ea3b62ccc4b-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":261.09999990463257,"connectEnd":261.09999990463257,"connectStart":261.09999990463257,"domainLookupEnd":261.09999990463257,"domainLookupStart":261.09999990463257,"fetchStart":261.09999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":285.40000009536743,"responseEnd":324.69999980926514,"responseStart":323.59999990463257,"secureConnectionStart":261.09999990463257},{"duration":341.09999990463257,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":279,"connectEnd":279,"connectStart":279,"domainLookupEnd":279,"domainLookupStart":279,"fetchStart":279,"redirectEnd":0,"redirectStart":0,"requestStart":302.69999980926514,"responseEnd":620.0999999046326,"responseStart":606.5999999046326,"secureConnectionStart":279},{"duration":341.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":279.09999990463257,"connectEnd":279.09999990463257,"connectStart":279.09999990463257,"domainLookupEnd":279.09999990463257,"domainLookupStart":279.09999990463257,"fetchStart":279.09999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":308,"responseEnd":620.5999999046326,"responseStart":607.2999997138977,"secureConnectionStart":279.09999990463257},{"duration":189.40000009536743,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":574.5999999046326,"connectEnd":574.5999999046326,"connectStart":574.5999999046326,"domainLookupEnd":574.5999999046326,"domainLookupStart":574.5999999046326,"fetchStart":574.5999999046326,"redirectEnd":0,"redirectStart":0,"requestStart":728.2999997138977,"responseEnd":764,"responseStart":763.2999997138977,"secureConnectionStart":574.5999999046326},{"duration":280.90000009536743,"initiatorType":"script","name":"https://www.google-analytics.com/analytics.js","startTime":811,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":811,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1091.9000000953674,"responseStart":0,"secureConnectionStart":0},{"duration":149.90000009536743,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":829.0999999046326,"connectEnd":829.0999999046326,"connectStart":829.0999999046326,"domainLookupEnd":829.0999999046326,"domainLookupStart":829.0999999046326,"fetchStart":829.0999999046326,"redirectEnd":0,"redirectStart":0,"requestStart":944.0999999046326,"responseEnd":979,"responseStart":978.4000000953674,"secureConnectionStart":829.0999999046326},{"duration":142.40000009536743,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/e65b778d185daf5aee24936755b43da6/_/download/contextbatch/js/browser-metrics-plugin.contrib,-_super,-project.issue.navigator,-jira.view.issue,-atl.general/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true&whisper-enabled=true","startTime":1105,"connectEnd":1105,"connectStart":1105,"domainLookupEnd":1105,"domainLookupStart":1105,"fetchStart":1105,"redirectEnd":0,"redirectStart":0,"requestStart":1228.0999999046326,"responseEnd":1247.4000000953674,"responseStart":1246.7999997138977,"secureConnectionStart":1105}],"fetchStart":0,"domainLookupStart":23,"domainLookupEnd":30,"connectStart":30,"connectEnd":53,"secureConnectionStart":40,"requestStart":53,"responseStart":253,"responseEnd":278,"domLoading":257,"domInteractive":1188,"domContentLoadedEventStart":1188,"domContentLoadedEventEnd":1242,"domComplete":1367,"loadEventStart":1367,"loadEventEnd":1367,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":1108.9000000953674},{"name":"bigPipe.sidebar-id.end","time":1109.6999998092651},{"name":"bigPipe.activity-panel-pipe-id.start","time":1109.9000000953674},{"name":"bigPipe.activity-panel-pipe-id.end","time":1110.7999997138977},{"name":"activityTabFullyLoaded","time":1260.2999997138977}],"measures":[],"correlationId":"a6a3f07e704664","effectiveType":"4g","downlink":10,"rtt":0,"serverDuration":121,"dbReadsTimeInMs":17,"dbConnsTimeInMs":27,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}
Bug
Critical
OK to push.