I have a MariaDB SQL query (actually very big) using chaining WITH statements like this :
WITH tbl_base AS (
|
|
-- Base level sub-query on a table containing about 22000 rows, using LEFT JOIN on 5 other tables, details not relevant here
|
SELECT ……… FROM <many things>
|
|
), tbl_middle AS (
|
|
-- Intermediate level sub-query #1 for intermediate computing, details not relevant here
|
SELECT ……… FROM tbl_middle …
|
|
), tbl_states AS (
|
|
-- Intermediate level sub-query #2 for intermediate computing, details not relevant here
|
SELECT
|
……… AS state,
|
……… AS `elec`
|
FROM tbl_states …
|
|
), tbl_sums AS (
|
|
-- Final grouping query
|
SELECT
|
`state`,
|
SUM(NOT `elec`) AS `vls`,
|
SUM(`elec`) AS `vae`,
|
count(`state`) AS `all`
|
FROM tbl_states
|
GROUP BY `state`
|
ORDER BY `state`
|
|
)
|
|
-- Additional query which could be avoided if left as simple as this but it will not stay as simple as this
|
SELECT `state`, `vae`, `vls`, `both` FROM tbl_sums
|
This query makes statistics on two kinds of objects which can each have 8 different states (bad, longrun, lost, maint, ok, removed, run or warn), so the result is a tiny view of only 8 rows and 4 columns (state, vls, vae and both).
On my server it needs about 500 mS to execute, which is acceptable.
Now I want to add rows for creating some combined states, so I replaced the last SELECT by this UNION:
SELECT `state`, `vae`, `vls`, `both` FROM tbl_sums
|
UNION
|
SELECT 'all_sta',SUM(`vls`),SUM(`vae`),SUM(`both`) FROM tbl_sums WHERE `state` IN ('ok','warn','bad','maint')
|
UNION
|
SELECT 'all_run',SUM(`vls`),SUM(`vae`),SUM(`both`) FROM tbl_sums WHERE `state` IN ('run','longrun')
|
UNION
|
SELECT 'operative',SUM(`vls`),SUM(`vae`),SUM(`both`) FROM tbl_sums WHERE `state` IN ('ok','warn','run','longrun')
|
UNION
|
SELECT 'unusable',SUM(`vls`),SUM(`vae`),SUM(`both`) FROM tbl_sums WHERE `state` IN ('bad','maint')
|
UNION
|
SELECT 'present',SUM(`vls`),SUM(`vae`),SUM(`both`) FROM tbl_sums WHERE `state` IN ('ok','warn','bad','maint','run','longrun')
|
UNION
|
SELECT 'missing',SUM(`vls`),SUM(`vae`),SUM(`both`) FROM tbl_sums WHERE `state` IN ('removed','lost')
|
UNION
|
SELECT 'total',SUM(`vls`),SUM(`vae`),SUM(`both`) FROM tbl_sums
|
I works fine, but I am very surprised to observe that the query needs now about 4 seconds to run, thus 8 times the initial 500 mS, likely because I am reusing the last WITH element 8 times.
Apparently MariaDB execute this by computing 8 times the whole sub-query chain instead of temporarily storing the last one and reusing it 8 times, whereas I was expecting it to be able to reuse the last tiny result in not even a millisecond.
If I use EXPLAIN on the last query, I get a result like this (see joined picture) where we could clearly see that the whole sub-queries are effectively explored 8 times.
So I'd like to know if I made something bad, or if I should change some settings in order to have it working better, or if this is simply a bug in MariaDB (versions 10.3.29 or 10.5.11 in this case), as WITH is a quite new feature in MariaDB.
I know I could create a temporary table, but as it is very complicated to execute multiple statements for ONE result in my application, I was precisely expecting to use WITH in order to avoid it.
This is likely irrelevant, but the server is running on Linux Debian 11.
Also, I do that using MyISAM tables. Not tried with others.
{"report":{"fcp":1087,"ttfb":365.5,"pageVisibility":"visible","entityId":102681,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":0.5,"journeyId":"ceebc313-aebe-4490-86cd-7bfc7ec42c6a","navigationType":0,"readyForUser":1188.800000011921,"redirectCount":0,"resourceLoadedEnd":804.5,"resourceLoadedStart":371.39999997615814,"resourceTiming":[{"duration":134.4000000357628,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2bu7/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":371.39999997615814,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":371.39999997615814,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":505.80000001192093,"responseStart":0,"secureConnectionStart":0},{"duration":134.5,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2bu7/820016/12ta74/8679b4946efa1a0bb029a3a22206fb5d/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":371.5999999642372,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":371.5999999642372,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":506.0999999642372,"responseStart":0,"secureConnectionStart":0},{"duration":314.69999998807907,"initiatorType":"script","name":"https://jira.mariadb.org/s/fbf975c0cce4b1abf04784eeae9ba1f4-CDN/lu2bu7/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":371.89999997615814,"connectEnd":371.89999997615814,"connectStart":371.89999997615814,"domainLookupEnd":371.89999997615814,"domainLookupStart":371.89999997615814,"fetchStart":371.89999997615814,"redirectEnd":0,"redirectStart":0,"requestStart":512.0999999642372,"responseEnd":686.5999999642372,"responseStart":537.8000000119209,"secureConnectionStart":371.89999997615814},{"duration":432.4000000357628,"initiatorType":"script","name":"https://jira.mariadb.org/s/099b33461394b8015fc36c0a4b96e19f-CDN/lu2bu7/820016/12ta74/8679b4946efa1a0bb029a3a22206fb5d/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true","startTime":372.0999999642372,"connectEnd":372.0999999642372,"connectStart":372.0999999642372,"domainLookupEnd":372.0999999642372,"domainLookupStart":372.0999999642372,"fetchStart":372.0999999642372,"redirectEnd":0,"redirectStart":0,"requestStart":512.8999999761581,"responseEnd":804.5,"responseStart":538.8000000119209,"secureConnectionStart":372.0999999642372},{"duration":169.60000002384186,"initiatorType":"script","name":"https://jira.mariadb.org/s/94c15bff32baef80f4096a08aceae8bc-CDN/lu2bu7/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":372.19999998807907,"connectEnd":372.19999998807907,"connectStart":372.19999998807907,"domainLookupEnd":372.19999998807907,"domainLookupStart":372.19999998807907,"fetchStart":372.19999998807907,"redirectEnd":0,"redirectStart":0,"requestStart":513,"responseEnd":541.8000000119209,"responseStart":540,"secureConnectionStart":372.19999998807907},{"duration":169.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":372.39999997615814,"connectEnd":372.39999997615814,"connectStart":372.39999997615814,"domainLookupEnd":372.39999997615814,"domainLookupStart":372.39999997615814,"fetchStart":372.39999997615814,"redirectEnd":0,"redirectStart":0,"requestStart":513.8000000119209,"responseEnd":541.8999999761581,"responseStart":540.5,"secureConnectionStart":372.39999997615814},{"duration":172.4000000357628,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":372.5999999642372,"connectEnd":372.5999999642372,"connectStart":372.5999999642372,"domainLookupEnd":372.5999999642372,"domainLookupStart":372.5999999642372,"fetchStart":372.5999999642372,"redirectEnd":0,"redirectStart":0,"requestStart":515.0999999642372,"responseEnd":545,"responseStart":542.5999999642372,"secureConnectionStart":372.5999999642372},{"duration":138.10000002384186,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2bu7/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":372.89999997615814,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":372.89999997615814,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":511,"responseStart":0,"secureConnectionStart":0},{"duration":172.30000001192093,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":373,"connectEnd":373,"connectStart":373,"domainLookupEnd":373,"domainLookupStart":373,"fetchStart":373,"redirectEnd":0,"redirectStart":0,"requestStart":515.3999999761581,"responseEnd":545.3000000119209,"responseStart":543.1999999880791,"secureConnectionStart":373},{"duration":139.30000001192093,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2bu7/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":373.19999998807907,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":373.19999998807907,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":512.5,"responseStart":0,"secureConnectionStart":0},{"duration":184,"initiatorType":"script","name":"https://jira.mariadb.org/s/3339d87fa2538a859872f2df449bf8d0-CDN/lu2bu7/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":373.30000001192093,"connectEnd":373.30000001192093,"connectStart":373.30000001192093,"domainLookupEnd":373.30000001192093,"domainLookupStart":373.30000001192093,"fetchStart":373.30000001192093,"redirectEnd":0,"redirectStart":0,"requestStart":516.3000000119209,"responseEnd":557.3000000119209,"responseStart":543.5999999642372,"secureConnectionStart":373.30000001192093},{"duration":403.19999998807907,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":374.30000001192093,"connectEnd":374.30000001192093,"connectStart":374.30000001192093,"domainLookupEnd":374.30000001192093,"domainLookupStart":374.30000001192093,"fetchStart":374.30000001192093,"redirectEnd":0,"redirectStart":0,"requestStart":753.5999999642372,"responseEnd":777.5,"responseStart":772.5999999642372,"secureConnectionStart":374.30000001192093},{"duration":403.2999999523163,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":374.30000001192093,"connectEnd":374.30000001192093,"connectStart":374.30000001192093,"domainLookupEnd":374.30000001192093,"domainLookupStart":374.30000001192093,"fetchStart":374.30000001192093,"redirectEnd":0,"redirectStart":0,"requestStart":753.8999999761581,"responseEnd":777.5999999642372,"responseStart":773.3000000119209,"secureConnectionStart":374.30000001192093},{"duration":255.69999998807907,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":813.8000000119209,"connectEnd":813.8000000119209,"connectStart":813.8000000119209,"domainLookupEnd":813.8000000119209,"domainLookupStart":813.8000000119209,"fetchStart":813.8000000119209,"redirectEnd":0,"redirectStart":0,"requestStart":1037.199999988079,"responseEnd":1069.5,"responseStart":1068.699999988079,"secureConnectionStart":813.8000000119209}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":165,"responseStart":366,"responseEnd":369,"domLoading":369,"domInteractive":1294,"domContentLoadedEventStart":1295,"domContentLoadedEventEnd":1347,"domComplete":2124,"loadEventStart":2124,"loadEventEnd":2124,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":1273.0999999642372},{"name":"bigPipe.sidebar-id.end","time":1273.800000011921},{"name":"bigPipe.activity-panel-pipe-id.start","time":1274},{"name":"bigPipe.activity-panel-pipe-id.end","time":1275},{"name":"activityTabFullyLoaded","time":1367.300000011921}],"measures":[],"correlationId":"9dece4287e28f3","effectiveType":"4g","downlink":9.2,"rtt":0,"serverDuration":111,"dbReadsTimeInMs":13,"dbConnsTimeInMs":20,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}
Hello.
I posted this about 20 months ago.
Does someone even read it?