Type:
Task
Priority:
Minor
Resolution:
Unresolved
Fix Version/s:
None
I would like to get the parse tree of a SELECT statement after the names have been resolved. The goal from there is to develop an automated INDEX generator that comes close to optimal for the query.
Currently, I am experimenting with PHP and regexps and having moderately good success. The name resolution would take care of single-table queries (wherein I can't easily know what is a column versus a keyword, JOINs without fully qualified column names, etc. Also, the parse tree would take care of some edge cases that my regexps don't cover.
I understand that the MariaDB code builds a tree of objects to be fed to the Optimizer. The addon would extract the attributes that I need: constant/column/keyword, column name, table_id, datatype, etc.
I see different approaches for hooking this into the server: a plugin or container, or my preference: something similar to "Optimizer Trace" – turn on a flag, and some JSON is made available.
Yes, I do understand that there are a lot of variations in optimizing indexes – histograms, probes, BKA, MRR, filter percentage, etc. But you might be horrified to hear how many users don't know what an index is or say "but I indexed every column" or have not heard of a 'composite' index or don't know the difference between JOIN and LEFT JOIN or how an auto_increment PK might slow down a query.
I expect my 'addon' will be a useful crutch for novices – They extract the tree; my tool gives them an INDEX recommendation.
relates to
MDEV-17825
EXPLAIN new index suggestion mode
Open
Rick James
made changes -
2022-03-06 20:02
Summary
Expose syntac parse tree with name resolution
Expose syntax parse tree with name resolution
Daniel Black
made changes -
2022-04-14 01:03
Labels
innodb performance
energy innodb performance
{"report":{"fcp":972.3000000715256,"ttfb":213.70000004768372,"pageVisibility":"visible","entityId":107734,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":0.5,"journeyId":"58a37dc0-93f6-4803-9afb-337b94247d99","navigationType":0,"readyForUser":1070.3000000715256,"redirectCount":0,"resourceLoadedEnd":718.1000000238419,"resourceLoadedStart":221.70000004768372,"resourceTiming":[{"duration":166.29999995231628,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2bu7/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":221.70000004768372,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":221.70000004768372,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":388,"responseStart":0,"secureConnectionStart":0},{"duration":166.90000009536743,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2bu7/820016/12ta74/8679b4946efa1a0bb029a3a22206fb5d/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":222,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":222,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":388.90000009536743,"responseStart":0,"secureConnectionStart":0},{"duration":369.89999997615814,"initiatorType":"script","name":"https://jira.mariadb.org/s/fbf975c0cce4b1abf04784eeae9ba1f4-CDN/lu2bu7/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":222.10000002384186,"connectEnd":390.10000002384186,"connectStart":390.10000002384186,"domainLookupEnd":390.10000002384186,"domainLookupStart":390.10000002384186,"fetchStart":222.10000002384186,"redirectEnd":0,"redirectStart":0,"requestStart":390.3000000715256,"responseEnd":592,"responseStart":403.5,"secureConnectionStart":390.10000002384186},{"duration":481.7000000476837,"initiatorType":"script","name":"https://jira.mariadb.org/s/099b33461394b8015fc36c0a4b96e19f-CDN/lu2bu7/820016/12ta74/8679b4946efa1a0bb029a3a22206fb5d/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true","startTime":222.70000004768372,"connectEnd":222.70000004768372,"connectStart":222.70000004768372,"domainLookupEnd":222.70000004768372,"domainLookupStart":222.70000004768372,"fetchStart":222.70000004768372,"redirectEnd":0,"redirectStart":0,"requestStart":391,"responseEnd":704.4000000953674,"responseStart":412.3000000715256,"secureConnectionStart":222.70000004768372},{"duration":194.10000002384186,"initiatorType":"script","name":"https://jira.mariadb.org/s/94c15bff32baef80f4096a08aceae8bc-CDN/lu2bu7/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":223,"connectEnd":223,"connectStart":223,"domainLookupEnd":223,"domainLookupStart":223,"fetchStart":223,"redirectEnd":0,"redirectStart":0,"requestStart":392,"responseEnd":417.10000002384186,"responseStart":415.40000009536743,"secureConnectionStart":223},{"duration":194.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":223.20000004768372,"connectEnd":223.20000004768372,"connectStart":223.20000004768372,"domainLookupEnd":223.20000004768372,"domainLookupStart":223.20000004768372,"fetchStart":223.20000004768372,"redirectEnd":0,"redirectStart":0,"requestStart":393,"responseEnd":417.7000000476837,"responseStart":416.10000002384186,"secureConnectionStart":223.20000004768372},{"duration":200.89999997615814,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":223.30000007152557,"connectEnd":223.30000007152557,"connectStart":223.30000007152557,"domainLookupEnd":223.30000007152557,"domainLookupStart":223.30000007152557,"fetchStart":223.30000007152557,"redirectEnd":0,"redirectStart":0,"requestStart":396.90000009536743,"responseEnd":424.2000000476837,"responseStart":422.40000009536743,"secureConnectionStart":223.30000007152557},{"duration":169.69999992847443,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2bu7/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":223.40000009536743,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":223.40000009536743,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":393.10000002384186,"responseStart":0,"secureConnectionStart":0},{"duration":233.60000002384186,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":223.70000004768372,"connectEnd":223.70000004768372,"connectStart":223.70000004768372,"domainLookupEnd":223.70000004768372,"domainLookupStart":223.70000004768372,"fetchStart":223.70000004768372,"redirectEnd":0,"redirectStart":0,"requestStart":397.60000002384186,"responseEnd":457.3000000715256,"responseStart":455.7000000476837,"secureConnectionStart":223.70000004768372},{"duration":173,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2bu7/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":223.80000007152557,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":223.80000007152557,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":396.8000000715256,"responseStart":0,"secureConnectionStart":0},{"duration":206.20000004768372,"initiatorType":"script","name":"https://jira.mariadb.org/s/3339d87fa2538a859872f2df449bf8d0-CDN/lu2bu7/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":224,"connectEnd":224,"connectStart":224,"domainLookupEnd":224,"domainLookupStart":224,"fetchStart":224,"redirectEnd":0,"redirectStart":0,"requestStart":399.8000000715256,"responseEnd":430.2000000476837,"responseStart":423.10000002384186,"secureConnectionStart":224},{"duration":488.3000000715256,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":229.5,"connectEnd":229.5,"connectStart":229.5,"domainLookupEnd":229.5,"domainLookupStart":229.5,"fetchStart":229.5,"redirectEnd":0,"redirectStart":0,"requestStart":550.9000000953674,"responseEnd":717.8000000715256,"responseStart":711.8000000715256,"secureConnectionStart":229.5},{"duration":488.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":229.60000002384186,"connectEnd":229.60000002384186,"connectStart":229.60000002384186,"domainLookupEnd":229.60000002384186,"domainLookupStart":229.60000002384186,"fetchStart":229.60000002384186,"redirectEnd":0,"redirectStart":0,"requestStart":552.4000000953674,"responseEnd":718.1000000238419,"responseStart":712.4000000953674,"secureConnectionStart":229.60000002384186},{"duration":104.79999995231628,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":734.7000000476837,"connectEnd":734.7000000476837,"connectStart":734.7000000476837,"domainLookupEnd":734.7000000476837,"domainLookupStart":734.7000000476837,"fetchStart":734.7000000476837,"redirectEnd":0,"redirectStart":0,"requestStart":806.7000000476837,"responseEnd":839.5,"responseStart":838.9000000953674,"secureConnectionStart":734.7000000476837}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":31,"responseStart":213,"responseEnd":228,"domLoading":217,"domInteractive":1120,"domContentLoadedEventStart":1120,"domContentLoadedEventEnd":1163,"domComplete":1670,"loadEventStart":1670,"loadEventEnd":1671,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":1101.7000000476837},{"name":"bigPipe.sidebar-id.end","time":1102.5},{"name":"bigPipe.activity-panel-pipe-id.start","time":1102.6000000238419},{"name":"bigPipe.activity-panel-pipe-id.end","time":1103.9000000953674},{"name":"activityTabFullyLoaded","time":1180.9000000953674}],"measures":[],"correlationId":"c4fbed5791bd99","effectiveType":"4g","downlink":9,"rtt":0,"serverDuration":97,"dbReadsTimeInMs":14,"dbConnsTimeInMs":21,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}