Type:
Task
Priority:
Major
Resolution:
Fixed
This task is about adding ANALYZE [FORMAT=JSON] support for PK filtering
(MDEV-16188 ).
User interface
Examples of PK filtering for EXPLAIN and EXPLAIN FORMAT=JSON:
id 1
select_type SIMPLE
table lineitem
type range|filter
possible_keys PRIMARY,i_l_shipdate,i_l_receiptdate,i_l_orderkey,i_l_orderkey_quantity
key i_l_receiptdate|i_l_shipdate
key_len 4|4
ref NULL
rows 6 (0%)
Extra Using index condition; Using where; Using filter
The filter shows rows="6 (0%)"
in ANALYZE, there will be "r_rows" column.
It should match the rows column, that is, display "nnn (mmm%)" where mmm% - is
the observed selectivity of the filter. If the filter was discared, we should
indicate that in some way (exactly what to print is to be determined).
With FORMAT=JSON we get more date:
+ "rowid_filter": {
+ "range": {
+ "key": "i_l_shipdate",
+ "used_key_parts": ["l_shipDATE"]
+ },
+ "rows": 8,
+ "selectivity_pct": 0.1332
+ },
ANALYZE FORMAT=JSON output should include:
r_rows for the quick select that was used to populate the filter.
if the filter was discarded (because it was too big), this should be indicated.
The total time to populate (i.e. build) the filter should be included as well.
selectivity_pct should get "r_selectivity_pct" which will show the observed filter selectivity.
(I am not sure if we should include the time it took to make checks in the filter).
Implementation
class Explain_rowid_filter should get a pointer to a "Rowid filter tracker
object" (which will exist for all queries, both ANALYZE or not). (just like
e.g. Explain_update has a pointer to Filesort_tracker object).
Filesort_tracker is a good example of a complex tracker object.
There should be a Rowid_filter_tracker object with information about the query
execution. It should have:
a member to track the time to populate the filter.
a member to track the time/#rows for table access while populating the
filter.
A member which indicates whether filter construction finished or was aborted.
(Optionally) A member which indicates how big the buffer size was.
Two counters: one counts how many times the filter was invoked, the other
counts how many times the filter has filter out the record. Dividing one by
another produces r_selectivity_pct.
Tracking the time to build the filter
See Explain_select::time_tracker and its use in JOIN::exec.
Tracking table access and r_rows for building the filter.
At the moment, storage engine objects (like ha_innobase) have "tracker" objects
inside them (see handler::tracker).
The question is: does MDEV-16188 use the same handler object to build the
filter (phase #1) and to perform the "primary" table access by the main query
(phase #2) ?
If yes: we will need to switch handler::tracker objects after building the
filter has been finished (as we want time/#rows for phase #1 and phase #2 to be
accounted separately)
If no: then, we'll need to set the secondary handler object's h->tracker to
point to our tracker.
Printing
void Explain_rowid_filter::print_explain_json(Explain_query *query,
Json_writer *writer,
bool is_analyze)
has "is_analyze" argument. This controls whether analyze data should be
printed.
is part of
MDEV-16188
Use in-memory PK filters built from range index scans
Closed
{"report":{"fcp":1408.1000003814697,"ttfb":286.7000002861023,"pageVisibility":"visible","entityId":71680,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":0.5,"journeyId":"9b6e5b21-0187-4c09-a997-e56a895cfc94","navigationType":0,"readyForUser":1487.8000001907349,"redirectCount":0,"resourceLoadedEnd":1547.7000002861023,"resourceLoadedStart":292.30000019073486,"resourceTiming":[{"duration":639.5999999046326,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":292.30000019073486,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":292.30000019073486,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":931.9000000953674,"responseStart":0,"secureConnectionStart":0},{"duration":639.5999999046326,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true&whisper-enabled=true","startTime":292.6000003814697,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":292.6000003814697,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":932.2000002861023,"responseStart":0,"secureConnectionStart":0},{"duration":649.0999999046326,"initiatorType":"script","name":"https://jira.mariadb.org/s/0917945aaa57108d00c5076fea35e069-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":292.80000019073486,"connectEnd":292.80000019073486,"connectStart":292.80000019073486,"domainLookupEnd":292.80000019073486,"domainLookupStart":292.80000019073486,"fetchStart":292.80000019073486,"redirectEnd":0,"redirectStart":0,"requestStart":292.80000019073486,"responseEnd":941.9000000953674,"responseStart":941.9000000953674,"secureConnectionStart":292.80000019073486},{"duration":703.2000002861023,"initiatorType":"script","name":"https://jira.mariadb.org/s/2d8175ec2fa4c816e8023260bd8c1786-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true&whisper-enabled=true","startTime":293,"connectEnd":293,"connectStart":293,"domainLookupEnd":293,"domainLookupStart":293,"fetchStart":293,"redirectEnd":0,"redirectStart":0,"requestStart":293,"responseEnd":996.2000002861023,"responseStart":996.2000002861023,"secureConnectionStart":293},{"duration":706.9000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/s/a9324d6758d385eb45c462685ad88f1d-CDN/lu2cib/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":293.2000002861023,"connectEnd":293.2000002861023,"connectStart":293.2000002861023,"domainLookupEnd":293.2000002861023,"domainLookupStart":293.2000002861023,"fetchStart":293.2000002861023,"redirectEnd":0,"redirectStart":0,"requestStart":293.2000002861023,"responseEnd":1000.1000003814697,"responseStart":1000.1000003814697,"secureConnectionStart":293.2000002861023},{"duration":707.0999999046326,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":293.40000009536743,"connectEnd":293.40000009536743,"connectStart":293.40000009536743,"domainLookupEnd":293.40000009536743,"domainLookupStart":293.40000009536743,"fetchStart":293.40000009536743,"redirectEnd":0,"redirectStart":0,"requestStart":293.40000009536743,"responseEnd":1000.5,"responseStart":1000.5,"secureConnectionStart":293.40000009536743},{"duration":707.2999997138977,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":293.6000003814697,"connectEnd":293.6000003814697,"connectStart":293.6000003814697,"domainLookupEnd":293.6000003814697,"domainLookupStart":293.6000003814697,"fetchStart":293.6000003814697,"redirectEnd":0,"redirectStart":0,"requestStart":293.6000003814697,"responseEnd":1000.9000000953674,"responseStart":1000.9000000953674,"secureConnectionStart":293.6000003814697},{"duration":798.5999999046326,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2cib/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":293.7000002861023,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":293.7000002861023,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1092.3000001907349,"responseStart":0,"secureConnectionStart":0},{"duration":707.4000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":293.90000009536743,"connectEnd":293.90000009536743,"connectStart":293.90000009536743,"domainLookupEnd":293.90000009536743,"domainLookupStart":293.90000009536743,"fetchStart":293.90000009536743,"redirectEnd":0,"redirectStart":0,"requestStart":293.90000009536743,"responseEnd":1001.3000001907349,"responseStart":1001.3000001907349,"secureConnectionStart":293.90000009536743},{"duration":798.2999997138977,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":294.1000003814697,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":294.1000003814697,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1092.4000000953674,"responseStart":0,"secureConnectionStart":0},{"duration":707.6999998092651,"initiatorType":"script","name":"https://jira.mariadb.org/s/5d5e8fe91fbc506585e83ea3b62ccc4b-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":294.2000002861023,"connectEnd":294.2000002861023,"connectStart":294.2000002861023,"domainLookupEnd":294.2000002861023,"domainLookupStart":294.2000002861023,"fetchStart":294.2000002861023,"redirectEnd":0,"redirectStart":0,"requestStart":294.2000002861023,"responseEnd":1001.9000000953674,"responseStart":1001.9000000953674,"secureConnectionStart":294.2000002861023},{"duration":1252.0999999046326,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":295.2000002861023,"connectEnd":295.2000002861023,"connectStart":295.2000002861023,"domainLookupEnd":295.2000002861023,"domainLookupStart":295.2000002861023,"fetchStart":295.2000002861023,"redirectEnd":0,"redirectStart":0,"requestStart":295.2000002861023,"responseEnd":1547.3000001907349,"responseStart":1547.3000001907349,"secureConnectionStart":295.2000002861023},{"duration":1252.4000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":295.30000019073486,"connectEnd":295.30000019073486,"connectStart":295.30000019073486,"domainLookupEnd":295.30000019073486,"domainLookupStart":295.30000019073486,"fetchStart":295.30000019073486,"redirectEnd":0,"redirectStart":0,"requestStart":295.30000019073486,"responseEnd":1547.7000002861023,"responseStart":1547.7000002861023,"secureConnectionStart":295.30000019073486},{"duration":308,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":1105.4000000953674,"connectEnd":1105.4000000953674,"connectStart":1105.4000000953674,"domainLookupEnd":1105.4000000953674,"domainLookupStart":1105.4000000953674,"fetchStart":1105.4000000953674,"redirectEnd":0,"redirectStart":0,"requestStart":1105.4000000953674,"responseEnd":1413.4000000953674,"responseStart":1413.4000000953674,"secureConnectionStart":1105.4000000953674}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":124,"responseStart":287,"responseEnd":288,"domLoading":290,"domInteractive":1572,"domContentLoadedEventStart":1572,"domContentLoadedEventEnd":1623,"domComplete":2907,"loadEventStart":2907,"loadEventEnd":2909,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":1553.7000002861023},{"name":"bigPipe.sidebar-id.end","time":1554.8000001907349},{"name":"bigPipe.activity-panel-pipe-id.start","time":1555},{"name":"bigPipe.activity-panel-pipe-id.end","time":1557.6000003814697},{"name":"activityTabFullyLoaded","time":1631.2000002861023}],"measures":[],"correlationId":"10fb133d94c4f5","effectiveType":"4g","downlink":9.5,"rtt":0,"serverDuration":102,"dbReadsTimeInMs":11,"dbConnsTimeInMs":21,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}