The adaptive hash index is disabled for temporary tables in MariaDB Server 10.2+.
Starting with MariaDB Server 10.2, the btr_search_latch has been partitioned into btr_search_latches[] of innodb_adaptive_hash_index_parts (btr_ahi_parts) elements, default 8, ranging from 1 to 512. The adaptive hash index itself is still global; only the latch has been partitioned by essentially index_id (which is still global; in MySQL 5.7, on my suggestion, some previsions were implemented to allow each tablespace to have their own domain of index_id):
So, all requests for a particular index will always use the same btr_search_latches[] element.
Marko Mäkelä
added a comment - The adaptive hash index is disabled for temporary tables in MariaDB Server 10.2+.
Starting with MariaDB Server 10.2, the btr_search_latch has been partitioned into btr_search_latches[] of innodb_adaptive_hash_index_parts ( btr_ahi_parts ) elements, default 8, ranging from 1 to 512. The adaptive hash index itself is still global; only the latch has been partitioned by essentially index_id (which is still global; in MySQL 5.7, on my suggestion, some previsions were implemented to allow each tablespace to have their own domain of index_id):
UNIV_INLINE
rw_lock_t*
btr_get_search_latch( const dict_index_t* index)
{
ut_ad(index != NULL);
ulint ifold = ut_fold_ulint_pair( static_cast <ulint>(index->id),
static_cast <ulint>(index->space));
return (btr_search_latches[ifold % btr_ahi_parts]);
}
So, all requests for a particular index will always use the same btr_search_latches[] element.
I tested now a lot more workloads to find cases where the AHI increases performance. The biggest improvements could be found for strictly read-only workloads. But even then it depends a lot of the specific queries. From the sysbench OLTP query mix only the point-select workload benefits from the AHI. Depending on thread count I see up to 7% improvement when the AHI is enabled. None of the range queries profits from the AHI. Details are in attached spreadsheet AHI_point-select.ods
I also tested a modified read-only workload where rows are accessed via secondary key. Now point selects don't benefit from the AHI. Range queries benefit from the AHI at low thread counts but are considerably slower at thread counts higher than 8. The reason is the btr_search_latch that becomes extremely busy. This can be seen in detail in attached spreadsheet AHI_secondary_range.ods
Axel Schwenke
added a comment - - edited I tested now a lot more workloads to find cases where the AHI increases performance. The biggest improvements could be found for strictly read-only workloads. But even then it depends a lot of the specific queries. From the sysbench OLTP query mix only the point-select workload benefits from the AHI. Depending on thread count I see up to 7% improvement when the AHI is enabled. None of the range queries profits from the AHI. Details are in attached spreadsheet AHI_point-select.ods
I also tested a modified read-only workload where rows are accessed via secondary key. Now point selects don't benefit from the AHI. Range queries benefit from the AHI at low thread counts but are considerably slower at thread counts higher than 8. The reason is the btr_search_latch that becomes extremely busy. This can be seen in detail in attached spreadsheet AHI_secondary_range.ods
Another tested workload is TPC-C, using the Lua implementation for sysbench 1.0. Since this workload includes heavy writes (about 50% of the queries are writes) the btr_search_latch becomes the bottleneck. Even partitioning the AHI does not help with that. Here are the TPC-C queries per scond for different thread counts using AHI with 16/1 partitions vs. no AHI:
threads 16-AHI 1-AHI no-AHI
1 8489.5 9057.8 9599.2
2 15614 17754 18439
4 29015 34259 35078
8 46936 62416 64824
16 67218 96819 101095
32 103217 145655 154564
64 111819 157915 177283
128 114516 161011 175250
256 113827 148542 162780
Axel Schwenke
added a comment - Another tested workload is TPC-C, using the Lua implementation for sysbench 1.0. Since this workload includes heavy writes (about 50% of the queries are writes) the btr_search_latch becomes the bottleneck. Even partitioning the AHI does not help with that. Here are the TPC-C queries per scond for different thread counts using AHI with 16/1 partitions vs. no AHI:
threads 16-AHI 1-AHI no-AHI
1 8489.5 9057.8 9599.2
2 15614 17754 18439
4 29015 34259 35078
8 46936 62416 64824
16 67218 96819 101095
32 103217 145655 154564
64 111819 157915 177283
128 114516 161011 175250
256 113827 148542 162780
Now here is a final set of results, testing a mixture of point-selects (which benefit from AHI) and writes (which lead to contention of the btr_search_latch). The dataset and queries are from sysbench OLTP, the writes are updates on indexed/non-indexed columns. I vary the percentage of writes between 1% and 20%.
even with 1% writes the btr_search_latch contention is visible in the case of a single AHI partition
with 2% or more writes, the configuration with AHI disabled starts to shine
at 5% writes a single AHI partition delivers only little more than half the througput compared to no AHI - at least for thread counts up to 64
at very high thread counts the difference flattens out. This is due to other locks becoming busy (log_sys_write_mutex, LOCK_table_cache)
I'm trying to draw a conclusion: the AHI has the potential to increase performance by up to 10% - but only with perfectly matching workload. Further more this workload must be (almost) read-only. Even 2% writes is enough to cancel any positive effect of the AHI. I'd say it's OK to remove the AHI from InnoDB. It's causing more trouble than it's worth.
Axel Schwenke
added a comment - - edited Now here is a final set of results, testing a mixture of point-selects (which benefit from AHI) and writes (which lead to contention of the btr_search_latch). The dataset and queries are from sysbench OLTP, the writes are updates on indexed/non-indexed columns. I vary the percentage of writes between 1% and 20%.
For details, look at the attached spread sheet AHI_vs_writes.ods .
Some observations:
even with 1% writes the btr_search_latch contention is visible in the case of a single AHI partition
with 2% or more writes, the configuration with AHI disabled starts to shine
at 5% writes a single AHI partition delivers only little more than half the througput compared to no AHI - at least for thread counts up to 64
at very high thread counts the difference flattens out. This is due to other locks becoming busy (log_sys_write_mutex, LOCK_table_cache)
I'm trying to draw a conclusion: the AHI has the potential to increase performance by up to 10% - but only with perfectly matching workload. Further more this workload must be (almost) read-only. Even 2% writes is enough to cancel any positive effect of the AHI. I'd say it's OK to remove the AHI from InnoDB. It's causing more trouble than it's worth.
We recently encountered a case where a customer experienced a 50% performance drop after upgrading from 10.4 to 10.6.19. Specifically, their long-running queries saw execution times double, increasing from 1 hour to over 2 hours.
Upon investigation, we found the root cause: AHI (Adaptive Hash Index) was disabled by default in 10.6, whereas it was enabled by default in 10.4.
This seems to show that there are still scenarios, perhaps less common, where this feature has a significant impact on performance, so I decided to reopen the ticket
Patrizio Tamorri
added a comment - We recently encountered a case where a customer experienced a 50% performance drop after upgrading from 10.4 to 10.6.19. Specifically, their long-running queries saw execution times double, increasing from 1 hour to over 2 hours.
Upon investigation, we found the root cause: AHI (Adaptive Hash Index) was disabled by default in 10.6, whereas it was enabled by default in 10.4.
This seems to show that there are still scenarios, perhaps less common, where this feature has a significant impact on performance, so I decided to reopen the ticket
People
Axel Schwenke
Eugene Kosov (Inactive)
Votes:
1Vote for this issue
Watchers:
9Start watching this issue
Dates
Created:
Updated:
Git Integration
Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.
{"report":{"fcp":1910.6999998092651,"ttfb":825.1999998092651,"pageVisibility":"visible","entityId":70355,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":0.5,"journeyId":"93c06fb1-bc00-409d-99f7-ee69267a2b31","navigationType":0,"readyForUser":2045,"redirectCount":0,"resourceLoadedEnd":2289.8999996185303,"resourceLoadedStart":831.7999997138977,"resourceTiming":[{"duration":438.7000002861023,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":831.7999997138977,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":831.7999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1270.5,"responseStart":0,"secureConnectionStart":0},{"duration":438.69999980926514,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true&whisper-enabled=true","startTime":832.0999999046326,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":832.0999999046326,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1270.7999997138977,"responseStart":0,"secureConnectionStart":0},{"duration":499.2000002861023,"initiatorType":"script","name":"https://jira.mariadb.org/s/0917945aaa57108d00c5076fea35e069-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":832.2999997138977,"connectEnd":832.2999997138977,"connectStart":832.2999997138977,"domainLookupEnd":832.2999997138977,"domainLookupStart":832.2999997138977,"fetchStart":832.2999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":832.2999997138977,"responseEnd":1331.5,"responseStart":1331.3999996185303,"secureConnectionStart":832.2999997138977},{"duration":622.8000001907349,"initiatorType":"script","name":"https://jira.mariadb.org/s/2d8175ec2fa4c816e8023260bd8c1786-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true&whisper-enabled=true","startTime":832.3999996185303,"connectEnd":832.3999996185303,"connectStart":832.3999996185303,"domainLookupEnd":832.3999996185303,"domainLookupStart":832.3999996185303,"fetchStart":832.3999996185303,"redirectEnd":0,"redirectStart":0,"requestStart":832.3999996185303,"responseEnd":1455.1999998092651,"responseStart":1455.1999998092651,"secureConnectionStart":832.3999996185303},{"duration":625.8000001907349,"initiatorType":"script","name":"https://jira.mariadb.org/s/a9324d6758d385eb45c462685ad88f1d-CDN/lu2cib/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":832.6999998092651,"connectEnd":832.6999998092651,"connectStart":832.6999998092651,"domainLookupEnd":832.6999998092651,"domainLookupStart":832.6999998092651,"fetchStart":832.6999998092651,"redirectEnd":0,"redirectStart":0,"requestStart":832.6999998092651,"responseEnd":1458.5,"responseStart":1458.5,"secureConnectionStart":832.6999998092651},{"duration":626.4000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":832.8999996185303,"connectEnd":832.8999996185303,"connectStart":832.8999996185303,"domainLookupEnd":832.8999996185303,"domainLookupStart":832.8999996185303,"fetchStart":832.8999996185303,"redirectEnd":0,"redirectStart":0,"requestStart":832.8999996185303,"responseEnd":1459.2999997138977,"responseStart":1459.2999997138977,"secureConnectionStart":832.8999996185303},{"duration":626.6999998092651,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":833.0999999046326,"connectEnd":833.0999999046326,"connectStart":833.0999999046326,"domainLookupEnd":833.0999999046326,"domainLookupStart":833.0999999046326,"fetchStart":833.0999999046326,"redirectEnd":0,"redirectStart":0,"requestStart":833.0999999046326,"responseEnd":1459.7999997138977,"responseStart":1459.7999997138977,"secureConnectionStart":833.0999999046326},{"duration":678.4000000953674,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2cib/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":833.2999997138977,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":833.2999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1511.6999998092651,"responseStart":0,"secureConnectionStart":0},{"duration":626.9000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":833.3999996185303,"connectEnd":833.3999996185303,"connectStart":833.3999996185303,"domainLookupEnd":833.3999996185303,"domainLookupStart":833.3999996185303,"fetchStart":833.3999996185303,"redirectEnd":0,"redirectStart":0,"requestStart":833.3999996185303,"responseEnd":1460.2999997138977,"responseStart":1460.2999997138977,"secureConnectionStart":833.3999996185303},{"duration":678.2999997138977,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":833.5999999046326,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":833.5999999046326,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1511.8999996185303,"responseStart":0,"secureConnectionStart":0},{"duration":627.2000002861023,"initiatorType":"script","name":"https://jira.mariadb.org/s/5d5e8fe91fbc506585e83ea3b62ccc4b-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":833.7999997138977,"connectEnd":833.7999997138977,"connectStart":833.7999997138977,"domainLookupEnd":833.7999997138977,"domainLookupStart":833.7999997138977,"fetchStart":833.7999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":833.7999997138977,"responseEnd":1461,"responseStart":1461,"secureConnectionStart":833.7999997138977},{"duration":1447.1000003814697,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":834.8999996185303,"connectEnd":834.8999996185303,"connectStart":834.8999996185303,"domainLookupEnd":834.8999996185303,"domainLookupStart":834.8999996185303,"fetchStart":834.8999996185303,"redirectEnd":0,"redirectStart":0,"requestStart":834.8999996185303,"responseEnd":2282,"responseStart":2282,"secureConnectionStart":834.8999996185303},{"duration":1454.8999996185303,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":835,"connectEnd":835,"connectStart":835,"domainLookupEnd":835,"domainLookupStart":835,"fetchStart":835,"redirectEnd":0,"redirectStart":0,"requestStart":835,"responseEnd":2289.8999996185303,"responseStart":2289.8999996185303,"secureConnectionStart":835},{"duration":720.9000000953674,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":1561.5999999046326,"connectEnd":1561.5999999046326,"connectStart":1561.5999999046326,"domainLookupEnd":1561.5999999046326,"domainLookupStart":1561.5999999046326,"fetchStart":1561.5999999046326,"redirectEnd":0,"redirectStart":0,"requestStart":1561.5999999046326,"responseEnd":2282.5,"responseStart":2282.5,"secureConnectionStart":1561.5999999046326},{"duration":459.7000002861023,"initiatorType":"script","name":"https://www.google-analytics.com/analytics.js","startTime":1924.3999996185303,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":1924.3999996185303,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":2384.0999999046326,"responseStart":0,"secureConnectionStart":0}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":655,"responseStart":825,"responseEnd":829,"domLoading":829,"domInteractive":2334,"domContentLoadedEventStart":2334,"domContentLoadedEventEnd":2383,"domComplete":3401,"loadEventStart":3401,"loadEventEnd":3402,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":2296.8999996185303},{"name":"bigPipe.sidebar-id.end","time":2297.7999997138977},{"name":"bigPipe.activity-panel-pipe-id.start","time":2298},{"name":"bigPipe.activity-panel-pipe-id.end","time":2300.2999997138977},{"name":"activityTabFullyLoaded","time":2427.0999999046326}],"measures":[],"correlationId":"76df6be3cab6a4","effectiveType":"4g","downlink":9.2,"rtt":0,"serverDuration":113,"dbReadsTimeInMs":17,"dbConnsTimeInMs":27,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}
The adaptive hash index is disabled for temporary tables in MariaDB Server 10.2+.
Starting with MariaDB Server 10.2, the btr_search_latch has been partitioned into btr_search_latches[] of innodb_adaptive_hash_index_parts (btr_ahi_parts) elements, default 8, ranging from 1 to 512. The adaptive hash index itself is still global; only the latch has been partitioned by essentially index_id (which is still global; in MySQL 5.7, on my suggestion, some previsions were implemented to allow each tablespace to have their own domain of index_id):
UNIV_INLINE
rw_lock_t*
{
ut_ad(index != NULL);
}
So, all requests for a particular index will always use the same btr_search_latches[] element.