unsigned int width = 64; ulonglong = long long unsigned int; uint = unsigned int]: Assertion `n < width' failed.
The stack trace:
#3 0x00007ffff5221472 in __GI___assert_fail (assertion=0x555556c6cb4a "n < width", file=0x555556c6cb08 "/home/psergey/dev-git2/ MariaDBEnterprise-10.6.14/sql/sql_bitmap.h", line=77, function=0x555556c6d380 <Bitmap<64u>::bit_mask(unsigned int) const:: __PRETTY_FUNCTION__> "ulonglong Bitmap<width>::bit_mask(uint) const [with unsigned int width = 64; ulonglong = long long unsigned int; uint = unsigned int]") at assert.c:101
#4 0x0000555555d94d37 in Bitmap<64u>::bit_mask (this=0x7fff341d8918, n=64) at /home/psergey/dev-git2/MariaDBEnterprise-10.6.14/sql/ sql_bitmap.h:77
#5 0x0000555555e41d69 in Bitmap<64u>::set_bit (this=0x7fff341d8918, n=64) at /home/psergey/dev-git2/MariaDBEnterprise-10.6.14/sql/ sql_bitmap.h:109
#6 0x00005555564541d2 in Item_bool_func::get_mm_parts (this=0x7fff3419e690, param=0x7ffff40f3c50, field=0x7fff3407a2d0, type=Item_func::EQ_FUNC, value=0x7fff3418edd8) at /home/psergey/dev-git2/MariaDBEnterprise-10.6.14/sql/opt_range.cc:8659
#7 0x0000555556453d11 in Item_equal::get_mm_tree (this=0x7fff3419e690, param=0x7ffff40f3c50, cond_ptr=0x7fff3419e7c0) at /home/psergey/ dev-git2/MariaDBEnterprise-10.6.14/sql/opt_range.cc:8568
#8 0x00005555564530e5 in Item_cond_and::get_mm_tree (this=0x7fff3402a9c0, param=0x7ffff40f3c50, cond_ptr=0x7fff34185438) at /home/ psergey/dev-git2/MariaDBEnterprise-10.6.14/sql/opt_range.cc:8346
#9 0x0000555556446c01 in calculate_cond_selectivity_for_table (thd=0x7fff34000d78, table=0x7fff340578a8, cond=0x7fff34185438) at /home/ psergey/dev-git2/MariaDBEnterprise-10.6.14/sql/opt_range.cc:3517
#10 0x00007fff341c0d70 in ?? ()
#11 0x00007fff341c0d90 in ?? ()
#12 0x00007fff341c0db0 in ?? ()
#13 0x00007fff341c0dd0 in ?? ()
In the testcase I'm trying, calculate_cond_selectivity_for_table() is trying to get estimates for 168 columns:
Thread 14 "mysqld" hit Breakpoint 4, create_key_parts_for_pseudo_indexes (param=0x7ffff40f3c50, used_fields=0x7fff28057a78) at /home/ psergey/dev-git2/MariaDBEnterprise-10.6.14/sql/opt_range.cc:3183
(gdb) p keys
$54 = 168
the range optimizer can handle at most 64 indexes.
(This is all I see so far. There may be other issues)
ralf.gebhardt, the problem and the fix will be still valid for the ES.
There is only one thing: I might need to adjust the patch's testcase to use > 128 indexes (currently it uses 100). The rest of the patch is valid.
Sergei Petrunia
added a comment - ralf.gebhardt , the problem and the fix will be still valid for the ES.
There is only one thing: I might need to adjust the patch's testcase to use > 128 indexes (currently it uses 100). The rest of the patch is valid.
@spetruna, how do you plan to choose the fields that we should create a pseudo index for?
I assume we should first use fields that are used with an operator we can use with the range optimizer ?
It looks like table->cond_set that is used is already setting the bit's for 'field op constant'
If there are more than 128 of them, which ones to choose?
The first 128 that we find in create_key_parts_for_pseudo_index() ?
Another option would be to change SEL_TREE::keys_map to have an explicit key map allocated in get_mm_parts()
This would allows to have any number of pseudo keys.
Michael Widenius
added a comment - @spetruna, how do you plan to choose the fields that we should create a pseudo index for?
I assume we should first use fields that are used with an operator we can use with the range optimizer ?
It looks like table->cond_set that is used is already setting the bit's for 'field op constant'
If there are more than 128 of them, which ones to choose?
The first 128 that we find in create_key_parts_for_pseudo_index() ?
Another option would be to change SEL_TREE::keys_map to have an explicit key map allocated in get_mm_parts()
This would allows to have any number of pseudo keys.
Each SEL_TREE has a key bitmap.
When we do a tree_and() we merge the bitmaps (and thus assume each bit
maps to the same index).
For pseudo trees, do we ever try to merge or combine the bitmaps?
I spent some time looking at the code and it is probably safe as we never
combine two SEL_ARG trees generated from different get_mm_trees.
Michael Widenius
added a comment - Review comments:
In the test case, you should create MAX_INDEXES + 1 fields and conditions.
To do that, please add system variable, MAX_INDEXES. This can be genrally
usefull (simlar to version_malloc_library).
double rows;
+ double rows= table_records;
You can remove the rows variable and replace it with
a local variable in the only part where it is needed:
rows= records_in_column_ranges(¶m, idx, key);
->
double rows= records_in_column_ranges(¶m, idx, key);
I have one open question:
Each SEL_TREE has a key bitmap.
When we do a tree_and() we merge the bitmaps (and thus assume each bit
maps to the same index).
For pseudo trees, do we ever try to merge or combine the bitmaps?
I spent some time looking at the code and it is probably safe as we never
combine two SEL_ARG trees generated from different get_mm_trees.
create_key_parts_for_pseudo_indexes() uses this piece of code to check whether to create pseudo indexes for columns:
for (field_ptr= table->field; *field_ptr; field_ptr++)
{
Field *field= *field_ptr;
if (bitmap_is_set(used_fields, field->field_index) &&
is_eits_usable(field))
parts++;
}
So are you suggesting that outside of create_key_parts_for_pseudo_indexes there should be identical code that walks through the columns, checks "if_eits_usable" and collects a bitmap of first 64 columns, then calls create_key_parts_for_pseudo_indexes().
Since you request not to touch create_key_parts_for_pseudo_indexes(), it will check "if_eits_usable" for each passed column again.
Is that what you would prefer to see implemented? Please clarify.
Sergei Petrunia
added a comment - igor , look:
create_key_parts_for_pseudo_indexes() uses this piece of code to check whether to create pseudo indexes for columns:
for (field_ptr= table->field; *field_ptr; field_ptr++)
{
Field *field= *field_ptr;
if (bitmap_is_set(used_fields, field->field_index) &&
is_eits_usable(field))
parts++;
}
So are you suggesting that outside of create_key_parts_for_pseudo_indexes there should be identical code that walks through the columns, checks "if_eits_usable" and collects a bitmap of first 64 columns, then calls create_key_parts_for_pseudo_indexes().
Since you request not to touch create_key_parts_for_pseudo_indexes(), it will check "if_eits_usable" for each passed column again.
Is that what you would prefer to see implemented? Please clarify.
Sergei Petrunia
added a comment - igor could you please review the latest patch: https://github.com/MariaDB/server/commit/2949128be1ffd7c340676618b83145f4f3f3929b
I believe it addressed all of your input. it's pushed into bb-10.6- MDEV-33314
I don't see any problems with the patch. Yet as it's a crashing bug the patch should be applied to 10.5 as well.
Igor Babaev (Inactive)
added a comment - I don't see any problems with the patch. Yet as it's a crashing bug the patch should be applied to 10.5 as well.
The server could crash if a query's WHERE clause has conditions over more than 128 (Enterprise Server) or 64 (Community Server) columns such that column statistics is available for them.
Sergei Petrunia
added a comment - Notes for the changelog:
The server could crash if a query's WHERE clause has conditions over more than 128 (Enterprise Server) or 64 (Community Server) columns such that column statistics is available for them.
People
Sergei Petrunia
Sergei Petrunia
Votes:
2Vote for this issue
Watchers:
10Start watching this issue
Dates
Created:
Updated:
Resolved:
Git Integration
Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.
{"report":{"fcp":856.2999999523163,"ttfb":176.09999990463257,"pageVisibility":"visible","entityId":127405,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":1,"journeyId":"94322ca9-6e6d-420e-be11-ac905be66fba","navigationType":0,"readyForUser":924.5999999046326,"redirectCount":0,"resourceLoadedEnd":991.1999999284744,"resourceLoadedStart":181.89999997615814,"resourceTiming":[{"duration":226.29999995231628,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2bu7/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":181.89999997615814,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":181.89999997615814,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":408.1999999284744,"responseStart":0,"secureConnectionStart":0},{"duration":224.60000002384186,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2bu7/820016/12ta74/8679b4946efa1a0bb029a3a22206fb5d/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":183.89999997615814,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":183.89999997615814,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":408.5,"responseStart":0,"secureConnectionStart":0},{"duration":233.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/fbf975c0cce4b1abf04784eeae9ba1f4-CDN/lu2bu7/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":184,"connectEnd":184,"connectStart":184,"domainLookupEnd":184,"domainLookupStart":184,"fetchStart":184,"redirectEnd":0,"redirectStart":0,"requestStart":184,"responseEnd":417.5,"responseStart":417.5,"secureConnectionStart":184},{"duration":308.60000002384186,"initiatorType":"script","name":"https://jira.mariadb.org/s/099b33461394b8015fc36c0a4b96e19f-CDN/lu2bu7/820016/12ta74/8679b4946efa1a0bb029a3a22206fb5d/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true","startTime":184.29999995231628,"connectEnd":184.29999995231628,"connectStart":184.29999995231628,"domainLookupEnd":184.29999995231628,"domainLookupStart":184.29999995231628,"fetchStart":184.29999995231628,"redirectEnd":0,"redirectStart":0,"requestStart":184.29999995231628,"responseEnd":492.89999997615814,"responseStart":492.89999997615814,"secureConnectionStart":184.29999995231628},{"duration":312.09999990463257,"initiatorType":"script","name":"https://jira.mariadb.org/s/94c15bff32baef80f4096a08aceae8bc-CDN/lu2bu7/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":184.5,"connectEnd":184.5,"connectStart":184.5,"domainLookupEnd":184.5,"domainLookupStart":184.5,"fetchStart":184.5,"redirectEnd":0,"redirectStart":0,"requestStart":184.5,"responseEnd":496.59999990463257,"responseStart":496.5,"secureConnectionStart":184.5},{"duration":312.40000009536743,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":184.59999990463257,"connectEnd":184.59999990463257,"connectStart":184.59999990463257,"domainLookupEnd":184.59999990463257,"domainLookupStart":184.59999990463257,"fetchStart":184.59999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":184.59999990463257,"responseEnd":497,"responseStart":497,"secureConnectionStart":184.59999990463257},{"duration":312.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":184.79999995231628,"connectEnd":184.79999995231628,"connectStart":184.79999995231628,"domainLookupEnd":184.79999995231628,"domainLookupStart":184.79999995231628,"fetchStart":184.79999995231628,"redirectEnd":0,"redirectStart":0,"requestStart":184.79999995231628,"responseEnd":497.2999999523163,"responseStart":497.2999999523163,"secureConnectionStart":184.79999995231628},{"duration":371.6999999284744,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2bu7/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":184.89999997615814,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":184.89999997615814,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":556.5999999046326,"responseStart":0,"secureConnectionStart":0},{"duration":312.60000002384186,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":185.19999992847443,"connectEnd":185.19999992847443,"connectStart":185.19999992847443,"domainLookupEnd":185.19999992847443,"domainLookupStart":185.19999992847443,"fetchStart":185.19999992847443,"redirectEnd":0,"redirectStart":0,"requestStart":185.19999992847443,"responseEnd":497.7999999523163,"responseStart":497.7999999523163,"secureConnectionStart":185.19999992847443},{"duration":371.39999997615814,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2bu7/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":185.29999995231628,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":185.29999995231628,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":556.6999999284744,"responseStart":0,"secureConnectionStart":0},{"duration":312.89999997615814,"initiatorType":"script","name":"https://jira.mariadb.org/s/3339d87fa2538a859872f2df449bf8d0-CDN/lu2bu7/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":185.39999997615814,"connectEnd":185.39999997615814,"connectStart":185.39999997615814,"domainLookupEnd":185.39999997615814,"domainLookupStart":185.39999997615814,"fetchStart":185.39999997615814,"redirectEnd":0,"redirectStart":0,"requestStart":185.39999997615814,"responseEnd":498.2999999523163,"responseStart":498.2999999523163,"secureConnectionStart":185.39999997615814},{"duration":682.4000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":186.09999990463257,"connectEnd":186.09999990463257,"connectStart":186.09999990463257,"domainLookupEnd":186.09999990463257,"domainLookupStart":186.09999990463257,"fetchStart":186.09999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":186.09999990463257,"responseEnd":868.5,"responseStart":868.5,"secureConnectionStart":186.09999990463257},{"duration":801,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":190.19999992847443,"connectEnd":190.19999992847443,"connectStart":190.19999992847443,"domainLookupEnd":190.19999992847443,"domainLookupStart":190.19999992847443,"fetchStart":190.19999992847443,"redirectEnd":0,"redirectStart":0,"requestStart":190.19999992847443,"responseEnd":991.1999999284744,"responseStart":991.1999999284744,"secureConnectionStart":190.19999992847443},{"duration":301.2000000476837,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":567.6999999284744,"connectEnd":567.6999999284744,"connectStart":567.6999999284744,"domainLookupEnd":567.6999999284744,"domainLookupStart":567.6999999284744,"fetchStart":567.6999999284744,"redirectEnd":0,"redirectStart":0,"requestStart":567.6999999284744,"responseEnd":868.8999999761581,"responseStart":868.8999999761581,"secureConnectionStart":567.6999999284744},{"duration":170.29999995231628,"initiatorType":"script","name":"https://www.google-analytics.com/analytics.js","startTime":850.3999999761581,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":850.3999999761581,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1020.6999999284744,"responseStart":0,"secureConnectionStart":0}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":2,"responseStart":176,"responseEnd":181,"domLoading":179,"domInteractive":1049,"domContentLoadedEventStart":1049,"domContentLoadedEventEnd":1099,"domComplete":1472,"loadEventStart":1472,"loadEventEnd":1472,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":1023.3999999761581},{"name":"bigPipe.sidebar-id.end","time":1024.1999999284744},{"name":"bigPipe.activity-panel-pipe-id.start","time":1024.3999999761581},{"name":"bigPipe.activity-panel-pipe-id.end","time":1027},{"name":"activityTabFullyLoaded","time":1116.5999999046326}],"measures":[],"correlationId":"600155ac730c8a","effectiveType":"4g","downlink":9.6,"rtt":0,"serverDuration":109,"dbReadsTimeInMs":11,"dbConnsTimeInMs":20,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}
bb-10.6-
MDEV-33314, please review.