Implement the engine-defined options corresponding to useful table
options, including (options marked as ++ are included only because
of the "keep" decision on the corresponding spider system variables
in MDEV-27228):
Many of these table params have a twin system variable. Examples
include read_only_mode, net_read_timeout etc. The natural
approach to these params is to use HA_TOPTION_SYSVAR macro,
which takes care of the value overriding. This is also a natural
continuation of spider sysvar tickets like MDEV-27169 and MDEV-31524. However, table options can not fall back dynamically to
the current sysvar value, so we have to keep the overriding/fallback
mechanism and use the string type (HA_TOPTION_STRING) for these
values, where we can encode unspecified as the null string.
There's also a temporary problem with unspecified non-string table
options. Given that we will keep connection string parsing by
COMMENT or CONNECTION until MDEV-31146, It makes sense as a part of
the changes for this ticket, to ignore COMMENT/CONNECTION parsing
when any table option is used. However, it is not possible to tell
whether a non-string type table option has been specified. We fix
this by making these options also string type (so all spider options
are of the string type) and if any of them is non-null (i.e.
specified), we ignore COMMENT/CONNECTION for parsing. An alternative
fix also considered was to make the COMMENT string parsing override
these table options. However, this is an inferior choice given
that 1. this increases complexity in overriding; 2. only six
relatively uncommon table options are not sysvars and not strings
(priority, query_cache, query_cache_sync, force_bulk_delete,
force_bulk_update, table_count_mode); 3. all params are already
supplied as strings (e.g. read_only_mode "1") in the existing
comment string parsing so no surprise to users if they remain
strings, and in create table statements strings do not need to be
quoted anyway; 4. users might want to use comments for comments.
Even though after this task there are MDEV-28861 and MDEV-31146 that
duplicate and remove connection info encoded in comments, there will
still be connection string parser in the spider codebase, for spider
udfs like spider_direct_sql and spider_copy_tables.
Regarding the choice of the table options in the list above, some
undocumented parameters have unclear purpose. It is not in the scope
of this ticket to try to document every one of them. Examples
include force_bulk_update, force_bulk_delete, table_count_mode. Also worth noting is the parameter delete_all_rows_type, which is buggy at 11.2 (MDEV-31996), but
on face value its purpose is clear, so we keep it for now.
Attachments
Issue Links
blocks
MDEV-28861Spider: Deprecate table options by COMMENT
Closed
causes
MDEV-32486Assertion `!trx->alloc_line_no[id] || trx->alloc_line_no[id] == line_no' failed in spider_alloc_mem_calc
Closed
MDEV-32558ERROR 1429 (base) versus crash [SIGSEGV in spider_create_conn] (28856 patch) and ERROR 12719 infinite loop (base) versus ERROR 12518 table is read only (patch) on optimized builds in CLI
Closed
includes
MDEV-32157Backport cleanup commits from MDEV-28856 to lower versions
Closed
MDEV-32234Missing spaces in Spider variable descriptions
Closed
is blocked by
MDEV-31117Spider UBSAN runtime error: applying non-zero offset x to null pointer in st_spider_param_string_parse::restore_delims
Closed
MDEV-32486Assertion `!trx->alloc_line_no[id] || trx->alloc_line_no[id] == line_no' failed in spider_alloc_mem_calc
Closed
relates to
MDEV-27228Deprecate Spider plugin variables that result in excessive tweak
Stalled
MDEV-27106Spider: specify connection to data node by engine-defined attributes
Closed
MDEV-32492SIGSEGV in spider_conn_first_link_idx and others on DELETE, INSERT and SELECT
Closed
MDEV-32557"ERROR 12501 (HY000): The connect info 'MYSQL' is invalid" not repeatable in MDEV-28856 branch
Closed
MDEV-32585ASAN: stack-buffer-overflow in get_defaults_options on SELECT
Status update: many new issues were seen during the testing of this feature. Most of these were traced back to 1) missing MDEV-31524 and MDEV-30981 commits in the feature branch, 2) MDEV-32492 (and the iterations thereof, ref bug for details), which turned out to be a legacy bug which asserted itself more readily in the feature branch. The final re-test results of the updated branch of this feature (bb-11.3-mdev-28856-and-fixes) are near complete.
Roel Van de Paar
added a comment - - edited Status update: many new issues were seen during the testing of this feature. Most of these were traced back to 1) missing MDEV-31524 and MDEV-30981 commits in the feature branch, 2) MDEV-32492 (and the iterations thereof, ref bug for details), which turned out to be a legacy bug which asserted itself more readily in the feature branch. The final re-test results of the updated branch of this feature (bb-11.3-mdev-28856-and-fixes) are near complete.
Both MDEV-32558 (marked as duplicate of MDEV-32492 as the underlying issue is the same) as well as the newly created MDEV-32585 are bugs that show in optimized builds only. For MDEV-32558 after discussion we agreed it is acceptable to leave it and fix later in combination with MDEV-32492. MDEV-32585 is a WIP.
Roel Van de Paar
added a comment - Both MDEV-32558 (marked as duplicate of MDEV-32492 as the underlying issue is the same) as well as the newly created MDEV-32585 are bugs that show in optimized builds only. For MDEV-32558 after discussion we agreed it is acceptable to leave it and fix later in combination with MDEV-32492 . MDEV-32585 is a WIP.
Given the patches depend on the MDEV-31117 commits, the latter having
not yet reached 11.3 (highest version inclusion 11.1), the push is
blocked by the merge of 11.1->11.2->11.3.
Yuchen Pei
added a comment - Thanks for the testing Roel .
Given the patches depend on the MDEV-31117 commits, the latter having
not yet reached 11.3 (highest version inclusion 11.1), the push is
blocked by the merge of 11.1->11.2->11.3.
Yuchen Pei
added a comment - - edited squashed the following and pushed to 11.3 as 126157061b4376496c034a809ea4943e863d1465
cc08a83ef42 * MDEV-32486 Fix duplicate spider_bulk_malloc ID
99c3003ec78 * MDEV-32234 Fix missing space in Spider variable descriptions
80e4b349fc2 * MDEV-28856 fixes test to pass in --ps-protocol
a4031e4c051 * MDEV-28856 Add remaining Spider table options
People
Yuchen Pei
Nayuta Yanagisawa (Inactive)
Votes:
1Vote for this issue
Watchers:
9Start watching this issue
Dates
Created:
Updated:
Resolved:
Git Integration
Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.
{"report":{"fcp":1444.5,"ttfb":358.09999990463257,"pageVisibility":"visible","entityId":111943,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":0.5,"journeyId":"c495a260-8dcd-4388-af55-7f369d926979","navigationType":0,"readyForUser":1626.2000000476837,"redirectCount":0,"resourceLoadedEnd":2099,"resourceLoadedStart":379.89999985694885,"resourceTiming":[{"duration":457.10000014305115,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":379.89999985694885,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":379.89999985694885,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":837,"responseStart":0,"secureConnectionStart":0},{"duration":457.09999990463257,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true&whisper-enabled=true","startTime":380.2999999523163,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":380.2999999523163,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":837.3999998569489,"responseStart":0,"secureConnectionStart":0},{"duration":516.6000001430511,"initiatorType":"script","name":"https://jira.mariadb.org/s/0917945aaa57108d00c5076fea35e069-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":380.59999990463257,"connectEnd":380.59999990463257,"connectStart":380.59999990463257,"domainLookupEnd":380.59999990463257,"domainLookupStart":380.59999990463257,"fetchStart":380.59999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":380.59999990463257,"responseEnd":897.2000000476837,"responseStart":897.2000000476837,"secureConnectionStart":380.59999990463257},{"duration":599.4000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/s/2d8175ec2fa4c816e8023260bd8c1786-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true&whisper-enabled=true","startTime":381.2999999523163,"connectEnd":381.2999999523163,"connectStart":381.2999999523163,"domainLookupEnd":381.2999999523163,"domainLookupStart":381.2999999523163,"fetchStart":381.2999999523163,"redirectEnd":0,"redirectStart":0,"requestStart":381.2999999523163,"responseEnd":980.7000000476837,"responseStart":980.7000000476837,"secureConnectionStart":381.2999999523163},{"duration":603.3999998569489,"initiatorType":"script","name":"https://jira.mariadb.org/s/a9324d6758d385eb45c462685ad88f1d-CDN/lu2cib/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":381.5,"connectEnd":381.5,"connectStart":381.5,"domainLookupEnd":381.5,"domainLookupStart":381.5,"fetchStart":381.5,"redirectEnd":0,"redirectStart":0,"requestStart":381.5,"responseEnd":984.8999998569489,"responseStart":984.8999998569489,"secureConnectionStart":381.5},{"duration":603.7999999523163,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":381.7000000476837,"connectEnd":381.7000000476837,"connectStart":381.7000000476837,"domainLookupEnd":381.7000000476837,"domainLookupStart":381.7000000476837,"fetchStart":381.7000000476837,"redirectEnd":0,"redirectStart":0,"requestStart":381.7000000476837,"responseEnd":985.5,"responseStart":985.5,"secureConnectionStart":381.7000000476837},{"duration":604,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":381.89999985694885,"connectEnd":381.89999985694885,"connectStart":381.89999985694885,"domainLookupEnd":381.89999985694885,"domainLookupStart":381.89999985694885,"fetchStart":381.89999985694885,"redirectEnd":0,"redirectStart":0,"requestStart":381.89999985694885,"responseEnd":985.8999998569489,"responseStart":985.8999998569489,"secureConnectionStart":381.89999985694885},{"duration":686.7000000476837,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2cib/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":382.09999990463257,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":382.09999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1068.7999999523163,"responseStart":0,"secureConnectionStart":0},{"duration":604.2000000476837,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":382.2999999523163,"connectEnd":382.2999999523163,"connectStart":382.2999999523163,"domainLookupEnd":382.2999999523163,"domainLookupStart":382.2999999523163,"fetchStart":382.2999999523163,"redirectEnd":0,"redirectStart":0,"requestStart":382.2999999523163,"responseEnd":986.5,"responseStart":986.5,"secureConnectionStart":382.2999999523163},{"duration":686.3999998569489,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":382.5,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":382.5,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1068.8999998569489,"responseStart":0,"secureConnectionStart":0},{"duration":604.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/5d5e8fe91fbc506585e83ea3b62ccc4b-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":382.7000000476837,"connectEnd":382.7000000476837,"connectStart":382.7000000476837,"domainLookupEnd":382.7000000476837,"domainLookupStart":382.7000000476837,"fetchStart":382.7000000476837,"redirectEnd":0,"redirectStart":0,"requestStart":382.7000000476837,"responseEnd":987.2000000476837,"responseStart":987.2000000476837,"secureConnectionStart":382.7000000476837},{"duration":1145.7000000476837,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":389.2999999523163,"connectEnd":389.2999999523163,"connectStart":389.2999999523163,"domainLookupEnd":389.2999999523163,"domainLookupStart":389.2999999523163,"fetchStart":389.2999999523163,"redirectEnd":0,"redirectStart":0,"requestStart":389.2999999523163,"responseEnd":1535,"responseStart":1535,"secureConnectionStart":389.2999999523163},{"duration":1706.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":392.5,"connectEnd":392.5,"connectStart":392.5,"domainLookupEnd":392.5,"domainLookupStart":392.5,"fetchStart":392.5,"redirectEnd":0,"redirectStart":0,"requestStart":392.5,"responseEnd":2099,"responseStart":2099,"secureConnectionStart":392.5},{"duration":524,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":1081.5,"connectEnd":1081.5,"connectStart":1081.5,"domainLookupEnd":1081.5,"domainLookupStart":1081.5,"fetchStart":1081.5,"redirectEnd":0,"redirectStart":0,"requestStart":1081.5,"responseEnd":1605.5,"responseStart":1605.5,"secureConnectionStart":1081.5},{"duration":695.2000000476837,"initiatorType":"script","name":"https://www.google-analytics.com/analytics.js","startTime":1435.5,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":1435.5,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":2130.7000000476837,"responseStart":0,"secureConnectionStart":0}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":152,"responseStart":358,"responseEnd":390,"domLoading":361,"domInteractive":2135,"domContentLoadedEventStart":2135,"domContentLoadedEventEnd":2187,"domComplete":2846,"loadEventStart":2846,"loadEventEnd":2847,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":2101},{"name":"bigPipe.sidebar-id.end","time":2101.899999856949},{"name":"bigPipe.activity-panel-pipe-id.start","time":2102.0999999046326},{"name":"bigPipe.activity-panel-pipe-id.end","time":2104.0999999046326},{"name":"activityTabFullyLoaded","time":2212.7000000476837}],"measures":[],"correlationId":"746f87099992a7","effectiveType":"4g","downlink":9.7,"rtt":0,"serverDuration":143,"dbReadsTimeInMs":31,"dbConnsTimeInMs":43,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}
Status update: many new issues were seen during the testing of this feature. Most of these were traced back to 1) missing
MDEV-31524andMDEV-30981commits in the feature branch, 2)MDEV-32492(and the iterations thereof, ref bug for details), which turned out to be a legacy bug which asserted itself more readily in the feature branch. The final re-test results of the updated branch of this feature (bb-11.3-mdev-28856-and-fixes) are near complete.