Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Duplicate
-
10.3(EOL), 10.4(EOL), 10.5, 10.6, 10.7(EOL), 10.11
Description
The loading of the spider plugin causes a background thread to be created and the set of sql in storage/spider/spd_init_query.h to be run.
Running mysql_upgrade at the same time the initialization of spider is occurring can cause problems as MDEV-27068 shows.
One of the conflicts was
- ALTER TABLE proc MODIFY
- and spiders' {{ drop procedure}}
Other similar conflicts can occur on plugin and udf function installation
MDEV-19275 sql service for plugins means 10.7+ can simply this. Is it valid during initialization? Not 100% sure, but if it is, it would avoid the mariadb-upgrade conflicts.
Other simplifications are:
spider_fix_one_table looks like a simple modification of the ALTER TABLE syntax to use the column IF NOT EXISTS syntax of https://mariadb.com/kb/en/alter-table/ would negate the need for this function.
mysql.spider_fix_system_tables, a procedure with zero arguments that sets user variables could be inline. Or better, removed and use the executable comment syntax in https://mariadb.com/kb/en/comment-syntax/#executable-comment-syntax to run statements based on the server version.
Can the following be implemented another way install plugin [spider_rewrite soname 'ha_spider' (INSTALL PLUGIN has IF NOT EXISTS from 10.4)
Can the udf functions created another way?
Is there are reason (all of) this couldn't be part of mysql-upgrade? wrapped under conditional upgrade if spider is installed?
Attachments
Issue Links
- is blocked by
-
MDEV-27107
prevent two mariadb-upgrade running in parallel
-
- Closed
-
- is part of
-
-
- Closed
-
- relates to
-
-
- Closed
-
-
MDEV-27998
Delete spider_table_crd_thread_count and spider_table_sts_thread_count
-
- Stalled
-
-
MDEV-28553 Spider: clean up plugin initialization SQLs in spd_init_query.h
-
- Open
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Stalled
-
-
-
- Closed
-
Activity
Hi holyfoot, PTAL thanks:
- https://github.com/MariaDB/server/commit/27f33965ef0
- https://github.com/MariaDB/server/commit/cb0e0c915f6
The commit to fix the init bugs MDEV-22979 MDEV-27233 will be based on these ones
I wrote a testcase with plugin-load-add=ha_spider in a .cnf file, and each time the server starts, spider gets loaded fine:
# ~/source/mariadb-server/mdev-22979/src/storage/spider/mysql-test/spider/bugfix/t/plugin_load_add_repeatedly.test
|
|
|
--echo #
|
--echo # plugin-load-add=ha_spider
|
--echo #
|
|
|
--let SEARCH_PATTERN=\\[Note\\] Loading of SPIDER[a-zA-Z0-9_]* from ha_spider(|.so|.dll) is deferred to a temporary init file
|
--let SEARCH_FILE=`select regexp_replace(@@tmpdir, '^.*/','')`
|
--let SEARCH_FILE=$MYSQLTEST_VARDIR/log/$SEARCH_FILE.err
|
--source include/search_pattern_in_file.inc
|
select * from mysql.plugin;
|
create table t (c int) Engine=SPIDER;
|
--source include/restart_mysqld.inc
|
--source include/search_pattern_in_file.inc
|
drop table t;
|
|
|
# ~/source/mariadb-server/mdev-22979/src/storage/spider/mysql-test/spider/bugfix/t/plugin_load_add_repeatedly.cnf
|
|
|
[mysqld.1.1]
|
plugin-load-add=ha_spider.so
|
|
|
# the test result:
|
|
|
#
|
# plugin-load-add=ha_spider
|
#
|
FOUND 3 /\[Note\] Loading of SPIDER[a-zA-Z0-9_]* from ha_spider(|.so|.dll) is deferred to a temporary init file/ in mysqld.1.1.err
|
select * from mysql.plugin;
|
name dl
|
create table t (c int) Engine=SPIDER;
|
# restart
|
FOUND 6 /\[Note\] Loading of SPIDER[a-zA-Z0-9_]* from ha_spider(|.so|.dll) is deferred to a temporary init file/ in mysqld.1.1.err
|
drop table t;
|
ok to push.
It seems to me i already reviewed these patches. Hope i reviewed proper ones this time 
Thanks holyfoot. Yes we can consider this as part of MDEV-22979 as
some of the patches there are named after this ticket. Marking as
blocked by that ticket.
Updates on 2023-10-06: close as duplicate to MDEV-22979.
Thanks for the comment serg. As with the discussions at #spider, after refining on the first idea above, I think it is worth trying to re-interpret plugin-load-add=ha_spider as a "internal" file containing install soname ha_spider;. Here is a PoC: https://github.com/MariaDB/server/commit/603a4c218b7
This way we can get rid of the bg thread completely. It also means we can decouple the present issue from
MDEV-22979andMDEV-27233as by the time of executing the init queries in the internal file it will have all it needs. A clean-up of the queries is still useful though.