[MDEV-29870] Backport fixes to spider init bugs to 10.4-10.6 once they have SQL service Created: 2022-10-25  Updated: 2023-12-07  Resolved: 2023-12-07

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - Spider
Affects Version/s: 10.4, 10.5, 10.6
Fix Version/s: 10.4.33, 10.5.24, 10.6.17

Type: Bug Priority: Critical
Reporter: Nayuta Yanagisawa (Inactive) Assignee: Yuchen Pei
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Blocks
blocks MDEV-27103 mariadb-upgrade fails with 'System ta... Closed
blocks MDEV-30722 sporadic failure spider/bugfix.insert... Open
blocks MDEV-31101 spider/bugfix.mdev_29904 fails with "... Closed
blocks MDEV-32773 Syntax error upon Spider initializati... Open
is blocked by MDEV-27233 Server hangs when using --init-file w... Closed
is blocked by MDEV-27595 Backport SQL service, introduced by M... Closed
is blocked by MDEV-32753 Spider engine does not load in ORACLE... Closed
PartOf
includes MDEV-32903 Backport MDEV-32532 to 10.6 Closed
Relates
relates to MDEV-27233 Server hangs when using --init-file w... Closed
relates to MDEV-32804 Remove the (unimplemented and unused)... Closed

 Description   

This is the 10.4-10.6 part of the fix to spider init bugs (MDEV-22979, MDEV-27233, MDEV-28218). The patches (also for MDEV-27095) can be found in MDEV-22979.



 Comments   
Comment by Yuchen Pei [ 2023-05-07 ]

The patch for MDEV-22979 adapts the fix for MDEV-27233 as mentioned in the description of the present ticket. It is not clear to me sql service is needed for lower versions as lower versions have mechanisms to query a server in C++ which spider uses, like mysql_real_query() whose implementation was updated IIUC as part of the sql service task. Perhaps the reason for not doing that is due to [1]? In any case I will look into the backporting of the MDEV-22979 patch after it has been reviewed and approved.

[1] https://jira.mariadb.org/browse/MDEV-27595?focusedCommentId=211798&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-211798

Comment by Yuchen Pei [ 2023-09-15 ]

This issue is obsolete, as the original fix for MDEV-27233 was reverted, and now we are tracking spider init bugs mainly in MDEV-22979, and we are holding off the init bug fix waiting for MDEV-27595 to be done.

Comment by Yuchen Pei [ 2023-10-06 ]

MDEV-27595 is not done yet, and MDEV-22979 is pushed to 10.10. Re-opening

Comment by Yuchen Pei [ 2023-10-20 ]

A reminder that all tickets caused by MDEV-22979 will need to be
backported as well as a part of this ticket.

Comment by Yuchen Pei [ 2023-11-14 ]

Below is an initial set of patches for 10.4, containing fixes for
MDEV-22979 and tickets caused by MDEV-22979 including MDEV-32515,
MDEV-32507, MDEV-32485, MDEV-32683, MDEV-32753 (under review). Two
other tickets caused by MDEV-22979, namely MDEV-32532 and MDEV-32559
are not applicable. More specifically, MDEV-32532 does not reproduce
in 10.4, and MDEV-32559 (under review) implements a spider version
of the callback signal_ddl_recovery_done() which was introduced
in 10.6 by the commit 7762ee5dbec5c336628c06bbe950837257276e57

e3b59ed6c85 upstream/bb-10.4-mdev-29870 MDEV-32753 Make spider init queries compatible with oracle sql mode
789f8b9a265 MDEV-32683 Spider engine does not load with non-default alter-algorithm
d8655221836 MDEV-32485 Fix Spider upgrade failure caused by duplication in mysql.func
520afd496be MDEV-32507 Spider: Use $MTR_SUITE_DIR for init-file files
071e71e647d MDEV-32515 Use $MYSQLD_LAST_CMD in spider/bugfix.mdev_30370
d4c85f2617a MDEV-22979 MDEV-27233 MDEV-28218 Fixing spider init bugs
e5b61bb5835 MDEV-27095 clean up spd_init_query.h
a7f5dd5a3c0 MDEV-27095 installing one spider plugin should not trigger others

Comment by Yuchen Pei [ 2023-11-15 ]

I have a draft for 10.5, but it will not compile without a fix for
MDEV-27595:

45ea30868da upstream/bb-10.5-mdev-29870 MDEV-32753 Make spider init queries compatible with oracle sql mode
4f4424914d4 MDEV-32683 Spider engine does not load with non-default alter-algorithm
1dc47b71172 MDEV-32485 Fix Spider upgrade failure caused by duplication in mysql.func
b5bdfa00450 MDEV-32507 Spider: Use $MTR_SUITE_DIR for init-file files
38ed0fbe963 MDEV-32515 Use $MYSQLD_LAST_CMD in spider/bugfix.mdev_30370
fe6fcc50321 MDEV-32046 Adding ER_NET_READ_ERROR to spider/bugfix.mdev_27240
4ce39fcbb20 MDEV-22979 MDEV-27233 MDEV-28218 Fixing spider init bugs
d7516b187b9 MDEV-27095 clean up spd_init_query.h
efbcc42789c MDEV-27095 installing one spider plugin should not trigger others

holyfoot, do you have a 10.5 version for MDEV-27595?

I will need both 10.5 and 10.6 to test whether these fixes also fix
MDEV-27103.

Comment by Yuchen Pei [ 2023-11-16 ]

The current patches are located at a7f5dd5a3c0..764dc6cb6f5.

I cannot send for review yet, because

1. MDEV-32753 needs to be reviewed first so that the oracle mode
regression fix can be included for review in this ticket
2. A 10.6 version of the MDEV-27595 patches is needed, so that I could
check whether MDEV-32532 reproduces and if so, ask for a backport
of MDEV-32532, or just backport MDEV-32559. Since this blockage is
not these tickets but rather inclusion of patches in a higher
version, I'm marking this ticket blocked by MDEV-32601 non-release
merges

Comment by Yuchen Pei [ 2023-11-28 ]

Hi holyfoot, ptal thanks

9c6548ab754 upstream/bb-10.4-mdev-29870 MDEV-32753 Make spider init queries compatible with oracle sql mode
4dd8d0b9cc4 MDEV-32683 Spider engine does not load with non-default alter-algorithm
23cc261fabd MDEV-32485 Fix Spider upgrade failure caused by duplication in mysql.func
585fb7fcc8b MDEV-32507 Spider: Use $MTR_SUITE_DIR for init-file files
c6c6bc751f6 MDEV-32515 Use $MYSQLD_LAST_CMD in spider/bugfix.mdev_30370
2bdfd398134 MDEV-22979 MDEV-27233 MDEV-28218 Fixing spider init bugs
c6d4ef55b13 MDEV-27095 clean up spd_init_query.h
fb2f4e53e63 MDEV-27095 installing one spider plugin should not trigger others

Above is 10.4. For 10.5 it is the commit range 1af055a555c..5b847372069, based on your bb-10.5-sqlser-hf.

For 10.6 it is the commit range 2a8f83a559d..c5bab76559c based on your bb-10.6-sqlser-hf. Because of the introduction of ddl log in 10.6, two extra patches are includes, i.e. a fixup which should probably be squashed into your commit

0ac8873630e * MDEV-27595 [fixup] Move ddl_log_initialize() to before plugin_init()

as well as a backport of monty's fix for MDEV-32532

83736277b41 * MDEV-32532 Assertion failure in ddl_log_increment_phase_no_lock upon ..

Comment by Alexey Botchkov [ 2023-12-03 ]

Looks ok to push.
SerG said he merged the SQL service in 10.5/6 separately, so you could check with him about pushing into the main branches.

Comment by Yuchen Pei [ 2023-12-07 ]

Pushed the following to 10.4

ba94778d2c4 upstream/bb-10.4-mdev-29870 upstream/10.4 MDEV-32753 Make spider init queries compatible with oracle sql mode
69389c03b16 MDEV-32683 Spider engine does not load with non-default alter-algorithm
9259b4b8494 MDEV-32485 Fix Spider upgrade failure caused by duplication in mysql.func
30af9872599 MDEV-32507 Spider: Use $MTR_SUITE_DIR for init-file files
70283aca344 MDEV-32515 Use $MYSQLD_LAST_CMD in spider/bugfix.mdev_30370
d8f5d2bef06 MDEV-22979 MDEV-27233 MDEV-28218 Fixing spider init bugs
afe63ec6144 MDEV-27095 clean up spd_init_query.h
f0af56be015 MDEV-27095 installing one spider plugin should not trigger others

Generated at Thu Feb 08 10:11:55 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.