Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-29870

Backport fixes to spider init bugs to 10.4-10.6 once they have SQL service

Details

    Description

      This is the 10.4-10.6 part of the fix to spider init bugs (MDEV-22979, MDEV-27233, MDEV-28218). The patches (also for MDEV-27095) can be found in MDEV-22979.

      Attachments

        Issue Links

          blocks

          Bug - A problem which impairs or prevents the functions of the product. MDEV-27103 mariadb-upgrade fails with 'System table spider_tables is different version' => Can't create database 'performance_schema'

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-30722 sporadic failure spider/bugfix.insert_select

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MDEV-31101 spider/bugfix.mdev_29904 fails with "Server shutdown in progress" in CI

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-32773 Syntax error upon Spider initialization under ORACLE mode (a different one)

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          includes

          Bug - A problem which impairs or prevents the functions of the product. MDEV-32903 Backport MDEV-32532 to 10.6

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. MDEV-27233 Server hangs when using --init-file which loads Spider and creates a Spider table

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Task - A task that needs to be done. MDEV-27595 Backport SQL service, introduced by MDEV-19275

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-32753 Spider engine does not load in ORACLE mode

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-27233 Server hangs when using --init-file which loads Spider and creates a Spider table

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Task - A task that needs to be done. MDEV-32804 Remove the (unimplemented and unused) spider_rewrite_plugin

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) created issue -
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            Field Original Value New Value
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            Description This is the 10.4-10.7 part of MDEV-27233. The fix of MDEV-27233 requires SQL service. Backport the fix once
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            Description This is the 10.4-10.7 part of MDEV-27233. The fix of MDEV-27233 requires SQL service. Backport the fix once This is the 10.4-10.7 part of MDEV-27233. The fix of MDEV-27233 requires SQL service. Backport the fix once MDEV-27595 has been completed.
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            ycp Yuchen Pei made changes -
            Assignee Nayuta Yanagisawa [ JIRAUSER47117 ] Yuchen Pei [ JIRAUSER52627 ]
            julien.fritsch Julien Fritsch made changes -
            Priority Major [ 3 ] Critical [ 2 ]
            ycp Yuchen Pei added a comment -

            The patch for MDEV-22979 adapts the fix for MDEV-27233 as mentioned in the description of the present ticket. It is not clear to me sql service is needed for lower versions as lower versions have mechanisms to query a server in C++ which spider uses, like mysql_real_query() whose implementation was updated IIUC as part of the sql service task. Perhaps the reason for not doing that is due to [1]? In any case I will look into the backporting of the MDEV-22979 patch after it has been reviewed and approved.

            [1] https://jira.mariadb.org/browse/MDEV-27595?focusedCommentId=211798&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-211798

            ycp Yuchen Pei added a comment - The patch for MDEV-22979 adapts the fix for MDEV-27233 as mentioned in the description of the present ticket. It is not clear to me sql service is needed for lower versions as lower versions have mechanisms to query a server in C++ which spider uses, like mysql_real_query() whose implementation was updated IIUC as part of the sql service task. Perhaps the reason for not doing that is due to [1] ? In any case I will look into the backporting of the MDEV-22979 patch after it has been reviewed and approved. [1] https://jira.mariadb.org/browse/MDEV-27595?focusedCommentId=211798&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-211798
            ycp Yuchen Pei added a comment -

            This issue is obsolete, as the original fix for MDEV-27233 was reverted, and now we are tracking spider init bugs mainly in MDEV-22979, and we are holding off the init bug fix waiting for MDEV-27595 to be done.

            ycp Yuchen Pei added a comment - This issue is obsolete, as the original fix for MDEV-27233 was reverted, and now we are tracking spider init bugs mainly in MDEV-22979 , and we are holding off the init bug fix waiting for MDEV-27595 to be done.
            ycp Yuchen Pei made changes -
            Fix Version/s N/A [ 14700 ]
            Fix Version/s 10.4 [ 22408 ]
            Fix Version/s 10.5 [ 23123 ]
            Fix Version/s 10.6 [ 24028 ]
            Resolution Won't Do [ 10201 ]
            Status Open [ 1 ] Closed [ 6 ]
            ycp Yuchen Pei added a comment -

            MDEV-27595 is not done yet, and MDEV-22979 is pushed to 10.10. Re-opening

            ycp Yuchen Pei added a comment - MDEV-27595 is not done yet, and MDEV-22979 is pushed to 10.10. Re-opening
            ycp Yuchen Pei made changes -
            Resolution Won't Do [ 10201 ]
            Status Closed [ 6 ] Stalled [ 10000 ]
            ycp Yuchen Pei made changes -
            Summary Backport MDEV-27233 to 10.4-10.6 once they have SQL service Backport fixes to spider init bugs to 10.4-10.6 once they have SQL service
            ycp Yuchen Pei made changes -
            Fix Version/s 10.4 [ 22408 ]
            Fix Version/s 10.5 [ 23123 ]
            Fix Version/s 10.6 [ 24028 ]
            Fix Version/s N/A [ 14700 ]
            ycp Yuchen Pei made changes -
            Description This is the 10.4-10.7 part of MDEV-27233. The fix of MDEV-27233 requires SQL service. Backport the fix once MDEV-27595 has been completed. This is the 10.4-10.6 part of the fix to spider init bugs (MDEV-22979, MDEV-27233, MDEV-28218). The patches (also for MDEV-27095) can be found in MDEV-22979.
            ycp Yuchen Pei added a comment -

            A reminder that all tickets caused by MDEV-22979 will need to be
            backported as well as a part of this ticket.

            ycp Yuchen Pei added a comment - A reminder that all tickets caused by MDEV-22979 will need to be backported as well as a part of this ticket.
            ycp Yuchen Pei made changes -
            ycp Yuchen Pei made changes -
            ycp Yuchen Pei made changes -
            Status Stalled [ 10000 ] In Progress [ 3 ]
            ycp Yuchen Pei made changes -
            ycp Yuchen Pei made changes -
            ycp Yuchen Pei added a comment - - edited

            Below is an initial set of patches for 10.4, containing fixes for
            MDEV-22979 and tickets caused by MDEV-22979 including MDEV-32515,
            MDEV-32507, MDEV-32485, MDEV-32683, MDEV-32753 (under review). Two
            other tickets caused by MDEV-22979, namely MDEV-32532 and MDEV-32559
            are not applicable. More specifically, MDEV-32532 does not reproduce
            in 10.4, and MDEV-32559 (under review) implements a spider version
            of the callback signal_ddl_recovery_done() which was introduced
            in 10.6 by the commit 7762ee5dbec5c336628c06bbe950837257276e57

            e3b59ed6c85 upstream/bb-10.4-mdev-29870 MDEV-32753 Make spider init queries compatible with oracle sql mode
            		 
            789f8b9a265 MDEV-32683 Spider engine does not load with non-default alter-algorithm
            		 
            d8655221836 MDEV-32485 Fix Spider upgrade failure caused by duplication in mysql.func
            		 
            520afd496be MDEV-32507 Spider: Use $MTR_SUITE_DIR for init-file files
            		 
            071e71e647d MDEV-32515 Use $MYSQLD_LAST_CMD in spider/bugfix.mdev_30370
            		 
            d4c85f2617a MDEV-22979 MDEV-27233 MDEV-28218 Fixing spider init bugs
            		 
            e5b61bb5835 MDEV-27095 clean up spd_init_query.h
            		 
            a7f5dd5a3c0 MDEV-27095 installing one spider plugin should not trigger others

            ycp Yuchen Pei added a comment - - edited Below is an initial set of patches for 10.4, containing fixes for MDEV-22979 and tickets caused by MDEV-22979 including MDEV-32515 , MDEV-32507 , MDEV-32485 , MDEV-32683 , MDEV-32753 (under review). Two other tickets caused by MDEV-22979 , namely MDEV-32532 and MDEV-32559 are not applicable. More specifically, MDEV-32532 does not reproduce in 10.4, and MDEV-32559 (under review) implements a spider version of the callback signal_ddl_recovery_done() which was introduced in 10.6 by the commit 7762ee5dbec5c336628c06bbe950837257276e57 e3b59ed6c85 upstream/bb-10.4-mdev-29870 MDEV-32753 Make spider init queries compatible with oracle sql mode 789f8b9a265 MDEV-32683 Spider engine does not load with non-default alter-algorithm d8655221836 MDEV-32485 Fix Spider upgrade failure caused by duplication in mysql.func 520afd496be MDEV-32507 Spider: Use $MTR_SUITE_DIR for init-file files 071e71e647d MDEV-32515 Use $MYSQLD_LAST_CMD in spider/bugfix.mdev_30370 d4c85f2617a MDEV-22979 MDEV-27233 MDEV-28218 Fixing spider init bugs e5b61bb5835 MDEV-27095 clean up spd_init_query.h a7f5dd5a3c0 MDEV-27095 installing one spider plugin should not trigger others
            ycp Yuchen Pei made changes -
            ycp Yuchen Pei added a comment -

            I have a draft for 10.5, but it will not compile without a fix for
            MDEV-27595:

            45ea30868da upstream/bb-10.5-mdev-29870 MDEV-32753 Make spider init queries compatible with oracle sql mode
            		 
            4f4424914d4 MDEV-32683 Spider engine does not load with non-default alter-algorithm
            		 
            1dc47b71172 MDEV-32485 Fix Spider upgrade failure caused by duplication in mysql.func
            		 
            b5bdfa00450 MDEV-32507 Spider: Use $MTR_SUITE_DIR for init-file files
            		 
            38ed0fbe963 MDEV-32515 Use $MYSQLD_LAST_CMD in spider/bugfix.mdev_30370
            		 
            fe6fcc50321 MDEV-32046 Adding ER_NET_READ_ERROR to spider/bugfix.mdev_27240
            		 
            4ce39fcbb20 MDEV-22979 MDEV-27233 MDEV-28218 Fixing spider init bugs
            		 
            d7516b187b9 MDEV-27095 clean up spd_init_query.h
            		 
            efbcc42789c MDEV-27095 installing one spider plugin should not trigger others

            holyfoot, do you have a 10.5 version for MDEV-27595?

            I will need both 10.5 and 10.6 to test whether these fixes also fix
            MDEV-27103.

            ycp Yuchen Pei added a comment - I have a draft for 10.5, but it will not compile without a fix for MDEV-27595 : 45ea30868da upstream/bb-10.5-mdev-29870 MDEV-32753 Make spider init queries compatible with oracle sql mode 4f4424914d4 MDEV-32683 Spider engine does not load with non-default alter-algorithm 1dc47b71172 MDEV-32485 Fix Spider upgrade failure caused by duplication in mysql.func b5bdfa00450 MDEV-32507 Spider: Use $MTR_SUITE_DIR for init-file files 38ed0fbe963 MDEV-32515 Use $MYSQLD_LAST_CMD in spider/bugfix.mdev_30370 fe6fcc50321 MDEV-32046 Adding ER_NET_READ_ERROR to spider/bugfix.mdev_27240 4ce39fcbb20 MDEV-22979 MDEV-27233 MDEV-28218 Fixing spider init bugs d7516b187b9 MDEV-27095 clean up spd_init_query.h efbcc42789c MDEV-27095 installing one spider plugin should not trigger others holyfoot , do you have a 10.5 version for MDEV-27595 ? I will need both 10.5 and 10.6 to test whether these fixes also fix MDEV-27103 .
            ycp Yuchen Pei made changes -
            ycp Yuchen Pei added a comment -

            The current patches are located at a7f5dd5a3c0..764dc6cb6f5.

            I cannot send for review yet, because

            1. MDEV-32753 needs to be reviewed first so that the oracle mode
            regression fix can be included for review in this ticket
            2. A 10.6 version of the MDEV-27595 patches is needed, so that I could
            check whether MDEV-32532 reproduces and if so, ask for a backport
            of MDEV-32532, or just backport MDEV-32559. Since this blockage is
            not these tickets but rather inclusion of patches in a higher
            version, I'm marking this ticket blocked by MDEV-32601 non-release
            merges

            ycp Yuchen Pei added a comment - The current patches are located at a7f5dd5a3c0..764dc6cb6f5. I cannot send for review yet, because 1. MDEV-32753 needs to be reviewed first so that the oracle mode regression fix can be included for review in this ticket 2. A 10.6 version of the MDEV-27595 patches is needed, so that I could check whether MDEV-32532 reproduces and if so, ask for a backport of MDEV-32532 , or just backport MDEV-32559 . Since this blockage is not these tickets but rather inclusion of patches in a higher version, I'm marking this ticket blocked by MDEV-32601 non-release merges
            ycp Yuchen Pei made changes -
            ycp Yuchen Pei made changes -
            Status In Progress [ 3 ] Stalled [ 10000 ]
            ycp Yuchen Pei made changes -
            ycp Yuchen Pei added a comment -

            Hi holyfoot, ptal thanks

            9c6548ab754 upstream/bb-10.4-mdev-29870 MDEV-32753 Make spider init queries compatible with oracle sql mode
            		 
            4dd8d0b9cc4 MDEV-32683 Spider engine does not load with non-default alter-algorithm
            		 
            23cc261fabd MDEV-32485 Fix Spider upgrade failure caused by duplication in mysql.func
            		 
            585fb7fcc8b MDEV-32507 Spider: Use $MTR_SUITE_DIR for init-file files
            		 
            c6c6bc751f6 MDEV-32515 Use $MYSQLD_LAST_CMD in spider/bugfix.mdev_30370
            		 
            2bdfd398134 MDEV-22979 MDEV-27233 MDEV-28218 Fixing spider init bugs
            		 
            c6d4ef55b13 MDEV-27095 clean up spd_init_query.h
            		 
            fb2f4e53e63 MDEV-27095 installing one spider plugin should not trigger others

            Above is 10.4. For 10.5 it is the commit range 1af055a555c..5b847372069, based on your bb-10.5-sqlser-hf.

            For 10.6 it is the commit range 2a8f83a559d..c5bab76559c based on your bb-10.6-sqlser-hf. Because of the introduction of ddl log in 10.6, two extra patches are includes, i.e. a fixup which should probably be squashed into your commit

            0ac8873630e * MDEV-27595 [fixup] Move ddl_log_initialize() to before plugin_init()

            as well as a backport of monty's fix for MDEV-32532

            83736277b41 * MDEV-32532 Assertion failure in ddl_log_increment_phase_no_lock upon ..

            ycp Yuchen Pei added a comment - Hi holyfoot , ptal thanks 9c6548ab754 upstream/bb-10.4-mdev-29870 MDEV-32753 Make spider init queries compatible with oracle sql mode 4dd8d0b9cc4 MDEV-32683 Spider engine does not load with non-default alter-algorithm 23cc261fabd MDEV-32485 Fix Spider upgrade failure caused by duplication in mysql.func 585fb7fcc8b MDEV-32507 Spider: Use $MTR_SUITE_DIR for init-file files c6c6bc751f6 MDEV-32515 Use $MYSQLD_LAST_CMD in spider/bugfix.mdev_30370 2bdfd398134 MDEV-22979 MDEV-27233 MDEV-28218 Fixing spider init bugs c6d4ef55b13 MDEV-27095 clean up spd_init_query.h fb2f4e53e63 MDEV-27095 installing one spider plugin should not trigger others Above is 10.4. For 10.5 it is the commit range 1af055a555c..5b847372069, based on your bb-10.5-sqlser-hf. For 10.6 it is the commit range 2a8f83a559d..c5bab76559c based on your bb-10.6-sqlser-hf. Because of the introduction of ddl log in 10.6, two extra patches are includes, i.e. a fixup which should probably be squashed into your commit 0ac8873630e * MDEV-27595 [fixup] Move ddl_log_initialize() to before plugin_init() as well as a backport of monty 's fix for MDEV-32532 83736277b41 * MDEV-32532 Assertion failure in ddl_log_increment_phase_no_lock upon ..
            ycp Yuchen Pei made changes -
            Assignee Yuchen Pei [ JIRAUSER52627 ] Alexey Botchkov [ holyfoot ]
            Status Stalled [ 10000 ] In Review [ 10002 ]
            ycp Yuchen Pei made changes -
            holyfoot Alexey Botchkov added a comment -

            Looks ok to push.
            SerG said he merged the SQL service in 10.5/6 separately, so you could check with him about pushing into the main branches.

            holyfoot Alexey Botchkov added a comment - Looks ok to push. SerG said he merged the SQL service in 10.5/6 separately, so you could check with him about pushing into the main branches.
            holyfoot Alexey Botchkov made changes -
            Assignee Alexey Botchkov [ holyfoot ] Yuchen Pei [ JIRAUSER52627 ]
            Status In Review [ 10002 ] Stalled [ 10000 ]
            ycp Yuchen Pei added a comment -

            Pushed the following to 10.4

            ba94778d2c4 upstream/bb-10.4-mdev-29870 upstream/10.4 MDEV-32753 Make spider init queries compatible with oracle sql mode
            		 
            69389c03b16 MDEV-32683 Spider engine does not load with non-default alter-algorithm
            		 
            9259b4b8494 MDEV-32485 Fix Spider upgrade failure caused by duplication in mysql.func
            		 
            30af9872599 MDEV-32507 Spider: Use $MTR_SUITE_DIR for init-file files
            		 
            70283aca344 MDEV-32515 Use $MYSQLD_LAST_CMD in spider/bugfix.mdev_30370
            		 
            d8f5d2bef06 MDEV-22979 MDEV-27233 MDEV-28218 Fixing spider init bugs
            		 
            afe63ec6144 MDEV-27095 clean up spd_init_query.h
            		 
            f0af56be015 MDEV-27095 installing one spider plugin should not trigger others

            ycp Yuchen Pei added a comment - Pushed the following to 10.4 ba94778d2c4 upstream/bb-10.4-mdev-29870 upstream/10.4 MDEV-32753 Make spider init queries compatible with oracle sql mode 69389c03b16 MDEV-32683 Spider engine does not load with non-default alter-algorithm 9259b4b8494 MDEV-32485 Fix Spider upgrade failure caused by duplication in mysql.func 30af9872599 MDEV-32507 Spider: Use $MTR_SUITE_DIR for init-file files 70283aca344 MDEV-32515 Use $MYSQLD_LAST_CMD in spider/bugfix.mdev_30370 d8f5d2bef06 MDEV-22979 MDEV-27233 MDEV-28218 Fixing spider init bugs afe63ec6144 MDEV-27095 clean up spd_init_query.h f0af56be015 MDEV-27095 installing one spider plugin should not trigger others
            ycp Yuchen Pei made changes -
            Fix Version/s 10.4.33 [ 29516 ]
            Fix Version/s 10.4 [ 22408 ]
            Fix Version/s 10.5 [ 23123 ]
            Fix Version/s 10.6 [ 24028 ]
            Resolution Fixed [ 1 ]
            Status Stalled [ 10000 ] Closed [ 6 ]
            JIraAutomate JiraAutomate made changes -
            Fix Version/s 10.5.24 [ 29517 ]
            Fix Version/s 10.6.17 [ 29518 ]
            Fix Version/s 10.11.7 [ 29519 ]
            Fix Version/s 11.0.5 [ 29520 ]
            Fix Version/s 11.1.4 [ 29024 ]
            Fix Version/s 11.2.3 [ 29521 ]
            ycp Yuchen Pei made changes -
            Fix Version/s 11.1.4 [ 29024 ]
            Fix Version/s 10.11.7 [ 29519 ]
            Fix Version/s 11.0.5 [ 29520 ]
            Fix Version/s 11.2.3 [ 29521 ]

            People

              ycp Yuchen Pei
              nayuta-yanagisawa Nayuta Yanagisawa (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.