Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-16967

Spider CREATE TABLE Pushdown

Details

    Description

      Currently it is necessary for a Spidered table to be created separately on each node in the Spider cluster. This process by the end user is time-consuming, repetitive and error-prone.

      To simplify this, we can have the Spider node push down the CREATE TABLE statement to each of the data nodes. The statement pushed down to the data nodes would not have the COMMENT that is used to specify the connection information.

      In case of a failure on any of the data nodes, Spider would potentially need to do cleanup to synchronize the table definition across the Spider cluster. Since DDLs are not transactional, the table cleanup would need to be implemented in the Spider node without the benefit of a rollback to remove the table definition throughout the cluster.

      We would also add a new Spider configuration setting to specify :

      Whether CREATE OR REPLACE TABLE is appropriate on the data nodes.

      Attachments

        Issue Links

          blocks

          Task - A task that needs to be done. MDEV-14509 Spider-VP Spiral Patches

          • Major - Major loss of function.
          • Stalled
          relates to

          Task - A task that needs to be done. MDEV-16882 extend DDL syntax for Spider to pass-through to data nodes

          • Major - Major loss of function.
          • Confirmed

          Task - A task that needs to be done. MDEV-18313 Supports 'wrapper "mariadb"' for connection information

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Task - A task that needs to be done. MDEV-32804 Remove the (unimplemented and unused) spider_rewrite_plugin

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            jacob-mathew Jacob Mathew (Inactive) added a comment -

            Correct setup of vertically partitioned tables requires an additional level of care and precision over correctly setting up a Spider cluster, and is therefore even more trouble prone. We therefore want to complete the CREATE TABLE pushdown task before proceeding with the tasks pertaining to vertically partitioned tables.

            jacob-mathew Jacob Mathew (Inactive) added a comment - Correct setup of vertically partitioned tables requires an additional level of care and precision over correctly setting up a Spider cluster, and is therefore even more trouble prone. We therefore want to complete the CREATE TABLE pushdown task before proceeding with the tasks pertaining to vertically partitioned tables.
            Kentoku Kentoku Shiba (Inactive) added a comment - - edited

            I just pushed it to bb-10.4-spider-ks. Please review this.
            4d37898

            Kentoku Kentoku Shiba (Inactive) added a comment - - edited I just pushed it to bb-10.4-spider-ks. Please review this. 4d37898
            serg Sergei Golubchik added a comment -

            this commit doesn't change anything outside of storage/spider, so I'm reassigning it back to the original reviewer

            serg Sergei Golubchik added a comment - this commit doesn't change anything outside of storage/spider, so I'm reassigning it back to the original reviewer
            sachin.setiya.007 Sachin Setiya (Inactive) added a comment -

            Okay to push after a more elaborate commit message.

            sachin.setiya.007 Sachin Setiya (Inactive) added a comment - Okay to push after a more elaborate commit message.
            f_razzoli Federico Razzoli added a comment -

            If in some cases the operation fails and orphaned tables remain on some remote servers, it wouldn't be a big problem. As long as we can re-run a create-or-replace operation.

            f_razzoli Federico Razzoli added a comment - If in some cases the operation fails and orphaned tables remain on some remote servers, it wouldn't be a big problem. As long as we can re-run a create-or-replace operation.

            People

              Unassigned Unassigned
              jacob-mathew Jacob Mathew (Inactive)
              Votes:
              1 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.