Details

    • New Feature
    • Status: Stalled (View Workflow)
    • Major
    • Resolution: Unresolved
    • None
    • None
    • maxgui, REST-API
    • None
    • MXS-SPRINT-169, MXS-SPRINT-170, MXS-SPRINT-171

    Description

      Maxscale should have the ability to compare tables within a server and across servers/topologies. After comparison, it should provide a diff and have the ability to use that diff to sync the tables.

      This should also be made available in the IDE.

      (The scope of this ticket will be for data comparison and sync.)

      Attachments

        Issue Links

          Activity

            Hi @Duong

            Can we implement for both data compare and structure compare features would be good. I mean it should compare both data and compare all the schema objects (Schema/Table/Procs/Triggers/Any) from source to destination.

            naresh.chandra@copart.com Naresh Chandra added a comment - Hi @Duong Can we implement for both data compare and structure compare features would be good. I mean it should compare both data and compare all the schema objects (Schema/Table/Procs/Triggers/Any) from source to destination.
            naresh.chandra@copart.com Naresh Chandra added a comment - - edited

            Hi Duong,

            Can we add compare/sync option to external servers as well? I mean if we don't add servers in the maxscale conf file, but we want to compare the data and table structures or sync data and structures for external servers would be a good feature for us.

            EX: User should be able to add temporary source/destination db server's details in the GUI and then compare external database data/structures from source to destination in the GUI.

            naresh.chandra@copart.com Naresh Chandra added a comment - - edited Hi Duong, Can we add compare/sync option to external servers as well? I mean if we don't add servers in the maxscale conf file, but we want to compare the data and table structures or sync data and structures for external servers would be a good feature for us. EX: User should be able to add temporary source/destination db server's details in the GUI and then compare external database data/structures from source to destination in the GUI.

            Hi Duong,

            Can you please include above external feature as well?

            naresh.chandra@copart.com Naresh Chandra added a comment - Hi Duong, Can you please include above external feature as well?

            Hi Duong,

            Just a thought, can we implement to compare data between different database technologies, it can be a MariaDB, Postgres, MongoDB or any other technologies?

            naresh.chandra@copart.com Naresh Chandra added a comment - Hi Duong, Just a thought, can we implement to compare data between different database technologies, it can be a MariaDB, Postgres, MongoDB or any other technologies?

            naresh.chandra@copart.com
            I believe the above can be split into different tickets, but the scope for this ticket is compare and sync for data.
            MXS-4386 is for DDL

            kathrynsizemore Kathryn Sizemore added a comment - naresh.chandra@copart.com I believe the above can be split into different tickets, but the scope for this ticket is compare and sync for data. MXS-4386 is for DDL

            Kathryn Sizemore Sure, thanks for the update.

            naresh.chandra@copart.com Naresh Chandra added a comment - Kathryn Sizemore Sure, thanks for the update.
            markus makela markus makela added a comment -

            From the REST-API side of things, this requires a new endpoint to do the result comparison. Doing the diff computation in the browser is too inefficient given the fact that the whole result would have to be turned into JSON and then a diff be generated from that, the amount of browser memory required for this would be very high. Calculating an actual diff over the results might also not be necessary as that would potentially require storing both results in memory to compute it and if a natural "sort key" is available, is not needed. For the simple case of comparing two tables, comparing the results of the table ordered by the primary key should be simple enough to implement so that it can be done automatically. For arbitrary queries, the user would have to tell which field to use as the fake "primary key" that is used to compute the difference of the two results.

            The API should reuse the existing code from the ETL feature that allows two connections to be used at the same time. A new /sql/diff endpoint would then be given two connections and an SQL query and the returned result would be the diff of the query results.

            markus makela markus makela added a comment - From the REST-API side of things, this requires a new endpoint to do the result comparison. Doing the diff computation in the browser is too inefficient given the fact that the whole result would have to be turned into JSON and then a diff be generated from that, the amount of browser memory required for this would be very high. Calculating an actual diff over the results might also not be necessary as that would potentially require storing both results in memory to compute it and if a natural "sort key" is available, is not needed. For the simple case of comparing two tables, comparing the results of the table ordered by the primary key should be simple enough to implement so that it can be done automatically. For arbitrary queries, the user would have to tell which field to use as the fake "primary key" that is used to compute the difference of the two results. The API should reuse the existing code from the ETL feature that allows two connections to be used at the same time. A new /sql/diff endpoint would then be given two connections and an SQL query and the returned result would be the diff of the query results.

            People

              thien.ly Thien Ly
              manjot Manjot Singh (Inactive)
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.