Uploaded image for project: 'MariaDB Foundation Development'
  1. MariaDB Foundation Development
  2. MDBF-501

Enhance MyDumper for MariaDB

Details

    • Task
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • None
    • N/A
    • None
    • None

    Description

      MyDumper is a multi-threaded database dumping tool and MyLoader is the complimentary database restore tool (https://github.com/mydumper/mydumper).

      There are some enhancements that can be made to MyDumper / MyLoader specifically for MariaDB, we should work on those.

      Attachments

        Issue Links

          split to

          Task - A task that needs to be done. MDBF-513 Mydumper: Add SEQUENCEs support

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. MDBF-514 Mydumper: Add System Version Tables support

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Task - A task that needs to be done. MDBF-515 Mydumper: test DDL locking mechanism

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Activity

            Ascending order - Click to sort in descending order
            TheLinuxJedi Andrew Hutchings (Inactive) created issue -
            TheLinuxJedi Andrew Hutchings (Inactive) made changes -
            Field Original Value New Value
            Fix Version/s N/A [ 27305 ]
            TheLinuxJedi Andrew Hutchings (Inactive) made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            TheLinuxJedi Andrew Hutchings (Inactive) added a comment -

            Note from Monty: Check sequences work.

            TheLinuxJedi Andrew Hutchings (Inactive) added a comment - Note from Monty: Check sequences work.
            TheLinuxJedi Andrew Hutchings (Inactive) made changes -
            Description There are some enhancements that can be made to MyDumper / MyLoader specifically for MariaDB, we should work on those. MyDumper is a multi-threaded database dumping tool and MyLoader is the complimentary database restore tool (https://github.com/mydumper/mydumper).

            There are some enhancements that can be made to MyDumper / MyLoader specifically for MariaDB, we should work on those.
            TheLinuxJedi Andrew Hutchings (Inactive) added a comment -

            Marko recommends implementing MDEV-515 using.

            SET unique_checks=0,foreign_key_checks=0

            TheLinuxJedi Andrew Hutchings (Inactive) added a comment - Marko recommends implementing MDEV-515 using. SET unique_checks=0,foreign_key_checks=0
            TheLinuxJedi Andrew Hutchings (Inactive) added a comment -

            My own notes so far:

            • CMake warning on GLIB2 find package needs fixing
            • Build docs ON even if Sphinx not found
            • Assumes libmysqlclient found when it isn't, needs a CMake check
            • CMake compatibility warning needs fixing
            • SSL options won't compile against MariaDB Connector/C
            TheLinuxJedi Andrew Hutchings (Inactive) added a comment - My own notes so far: CMake warning on GLIB2 find package needs fixing Build docs ON even if Sphinx not found Assumes libmysqlclient found when it isn't, needs a CMake check CMake compatibility warning needs fixing SSL options won't compile against MariaDB Connector/C
            TheLinuxJedi Andrew Hutchings (Inactive) added a comment -

            Set checks for MDEV-515 already exist in myloader.

            Fixed a bunch of build issues with https://github.com/mydumper/mydumper/pull/914

            TheLinuxJedi Andrew Hutchings (Inactive) added a comment - Set checks for MDEV-515 already exist in myloader. Fixed a bunch of build issues with https://github.com/mydumper/mydumper/pull/914
            Permalink
            TheLinuxJedi Andrew Hutchings (Inactive) logged work - 2022-11-23 11:25
            • Time Spent:
              0.75d
               
              Whole bunch of build fixes.
            TheLinuxJedi Andrew Hutchings (Inactive) made changes -
            Worklog Id 105248 [ 105248 ]
            Remaining Estimate 0d [ 0 ]
            Time Spent 0.75d [ 21600 ]
            Permalink
            TheLinuxJedi Andrew Hutchings (Inactive) logged work - 2022-11-25 10:16
            • Time Spent:
              1d
               
              <No comment>
            TheLinuxJedi Andrew Hutchings (Inactive) added a comment -

            Added ASAN and TSAN: https://github.com/mydumper/mydumper/pull/917 both currently fail with master.

            Additionally myloader threads lock up: https://github.com/mydumper/mydumper/issues/915

            Next steps for me:

            • Test / fix sequences
            • Try to diagnose myloader thread lockup
            TheLinuxJedi Andrew Hutchings (Inactive) added a comment - Added ASAN and TSAN: https://github.com/mydumper/mydumper/pull/917 both currently fail with master. Additionally myloader threads lock up: https://github.com/mydumper/mydumper/issues/915 Next steps for me: Test / fix sequences Try to diagnose myloader thread lockup
            TheLinuxJedi Andrew Hutchings (Inactive) made changes -
            Worklog Id 105303 [ 105303 ]
            Time Spent 0.75d [ 21600 ] 1.75d [ 50400 ]
            Permalink
            TheLinuxJedi Andrew Hutchings (Inactive) logged work - 2022-12-02 15:04
            • Time Spent:
              0.25d
               
              <No comment>
            TheLinuxJedi Andrew Hutchings (Inactive) added a comment -

            Added pull request to enhance debugging: https://github.com/mydumper/mydumper/pull/929

            Closing this ticket and tracking the rest of the work in separate tickets.

            TheLinuxJedi Andrew Hutchings (Inactive) added a comment - Added pull request to enhance debugging: https://github.com/mydumper/mydumper/pull/929 Closing this ticket and tracking the rest of the work in separate tickets.
            TheLinuxJedi Andrew Hutchings (Inactive) made changes -
            Worklog Id 105645 [ 105645 ]
            Time Spent 1.75d [ 50400 ] 2d [ 57600 ]
            TheLinuxJedi Andrew Hutchings (Inactive) made changes -
            Component/s None [ 18105 ]
            Resolution Fixed [ 1 ]
            Status In Progress [ 3 ] Closed [ 6 ]
            TheLinuxJedi Andrew Hutchings (Inactive) made changes -
            TheLinuxJedi Andrew Hutchings (Inactive) made changes -
            TheLinuxJedi Andrew Hutchings (Inactive) made changes -
            julien.fritsch Julien Fritsch made changes -
            Workflow MariaDB v4 [ 172593 ] MariaDB Foundation v1 [ 188692 ]
            cvicentiu Vicențiu Ciorbaru made changes -
            Component/s None [ 18105 ]

            People

              TheLinuxJedi Andrew Hutchings (Inactive)
              TheLinuxJedi Andrew Hutchings (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0d
                  0d
                  Logged:
                  Time Spent - 2d
                  2d