Uploaded image for project: 'MariaDB Foundation Development'
  1. MariaDB Foundation Development
  2. MDBF-513

Mydumper: Add SEQUENCEs support

Details

    • Task
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • None
    • N/A
    • None
    • None

    Description

      Mydumper needs to support MariaDB's SEQUENCEs for backup and restoration.

      Attachments

        Issue Links

          relates to

          New Feature - A new feature of the product, which has yet to be developed. MDEV-30155 SEQUENCEs/VIEWS/TRIGGERS/PROCEDURES dumped with mariadb-dump cannot be restored in different DB

          • Critical - Crashes, loss of data, severe memory leak.
          • Confirmed
          split from

          Task - A task that needs to be done. MDBF-501 Enhance MyDumper for MariaDB

          • Major - Major loss of function.
          • Closed
          links to

          Web Link Mydumper GitHub #921

          Activity

            Ascending order - Click to sort in descending order
            Permalink
            TheLinuxJedi Andrew Hutchings (Inactive) logged work - 2022-12-02 15:06
            • Time Spent:
              2d
               
              <No comment>
            TheLinuxJedi Andrew Hutchings (Inactive) created issue -
            TheLinuxJedi Andrew Hutchings (Inactive) made changes -
            Field Original Value New Value
            TheLinuxJedi Andrew Hutchings (Inactive) made changes -
            Fix Version/s N/A [ 27305 ]
            TheLinuxJedi Andrew Hutchings (Inactive) made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            TheLinuxJedi Andrew Hutchings (Inactive) made changes -
            Worklog Id 105646 [ 105646 ]
            Remaining Estimate 0d [ 0 ]
            Time Spent 2d [ 57600 ]
            TheLinuxJedi Andrew Hutchings (Inactive) added a comment -

            Development is mostly done in https://github.com/linuxJedi/mydumper/tree/mariadb-sequences

            Current issue is with myloader, SEQUENCEs need to be restored before tables. This is not yet expressible in myloader which tries to restore everything in parallel. I have an idea on how to resolve this which I am passing by David Ducos.

            TheLinuxJedi Andrew Hutchings (Inactive) added a comment - Development is mostly done in https://github.com/linuxJedi/mydumper/tree/mariadb-sequences Current issue is with myloader, SEQUENCEs need to be restored before tables. This is not yet expressible in myloader which tries to restore everything in parallel. I have an idea on how to resolve this which I am passing by David Ducos.
            TheLinuxJedi Andrew Hutchings (Inactive) made changes -
            TheLinuxJedi Andrew Hutchings (Inactive) added a comment - - edited

            Second issue to contend with, a relative reference to a SEQUENCE in `nextval` turns into an absolute one including the schema name. This means that backup/restore into a different database breaks.

            Discussions ongoing on how to solve these two issues.

            TheLinuxJedi Andrew Hutchings (Inactive) added a comment - - edited Second issue to contend with, a relative reference to a SEQUENCE in `nextval` turns into an absolute one including the schema name. This means that backup/restore into a different database breaks. Discussions ongoing on how to solve these two issues.
            TheLinuxJedi Andrew Hutchings (Inactive) made changes -
            Permalink
            TheLinuxJedi Andrew Hutchings (Inactive) logged work - 2022-12-21 08:00
            • Time Spent:
              0.5d
               
              <No comment>
            TheLinuxJedi Andrew Hutchings (Inactive) added a comment -

            `nextval` issue is solved using a regex filter.

            David wants some retry code in there so tables that fail due to missing sequence are retried after the initial table threads have run.

            TheLinuxJedi Andrew Hutchings (Inactive) added a comment - `nextval` issue is solved using a regex filter. David wants some retry code in there so tables that fail due to missing sequence are retried after the initial table threads have run.
            TheLinuxJedi Andrew Hutchings (Inactive) made changes -
            Worklog Id 107292 [ 107292 ]
            Time Spent 2d [ 57600 ] 2.5d [ 72000 ]
            Permalink
            TheLinuxJedi Andrew Hutchings (Inactive) logged work - 2023-01-31 08:29
            • Time Spent:
              0.25d
               
              <No comment>
            TheLinuxJedi Andrew Hutchings (Inactive) added a comment -

            Some more goundwork has been laid and merged in. The MariaDB detection in mydumper now works correctly and the table detection queries for MariaDB have been changed so that sequences and system version tables can be detected correctly.

            TheLinuxJedi Andrew Hutchings (Inactive) added a comment - Some more goundwork has been laid and merged in. The MariaDB detection in mydumper now works correctly and the table detection queries for MariaDB have been changed so that sequences and system version tables can be detected correctly.
            TheLinuxJedi Andrew Hutchings (Inactive) made changes -
            Worklog Id 107531 [ 107531 ]
            Time Spent 2.5d [ 72000 ] 2.75d [ 79200 ]
            TheLinuxJedi Andrew Hutchings (Inactive) made changes -
            Remaining Estimate 0d [ 0 ] 3h [ 10800 ]
            Original Estimate 3h [ 10800 ]
            TheLinuxJedi Andrew Hutchings (Inactive) made changes -
            Status In Progress [ 3 ] Stalled [ 10000 ]
            TheLinuxJedi Andrew Hutchings (Inactive) added a comment -

            David has done the final parts of this, it is currently in review.

            TheLinuxJedi Andrew Hutchings (Inactive) added a comment - David has done the final parts of this, it is currently in review.
            TheLinuxJedi Andrew Hutchings (Inactive) made changes -
            Status Stalled [ 10000 ] In Review [ 10002 ]
            TheLinuxJedi Andrew Hutchings (Inactive) added a comment -

            This has been merged upstream.

            TheLinuxJedi Andrew Hutchings (Inactive) added a comment - This has been merged upstream.
            TheLinuxJedi Andrew Hutchings (Inactive) made changes -
            Component/s None [ 18105 ]
            Resolution Fixed [ 1 ]
            Status In Review [ 10002 ] Closed [ 6 ]
            Remaining Estimate 3h [ 10800 ] 0d [ 0 ]
            julien.fritsch Julien Fritsch made changes -
            Workflow MariaDB v4 [ 172908 ] MariaDB Foundation v1 [ 188702 ]
            cvicentiu Vicențiu Ciorbaru made changes -
            Component/s None [ 18105 ]

            People

              TheLinuxJedi Andrew Hutchings (Inactive)
              TheLinuxJedi Andrew Hutchings (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 3h Original Estimate - 3h
                  3h
                  Remaining:
                  Remaining Estimate - 0d
                  0d
                  Logged:
                  Time Spent - 2.75d
                  2.75d