Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-34529

Shrink the system tablespace when system tablespace contains MDEV-30671 leaked undo pages

Details

    Description

      Looks like the fix for MDEV-14795 didn't handle leaked undo pages (because even their extent has never been marked as freed). So MDEV-14795 assumes that it is used by server. This does NOT allow to shrink ibdata1 in a newer versions, even after a slow shutdown and even when UNDO is already moved to separate tablespaces (so instance re-creation is still required if the instance was ever affected by MDEV-30671.

      Attachments

        Issue Links

          Activity

            valerii Valerii Kravchuk created issue -
            thiru Thirunarayanan Balathandayuthapani made changes -
            Field Original Value New Value
            Assignee Thirunarayanan Balathandayuthapani [ thiru ]
            valerii Valerii Kravchuk made changes -
            Affects Version/s 11.2.1 [ 29034 ]

            I can repeat the issue.
            Followed the steps in MDEV-31234 in affected version.

            1) started the server with --innodb-immediate-scrub-data-uncompressed=1. --innodb_undo_tablespace=0.
            2) Ran sysbench. did slow shutdown and ended up with 1528 to_purge undo log pages. (leaked undo log corrupt pages)
            Start the latest 10.6 against the data directory, did slow shutdown

            2024-07-04 14:29:50 0 [Note] InnoDB: Truncating system tablespace from 4864 to 2496 pages
            2024-07-04 14:29:50 0 [Note] InnoDB: System tablespace truncated successfully
            

            Ran innochecksum tool against shrinked system tablespace and we end up with same

             to_purge Undo page state: 0 active, 128 cached, 1528 to_purge, 0 prepared, 148 other
            

            So verified what was exist before 2496 page, it was all leaked undo pages and previous extent state has been marked as "XDES_FREE_FRAG" and 12 pages has been used in that extent.

            #::2484         |               Undo log page                   |, Will be freed in purge when all undodata in it is removed, -
            #::2485         |               Undo log page                   |, Will be freed in purge when all undodata in it is removed, -
            #::2486         |               Undo log page                   |, Will be freed in purge when all undodata in it is removed, -
            #::2487         |               Undo log page                   |, Will be freed in purge when all undodata in it is removed, -
            #::2488         |               Undo log page                   |, Will be freed in purge when all undodata in it is removed, -
            #::2489         |               Undo log page                   |, Will be freed in purge when all undodata in it is removed, -
            #::2490         |               Undo log page                   |, Will be freed in purge when all undodata in it is removed, -
            #::2491         |               Undo log page                   |, Will be freed in purge when all undodata in it is removed, -
            #::2492         |               Undo log page                   |, Will be freed in purge when all undodata in it is removed, -
            #::2493         |               Undo log page                   |, Will be freed in purge when all undodata in it is removed, -
            #::2494         |               Undo log page                   |, Will be freed in purge when all undodata in it is removed, -
            #::2495         |               Undo log page                   |, Will be freed in purge when all undodata in it is removed, -
            

            MDEV-14795 didn't handle leaked undo pages because even their extent has never been marked as freed. So MDEV-14795 assumes that it is used by server.
            MDEV-14795 finds the last used extent in the system tablespace and truncates the remaining file.
            Attached the ibdata1 of the leaky undo pages for future reference.

            thiru Thirunarayanan Balathandayuthapani added a comment - - edited I can repeat the issue. Followed the steps in MDEV-31234 in affected version. 1) started the server with --innodb-immediate-scrub-data-uncompressed=1. --innodb_undo_tablespace=0. 2) Ran sysbench. did slow shutdown and ended up with 1528 to_purge undo log pages. (leaked undo log corrupt pages) Start the latest 10.6 against the data directory, did slow shutdown 2024-07-04 14:29:50 0 [Note] InnoDB: Truncating system tablespace from 4864 to 2496 pages 2024-07-04 14:29:50 0 [Note] InnoDB: System tablespace truncated successfully Ran innochecksum tool against shrinked system tablespace and we end up with same to_purge Undo page state: 0 active, 128 cached, 1528 to_purge, 0 prepared, 148 other So verified what was exist before 2496 page, it was all leaked undo pages and previous extent state has been marked as "XDES_FREE_FRAG" and 12 pages has been used in that extent. #::2484 | Undo log page |, Will be freed in purge when all undodata in it is removed, - #::2485 | Undo log page |, Will be freed in purge when all undodata in it is removed, - #::2486 | Undo log page |, Will be freed in purge when all undodata in it is removed, - #::2487 | Undo log page |, Will be freed in purge when all undodata in it is removed, - #::2488 | Undo log page |, Will be freed in purge when all undodata in it is removed, - #::2489 | Undo log page |, Will be freed in purge when all undodata in it is removed, - #::2490 | Undo log page |, Will be freed in purge when all undodata in it is removed, - #::2491 | Undo log page |, Will be freed in purge when all undodata in it is removed, - #::2492 | Undo log page |, Will be freed in purge when all undodata in it is removed, - #::2493 | Undo log page |, Will be freed in purge when all undodata in it is removed, - #::2494 | Undo log page |, Will be freed in purge when all undodata in it is removed, - #::2495 | Undo log page |, Will be freed in purge when all undodata in it is removed, - MDEV-14795 didn't handle leaked undo pages because even their extent has never been marked as freed. So MDEV-14795 assumes that it is used by server. MDEV-14795 finds the last used extent in the system tablespace and truncates the remaining file. Attached the ibdata1 of the leaky undo pages for future reference.
            thiru Thirunarayanan Balathandayuthapani made changes -
            Attachment undo_leak_data.tar.gz [ 73800 ]
            mariadb-jira-automation Jira Automation (IT) made changes -
            Zendesk Related Tickets 201662
            Zendesk active tickets 201662
            JIraAutomate JiraAutomate made changes -
            Fix Version/s 10.4 [ 22408 ]
            Fix Version/s 10.5 [ 23123 ]
            Fix Version/s 10.6 [ 24028 ]
            thiru Thirunarayanan Balathandayuthapani made changes -
            Fix Version/s 11.2 [ 28603 ]
            Fix Version/s 11.3 [ 28565 ]
            Fix Version/s 11.4 [ 29301 ]
            Fix Version/s 10.4 [ 22408 ]
            Fix Version/s 10.5 [ 23123 ]
            Fix Version/s 10.6 [ 24028 ]
            julien.fritsch Julien Fritsch made changes -
            Fix Version/s 11.3 [ 28565 ]
            thiru Thirunarayanan Balathandayuthapani made changes -
            Status Open [ 1 ] Confirmed [ 10101 ]
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            Priority Major [ 3 ] Critical [ 2 ]
            tiandiwonder Shaohua Wang added a comment -

            So we should fix the undo page leak bug?

            tiandiwonder Shaohua Wang added a comment - So we should fix the undo page leak bug?
            julien.fritsch Julien Fritsch made changes -
            Status Confirmed [ 10101 ] In Progress [ 3 ]
            thiru Thirunarayanan Balathandayuthapani made changes -
            Assignee Thirunarayanan Balathandayuthapani [ thiru ] Marko Mäkelä [ marko ]
            Status In Progress [ 3 ] In Review [ 10002 ]

            The undo page leak bug MDEV-31234 was fixed a long time ago. This fix is only freeing up the previously leaked pages in the system tablespace.

            marko Marko Mäkelä added a comment - The undo page leak bug MDEV-31234 was fixed a long time ago. This fix is only freeing up the previously leaked pages in the system tablespace.
            mleich Matthias Leich added a comment - - edited

            origin/11.2-MDEV-34529 c1fc29443a213367c7b0c521d0d9c1c1636bf48b 2024-10-10T19:21:33+05:30
            performed well in RQG testing. No new problems.
            No objections against pushing the fix.

            mleich Matthias Leich added a comment - - edited origin/11.2- MDEV-34529 c1fc29443a213367c7b0c521d0d9c1c1636bf48b 2024-10-10T19:21:33+05:30 performed well in RQG testing. No new problems. No objections against pushing the fix.
            marko Marko Mäkelä made changes -
            Assignee Marko Mäkelä [ marko ] Thirunarayanan Balathandayuthapani [ thiru ]
            Status In Review [ 10002 ] Stalled [ 10000 ]
            thiru Thirunarayanan Balathandayuthapani made changes -
            Component/s Storage Engine - InnoDB [ 10129 ]
            Fix Version/s 11.2.6 [ 29906 ]
            Fix Version/s 11.2 [ 28603 ]
            Fix Version/s 11.4 [ 29301 ]
            Resolution Fixed [ 1 ]
            Status Stalled [ 10000 ] Closed [ 6 ]
            JIraAutomate JiraAutomate made changes -
            Fix Version/s 11.4.4 [ 29907 ]
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            ralf.gebhardt Ralf Gebhardt made changes -
            thiru Thirunarayanan Balathandayuthapani made changes -

            People

              thiru Thirunarayanan Balathandayuthapani
              valerii Valerii Kravchuk
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.