Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-19229

Allow innodb_undo_tablespaces to be changed after database creation

Details

    Description

      To change the number of undo tablespaces, we need to reinitialize the data directory (dump + restore).
      InnoDB should allow changing number of undo tablespaces as a part of restart process.
      System tablespace contains InnoDB dictionary tables, doublewrite buffer, change buffer tree
      and undo log pages. By allowing this change, InnoDB can reduce the workload on system tablespace.

      Requirements

      A prior shutdown with SET GLOBAL innodb_fast_shutdown=0 must be executed before adding the undo log tablespaces.

      This is because the undo logs must be empty (no incomplete or XA PREPARE transactions, nothing to be purged) so that the old undo tablespaces can discarded and new ones created.

      Steps

      1. Check whether the existing undo log exists. If exists then give the warnings about the slow shutdown and start the server normally.
      2. If not exist then free the system tablespace rollback segment header page else throw message about slow shutdown and continue the normal server start operation
      3. Free the system tablespace rollback segment header page of the slots 1...127
      4. Reset the TRX_SYS page and reinitialize the system tablespace rollback segment slot(0th slot), doublewrite information,
        update the binlog info and WSREP info in system rollback segment header page (0th slot page)
        Step(3) and (4) should happen within a single mini-transaction
      5. Delete the old undo tablespaces if any
      6. Make checkpoint to get rid of old undo tablespace redo log records
      7. Read the latest MAX_SPACE_ID from dictionary header page
      8. Create the new specified undo log tablespace and initialize the page0 of all undo tablespaces
        Step (7) and Step (8) should happen within a single mini-transaction
      9. Make the checkpoint again to make sure that next startup or backup reads the undo log tablespaces before
        opening the redo log records
      10. Create the rollback segment for each rollback segment in a round robin fashion

      Attachments

        Issue Links

          blocks

          Task - A task that needs to be done. MDEV-29986 Set innodb_undo_tablespaces=3 by default

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed
          causes

          Bug - A problem which impairs or prevents the functions of the product. MDEV-30122 mariabackup.skip_innodb crashes when innodb_undo_tablespaces > 0

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-30158 InnoDB fails to start ther server 10.11 when innodb_undo_tablespaces mismatch

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-30311 system-wide max transaction id corrupted (from MySQL-5.7 upgrade)

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-32974 Member fails to join due to old seqno in GTID

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-34200 InnoDB tries to write to read-only system tablespace in buf_dblwr_t::init_or_load_pages()

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. MDEV-21216 InnoDB performs dirty read of TRX_SYS page before crash recovery

          • Major - Major loss of function.
          • Closed
          is duplicated by

          Task - A task that needs to be done. MDEV-10373 Possibility to implement UNDO tablespace in existing intance

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Task - A task that needs to be done. MDEV-17555 Allow moving UNDO segments into separate UNDO tablespaces without total rebuild of the data directory from a logical backup

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-27121 mariabackup incompatible with disabled dedicated undo log tablespaces

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. MDEV-29983 Deprecate innodb_file_per_table

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Task - A task that needs to be done. MDEV-14481 Execute InnoDB crash recovery in the background

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. MDEV-14795 InnoDB system tablespace cannot be shrunk

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-21952 ibdata1 file size growing in MariaDB

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-31488 Restart on backupped data fails with InnoDB: Failing assertion: !i || prev_id + 1 == space_id at srv0start.cc line 803

          • Major - Major loss of function.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            thiru Thirunarayanan Balathandayuthapani created issue -
            thiru Thirunarayanan Balathandayuthapani made changes -
            Field Original Value New Value
            Status Open [ 1 ] Confirmed [ 10101 ]
            marko Marko Mäkelä made changes -
            Description Requirements:
            ==============
            (1) Slow shutdown should have happened before increasing the undo log tablespaces.

            This patch should come to effect when there is an increase in the *innodb_undo_tablespaces* variable.

            Steps:
            =======

            (1) Check whether the existing undo log exists. If exists then give the warnings about the slow shutdown and start the server normally.
            (2) If not exist then create the specified number of undo tablespace.
            (3) Create the rollback segment for each rollback segment in a round robin fashion
            (4) Step (2), (3) should happen with the same MTR.
            		h1. Requirements
            * A prior shutdown with {{SET GLOBAL innodb_fast_shutdown=0}} must be executed before adding the undo log tablespaces.

            This is because the undo logs must be empty (no incomplete or XA PREPARE transactions, nothing to be purged) so that the old undo tablespaces can discarded and new ones created.

            This patch should come to effect when there is an increase in the *innodb_undo_tablespaces* variable.

            h1. Steps
            # Check whether the existing undo log exists. If exists then give the warnings about the slow shutdown and start the server normally.
            # If not exist then create the specified number of undo tablespace.
            # Create the rollback segment for each rollback segment in a round robin fashion
            # The modifications must happen atomically within a single mini-transaction.
            marko Marko Mäkelä made changes -
            thiru Thirunarayanan Balathandayuthapani made changes -
            Status Confirmed [ 10101 ] In Progress [ 3 ]
            marko Marko Mäkelä made changes -
            thiru Thirunarayanan Balathandayuthapani made changes -
            Status In Progress [ 3 ] Stalled [ 10000 ]
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 95312 ] MariaDB v4 [ 143586 ]
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            Fix Version/s 10.3 [ 22126 ]
            Fix Version/s 10.5 [ 23123 ]
            Fix Version/s 10.6 [ 24028 ]
            Fix Version/s 10.7 [ 24805 ]
            Fix Version/s 10.8 [ 26121 ]
            Fix Version/s 10.9 [ 26905 ]
            Fix Version/s 10.10 [ 27530 ]
            Affects Version/s 10.3 [ 22126 ]
            Affects Version/s 10.4 [ 22408 ]
            Affects Version/s 10.5 [ 23123 ]
            Affects Version/s 10.6 [ 24028 ]
            Affects Version/s 10.7 [ 24805 ]
            Affects Version/s 10.8 [ 26121 ]
            Affects Version/s 10.9 [ 26905 ]
            Affects Version/s 10.10 [ 27530 ]
            Priority Major [ 3 ] Critical [ 2 ]
            Summary Increase the undo log tablespace during startup Allow innodb_undo_tablespaces to be changed after database creation
            thiru Thirunarayanan Balathandayuthapani made changes -
            Status Stalled [ 10000 ] In Progress [ 3 ]
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            thiru Thirunarayanan Balathandayuthapani made changes -
            Assignee Thirunarayanan Balathandayuthapani [ thiru ] Marko Mäkelä [ marko ]
            Status In Progress [ 3 ] In Review [ 10002 ]
            marko Marko Mäkelä made changes -
            Fix Version/s 10.3 [ 22126 ]
            Fix Version/s 10.4 [ 22408 ]
            Fix Version/s 10.5 [ 23123 ]
            Summary Allow innodb_undo_tablespaces to be changed after database creation Allow innodb_undo_tablespaces to be increased after database creation
            marko Marko Mäkelä made changes -
            Assignee Marko Mäkelä [ marko ] Thirunarayanan Balathandayuthapani [ thiru ]
            Status In Review [ 10002 ] Stalled [ 10000 ]
            thiru Thirunarayanan Balathandayuthapani made changes -
            Status Stalled [ 10000 ] In Progress [ 3 ]
            thiru Thirunarayanan Balathandayuthapani made changes -
            Assignee Thirunarayanan Balathandayuthapani [ thiru ] Marko Mäkelä [ marko ]
            Status In Progress [ 3 ] In Review [ 10002 ]
            marko Marko Mäkelä made changes -
            Assignee Marko Mäkelä [ marko ] Thirunarayanan Balathandayuthapani [ thiru ]
            Status In Review [ 10002 ] Stalled [ 10000 ]
            thiru Thirunarayanan Balathandayuthapani made changes -
            Assignee Thirunarayanan Balathandayuthapani [ thiru ] Marko Mäkelä [ marko ]
            Status Stalled [ 10000 ] In Review [ 10002 ]
            marko Marko Mäkelä made changes -
            Assignee Marko Mäkelä [ marko ] Thirunarayanan Balathandayuthapani [ thiru ]
            Status In Review [ 10002 ] Stalled [ 10000 ]
            thiru Thirunarayanan Balathandayuthapani made changes -
            Assignee Thirunarayanan Balathandayuthapani [ thiru ] Marko Mäkelä [ marko ]
            Status Stalled [ 10000 ] In Review [ 10002 ]
            marko Marko Mäkelä made changes -
            Summary Allow innodb_undo_tablespaces to be increased after database creation Allow innodb_undo_tablespaces to be changed after database creation
            thiru Thirunarayanan Balathandayuthapani made changes -
            Description h1. Requirements
            * A prior shutdown with {{SET GLOBAL innodb_fast_shutdown=0}} must be executed before adding the undo log tablespaces.

            This is because the undo logs must be empty (no incomplete or XA PREPARE transactions, nothing to be purged) so that the old undo tablespaces can discarded and new ones created.

            This patch should come to effect when there is an increase in the *innodb_undo_tablespaces* variable.

            h1. Steps
            # Check whether the existing undo log exists. If exists then give the warnings about the slow shutdown and start the server normally.
            # If not exist then create the specified number of undo tablespace.
            # Create the rollback segment for each rollback segment in a round robin fashion
            # The modifications must happen atomically within a single mini-transaction.
              To change the number of undo tablespaces, we need to reinitialize the data directory (dump + restore).
            InnoDB should allow changing number of undo tablespaces as a part of restart process.
            System tablespace contains InnoDB dictionary tables, doublewrite buffer, change buffer tree
            and undo log pages. By allowing this change, InnoDB can reduce the workload on system tablespace.

            h1. Requirements
            * A prior shutdown with {{SET GLOBAL innodb_fast_shutdown=0}} must be executed before adding the undo log tablespaces.

            This is because the undo logs must be empty (no incomplete or XA PREPARE transactions, nothing to be purged) so that the old undo tablespaces can discarded and new ones created.

            This patch should come to effect when there is an increase in the *innodb_undo_tablespaces* variable.

            h1. Steps

            # Check whether the existing undo log exists. If exists then give the warnings about the slow shutdown and start the server normally.
            # If not exist then create the specified number of undo tablespace.
            # Create the rollback segment for each rollback segment in a round robin fashion
            # The modifications must happen atomically within a single mini-transaction.
            thiru Thirunarayanan Balathandayuthapani made changes -
            Description
              To change the number of undo tablespaces, we need to reinitialize the data directory (dump + restore).
            InnoDB should allow changing number of undo tablespaces as a part of restart process.
            System tablespace contains InnoDB dictionary tables, doublewrite buffer, change buffer tree
            and undo log pages. By allowing this change, InnoDB can reduce the workload on system tablespace.

            h1. Requirements
            * A prior shutdown with {{SET GLOBAL innodb_fast_shutdown=0}} must be executed before adding the undo log tablespaces.

            This is because the undo logs must be empty (no incomplete or XA PREPARE transactions, nothing to be purged) so that the old undo tablespaces can discarded and new ones created.

            This patch should come to effect when there is an increase in the *innodb_undo_tablespaces* variable.

            h1. Steps

            # Check whether the existing undo log exists. If exists then give the warnings about the slow shutdown and start the server normally.
            # If not exist then create the specified number of undo tablespace.
            # Create the rollback segment for each rollback segment in a round robin fashion
            # The modifications must happen atomically within a single mini-transaction.             		  To change the number of undo tablespaces, we need to reinitialize the data directory (dump + restore).
            InnoDB should allow changing number of undo tablespaces as a part of restart process.
            System tablespace contains InnoDB dictionary tables, doublewrite buffer, change buffer tree
            and undo log pages. By allowing this change, InnoDB can reduce the workload on system tablespace.

            h1. Requirements
            A prior shutdown with {{SET GLOBAL innodb_fast_shutdown=0}} must be executed before adding the undo log tablespaces.

            This is because the undo logs must be empty (no incomplete or XA PREPARE transactions, nothing to be purged) so that the old undo tablespaces can discarded and new ones created.


            h1. Steps

            # Check whether the existing undo log exists. If exists then give the warnings about the slow shutdown and start the server normally.
            # If not exist then create the specified number of undo tablespace.
            # Create the rollback segment for each rollback segment in a round robin fashion
            # The modifications must happen atomically within a single mini-transaction.
            thiru Thirunarayanan Balathandayuthapani made changes -
            Description   To change the number of undo tablespaces, we need to reinitialize the data directory (dump + restore).
            InnoDB should allow changing number of undo tablespaces as a part of restart process.
            System tablespace contains InnoDB dictionary tables, doublewrite buffer, change buffer tree
            and undo log pages. By allowing this change, InnoDB can reduce the workload on system tablespace.

            h1. Requirements
            A prior shutdown with {{SET GLOBAL innodb_fast_shutdown=0}} must be executed before adding the undo log tablespaces.

            This is because the undo logs must be empty (no incomplete or XA PREPARE transactions, nothing to be purged) so that the old undo tablespaces can discarded and new ones created.


            h1. Steps

            # Check whether the existing undo log exists. If exists then give the warnings about the slow shutdown and start the server normally.
            # If not exist then create the specified number of undo tablespace.
            # Create the rollback segment for each rollback segment in a round robin fashion
            # The modifications must happen atomically within a single mini-transaction.             		  To change the number of undo tablespaces, we need to reinitialize the data directory (dump + restore).
            InnoDB should allow changing number of undo tablespaces as a part of restart process.
            System tablespace contains InnoDB dictionary tables, doublewrite buffer, change buffer tree
            and undo log pages. By allowing this change, InnoDB can reduce the workload on system tablespace.

            h1. Requirements
            A prior shutdown with {{SET GLOBAL innodb_fast_shutdown=0}} must be executed before adding the undo log tablespaces.

            This is because the undo logs must be empty (no incomplete or XA PREPARE transactions, nothing to be purged) so that the old undo tablespaces can discarded and new ones created.


            h1. Steps

            # Check whether the existing undo log exists. If exists then give the warnings about the slow shutdown and start the server normally.
            # If not exist then free the system tablespace rollback segment header page else throw message about slow shutdown and continue the normal server start operation
            # Free the system tablespace rollback segment header page of the slots 1...127
            # Set the rollback segment slots (1...127) to
            # Create the rollback segment for each rollback segment in a round robin fashion
            # The modifications must happen atomically within a single mini-transaction.
            marko Marko Mäkelä made changes -
            thiru Thirunarayanan Balathandayuthapani made changes -
            Description   To change the number of undo tablespaces, we need to reinitialize the data directory (dump + restore).
            InnoDB should allow changing number of undo tablespaces as a part of restart process.
            System tablespace contains InnoDB dictionary tables, doublewrite buffer, change buffer tree
            and undo log pages. By allowing this change, InnoDB can reduce the workload on system tablespace.

            h1. Requirements
            A prior shutdown with {{SET GLOBAL innodb_fast_shutdown=0}} must be executed before adding the undo log tablespaces.

            This is because the undo logs must be empty (no incomplete or XA PREPARE transactions, nothing to be purged) so that the old undo tablespaces can discarded and new ones created.


            h1. Steps

            # Check whether the existing undo log exists. If exists then give the warnings about the slow shutdown and start the server normally.
            # If not exist then free the system tablespace rollback segment header page else throw message about slow shutdown and continue the normal server start operation
            # Free the system tablespace rollback segment header page of the slots 1...127
            # Set the rollback segment slots (1...127) to
            # Create the rollback segment for each rollback segment in a round robin fashion
            # The modifications must happen atomically within a single mini-transaction.             		  To change the number of undo tablespaces, we need to reinitialize the data directory (dump + restore).
            InnoDB should allow changing number of undo tablespaces as a part of restart process.
            System tablespace contains InnoDB dictionary tables, doublewrite buffer, change buffer tree
            and undo log pages. By allowing this change, InnoDB can reduce the workload on system tablespace.

            h1. Requirements
            A prior shutdown with {{SET GLOBAL innodb_fast_shutdown=0}} must be executed before adding the undo log tablespaces.

            This is because the undo logs must be empty (no incomplete or XA PREPARE transactions, nothing to be purged) so that the old undo tablespaces can discarded and new ones created.


            h1. Steps

            # Check whether the existing undo log exists. If exists then give the warnings about the slow shutdown and start the server normally.
            # If not exist then free the system tablespace rollback segment header page else throw message about slow shutdown and continue the normal server start operation
            # Free the system tablespace rollback segment header page of the slots 1...127
            # Reset the TRX_SYS page and reinitialize the system tablespace rollback segment slot(0th slot), doublewrite information,
               update the binlog info and WSREP info in system rollback segment header page (0th slot page)
             Step(3) and (4) should happen within a single mini-transaction
            # Delete the old undo tablespaces if any
            # Make checkpoint to get rid of old undo tablespace redo log records
            # Read the latest MAX_SPACE_ID from dictionary header page
            # Create the new specified undo log tablespace and initialize the page0 of all undo tablespaces
            Step (7) and Step (8) should happen within a single mini-transaction
            # Make the checkpoint again to make sure that next startup or backup reads the undo log tablespaces before
            opening the redo log records
            # Create the rollback segment for each rollback segment in a round robin fashion
            thiru Thirunarayanan Balathandayuthapani made changes -
            Status In Review [ 10002 ] In Testing [ 10301 ]
            thiru Thirunarayanan Balathandayuthapani made changes -
            Assignee Marko Mäkelä [ marko ] Matthias Leich [ mleich ]
            mleich Matthias Leich made changes -
            Assignee Matthias Leich [ mleich ] Thirunarayanan Balathandayuthapani [ thiru ]
            Status In Testing [ 10301 ] Stalled [ 10000 ]
            marko Marko Mäkelä made changes -
            Fix Version/s 10.11 [ 27614 ]
            Fix Version/s 10.6 [ 24028 ]
            Fix Version/s 10.7 [ 24805 ]
            Fix Version/s 10.8 [ 26121 ]
            Fix Version/s 10.9 [ 26905 ]
            Fix Version/s 10.10 [ 27530 ]
            Affects Version/s 10.11 [ 27614 ]
            thiru Thirunarayanan Balathandayuthapani made changes -
            Fix Version/s 10.11.1 [ 28454 ]
            Fix Version/s 10.11 [ 27614 ]
            Resolution Fixed [ 1 ]
            Status Stalled [ 10000 ] Closed [ 6 ]
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            julien.fritsch Julien Fritsch made changes -
            marko Marko Mäkelä made changes -
            ralf.gebhardt Ralf Gebhardt made changes -
            Affects Version/s 10.3 [ 22126 ]
            Affects Version/s 10.4 [ 22408 ]
            Affects Version/s 10.5 [ 23123 ]
            Affects Version/s 10.6 [ 24028 ]
            Affects Version/s 10.7 [ 24805 ]
            Affects Version/s 10.8 [ 26121 ]
            Affects Version/s 10.9 [ 26905 ]
            Affects Version/s 10.10 [ 27530 ]
            Affects Version/s 10.11 [ 27614 ]
            Issue Type Bug [ 1 ] Task [ 3 ]
            marko Marko Mäkelä made changes -
            thiru Thirunarayanan Balathandayuthapani made changes -
            ralf.gebhardt Ralf Gebhardt made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            thiru Thirunarayanan Balathandayuthapani made changes -
            mariadb-jira-automation Jira Automation (IT) made changes -
            Zendesk Related Tickets 131727

            People

              thiru Thirunarayanan Balathandayuthapani
              thiru Thirunarayanan Balathandayuthapani
              Votes:
              3 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.