Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-27812

Allow innodb_log_file_size to change without server restart

Details

    Description

      Currently, if the parameters of the redo log change, InnoDB in the MariaDB Community Server will rebuild the redo log at server startup.

      MariaDB Enterprise Server 10.5 and 10.6 allow dynamic tuning of the redo log parameters, rebuilding the redo log in a crash-safe manner without restarting the server.

      SET GLOBAL innodb_log_file_size=…;
      

      Before MDEV-14425 changed the redo log file format in 10.8, we were be unable to enable or disable innodb_encrypt_log without server restart, because starting with MDEV-12041 in MariaDB 10.4, the encrypted redo log blocks will have 4 bytes less payload per 512-byte log block.

      Log resizing is tied to checkpoints. We can start writing a second redo log ib_logfile101 with the requested new size, starting from something close to the last written log sequence number. On log checkpoint completion, we can switch files, provided that the checkpoint LSN was not earlier than the start LSN of the resized log file. When resizing to a small log file during a heavy write workload, multiple checkpoints may be necessary.

      While technically it would be possible to rebuild the log for changing innodb_encrypt_log, this task does not implement it, because it would require a non-trivial transformation between the log record streams that are being written to the current log file (ib_logfile0) and the future log file (ib_logfile101 that will replace ib_logfile0).

      Rebuilding the log file will obviously cause disruption to mariadb-backup --backup, because the old log file will stop receiving writes once the server has switched to another log file. This could be addressed in MDEV-14992 by letting the server provide a log record stream directly.

      Attachments

        Issue Links

          Activity

            marko Marko Mäkelä created issue -
            marko Marko Mäkelä made changes -
            Field Original Value New Value
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            marko Marko Mäkelä made changes -
            Assignee Marko Mäkelä [ marko ] Vladislav Vaintroub [ wlad ]
            Status In Progress [ 3 ] In Review [ 10002 ]
            marko Marko Mäkelä made changes -
            Description Currently, if the parameters of the redo log change, InnoDB in the MariaDB Community Server will rebuild the redo log at server startup.

            MariaDB Enterprise Server 10.5 and 10.6 allow dynamic tuning of the redo log parameters, rebuilding the redo log in a crash-safe manner without restarting the server.
            {code:sql}
            SET GLOBAL innodb_log_file_size=…;
            {code}
            Before MDEV-14425 changed the redo log file format in 10.8, we were be unable to enable or disable {{innodb_encrypt_log}} without server restart, because starting with MDEV-12041 in MariaDB 10.4, the encrypted redo log blocks will have 4 bytes less payload per 512-byte log block.

            Log resizing can be tied to log checkpoints. We can start writing a second redo log with the requested new parameters (encryption and size), and on log checkpoint completion, we can switch files.

            Rebuilding the log file will obviously cause disruption to {{mariadb-backup --backup}}, because the old log file will stop receiving writes once the server has switched to another log file. This could be addressed in MDEV-14992 by letting the server provide a log record stream directly.
            Currently, if the parameters of the redo log change, InnoDB in the MariaDB Community Server will rebuild the redo log at server startup.

            MariaDB Enterprise Server 10.5 and 10.6 allow dynamic tuning of the redo log parameters, rebuilding the redo log in a crash-safe manner without restarting the server.
            {code:sql}
            SET GLOBAL innodb_log_file_size=…;
            {code}
            Before MDEV-14425 changed the redo log file format in 10.8, we were be unable to enable or disable {{innodb_encrypt_log}} without server restart, because starting with MDEV-12041 in MariaDB 10.4, the encrypted redo log blocks will have 4 bytes less payload per 512-byte log block.

            Log resizing is tied to checkpoints. We can start writing a second redo log {{ib_logfile101}} with the requested new size, starting from something close to the last written log sequence number. On log checkpoint completion, we can switch files, provided that the checkpoint LSN was not earlier than the start LSN of the resized log file. When resizing to a small log file during a heavy write workload, multiple checkpoints may be necessary.

            While technically it would be possible to rebuild the log for changing {{innodb_encrypt_log}}, this task does not implement it, because it would require a non-trivial transformation between the log record streams that are being written to the current log file ({{ib_logfile0}}) and the future log file ({{ib_logfile101}} that will replace {{ib_logfile0}}).

            Rebuilding the log file will obviously cause disruption to {{mariadb-backup --backup}}, because the old log file will stop receiving writes once the server has switched to another log file. This could be addressed in MDEV-14992 by letting the server provide a log record stream directly.
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            wlad Vladislav Vaintroub made changes -
            Status In Review [ 10002 ] Stalled [ 10000 ]
            wlad Vladislav Vaintroub made changes -
            Assignee Vladislav Vaintroub [ wlad ] Marko Mäkelä [ marko ]
            marko Marko Mäkelä made changes -
            Status Stalled [ 10000 ] In Testing [ 10301 ]
            marko Marko Mäkelä made changes -
            Assignee Marko Mäkelä [ marko ] Matthias Leich [ mleich ]
            marko Marko Mäkelä made changes -
            issue.field.resolutiondate 2022-03-02 15:24:23.0 2022-03-02 15:24:23.949
            marko Marko Mäkelä made changes -
            Fix Version/s 10.9.0 [ 27113 ]
            Fix Version/s 10.9 [ 26905 ]
            Assignee Matthias Leich [ mleich ] Marko Mäkelä [ marko ]
            Resolution Fixed [ 1 ]
            Status In Testing [ 10301 ] Closed [ 6 ]
            serg Sergei Golubchik made changes -
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            marko Marko Mäkelä made changes -
            jplindst Jan Lindström (Inactive) made changes -
            ralf.gebhardt Ralf Gebhardt made changes -
            Labels usability Preview_10.9 usability
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -

            People

              marko Marko Mäkelä
              marko Marko Mäkelä
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.