Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-13039

innodb_fast_shutdown=0 may fail to purge all undo logs

Details

    Description

      When a slow shutdown is performed soon after creating some activity for fts_optimize_thread(), it is possible that the optimize thread is starting new transactions after the purge has finished. This is violating the specification of innodb_fast_shutdown, namely that the purge must be completed. (None of the history of new transactions would be purged.)

      The proper fix would seem to declare a flag that indicates whether undo-log-generating background threads are active:

      /** Shut down background threads that can generate undo log. */
      void
      srv_shutdown_bg_undo_sources()
      {
      	if (srv_bg_undo_sources) {
      		ut_ad(!srv_read_only_mode);
      		fts_optimize_shutdown();
      		dict_stats_shutdown();
      		srv_bg_undo_sources = false;
      	}
      }
      

      As long as this flag is set, srv_purge_should_exit() must return false.

      Attachments

        Issue Links

          Activity

            bb-10.2-marko
            I plan to backport this back to 10.0, along with some MDEV-13015 compatibility tests and fixes.

            marko Marko Mäkelä added a comment - bb-10.2-marko I plan to backport this back to 10.0, along with some MDEV-13015 compatibility tests and fixes.

            patch is ok to push, but these shutdown changes are hard as so many timing things can happen, buildbot has not yet tested this thoroughly, please wait that first.

            jplindst Jan Lindström (Inactive) added a comment - patch is ok to push, but these shutdown changes are hard as so many timing things can happen, buildbot has not yet tested this thoroughly, please wait that first.

            This is also related to MDEV-12091, because in slow shutdown, the purge threads may exit prematurely while the rollback of recovered incomplete transactions is in progress. Thus, the slow shutdown could fail to purge any undo log that would become purgeable after the rollback finishes.

            marko Marko Mäkelä added a comment - This is also related to MDEV-12091 , because in slow shutdown, the purge threads may exit prematurely while the rollback of recovered incomplete transactions is in progress. Thus, the slow shutdown could fail to purge any undo log that would become purgeable after the rollback finishes.

            Revised fix (with test) on bb-10.2-marko

            marko Marko Mäkelä added a comment - Revised fix (with test) on bb-10.2-marko

            Changes look otherwise good expect based on bb that checkpoint writing on startup should be restored.

            jplindst Jan Lindström (Inactive) added a comment - Changes look otherwise good expect based on bb that checkpoint writing on startup should be restored.

            Thanks. I filed a follow-up task: MDEV-13044 Do not create a redo log checkpoint at startup

            marko Marko Mäkelä added a comment - Thanks. I filed a follow-up task: MDEV-13044 Do not create a redo log checkpoint at startup

            In all InnoDB versions, the background DROP TABLE queue (which is blatantly breaking ACID properties) is potentially interfering with the purge at slow shutdown.

            Starting with MariaDB 10.0, the dict_stats_thread() and fts_optimize_thread() could create undo log records that would fail to be purged at slow shutdown.

            marko Marko Mäkelä added a comment - In all InnoDB versions, the background DROP TABLE queue (which is blatantly breaking ACID properties) is potentially interfering with the purge at slow shutdown. Starting with MariaDB 10.0, the dict_stats_thread() and fts_optimize_thread() could create undo log records that would fail to be purged at slow shutdown.

            People

              marko Marko Mäkelä
              marko Marko Mäkelä
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.