Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-15158

On commit, do not write to the TRX_SYS page

Details

    Description

      As noted in a commit message, we should not avoid writing anything to the TRX_SYS page during normal operation.

      We still write some binlog and Galera write-set replication (WSREP) information on each commit to the TRX_SYS page. We do not really have to write this to that page; we can instead use rollback segment header pages (which commit is already modifying anyway).

      To facilitate upgrade from older MySQL or MariaDB versions, we must read the information from the TRX_SYS page. Normally this information would be overridden by information that we find in the rollback segment header pages.

      We choose pick the newest information from the rollback segment header pages. It is probably simplest to assign a 64-bit sequence number for this purpose, say, TRX_RSEG_COMMIT_INFO. On startup, we would only read this number if TRX_RSEG_FORMAT contains 0. When TRX_RSEG_FORMAT was introduced by repurposing TRX_RSEG_MAX_SIZE we would initialize the unused part of the rollback segment header page to 0.

      If TRX_RSEG_FORMAT is 0 and TRX_RSEG_COMMIT_INFO is not 0, we would read the binlog and WSREP metadata into a global variable unless we already saw a bigger TRX_RSEG_COMMIT_INFO in some other rollback segment header page.

      Note: 64 bits should be more than enough, because we can only have at most 1<<47 transactions (DB_TRX_ID is 48 bits, and trx_sys.get_max_trx_id() increments at transaction start and commit (actually twice at commit if the MDEV-12894 mysql.transaction_registry table is being udpated).

      Upgrade will trivially work: we will first read from the TRX_SYS page, and not find anything in the rollback segment header pages.

      Downgrade is trivial too: After MDEV-15132, any new transactions will be blocked, because the rollback segment headers will contain 0 in TRX_RSEG_MAX_SIZE.

      Attachments

        Issue Links

          Activity

            marko Marko Mäkelä created issue -
            marko Marko Mäkelä made changes -
            Field Original Value New Value
            Epic Link MDEV-14442 [ 64369 ]
            marko Marko Mäkelä made changes -
            thiru Thirunarayanan Balathandayuthapani made changes -
            Status Open [ 1 ] In Progress [ 3 ]

            Discussed with marko about where to add rollback segment information. But we need to find where there is a need
            for storing "binlog information". InnoDB have function to print binlog info, nothing more. mariabackup code uses the
            binlog information. There is no test coverage for it to prove it is not garbage. Figuring out how to make test case to
            reach the code path.

            thiru Thirunarayanan Balathandayuthapani added a comment - Discussed with marko about where to add rollback segment information. But we need to find where there is a need for storing "binlog information". InnoDB have function to print binlog info, nothing more. mariabackup code uses the binlog information. There is no test coverage for it to prove it is not garbage. Figuring out how to make test case to reach the code path.

            thiru, thank you! You made a good prototype and also a new test mariabackup.binlog.
            My review actually became a rewrite. I simplified the code and the persistent data structures.
            It turns out that we can do without any magic numbers or new counters. We can simply use TRX_RSEG_MAX_TRX_ID as the counter.
            The semantics is a bit fuzzy when the binlog or Galera write-set replication is disabled ‘on the fly’ so that some rollback segment header pages contain the information while others do not, but I think that it is fine for the generic use case (binlog or WSREP is constantly enabled).

            marko Marko Mäkelä added a comment - thiru , thank you! You made a good prototype and also a new test mariabackup.binlog . My review actually became a rewrite. I simplified the code and the persistent data structures. It turns out that we can do without any magic numbers or new counters. We can simply use TRX_RSEG_MAX_TRX_ID as the counter. The semantics is a bit fuzzy when the binlog or Galera write-set replication is disabled ‘on the fly’ so that some rollback segment header pages contain the information while others do not, but I think that it is fine for the generic use case (binlog or WSREP is constantly enabled).

            My rewrite of Thiru’s work ran fine on buildbot.
            By manual testing I found and fixed a bug on upgrade: it turns out that in trx_purge_add_undo_to_history() must upgrade the TRX_RSEG_FORMAT, or otherwise the information could be ignored on a restart that follows an upgrade.

            marko Marko Mäkelä added a comment - My rewrite of Thiru’s work ran fine on buildbot. By manual testing I found and fixed a bug on upgrade: it turns out that in trx_purge_add_undo_to_history() must upgrade the TRX_RSEG_FORMAT , or otherwise the information could be ignored on a restart that follows an upgrade.
            marko Marko Mäkelä made changes -
            issue.field.resolutiondate 2018-02-20 19:40:38.0 2018-02-20 19:40:38.335
            marko Marko Mäkelä made changes -
            Fix Version/s 10.3.5 [ 22905 ]
            Fix Version/s 10.3 [ 22126 ]
            Assignee Thirunarayanan B [ thiru ] Marko Mäkelä [ marko ]
            Resolution Fixed [ 1 ]
            Status In Progress [ 3 ] Closed [ 6 ]
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 85353 ] MariaDB v4 [ 133462 ]
            marko Marko Mäkelä made changes -

            People

              marko Marko Mäkelä
              marko Marko Mäkelä
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.