Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-14892

Porting Rdb_binlog_manager to MariaDB

Details

    Description

      Looking at Rdb_binlog_manager in FB/MySQL.

      Feature 1: store the master position in MyRocks

      It stores the master position in this record:

      key: BINLOG_INFO_INDEX_NUMBER
      value:  
        - binlog_file:position
        - Last GTID (isn't that a function of binlog position?)
      

      Uses for that information:

      • It is visible through INFORMATION_SCHEMA.ROCKSDB_GLOBAL_INFO.

      Re #2, InnoDB in FB/MySQL has similar code in is_binlog_advanced (and MariaDB's InnoDB doesn't have it)

      Feature 2: keep mysql.slave_gtid_info up to date

      When slave-gtid-info=optimized option is used, mysql.slave_gtid_info is kept up-todate directly by the storage engines. (See https://github.com/facebook/mysql-5.6/commit/0f402cb8381ba9d8e71c93085a22e5a12589b6e3)

      (Q: and that option is not used, both the SE and the SQL layer do the updates?)

      Proposed solution

      • Neither of the above features are essential (or storage engine-only)
      • MariaDB has an error when trying to use a part of Rdb_binlog_manager (as it doesn't have binlog position stored). We need to disable it.

      Attachments

        Issue Links

          Activity

            psergei Sergei Petrunia added a comment - - edited

            Take-aways from discussion with Andrei:

            • The server keeps its own binlog and storage engine(s) in sync by doing XA. Slave SQL thread participates in XA, so the slave is in sync with its own binlog.
            • Slave's position in the relay log was traditionally stored in relay_log.info but that is getting phased out in favor of mysql.gtid_slave_pos table. That table uses a transactional engine, which means XA will keep it in sync.
            psergei Sergei Petrunia added a comment - - edited Take-aways from discussion with Andrei: The server keeps its own binlog and storage engine(s) in sync by doing XA. Slave SQL thread participates in XA, so the slave is in sync with its own binlog. Slave's position in the relay log was traditionally stored in relay_log.info but that is getting phased out in favor of mysql.gtid_slave_pos table. That table uses a transactional engine, which means XA will keep it in sync.

            Notes from discussion with Herman:

            • Another reason to keep master gtid and binlog position in the storage engine is online (that is, non-blocking) backup. Suppose somebody takes a snapshot and copies it over to another machine. There, they restore it. How do they know which binlog position to start from? BINLOG_INFO_INDEX_NUMBER gives them that. (Taking a snapshot here is assumed to be done with @@rocksdb_create_checkpoint IIRC).
            psergei Sergei Petrunia added a comment - Notes from discussion with Herman: Another reason to keep master gtid and binlog position in the storage engine is online (that is, non-blocking) backup. Suppose somebody takes a snapshot and copies it over to another machine. There, they restore it. How do they know which binlog position to start from? BINLOG_INFO_INDEX_NUMBER gives them that. (Taking a snapshot here is assumed to be done with @@rocksdb_create_checkpoint IIRC).

            As part of fix for MDEV-14372, removed a possible crash when Rdb_binlog_manager attempts to read BINLOG_INFO_INDEX_NUMBER value which is empty.

            psergei Sergei Petrunia added a comment - As part of fix for MDEV-14372 , removed a possible crash when Rdb_binlog_manager attempts to read BINLOG_INFO_INDEX_NUMBER value which is empty.

            People

              psergei Sergei Petrunia
              psergei Sergei Petrunia
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.