Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-36907

GTID included in gtid_slave_pos even when transaction is skipped by replicate-do-db

    XMLWordPrintable

Details

    • Bug
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • 10.3.27, 10.6.21
    • None
    • Replication
    • None
    • Can result in data loss

    Description

      Summary:
      GTID is recorded despite transaction being skipped by replicate-do-db, leading to data inconsistency on replica

      Description:
      We have encountered an issue in MariaDB replication where GTIDs from skipped transactions are still recorded in gtid_slave_pos, resulting in data inconsistency. This issue was reproduced in:

      10.3 → 10.3 replication

      10.3 → 10.6 replication

      This suggests that the issue is not specific to the version upgrade, but to the replication filtering mechanism using replicate-do-db.

      Test Procedure
      1. Inserted test data separately into:

      Three shared/common databases: sso, drd, cmn

      A dedicated AS-IS master database (ex: eng)

      2. Configured replication from AS-IS master to TO-BE master, using the following replicate-do-db configuration:

      The intent was to only replicate eng DB changes.

      3. However, after applying this setup, we observed:

      Transactions for sso, drd, cmn were correctly skipped

      But the GTIDs for these skipped transactions were still recorded in gtid_slave_pos

      As a result, the TO-BE master appeared to be fully replicated (based on GTID), but actually lacked the data for the skipped DBs.

      Problem
      replicate-do-db filters transactions after they are received by the replication thread.

      Skipped transactions still increment GTID state, even though the associated data is not applied.

      This results in GTID drift and data loss, since the replica is unaware that it has not applied the data for the skipped databases.

      Expected Behavior
      GTIDs from skipped transactions should not be recorded in gtid_slave_pos.

      This would prevent replicas from incorrectly thinking they are fully synced with the master.

      Replication Configuration
      Below is the actual replication configuration used during testing:

      AS-IS Master Configuration:
      log-slave-updates
      gtid_domain_id = 4
      server_id = 4

      sso.replicate-do-db = 'sso'
      drd.replicate-do-db = 'drd'
      cmn.replicate-do-db = 'cmn'

      TO-BE Master Configuration:
      log-slave-updates
      gtid_domain_id = 4
      server_id = 5

      sso.replicate-do-db = 'sso'
      drd.replicate-do-db = 'drd'
      cmn.replicate-do-db = 'cmn'
      eng.replicate-do-db = 'eng'

      Despite having replicate-do-db filters, the GTIDs for transactions from sso, drd, and cmn were still added to gtid_slave_pos on the TO-BE master, even though the data was not replicated.

      Request
      IIs this behavior expected and intentional?

      If so, how should we handle GTID filtering when using replicate-do-db, to ensure replicas don’t fall into inconsistent states?

      If not expected, can this be treated as a bug?

      We have attached the following for your reference:

      The exact my.cnf configuration files used during the test

      Relevant MariaDB logs captured during our internal testing

      MariaDB logs provided by the customer environment where this issue was first observed

      Thank you for your time and support.

      Attachments

        1. mariadb.err_250523
          41 kB
          baek seung ho
        2. mariadb.err_250527
          55 kB
          baek seung ho
        3. mariadb.err_250528
          27 kB
          baek seung ho
        4. my_as-is_master.cnf
          3 kB
          baek seung ho
        5. my_cmn.cnf
          2 kB
          baek seung ho
        6. my_drd.cnf
          2 kB
          baek seung ho
        7. my_sso.cnf
          2 kB
          baek seung ho
        8. my_to-be_master.cnf
          3 kB
          baek seung ho
        9. test_slave1.txt
          1.90 MB
          baek seung ho
        10. test master.txt
          60 kB
          baek seung ho
        11. test slave2.txt
          503 kB
          baek seung ho

        Activity

          People

            Unassigned Unassigned
            supbaek baek seung ho
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.