Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-41232

ROW replication silently advances past a missing after-image column despite ERROR_IF_MISSING_FIELD

    XMLWordPrintable

Details

    • Bug
    • Status: Open (View Workflow)
    • Critical
    • Resolution: Unresolved
    • 12.3.3
    • None
    • Replication
    • Can result in hang or crash

    Description

      With `binlog_row_metadata=FULL` and
      `slave_type_conversions=ERROR_IF_MISSING_FIELD`, the replica correctly rejects
      a FULL row event that updates a source-only column. The semantically equivalent
      MINIMAL row event is silently accepted: the SQL thread advances to the end of
      the event with no error, but the target row is unchanged because the updated
      column does not exist there.

      This makes the result depend on row-image shape and permits silent source/
      replica divergence under a mode that explicitly requests an error for missing
      fields.

      Steps to reproduce

      #!/usr/bin/env bash
      set -euo pipefail
      IMAGE='mariadb@sha256:dd9b303aed4f4890ed09f766d8ca9ddfd176c0c6f6267feff53b3192ec65a979'
      OWNER="mdev-mcf206-$$"
      S="mdev-mcf206-source-$$"; R="mdev-mcf206-replica-$$"; NAMES=("$S" "$R")
      cleanup() {
        for C in "${NAMES[@]}"; do
          if docker inspect --format '{{ index .Config.Labels "io.encryptiondbfuzz.owner" }}' \
               "$C" 2>/dev/null | grep -Fqx "$OWNER"; then
            docker rm -f "$C" >/dev/null 2>&1 || true
          fi
        done
      }
      trap cleanup EXIT
      run_server() {
        NAME=$1; NETWORK=$2; PORT=$3; ID=$4; shift 4
        docker run -d --name "$NAME" --label "io.encryptiondbfuzz.owner=$OWNER" \
          --network "$NETWORK" --read-only --cap-drop ALL \
          --security-opt no-new-privileges=true --pids-limit 180 --memory 900m \
          --cpus 1 --user 999:999 \
          --tmpfs /var/lib/mysql:rw,nosuid,nodev,size=400m,uid=999,gid=999 \
          --tmpfs /run/mysqld:rw,nosuid,nodev,size=16m,uid=999,gid=999 \
          --tmpfs /tmp:rw,nosuid,nodev,size=64m,uid=999,gid=999 \
          --env MARIADB_ALLOW_EMPTY_ROOT_PASSWORD=1 "$IMAGE" \
          --port="$PORT" --bind-address=127.0.0.1 --skip-name-resolve \
          --server-id="$ID" "$@" >/dev/null
      }
      sql() { docker exec --user 999:999 "$1" mariadb --protocol=socket --socket=/run/mysqld/mysqld.sock -uroot --batch --skip-column-names -e "$2"; }
      ready() { for I in $(seq 1 90); do sql "$1" "SELECT @@port" 2>/dev/null | grep -qx "$2" && return 0; sleep 1; done; return 1; }
       
      run_server "$S" none 3307 11 --log-bin=mariadb-bin --binlog-format=ROW --binlog-row-metadata=FULL
      ready "$S" 3307
      run_server "$R" "container:$S" 3306 22 --relay-log=relay-bin
      ready "$R" 3306
      sql "$S" "CREATE USER repl@'127.0.0.1' IDENTIFIED BY 'replpass'; GRANT REPLICATION SLAVE ON *.* TO repl@'127.0.0.1'; CREATE DATABASE edbf; CREATE TABLE edbf.t(id INT PRIMARY KEY,missing_col INT,kept_col INT) ENGINE=InnoDB; INSERT INTO edbf.t VALUES(1,10,100)"
      sql "$R" "CREATE DATABASE edbf; CREATE TABLE edbf.t(id INT PRIMARY KEY,kept_col INT) ENGINE=InnoDB; INSERT INTO edbf.t VALUES(1,100); SET GLOBAL slave_type_conversions='ERROR_IF_MISSING_FIELD'"
      read FILE POS <<EOF
      $(sql "$S" 'SHOW MASTER STATUS' | awk '{print $1, $2}')
      EOF
      sql "$R" "CHANGE MASTER TO MASTER_HOST='127.0.0.1',MASTER_PORT=3307,MASTER_USER='repl',MASTER_PASSWORD='replpass',MASTER_LOG_FILE='$FILE',MASTER_LOG_POS=$POS,MASTER_CONNECT_RETRY=1; START SLAVE"
      for I in $(seq 1 60); do sql "$R" 'SHOW SLAVE STATUS' | awk -F '\t' '{print $12 "/" $13}' | grep -qx 'Yes/Yes' && break; sleep 1; done
       
      sql "$S" "SET SESSION binlog_row_image='MINIMAL'; UPDATE edbf.t SET missing_col=11 WHERE id=1"
      MIN_POS=$(sql "$S" 'SHOW MASTER STATUS' | awk '{print $2}')
      for I in $(seq 1 60); do
        EXEC=$(sql "$R" 'SHOW SLAVE STATUS' | awk -F '\t' '{print $22}')
        [ "$EXEC" -ge "$MIN_POS" ] && break
        sleep 0.25
      done
      docker exec --user 999:999 "$R" mariadb --protocol=socket \
        --socket=/run/mysqld/mysqld.sock -uroot -e 'SHOW SLAVE STATUS\G' | \
        grep -E 'Slave_IO_Running:|Slave_SQL_Running:|Last_SQL_Errno:|Exec_Master_Log_Pos:'
      sql "$R" "SELECT CONCAT('minimal_row=',id,',',kept_col) FROM edbf.t"
       
      sql "$S" "SET SESSION binlog_row_image='FULL'; UPDATE edbf.t SET missing_col=12 WHERE id=1"
      for I in $(seq 1 60); do
        STOPPED=$(sql "$R" 'SHOW SLAVE STATUS' | awk -F '\t' '{print $13 ":" $36}')
        [ "$STOPPED" = 'No:4254' ] && break
        sleep 0.25
      done
      docker exec --user 999:999 "$R" mariadb --protocol=socket \
        --socket=/run/mysqld/mysqld.sock -uroot -e 'SHOW SLAVE STATUS\G' | \
        grep -E 'Slave_IO_Running:|Slave_SQL_Running:|Last_SQL_Errno:|Last_SQL_Error:|Exec_Master_Log_Pos:'
      sql "$R" "SELECT CONCAT('full_row=',id,',',kept_col) FROM edbf.t"
      

      Actual result

      The significant result was:

      minimal_status=Yes/Yes, errno=0, exec=1501
      minimal_row=1,100
      full_status=Yes/No, errno=4254, error=Table structure for binlog event is not compatible with the table definition on this slave: Column 'missing_col' missing from table 'edbf.t'
      full_row=1,100
      

      The MINIMAL transaction was counted as executed and the SQL thread remained
      healthy, although its only update was not represented on the replica. The FULL
      control stopped with the expected error. Neither server crashed.

      Expected result

      `ERROR_IF_MISSING_FIELD` should reject every row event that uses a source
      column absent from the replica, including columns present only in an UPDATE
      after-image. The SQL thread must not advance silently past the transaction.

      Attachments

        Activity

          People

            Deepthi ES Deepthi Eranti Sreenivas
            csfuzz csfuzz
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.