Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-38638

GTT: SBR and MBR fall back to ROW events for all DML using GTT and base tables + binlog Bloat

    XMLWordPrintable

Details

    • Unexpected results
    • Q1/2026 Server Development

    Description

      In MDEV-36099 it was decided to implement ROW-based fallback in SBR where necessary to do so for regular TT tables.

      In GTT (MDEV-35915) however we see that SBR and MBR fall back to ROW events for all DML where both a GTT and base table is used.

      Original testcase showing a commonplace example:

      --source include/have_innodb.inc
      --source include/have_binlog_format_statement.inc
      --source include/master-slave.inc
       
      --connection master
      CREATE TABLE t1 (c INT) ENGINE=InnoDB;
      CREATE GLOBAL TEMPORARY TABLE gtt1 (c INT) ENGINE=InnoDB ON COMMIT PRESERVE ROWS;
      INSERT INTO gtt1 VALUES (1),(2);
      INSERT INTO t1 SELECT c FROM gtt1;
      SELECT * FROM t1;  # 1
       
      --source include/sync_slave_sql_with_master.inc
      --connection slave
      SELECT * FROM t1;  # 2
       
      --connection master
      SHOW BINLOG EVENTS;
      TRUNCATE TABLE gtt1;
      DROP TABLE t1, gtt1;
      --source include/rpl_end.inc
      

      Leads to:

      MDEV-35915-v11b CS 12.2.0 8934bac9ea05b79f7a5109d9e980cb939a24d4e3 (Debug, Clang 21.1.3-20250923) Build 19/01/2026

      SELECT * FROM t1;
      c
      1
      2
      include/sync_slave_sql_with_master.inc
      connection slave;
      SELECT * FROM t1;
      c
      1
      2
      ...
      master-bin.000001	724	Annotate_rows	1	0	INSERT INTO t1 SELECT c FROM gtt1
      master-bin.000001	780	Table_map	1	0	table_id: 34 (test.t1)
      master-bin.000001	825	Write_rows_v1	1	0	table_id: 34 flags: STMT_END_F
      

      Confirmed in binlog:

      MDEV-35915-v11b CS 12.2.0 8934bac9ea05b79f7a5109d9e980cb939a24d4e3 (Debug, Clang 21.1.3-20250923) Build 19/01/2026

      mariadb-test$ ../bin/mariadb-binlog --base64-output=decode-rows -vvv var/mysqld.1/data/master-bin.00[0-9][0-9][0-9][0-9] | grep '^###'
      ### INSERT INTO `test`.`t1`
      ### SET
      ###   @1=1 /* INT meta=0 nullable=1 is_null=0 */
      ### INSERT INTO `test`.`t1`
      ### SET
      ###   @1=2 /* INT meta=0 nullable=1 is_null=0 */
      

      Changing SBR to MBR results in the same outcome.

      The same testcase, with only a GTT to TT change, does not result in such row based events.

      Furthermore, there are serious concerns with binary log bloat:

      --source include/master-slave.inc
      --source include/have_innodb.inc
      --source include/have_binlog_format_statement.inc
      --source include/have_sequence.inc
       
      CREATE TABLE t  (id INT PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t1 (id INT PRIMARY KEY, data CHAR(200)) ENGINE=InnoDB;
      CREATE TABLE t2 (id INT PRIMARY KEY, data CHAR(200)) ENGINE=InnoDB;
      CREATE GLOBAL TEMPORARY TABLE gtt (id INT PRIMARY KEY) ENGINE=InnoDB ON COMMIT PRESERVE ROWS;
       
      INSERT INTO t  SELECT seq FROM seq_1_to_1000;
      INSERT INTO t1 SELECT seq, REPEAT('x', 200) FROM seq_1_to_1000;
      INSERT INTO t2 SELECT seq, REPEAT('x', 200) FROM seq_1_to_1000;
      INSERT INTO gtt SELECT seq FROM seq_1_to_1000;
       
      FLUSH BINARY LOGS;
       
      --let $start= query_get_value(SHOW MASTER STATUS, Position, 1)
      UPDATE t1 JOIN t ON t1.id = t.id SET t1.data = REPEAT('y', 200);
      --let $end= query_get_value(SHOW MASTER STATUS, Position, 1)
      --let $sbr= `SELECT $end - $start`
       
      FLUSH BINARY LOGS;
       
      --let $start= query_get_value(SHOW MASTER STATUS, Position, 1)
      --disable_warnings
      UPDATE t2 JOIN gtt ON t2.id = gtt.id SET t2.data = REPEAT('z', 200);
      --enable_warnings
      --let $end= query_get_value(SHOW MASTER STATUS, Position, 1)
      --let $row= `SELECT $end - $start`
       
      --let $ratio= `SELECT $row DIV $sbr`
      --echo # Bloat Factor: $ratio x larger
       
      --connection master
      TRUNCATE TABLE gtt;
      DROP TABLE t, t1, t2, gtt;
      --source include/rpl_end.inc
      

      Leads to: Bloat Factor: 1962 x larger. The same testcase with GTT changed to TT does not provide any bloat/increase (factor 1 x). Same result for MBR.

      Attachments

        Issue Links

          Activity

            People

              monty Michael Widenius
              Roel Roel Van de Paar
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.