Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-38456

Replication using new binlog aborts with ER_GTID_STRICT_OUT_OF_ORDER

    XMLWordPrintable

Details

    • Can result in unexpected behaviour
    • Q1/2026 Server Maintenance

    Description

      knielsen_binlog_in_engine_12.3 bc956664992d2ff0bf6f5b6675d9597c9530d83a

      2025-12-30 20:08:18 160 [ERROR] Slave SQL: An attempt was made to binlog GTID 0-1001-3893 which would create an out-of-order sequence number with existing GTID 0-1001-3894, and gtid strict mo
      de is enabled, Gtid 0-1001-3893, Internal MariaDB error code: 1950
      2025-12-30 20:08:18 160 [Warning] Slave: An attempt was made to binlog GTID 0-1001-3893 which would create an out-of-order sequence number with existing GTID 0-1001-3894, and gtid strict mode
       is enabled Error_code: 1950
      2025-12-30 20:08:18 160 [ERROR] Error running query, slave SQL thread aborted. Fix the problem, and restart the slave SQL thread with "SLAVE START". We stopped at log 'binlog-000118.ibb' posi
      tion 4; GTID position '0-1001-3894'
      

      The test configuration is parallel MS replication with binlog-storage-engine=InnoDB, where the slave threads are periodically stopped with STOP SLAVE and restarted with START SLAVE UNTIL. The master executes a concurrent mix of DML and admin statements in several threads. Both instances are running without interruption, there is no crash recovery on either the master or the slave. The full command line is below. It usually fails for me within 10-20 attempts (the command line runs 20 trials, until it hits the error in question), but it's not very reliable, better to run more before giving up. Failures observed while running on SSD and HDD. I didn't try running in shm and recommend caution while doing so, as it may require quite a lot of disk space.
      I couldn't reproduce the error without binlog-storage-engine, but given that the test is not very reliable, there is no guarantee that it is new-binlog-specific.

      MariaDB/randgen 85500ee499b172c1a158e8e632c6b503a3a7ced4

      perl ./run.pl --base-port=14120 --basedir=/home/binlog/MDEV-38456/knielsen_binlog_in_engine_12.3 --compatibility=999999 --duration=200 --engine=InnoDB --filter=conf/ff/replication.ff --filter=conf/preview/new_binlog.ff --gendata=advanced  --gendata=conf/zz/oltp-aria.zz --gendata=simple --grammar=conf/yy/admin.yy    --grammar=conf/yy/oltp-write.yy --mysqld=--binlog_annotate_row_events=OFF --mysqld=--binlog_cache_size=1048576 --mysqld=--binlog_expire_logs_seconds=432000 --mysqld=--binlog_format=ROW --mysqld=--binlog_stmt_cache_size=131072 --mysqld=--binlog_storage_engine=innodb --mysqld=--character_set_server=utf8mb3 --mysqld=--collation_server=utf8mb3_general_ci   --mysqld=--explicit_defaults_for_timestamp=ON --mysqld=--gtid_strict_mode=ON   --mysqld=--innodb-lock-wait-timeout=5 --mysqld=--innodb_autoinc_lock_mode=2 --mysqld=--innodb_buffer_pool_size=2G --mysqld=--innodb_flush_neighbors=0 --mysqld=--innodb_io_capacity=400 --mysqld=--innodb_io_capacity_max=4000 --mysqld=--innodb_lock_wait_timeout=180 --mysqld=--innodb_log_buffer_size=67108864 --mysqld=--innodb_log_file_size=4294967296 --mysqld=--innodb_max_dirty_pages_pct=50.0 --mysqld=--innodb_open_files=2048 --mysqld=--innodb_print_all_deadlocks=ON --mysqld=--innodb_read_io_threads=8 --mysqld=--innodb_sort_buffer_size=12582912 --mysqld=--innodb_write_io_threads=8  --mysqld=--key_buffer_size=67108864 --mysqld=--lock-wait-timeout=10 --mysqld=--lock_wait_timeout=10800 --mysqld=--log_bin --mysqld=--log_bin_compress=ON --mysqld=--log_bin_trust_function_creators=ON --mysqld=--log_slave_updates=ON --mysqld=--log_slow_verbosity=innodb,query_plan --mysqld=--long_query_time=30 --mysqld=--loose-shutdown_wait_for_slaves=ON  --mysqld=--lower_case_table_names=1 --mysqld=--master_verify_checksum=ON --mysqld=--max-statement-time=20 --mysqld=--max_allowed_packet=67108864 --mysqld=--max_binlog_size=268435456 --mysqld=--max_connect_errors=10000 --mysqld=--max_connections=10000 --mysqld=--max_heap_table_size=33554432 --mysqld=--max_password_errors=5 --mysqld=--max_relay_log_size=268435456 --mysqld=--myisam_recover_options=BACKUP,FORCE --mysqld=--myisam_sort_buffer_size=67108864 --mysqld=--open_files_limit=278528 --mysqld=--optimizer_search_depth=0 --mysqld=--optimizer_switch=mrr=on,mrr_cost_based=on    --mysqld=--plugin-maturity=experimental --mysqld=--plugin_maturity=stable  --mysqld=--relay_log_recovery=ON --mysqld=--replicate_events_marked_for_skip=FILTER_ON_MASTER --mysqld=--slave_net_timeout=10 --mysqld=--slave_parallel_max_queued=67108864 --mysqld=--slave_parallel_threads=4 --mysqld=--slave_parallel_workers=4 --mysqld=--slow_query_log=ON  --mysqld=--sync_binlog=1 --mysqld=--sync_master_info=1  --mysqld=--transaction_isolation=READ-COMMITTED  --mysqld=--wait_timeout=3600 --queries=1000000 --reporter=ReplicationStartUntil --reporters=Backtrace,Deadlock --scenario-use-gtid --scenario=Replication   --server1-mysqld=--log_bin --server1-mysqld=--read_only=0 --threads=4 --vardir=/home/binlog/MDEV-38456/var-rr --seed=1766748636 --trials=20 --output="Slave SQL: An attempt was made to binlog" --rr
      

      The test is rr-able, although I don't know if an rr profile is helpful for failures like this. In any case, it will be provided, as well as full datadirs, logs, etc.

      Attachments

        Issue Links

          Activity

            People

              knielsen Kristian Nielsen
              elenst Elena Stepanova
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.