[MDEV-29989] binlog_do_db option breaks importing sql dumps Created: 2022-11-09  Updated: 2023-12-11  Resolved: 2023-09-26

Status: Closed
Project: MariaDB Server
Component/s: Server
Affects Version/s: 10.4, 10.5, 10.6, 10.9, 10.10, 10.11
Fix Version/s: 10.4.32, 10.5.23, 10.6.16, 10.10.7, 10.11.6, 11.0.4, 11.1.3

Type: Bug Priority: Critical
Reporter: Anton Avramov Assignee: Thirunarayanan Balathandayuthapani
Resolution: Fixed Votes: 1
Labels: regression
Environment:

Debian GNU/Linux 10 (buster)


Attachments: File binlog-do-db-bug.sh    
Issue Links:
Duplicate
is duplicated by MDEV-30834 restoring a dump created by mysqldump... Closed
is duplicated by MDEV-32442 Tables empty after mysqldump restore ... Closed
Relates
relates to MDEV-24621 In bulk insert, pre-sort and build in... Closed

 Description   

If you have binlog enabled and binlog_do_db set to some table and then make a mariadb-dump and import the dump to some database different of the one set in binlog_do_db, then all the Innodb tables in that database have no records.

I'm attaching a bash script to demonstrate the problem.
The result from all counts should e 1, but when binlog_do_db is set the result for the innodb table changes.

That behaviour wasn't observed in 10.5 versions. It appeared after upgrade to 10.8.
Cannot report for the versions in between, unfortunately.



 Comments   
Comment by Sergei Golubchik [ 2022-11-18 ]

Sorry, I wasn't able to repeat the bug with your script. It shows

+------------+
| innodb_bug |
+------------+
|          1 |
+------------+

on

# mysql -e 'select version()'
+-------------------------------------+
| version()                           |
+-------------------------------------+
| 10.8.6-MariaDB-1:10.8.6+maria~deb10 |
+-------------------------------------+

The issue will be closed as not reproducible, but don't worry, if you provide more info we'll reopen it

Comment by Anton Avramov [ 2022-11-18 ]

That is very interesting.
I've just updated from 10.8.5 to: 10.8.6-MariaDB-1:10.8.6+maria~deb10-log and the bug is still reproducible.

I'll try to find out what else it depends on.

Comment by Anton Avramov [ 2022-11-18 ]

Got it. bin_log should be enabled. It is disabled by default.
Can you reopen the bug or should I file a new one?

Comment by Anton Avramov [ 2022-11-18 ]

I've updated the script, so it should be reproducible now

Comment by Sergei Golubchik [ 2022-11-18 ]

this seems related to innodb bulk insert.

set unique_checks=0, foreign_key_checks=0;
create table t1 (c int(11) not null auto_increment primary key) engine=innodb;
insert into t1 values (1);
select count(*) as innodb_ok from t1;

this shows the table empty. If the server is started with --log-bin --binlog-do-db=foo
Removing at least one of the two assignments makes insert to work.

Comment by Jan Middelkoop [ 2023-09-06 ]

Hi. I can confirm this issue still exists in MariaDB, tested on MariaDB 10.11 on AlmaLinux 9.2.

Comment by Marko Mäkelä [ 2023-09-11 ]

This bug does not seem to affect MariaDB Server 10.6, which was the first major version to include MDEV-515. This could be related to the follow-up change MDEV-24621.

Comment by Thirunarayanan Balathandayuthapani [ 2023-09-20 ]

During commit, InnoDB does apply the buffered bulk insert operation in innodb_prepare_commit_versioned().
But if we add the option like --log-bin and --binlog_do_db=<non-default database> then
we never call trx_end_id= ha_info->ht()->prepare_commit_versioned(thd, &trx_start_id);.

#if 1 // FIXME: This should be done in ha_prepare().
  if (rw_trans || (thd->lex->sql_command == SQLCOM_ALTER_TABLE &&
                   thd->lex->alter_info.flags & ALTER_ADD_SYSTEM_VERSIONING &&
                   is_real_trans))
  {
    ulonglong trx_start_id= 0, trx_end_id= 0;
    for (Ha_trx_info *ha_info= trans->ha_list; ha_info; ha_info= ha_info->next())
    {
      if (ha_info->ht()->prepare_commit_versioned)
      {
        trx_end_id= ha_info->ht()->prepare_commit_versioned(thd, &trx_start_id);
 
        if (trx_end_id == ULONGLONG_MAX)
        {
          my_error(ER_ERROR_DURING_COMMIT, MYF(0), 1);
          goto err;
        }
 
        if (trx_end_id)
          break; // FIXME: use a common ID for cross-engine transactions
      }
    }

But rw_trans is disabled if we use the above option.

 /* rw_trans is TRUE when we in a transaction changing data */
  bool rw_trans= is_real_trans &&
                 (rw_ha_count > (thd->is_current_stmt_binlog_disabled()?0U:1U));

binlog_do_db disables the binlog for test database. So it sets rw_trans to false.
rw_trans indicates only binlog writes.

To verify the claim, ran the versioning suite with

./mtr --suite=versioning --mysqld=--log-bin=1 --mysqld=--binlog_do_db=foo
 
Completed: Failed 28/100 tests, 72.00% were successful.
 
Failing test(s): versioning.delete_history versioning.select versioning.select2 versioning.update versioning.trx_id versioning.commit_id versioning.rpl versioning.rpl_mix versioning.rpl_row versioning.rpl_stmt

I think prepare_commit_versioned should be called irrespective of binlog state. So I added the patch in bb-10.10-MDEV-28699 and bb-10.4-MDEV-28699 too.

Generated at Thu Feb 08 10:12:48 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.