Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-29989

binlog_do_db option breaks importing sql dumps

Details

    Description

      If you have binlog enabled and binlog_do_db set to some table and then make a mariadb-dump and import the dump to some database different of the one set in binlog_do_db, then all the Innodb tables in that database have no records.

      I'm attaching a bash script to demonstrate the problem.
      The result from all counts should e 1, but when binlog_do_db is set the result for the innodb table changes.

      That behaviour wasn't observed in 10.5 versions. It appeared after upgrade to 10.8.
      Cannot report for the versions in between, unfortunately.

      Attachments

        Issue Links

          Activity

            serg Sergei Golubchik added a comment - - edited

            Sorry, I wasn't able to repeat the bug with your script. It shows

            +------------+
            | innodb_bug |
            +------------+
            |          1 |
            +------------+
            

            on

            # mysql -e 'select version()'
            +-------------------------------------+
            | version()                           |
            +-------------------------------------+
            | 10.8.6-MariaDB-1:10.8.6+maria~deb10 |
            +-------------------------------------+
            

            The issue will be closed as not reproducible, but don't worry, if you provide more info we'll reopen it

            serg Sergei Golubchik added a comment - - edited Sorry, I wasn't able to repeat the bug with your script. It shows +------------+ | innodb_bug | +------------+ | 1 | +------------+ on # mysql -e 'select version()' +-------------------------------------+ | version() | +-------------------------------------+ | 10.8.6-MariaDB-1:10.8.6+maria~deb10 | +-------------------------------------+ The issue will be closed as not reproducible, but don't worry, if you provide more info we'll reopen it
            lukav Anton Avramov added a comment -

            That is very interesting.
            I've just updated from 10.8.5 to: 10.8.6-MariaDB-1:10.8.6+maria~deb10-log and the bug is still reproducible.

            I'll try to find out what else it depends on.

            lukav Anton Avramov added a comment - That is very interesting. I've just updated from 10.8.5 to: 10.8.6-MariaDB-1:10.8.6+maria~deb10-log and the bug is still reproducible. I'll try to find out what else it depends on.
            lukav Anton Avramov added a comment -

            Got it. bin_log should be enabled. It is disabled by default.
            Can you reopen the bug or should I file a new one?

            lukav Anton Avramov added a comment - Got it. bin_log should be enabled. It is disabled by default. Can you reopen the bug or should I file a new one?
            lukav Anton Avramov added a comment -

            I've updated the script, so it should be reproducible now

            lukav Anton Avramov added a comment - I've updated the script, so it should be reproducible now

            this seems related to innodb bulk insert.

            set unique_checks=0, foreign_key_checks=0;
            create table t1 (c int(11) not null auto_increment primary key) engine=innodb;
            insert into t1 values (1);
            select count(*) as innodb_ok from t1;
            

            this shows the table empty. If the server is started with --log-bin --binlog-do-db=foo
            Removing at least one of the two assignments makes insert to work.

            serg Sergei Golubchik added a comment - this seems related to innodb bulk insert. set unique_checks=0, foreign_key_checks=0; create table t1 (c int (11) not null auto_increment primary key ) engine=innodb; insert into t1 values (1); select count (*) as innodb_ok from t1; this shows the table empty. If the server is started with --log-bin --binlog-do-db=foo Removing at least one of the two assignments makes insert to work.

            Hi. I can confirm this issue still exists in MariaDB, tested on MariaDB 10.11 on AlmaLinux 9.2.

            janmiddelkoop Jan Middelkoop added a comment - Hi. I can confirm this issue still exists in MariaDB, tested on MariaDB 10.11 on AlmaLinux 9.2.

            This bug does not seem to affect MariaDB Server 10.6, which was the first major version to include MDEV-515. This could be related to the follow-up change MDEV-24621.

            marko Marko Mäkelä added a comment - This bug does not seem to affect MariaDB Server 10.6, which was the first major version to include MDEV-515 . This could be related to the follow-up change MDEV-24621 .

            During commit, InnoDB does apply the buffered bulk insert operation in innodb_prepare_commit_versioned().
            But if we add the option like --log-bin and --binlog_do_db=<non-default database> then
            we never call trx_end_id= ha_info->ht()->prepare_commit_versioned(thd, &trx_start_id);.

            #if 1 // FIXME: This should be done in ha_prepare().
              if (rw_trans || (thd->lex->sql_command == SQLCOM_ALTER_TABLE &&
                               thd->lex->alter_info.flags & ALTER_ADD_SYSTEM_VERSIONING &&
                               is_real_trans))
              {
                ulonglong trx_start_id= 0, trx_end_id= 0;
                for (Ha_trx_info *ha_info= trans->ha_list; ha_info; ha_info= ha_info->next())
                {
                  if (ha_info->ht()->prepare_commit_versioned)
                  {
                    trx_end_id= ha_info->ht()->prepare_commit_versioned(thd, &trx_start_id);
             
                    if (trx_end_id == ULONGLONG_MAX)
                    {
                      my_error(ER_ERROR_DURING_COMMIT, MYF(0), 1);
                      goto err;
                    }
             
                    if (trx_end_id)
                      break; // FIXME: use a common ID for cross-engine transactions
                  }
                }
            

            But rw_trans is disabled if we use the above option.

             /* rw_trans is TRUE when we in a transaction changing data */
              bool rw_trans= is_real_trans &&
                             (rw_ha_count > (thd->is_current_stmt_binlog_disabled()?0U:1U));
            

            binlog_do_db disables the binlog for test database. So it sets rw_trans to false.
            rw_trans indicates only binlog writes.

            To verify the claim, ran the versioning suite with

            ./mtr --suite=versioning --mysqld=--log-bin=1 --mysqld=--binlog_do_db=foo
             
            Completed: Failed 28/100 tests, 72.00% were successful.
             
            Failing test(s): versioning.delete_history versioning.select versioning.select2 versioning.update versioning.trx_id versioning.commit_id versioning.rpl versioning.rpl_mix versioning.rpl_row versioning.rpl_stmt
            
            

            I think prepare_commit_versioned should be called irrespective of binlog state. So I added the patch in bb-10.10-MDEV-28699 and bb-10.4-MDEV-28699 too.

            thiru Thirunarayanan Balathandayuthapani added a comment - During commit, InnoDB does apply the buffered bulk insert operation in innodb_prepare_commit_versioned(). But if we add the option like --log-bin and --binlog_do_db=<non-default database> then we never call trx_end_id= ha_info->ht()->prepare_commit_versioned(thd, &trx_start_id); . #if 1 // FIXME: This should be done in ha_prepare(). if (rw_trans || (thd->lex->sql_command == SQLCOM_ALTER_TABLE && thd->lex->alter_info.flags & ALTER_ADD_SYSTEM_VERSIONING && is_real_trans)) { ulonglong trx_start_id= 0, trx_end_id= 0; for (Ha_trx_info *ha_info= trans->ha_list; ha_info; ha_info= ha_info->next()) { if (ha_info->ht()->prepare_commit_versioned) { trx_end_id= ha_info->ht()->prepare_commit_versioned(thd, &trx_start_id);   if (trx_end_id == ULONGLONG_MAX) { my_error(ER_ERROR_DURING_COMMIT, MYF(0), 1); goto err; }   if (trx_end_id) break; // FIXME: use a common ID for cross-engine transactions } } But rw_trans is disabled if we use the above option. /* rw_trans is TRUE when we in a transaction changing data */ bool rw_trans= is_real_trans && (rw_ha_count > (thd->is_current_stmt_binlog_disabled()?0U:1U)); binlog_do_db disables the binlog for test database. So it sets rw_trans to false. rw_trans indicates only binlog writes. To verify the claim, ran the versioning suite with ./mtr --suite=versioning --mysqld=--log-bin=1 --mysqld=--binlog_do_db=foo   Completed: Failed 28/100 tests, 72.00% were successful.   Failing test(s): versioning.delete_history versioning.select versioning.select2 versioning.update versioning.trx_id versioning.commit_id versioning.rpl versioning.rpl_mix versioning.rpl_row versioning.rpl_stmt I think prepare_commit_versioned should be called irrespective of binlog state. So I added the patch in bb-10.10- MDEV-28699 and bb-10.4- MDEV-28699 too.

            People

              thiru Thirunarayanan Balathandayuthapani
              lukav Anton Avramov
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.