Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-13568

gcol.innodb_virtual_debug_purge failed in buildbot with wrong result

Details

    Description

      http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/9134/steps/test_2/logs/stdio

      gcol.innodb_virtual_debug_purge 'innodb' w3 [ fail ]
              Test ended at 2017-08-10 12:36:03
       
      CURRENT_TEST: gcol.innodb_virtual_debug_purge
      --- /mnt/buildbot/build/mariadb-10.2.8/mysql-test/suite/gcol/r/innodb_virtual_debug_purge.result	2017-08-10 09:03:15.000000000 +0300
      +++ /mnt/buildbot/build/mariadb-10.2.8/mysql-test/suite/gcol/r/innodb_virtual_debug_purge.reject	2017-08-10 12:36:03.000000000 +0300
      @@ -149,7 +149,7 @@
       connection default;
       select variable_value>1 from information_schema.global_status where variable_name='innodb_purge_trx_id_age';
       variable_value>1
      -1
      +0
       disconnect con1;
       start transaction with consistent snapshot;
       commit;
       
      mysqltest: Result content mismatch
      

      Attachments

        Issue Links

          Activity

            alice Alice Sherepa added a comment - the same in 10.3 http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/9336/steps/test_3/logs/stdio

            I debugged this a little.
            It seems to me that the conditions should be variable_value>0 instead of variable_value>1.

            But there is more to it. I tried the following experimental change to the test:

            diff --git a/mysql-test/suite/gcol/t/innodb_virtual_debug_purge.test b/mysql-test/suite/gcol/t/innodb_virtual_debug_purge.test
            index 2668e26c976..e6f8c9bb962 100644
            --- a/mysql-test/suite/gcol/t/innodb_virtual_debug_purge.test
            +++ b/mysql-test/suite/gcol/t/innodb_virtual_debug_purge.test
            @@ -186,12 +186,13 @@ connect(con1, localhost, root);
             lock table t write;
             disconnect prevent_purge;
             connection default;
            -select variable_value>1 from information_schema.global_status where variable_name='innodb_purge_trx_id_age';
            +--source ../../innodb/include/wait_all_purged.inc
            +select variable_value>0 from information_schema.global_status where variable_name='innodb_purge_trx_id_age';
             disconnect con1;
             start transaction with consistent snapshot;
             commit;
             --source ../../innodb/include/wait_all_purged.inc
            -select variable_value>1 from information_schema.global_status where variable_name='innodb_purge_trx_id_age';
            +select variable_value>0 from information_schema.global_status where variable_name='innodb_purge_trx_id_age';
             set global debug_dbug=@old_dbug;
             drop table t;
             
            

            I was expecting the wait_all_purged.inc to time out, to report that 1 transaction was not purged. But instead, it reported that everything was purged:

            InnoDB		0 transactions not purged
            

            The intention of the test seems to be to demonstrate that the LOCK TABLES is preventing purge from evaluating the INDEX(c(100)):

            create table t (a blob, b blob, c blob as (concat(a,b)), h varchar(10), index (c(100)));
            insert t(a,b,h) values (repeat('g', 16000), repeat('x', 16000), "kk");
            insert t(a,b,h) values (repeat('a', 16000), repeat('b', 16000), "mm");
            set global debug_dbug="+d,ib_purge_virtual_index_callback";
            connect(prevent_purge, localhost, root);
            start transaction with consistent snapshot;
            connection default;
            update t set a = repeat('m', 16000) where a like "aaa%";
            connect(con1, localhost, root);
            lock table t write;
            disconnect prevent_purge;
            connection default;
            select variable_value>1 from information_schema…
            

            (The connection prevent_purge was introduced in an earlier cleanup by me.)
            If I move the disconnect prevent_purge after the condition, then the condition will always return the same result (that the purge was not run).
            So, it seems that the test is demonstrating that the LOCK TABLE is not having the effect that it might have in MySQL 5.7.

            For what it is worth, I do not see an obvious problem if purge is ignoring the LOCK TABLE. It would only be consistent with the existing behaviour that purge is only removing history that cannot be observed, and by definition, cannot be locked by active transactions.

            marko Marko Mäkelä added a comment - I debugged this a little. It seems to me that the conditions should be variable_value>0 instead of variable_value>1. But there is more to it. I tried the following experimental change to the test: diff --git a/mysql-test/suite/gcol/t/innodb_virtual_debug_purge.test b/mysql-test/suite/gcol/t/innodb_virtual_debug_purge.test index 2668e26c976..e6f8c9bb962 100644 --- a/mysql-test/suite/gcol/t/innodb_virtual_debug_purge.test +++ b/mysql-test/suite/gcol/t/innodb_virtual_debug_purge.test @@ -186,12 +186,13 @@ connect(con1, localhost, root); lock table t write; disconnect prevent_purge; connection default; -select variable_value>1 from information_schema.global_status where variable_name='innodb_purge_trx_id_age'; +--source ../../innodb/include/wait_all_purged.inc +select variable_value>0 from information_schema.global_status where variable_name='innodb_purge_trx_id_age'; disconnect con1; start transaction with consistent snapshot; commit; --source ../../innodb/include/wait_all_purged.inc -select variable_value>1 from information_schema.global_status where variable_name='innodb_purge_trx_id_age'; +select variable_value>0 from information_schema.global_status where variable_name='innodb_purge_trx_id_age'; set global debug_dbug=@old_dbug; drop table t; I was expecting the wait_all_purged.inc to time out, to report that 1 transaction was not purged. But instead, it reported that everything was purged: InnoDB 0 transactions not purged The intention of the test seems to be to demonstrate that the LOCK TABLES is preventing purge from evaluating the INDEX(c(100)): create table t (a blob, b blob, c blob as (concat(a,b)), h varchar (10), index (c(100))); insert t(a,b,h) values (repeat( 'g' , 16000), repeat( 'x' , 16000), "kk" ); insert t(a,b,h) values (repeat( 'a' , 16000), repeat( 'b' , 16000), "mm" ); set global debug_dbug= "+d,ib_purge_virtual_index_callback" ; connect (prevent_purge, localhost, root); start transaction with consistent snapshot; connection default ; update t set a = repeat( 'm' , 16000) where a like "aaa%" ; connect (con1, localhost, root); lock table t write; disconnect prevent_purge; connection default ; select variable_value>1 from information_schema… (The connection prevent_purge was introduced in an earlier cleanup by me.) If I move the disconnect prevent_purge after the condition, then the condition will always return the same result (that the purge was not run). So, it seems that the test is demonstrating that the LOCK TABLE is not having the effect that it might have in MySQL 5.7. For what it is worth, I do not see an obvious problem if purge is ignoring the LOCK TABLE. It would only be consistent with the existing behaviour that purge is only removing history that cannot be observed, and by definition, cannot be locked by active transactions.

            serg, if you believe that the LOCK TABLE can safely be disregarded by the InnoDB purge, then we could close this by patching the test as shown in my previous comment.

            marko Marko Mäkelä added a comment - serg , if you believe that the LOCK TABLE can safely be disregarded by the InnoDB purge, then we could close this by patching the test as shown in my previous comment.

            serg, the only locking guarantee that InnoDB purge needs is that the table definition cannot be moved or removed (by DROP TABLE, RENAME TABLE, ALTER TABLE or similar) while the single undo log record is being processed. If we can guarantee this, I do not see a reason to have LOCK TABLES block the purge. Purge normally disregards any InnoDB transactional locks. For exporting tables, there is a special command (FLUSH TABLES FOR EXPORT) that guarantees that purge will be inactive.

            marko Marko Mäkelä added a comment - serg , the only locking guarantee that InnoDB purge needs is that the table definition cannot be moved or removed (by DROP TABLE, RENAME TABLE, ALTER TABLE or similar) while the single undo log record is being processed. If we can guarantee this, I do not see a reason to have LOCK TABLES block the purge. Purge normally disregards any InnoDB transactional locks. For exporting tables, there is a special command (FLUSH TABLES FOR EXPORT) that guarantees that purge will be inactive.

            serg, I cannot think of any valid purpose that the

            select variable_value>0 from information_schema.global_status where variable_name='innodb_purge_trx_id_age';
            

            might be serving. Adding the wait_all_purged.inc should be both necessary and sufficient.

            If we expect foo>0 to return 0, we might as well remove the >0 so that in case of failure, we will get a more accurate failure result (some actual number instead of the comparison result 1).

            marko Marko Mäkelä added a comment - serg , I cannot think of any valid purpose that the select variable_value>0 from information_schema.global_status where variable_name= 'innodb_purge_trx_id_age' ; might be serving. Adding the wait_all_purged.inc should be both necessary and sufficient. If we expect foo>0 to return 0 , we might as well remove the >0 so that in case of failure, we will get a more accurate failure result (some actual number instead of the comparison result 1 ).

            I revised the test. The MariaDB behaviour is more consistent than the one in MySQL 5.7:
            purge will not yield to any locks, not even to LOCK TABLES in this special case when indexed virtual columns exist.

            marko Marko Mäkelä added a comment - I revised the test. The MariaDB behaviour is more consistent than the one in MySQL 5.7: purge will not yield to any locks, not even to LOCK TABLES in this special case when indexed virtual columns exist.

            People

              marko Marko Mäkelä
              elenst Elena Stepanova
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.