Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-15203

innodb.innodb_max_recordsize_64k failed in buildbot with wrong result

Details

    Description

      http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/11476/steps/mtr_nm/logs/stdio

      innodb.innodb_max_recordsize_64k 'xtradb' w3 [ fail ]
              Test ended at 2018-02-03 14:09:55
       
      CURRENT_TEST: innodb.innodb_max_recordsize_64k
      --- /mnt/buildbot/build/mariadb-10.1.31/mysql-test/suite/innodb/r/innodb_max_recordsize_64k.result	2018-02-03 11:41:55.000000000 +0200
      +++ /mnt/buildbot/build/mariadb-10.1.31/mysql-test/suite/innodb/r/innodb_max_recordsize_64k.reject	2018-02-03 14:09:55.000000000 +0200
      @@ -539,10 +539,10 @@
       test.t	analyze	status	OK
       SELECT stat_value FROM mysql.innodb_index_stats where database_name = 'test' and table_name= 't' and stat_name='n_leaf_pages';
       stat_value
      -4
      +2
       SELECT clustered_index_size from mysql.innodb_table_stats where database_name = 'test' and table_name= 't';
       clustered_index_size
      -5
      +3
       DROP TABLE t;
       CREATE TABLE t(col BLOB) ENGINE=InnoDB ROW_FORMAT=REDUNDANT;
       SHOW WARNINGS;
       
      mysqltest: Result content mismatch
      

      Attachments

        Issue Links

          Activity

            elenst, would it be possible to narrow down the test to the relevant part (see below) and repeat it to establish how often it can fail?

            The test result differs immediately after the test creates, populates and analyzes the first table named test.t.
            Because the table creation and population only involves INSERT statements and no UPDATE or DELETE, the result should actually be deterministic, with the purge playing no role whatsoever. There are no secondary indexes, so change buffering should not kick in either.
            This is the corresponding section of the test input:

            CREATE TABLE t(col BLOB) ENGINE=InnoDB ROW_FORMAT=DYNAMIC;
            SHOW WARNINGS;
            INSERT INTO t VALUES (REPEAT('a',16384));
            INSERT INTO t VALUES (REPEAT('a',32768));
            INSERT INTO t VALUES (REPEAT('a',65535));
            SELECT LENGTH(col) FROM t;
            FLUSH TABLE t;
            ANALYZE TABLE t;
            # retrieve the number of leaf pages
            SELECT stat_value FROM mysql.innodb_index_stats where database_name = 'test' and table_name= 't' and stat_name='n_leaf_pages';
            SELECT clustered_index_size from mysql.innodb_table_stats where database_name = 'test' and table_name= 't';
            DROP TABLE t;
            

            Perhaps this intermittent failure to compute correct statistics could explain MDEV-7142?
            Note: while the other statistics can be estimates, the number of pages should always be accurate, from btr_get_size().
            In this table, we expet all 3 records to be stored in the root page, and 1+1+2 BLOB pages being referred to by the records. Somehow there appear to be 2 fewer BLOB pages than expected.

            marko Marko Mäkelä added a comment - elenst , would it be possible to narrow down the test to the relevant part (see below) and repeat it to establish how often it can fail? The test result differs immediately after the test creates, populates and analyzes the first table named test.t . Because the table creation and population only involves INSERT statements and no UPDATE or DELETE , the result should actually be deterministic, with the purge playing no role whatsoever. There are no secondary indexes, so change buffering should not kick in either. This is the corresponding section of the test input: CREATE TABLE t(col BLOB) ENGINE=InnoDB ROW_FORMAT= DYNAMIC ; SHOW WARNINGS; INSERT INTO t VALUES (REPEAT( 'a' ,16384)); INSERT INTO t VALUES (REPEAT( 'a' ,32768)); INSERT INTO t VALUES (REPEAT( 'a' ,65535)); SELECT LENGTH(col) FROM t; FLUSH TABLE t; ANALYZE TABLE t; # retrieve the number of leaf pages SELECT stat_value FROM mysql.innodb_index_stats where database_name = 'test' and table_name= 't' and stat_name= 'n_leaf_pages' ; SELECT clustered_index_size from mysql.innodb_table_stats where database_name = 'test' and table_name= 't' ; DROP TABLE t; Perhaps this intermittent failure to compute correct statistics could explain MDEV-7142 ? Note: while the other statistics can be estimates, the number of pages should always be accurate, from btr_get_size() . In this table, we expet all 3 records to be stored in the root page, and 1+1+2 BLOB pages being referred to by the records. Somehow there appear to be 2 fewer BLOB pages than expected.

            Elena Stepanova, would it be possible to narrow down the test to the relevant part (see below) and repeat it to establish how often it can fail?

            It was a one time occurrence so far, and I ran 1100 repetitions locally without any success.
            For the small test case above, I've so far always gotten the same result:

            SELECT stat_value FROM mysql.innodb_index_stats where database_name = 'test' and table_name= 't' and stat_name='n_leaf_pages';
            stat_value
            3
            SELECT clustered_index_size from mysql.innodb_table_stats where database_name = 'test' and table_name= 't';
            clustered_index_size
            4
            

            elenst Elena Stepanova added a comment - Elena Stepanova, would it be possible to narrow down the test to the relevant part (see below) and repeat it to establish how often it can fail? It was a one time occurrence so far, and I ran 1100 repetitions locally without any success. For the small test case above, I've so far always gotten the same result: SELECT stat_value FROM mysql.innodb_index_stats where database_name = 'test' and table_name= 't' and stat_name= 'n_leaf_pages' ; stat_value 3 SELECT clustered_index_size from mysql.innodb_table_stats where database_name = 'test' and table_name= 't' ; clustered_index_size 4

            I think that this could be explained by the race condition bug MDEV-10682, which was fixed in 2021 in MariaDB Server 10.2.38, …, and not in 10.1 which had already reached its end of life.

            These tests failures were rather rare on buildbot. The latest one where stat_value occurs in the output was this one:

            10.1 c383418cbf754c654a9a705ddb73a107

            innodb.innodb_max_recordsize_64k 'xtradb' w3 [ fail ]
                    Test ended at 2018-02-03 14:09:55
             
            CURRENT_TEST: innodb.innodb_max_recordsize_64k
            --- /mnt/buildbot/build/mariadb-10.1.31/mysql-test/suite/innodb/r/innodb_max_recordsize_64k.result	2018-02-03 11:41:55.000000000 +0200
            +++ /mnt/buildbot/build/mariadb-10.1.31/mysql-test/suite/innodb/r/innodb_max_recordsize_64k.reject	2018-02-03 14:09:55.000000000 +0200
            @@ -539,10 +539,10 @@
             test.t	analyze	status	OK
             SELECT stat_value FROM mysql.innodb_index_stats where database_name = 'test' and table_name= 't' and stat_name='n_leaf_pages';
             stat_value
            -4
            +2
             SELECT clustered_index_size from mysql.innodb_table_stats where database_name = 'test' and table_name= 't';
             clustered_index_size
            -5
            +3
             DROP TABLE t;
             CREATE TABLE t(col BLOB) ENGINE=InnoDB ROW_FORMAT=REDUNDANT;
             SHOW WARNINGS;
            

            marko Marko Mäkelä added a comment - I think that this could be explained by the race condition bug MDEV-10682 , which was fixed in 2021 in MariaDB Server 10.2.38, …, and not in 10.1 which had already reached its end of life. These tests failures were rather rare on buildbot. The latest one where stat_value occurs in the output was this one: 10.1 c383418cbf754c654a9a705ddb73a107 innodb.innodb_max_recordsize_64k 'xtradb' w3 [ fail ] Test ended at 2018-02-03 14:09:55   CURRENT_TEST: innodb.innodb_max_recordsize_64k --- /mnt/buildbot/build/mariadb-10.1.31/mysql-test/suite/innodb/r/innodb_max_recordsize_64k.result 2018-02-03 11:41:55.000000000 +0200 +++ /mnt/buildbot/build/mariadb-10.1.31/mysql-test/suite/innodb/r/innodb_max_recordsize_64k.reject 2018-02-03 14:09:55.000000000 +0200 @@ -539,10 +539,10 @@ test.t analyze status OK SELECT stat_value FROM mysql.innodb_index_stats where database_name = 'test' and table_name= 't' and stat_name='n_leaf_pages'; stat_value -4 +2 SELECT clustered_index_size from mysql.innodb_table_stats where database_name = 'test' and table_name= 't'; clustered_index_size -5 +3 DROP TABLE t; CREATE TABLE t(col BLOB) ENGINE=InnoDB ROW_FORMAT=REDUNDANT; SHOW WARNINGS;

            People

              marko Marko Mäkelä
              elenst Elena Stepanova
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.