Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-4024

Found Index PRIMARY whose column info does not match that of MySQL

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • 10.0.1, 5.5.30, 5.3.12
    • 10.0.5
    • None
    • None
    • Linux (RHEL6)

    Description

      Updated by Elena
      (I will leave the initial description intact at the end of this field, for the reference.)

      The problem actually happens even inside the same 10.0 version, e.g. on the current 10.0-base, on an attempt to switch from XtraDB to InnoDB; also reproducible on an upgrade from 5.5, as initially described, and apparently on crossgrade from 5.6 too.

      For some data types, if a table was created with XtraDB, an attempt to open it with InnoDB produces the described error. To my understanding, there is a mismatch in types inside InnoDB comparing to MySQL.

      It doesn't however happen on an upgrade to MySQL 5.6.5, so I assume it's a genuine bug, probably a result of 5.6 InnoDB merge.

      I've got the problem for the following data types:
      ENUM
      SET
      YEAR
      YEAR(2)

      The erroneous mapping for all of them is the same, something about FIXVARBINARY vs INT, if I'm not mistaken.

      The provided test case needs to be run with mysqld=--innodb command line option, assuming that XtraDB is built-in (it cannot have include/xtradb.so due to the specifics of the test). InnoDB is supposed to be present as a plugin, as it now happens in 10.0-base.
      The test creates tables with a PK of various data types, then shuts down the server, restarts it with InnoDB plugin, runs mysql_upgrade and then runs SHOW CREATE for each table. As usual, MTR collects errors from the log file at the end. The following are produced as of 10.0-base revno 3467:

      130112  5:18:59 [ERROR] Found index PRIMARY whose column info does not match that of MySQL.
      130112  5:18:59 [ERROR] Build InnoDB index translation table for Table ./test/t_enum failed
      130112  5:18:59 [ERROR] Found index PRIMARY whose column info does not match that of MySQL.
      130112  5:18:59 [ERROR] Build InnoDB index translation table for Table ./test/t_set failed
      130112  5:18:59 [ERROR] Found index PRIMARY whose column info does not match that of MySQL.
      130112  5:18:59 [ERROR] Build InnoDB index translation table for Table ./test/t_year failed
      130112  5:18:59 [ERROR] Found index PRIMARY whose column info does not match that of MySQL.
      130112  5:18:59 [ERROR] Build InnoDB index translation table for Table ./test/t_year2 failed

      Test case

       
      # Run as mysql-test-run.pl --mysqld=--innodb <test name>
      # (assuming that XtraDB is built in, and InnoDB exists as a plugin ha_innodb.so)
       
      CREATE TABLE t_int (f1 INT PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t_smallint (f1 SMALLINT PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t_tinyint (f1 TINYINT PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t_mediumint (f1 MEDIUMINT PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t_bigint (f1 BIGINT PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t_numeric (f1 NUMERIC(6,2) PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t_decimal (f1 DECIMAL(6,2) PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t_float (f1 FLOAT(6,2) PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t_double (f1 DOUBLE(6,2) PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t_bit (f1 BIT(2) PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t_date (f1 DATE PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t_datetime (f1 DATETIME PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t_datetime4 (f1 DATETIME(4) PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t_timestamp (f1 TIMESTAMP PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t_timestamp3 (f1 TIMESTAMP(3) PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t_time (f1 TIME PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t_time2 (f1 TIME(2) PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t_year (f1 YEAR PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t_year2 (f1 YEAR(2) PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t_char (f1 CHAR(3) PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t_varchar (f1 VARCHAR(8) PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t_binary (f1 BINARY(3) PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t_varbinary (f1 VARBINARY(3) PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t_enum (f1 ENUM('a','b') PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t_set (f1 SET('a','b') PRIMARY KEY) ENGINE=InnoDB;
      CREATE TABLE t_text (f1 TEXT, PRIMARY KEY (f1(256))) ENGINE=InnoDB;
      CREATE TABLE t_blob (f1 BLOB, PRIMARY KEY (f1(256))) ENGINE=InnoDB;
       
       
      let $expect_file= $MYSQLTEST_VARDIR/tmp/mysqld.1.expect;
      --error 0,1
      --remove_file $expect_file
      --write_file $expect_file
      wait
      EOF
       
      --echo #
      --echo # Shutdown the server
      --echo #
       
      --shutdown_server 10
      --source include/wait_until_disconnected.inc
       
      --append_file $expect_file
      restart: --ignore-builtin-innodb --plugin-load=ha_innodb.so --innodb 
      EOF
      --enable_reconnect
      --source include/wait_until_connected_again.inc
       
      --echo #
      --echo # Run mysql_upgrade
      --echo #
       
      --exec $MYSQL_UPGRADE -uroot
       
      SHOW CREATE TABLE t_int;
      SHOW CREATE TABLE t_smallint;
      SHOW CREATE TABLE t_tinyint;
      SHOW CREATE TABLE t_mediumint;
      SHOW CREATE TABLE t_bigint;
      SHOW CREATE TABLE t_numeric;
      SHOW CREATE TABLE t_decimal;
      SHOW CREATE TABLE t_float;
      SHOW CREATE TABLE t_double;
      SHOW CREATE TABLE t_bit;
      SHOW CREATE TABLE t_date;
      SHOW CREATE TABLE t_datetime;
      SHOW CREATE TABLE t_datetime4;
      SHOW CREATE TABLE t_timestamp;
      SHOW CREATE TABLE t_timestamp3;
      SHOW CREATE TABLE t_time;
      SHOW CREATE TABLE t_time2;
      SHOW CREATE TABLE t_year;
      SHOW CREATE TABLE t_year2;
      SHOW CREATE TABLE t_char;
      SHOW CREATE TABLE t_varchar;
      SHOW CREATE TABLE t_binary;
      SHOW CREATE TABLE t_varbinary;
      SHOW CREATE TABLE t_enum;
      SHOW CREATE TABLE t_set;
      SHOW CREATE TABLE t_text;
      SHOW CREATE TABLE t_blob;

      ==============================

      Initial description

      After upgrading from MySQL (Percona) 5.5.x to MariaDB-10.x, after a while these errors seem to appear in the logs:

      130111 3:48:39 [ERROR] Found index PRIMARY whose column info does not match that of MySQL.
      130111 3:48:39 [ERROR] Build InnoDB index translation table for Table ./dbname/tablename#P#p201209 failed

      There is no obvious indication that the data is actually corrupted, but primary keys being broken seems like a potentially dangerous issue.

      I have only seen this happen on partitioned tables (but most of my tables are partitioned so I wouldn't want to categorically say it doesn't happen on other tables).

      I have another server containing the exact same data that has been initialized from empty from a mysqldump (the one reporting the error was upgraded in-place), and that one isn't producing the above error in the logs.

      Attachments

        Issue Links

          Activity

            jeremycole Jeremy Cole added a comment -

            We found this bug during upgrade. Please see attached a patch for the bug. However, fixing it is not quite as trivial as restoring the correct behavior. Each table affected must be adjusted back to the correct type/value. This could be done by the user (using ALTER TABLE ... ENGINE=InnoDB) or a patch could be made to automatically adjust this type (without copying any data) on e.g. table open.

            I would find it rather irresponsible to leave this bug in MariaDB. Aside from its disabling the index translation table and generating complaintive log messages, it has a reasonably high chance of causing future more severe breakage if InnoDB changes its behavior and suddenly starts failing to understand ENUM/SET data in DATA_FIXBINARY columns. The fix is in theory quite simple (an InnoDB data dictionary fixup one time per table) – it should be done sooner rather than later.

            The commit message from my patch follows, since it contains a reasonably complete explanation:

            Use correct InnoDB storage type for 1-byte ENUM and SET

            In MariaDB 5.3, the get_innobase_type_from_mysql_type function was
            rewritten in revision 2661.777.35, but this rewrite failed to account
            for the previous special handling of ENUM, SET, and TINYINT UNSIGNED.
            These types have previously been mapped to InnoDB's type DATA_INT with
            flag DATA_UNSIGNED. After this commit they were mapped to DATA_FIXBINARY
            instead. A fix to revert to the old type for TINYINT UNSIGNED was later
            made in revision 2661.777.47, but ENUM and SET were not fixed.

            This InnoDB internal data type difference makes upgrades from older
            versions (using data files without rebuild) incompatible in a subtle
            way. Errors of the form "Found index <name> whose column info does not
            match that of MySQL." will be logged when tables are opened. InnoDB will
            fail to build a per-table index "translation table" (share->idx_trans_tbl)
            which will cause all future index usage to fall back to finding the
            index by walking the index list and comparing index name instead of
            using a fast lookup table.

            Query results should be unaffected by this bug, because the on-disk
            representation of the two types is identical, and all usage by the MySQL
            layer is unaffected.

            This change additionally sorts the get_innobase_type_from_mysql_type
            cases for integer types from smallest-to-largest, and adds an
            explanatory comment about TINYINT UNSIGNED, ENUM, and SET.

            (Note that some errors in revisions 2661.777.35 and 2661.777.47 were fixed
            in later merges, so the diffs in those revisions do not accurately represent
            the final state.)

            See the Bazaar revisions on Launchpad:

            jeremycole Jeremy Cole added a comment - We found this bug during upgrade. Please see attached a patch for the bug. However, fixing it is not quite as trivial as restoring the correct behavior. Each table affected must be adjusted back to the correct type/value. This could be done by the user (using ALTER TABLE ... ENGINE=InnoDB) or a patch could be made to automatically adjust this type (without copying any data) on e.g. table open. I would find it rather irresponsible to leave this bug in MariaDB. Aside from its disabling the index translation table and generating complaintive log messages, it has a reasonably high chance of causing future more severe breakage if InnoDB changes its behavior and suddenly starts failing to understand ENUM/SET data in DATA_FIXBINARY columns. The fix is in theory quite simple (an InnoDB data dictionary fixup one time per table) – it should be done sooner rather than later. The commit message from my patch follows, since it contains a reasonably complete explanation: Use correct InnoDB storage type for 1-byte ENUM and SET In MariaDB 5.3, the get_innobase_type_from_mysql_type function was rewritten in revision 2661.777.35, but this rewrite failed to account for the previous special handling of ENUM, SET, and TINYINT UNSIGNED. These types have previously been mapped to InnoDB's type DATA_INT with flag DATA_UNSIGNED. After this commit they were mapped to DATA_FIXBINARY instead. A fix to revert to the old type for TINYINT UNSIGNED was later made in revision 2661.777.47, but ENUM and SET were not fixed. This InnoDB internal data type difference makes upgrades from older versions (using data files without rebuild) incompatible in a subtle way. Errors of the form "Found index <name> whose column info does not match that of MySQL." will be logged when tables are opened. InnoDB will fail to build a per-table index "translation table" (share->idx_trans_tbl) which will cause all future index usage to fall back to finding the index by walking the index list and comparing index name instead of using a fast lookup table. Query results should be unaffected by this bug, because the on-disk representation of the two types is identical, and all usage by the MySQL layer is unaffected. This change additionally sorts the get_innobase_type_from_mysql_type cases for integer types from smallest-to-largest, and adds an explanatory comment about TINYINT UNSIGNED, ENUM, and SET. (Note that some errors in revisions 2661.777.35 and 2661.777.47 were fixed in later merges, so the diffs in those revisions do not accurately represent the final state.) See the Bazaar revisions on Launchpad: http://bazaar.launchpad.net/~maria-captains/maria/10.0-base/revision/2661.777.35 http://bazaar.launchpad.net/~maria-captains/maria/10.0-base/revision/2661.777.47
            jeremycole Jeremy Cole added a comment -

            Note that the patch contributed here (innodb_1byte_data_int.patch.txt) is given under the New BSD License.

            jeremycole Jeremy Cole added a comment - Note that the patch contributed here (innodb_1byte_data_int.patch.txt) is given under the New BSD License.
            jeremycole Jeremy Cole added a comment -

            What's the fix?

            jeremycole Jeremy Cole added a comment - What's the fix?
            serg Sergei Golubchik added a comment - http://bazaar.launchpad.net/~maria-captains/maria/10.0/revision/3874
            jeremycole Jeremy Cole added a comment -

            That is not really an adequate fix, IMHO. Also, why strip out useful explanatory comments about these special cases that I provided in my patch?

            Additionally, this bug also affects YEAR type which is internally stored as TINYINT UNSIGNED equivalent.

            jeremycole Jeremy Cole added a comment - That is not really an adequate fix, IMHO. Also, why strip out useful explanatory comments about these special cases that I provided in my patch? Additionally, this bug also affects YEAR type which is internally stored as TINYINT UNSIGNED equivalent.

            People

              serg Sergei Golubchik
              gordan Gordan Bobic
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.