Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-31086

MODIFY COLUMN can break FK constraints, and lead to unrestorable dumps

Details

    Description

      When creating a foreign key constraint the referenced columns on both sides need to have the same data types, including same character set. E.g.

      CREATE TABLE t1(
        id SERIAL,
         msg VARCHAR(100) CHARACTER SET utf8mb3, 
        KEY(msg)
      );
       
      CREATE TABLE t2(
        id SERIAL,
        msg varchar(100) CHARACTER SET utf8mb4,
        CONSTRAINT fk_t1 FOREIGN KEY (msg) REFERENCES t1 (msg)
      );
      

      will fail with "Foreign key constraint is incorrectly formed" due to the mix of utf8mb3 and utf8mb4. Even with foreign_key_checks=OFF this will still not work.

      Changing the character set to "utf8mb3" I can create the 2nd table now, as expected:

      CREATE TABLE t2(
        id SERIAL,
        msg varchar(100) CHARACTER SET utf8mb3,
        CONSTRAINT fk_t1 FOREIGN KEY (msg) REFERENCES t1 (msg)
      );
      

      Now the problem is that starting with 10.6 I can change the character set "under the hood" without getting an error at all, regardless of foreign_key_checks being ON or OFF:

      ALTER TABLE t2
        MODIFY COLUMN msg VARCHAR(100) CHARACTER SET utf8mb4;
      

      With 10.5 and earlier the situation is a bit better, the MODIFY COLUMN will lead to

      Cannot change column 'msg': used in a foreign key constraint 'fk_t1'

      when foreign_key_checks are ON, but will succeed when checks are OFF; unlike the CREATE TABLE above that will fail regardless of checks being ON or OFF.

      And with that we now have a table with a broken foreign key constraint, and a schema that we can dump with mysqldump, but not restore as the dump now contains a CREATE TABLE statement with a FK CONSTRAINT that will lead to an

      Foreign key constraint is incorrectly formed

      error even though mysqldump takes care to turn off foreign_key_checks for the duration of a restore.

      So two things need to be fixed here IMHO:

      • on 10.6 or later: MODIFY TABLE definitely needs to fail if foreign_key_checks are ON
      • on all versions: with CREATE TABLE not allowing to create a FK constraint with column type mismatches regardless of foreign_key_checks, MODIFY COLUMN should not allow for this even with checks turned OFF either; or both should allow this when checks are OFF, for symmetry reasons

      Attachments

        Issue Links

          Activity

            hholzgra Hartmut Holzgraefe created issue -
            hholzgra Hartmut Holzgraefe made changes -
            Field Original Value New Value
            Affects Version/s 10.10.3 [ 28521 ]
            Affects Version/s 10.9.5 [ 28519 ]
            Affects Version/s 10.8.7 [ 28517 ]
            Affects Version/s 10.11.1 [ 28454 ]
            Affects Version/s 10.7 [ 24805 ]
            julien.fritsch Julien Fritsch made changes -
            Fix Version/s 10.4 [ 22408 ]
            Fix Version/s 10.5 [ 23123 ]
            Fix Version/s 10.6 [ 24028 ]
            hholzgra Hartmut Holzgraefe made changes -
            Description When creating a foreign key constraint the referenced columns on both sides need to have the same data types, including same character set. E.g.

            {code}
            CREATE TABLE t1(
              id SERIAL,
               msg VARCHAR(100) CHARACTER SET utf8mb3,
              KEY(msg)
            );

            CREATE TABLE t2(
              id SERIAL,
              msg varchar(100) CHARACTER SET utf8mb4,
              CONSTRAINT fk_t1 FOREIGN KEY (msg) REFERENCES t1 (msg)
            );
            {code}

            will fail with "Foreign key constraint is incorrectly formed" due to the mix of utf8mb3 and utf8mb4. Even with {{foreign_key_checks=OFF}} this will still not work.


            Changing the character set to "utf8mb3" I can create the 2nd table now, as expected:

            {code}
            CREATE TABLE t2(
              id SERIAL,
              msg varchar(100) CHARACTER SET utf8mb3,
              CONSTRAINT fk_t1 FOREIGN KEY (msg) REFERENCES t1 (msg)
            );
            {code}

            Now the problem is that I can change the character set "under the hood" without getting an error at all:

            {code}
            ALTER TABLE t2
              MODIFY COLUMN msg VARCHAR(100) CHARACTER SET utf8mb4;
            {code}

            And with that we now have a table with a broken foreign key constraint, and a schema that we can dump with {{mysqldump}}, but not restore as the dump now contains a CREATE TABLE statement with a FK CONSTRAINT that will lead to an

             "Foreign key constraint is incorrectly formed"

            error even though mysqldump takes care to turn off {{foreign_key_checks}} for the duration of a restore.

            So either the "same data type" check needs to turned off with {{foreign_key_checks}} like the rest of the FK checks, or the same check must be applied on ALTER TABLE...MODIFY COLUMN and similar operations, too.
            When creating a foreign key constraint the referenced columns on both sides need to have the same data types, including same character set. E.g.

            {code}
            CREATE TABLE t1(
              id SERIAL,
               msg VARCHAR(100) CHARACTER SET utf8mb3,
              KEY(msg)
            );

            CREATE TABLE t2(
              id SERIAL,
              msg varchar(100) CHARACTER SET utf8mb4,
              CONSTRAINT fk_t1 FOREIGN KEY (msg) REFERENCES t1 (msg)
            );
            {code}

            will fail with "Foreign key constraint is incorrectly formed" due to the mix of utf8mb3 and utf8mb4. Even with {{foreign_key_checks=OFF}} this will still not work.


            Changing the character set to "utf8mb3" I can create the 2nd table now, as expected:

            {code}
            CREATE TABLE t2(
              id SERIAL,
              msg varchar(100) CHARACTER SET utf8mb3,
              CONSTRAINT fk_t1 FOREIGN KEY (msg) REFERENCES t1 (msg)
            );
            {code}

            Now the problem is that starting with 10.6 I can change the character set "under the hood" without getting an error at all, regardless of {{foreign_key_checks}} being ON or OFF:

            {code}
            ALTER TABLE t2
              MODIFY COLUMN msg VARCHAR(100) CHARACTER SET utf8mb4;
            {code}

            With 10.5 and earlier the situation is a bit better, the {{MODIFY COLUMN}} will lead to

            {{Cannot change column 'msg': used in a foreign key constraint 'fk_t1'}}

            when {{foreign_key_checks}} are ON, but will succeed when checks are OFF; unlike the CREATE TABLE above that will fail regardless of checks being ON or OFF.

            And with that we now have a table with a broken foreign key constraint, and a schema that we can dump with {{mysqldump}}, but not restore as the dump now contains a CREATE TABLE statement with a FK CONSTRAINT that will lead to an

            {{Foreign key constraint is incorrectly formed}}

            error even though mysqldump takes care to turn off {{foreign_key_checks}} for the duration of a restore.

            So two things need to be fixed here IMHO:

            * on 10.6 or later: MODIFY TABLE definitely needs to fail if {{foreign_key_checks}} are ON

            * on all versions: with CREATE TABLE not allowing to create a FK constraint with column type mismatches regardless of {{foreign_key_checks}}, MODIFY COLUMN should not allow for this even with checks turned OFF either; or both should allow this when checks are OFF, for symmetry reasons
            julien.fritsch Julien Fritsch made changes -
            Labels foreign-keys
            serg Sergei Golubchik made changes -
            Assignee Marko Mäkelä [ marko ]
            marko Marko Mäkelä made changes -
            Assignee Marko Mäkelä [ marko ] Thirunarayanan Balathandayuthapani [ thiru ]
            marko Marko Mäkelä made changes -
            Status Open [ 1 ] Confirmed [ 10101 ]
            thiru Thirunarayanan Balathandayuthapani made changes -
            Status Confirmed [ 10101 ] In Progress [ 3 ]
            thiru Thirunarayanan Balathandayuthapani made changes -
            Assignee Thirunarayanan Balathandayuthapani [ thiru ] Marko Mäkelä [ marko ]
            Status In Progress [ 3 ] In Review [ 10002 ]
            marko Marko Mäkelä made changes -
            Assignee Marko Mäkelä [ marko ] Thirunarayanan Balathandayuthapani [ thiru ]
            Status In Review [ 10002 ] Stalled [ 10000 ]
            julien.fritsch Julien Fritsch made changes -
            Priority Major [ 3 ] Critical [ 2 ]
            thiru Thirunarayanan Balathandayuthapani made changes -
            Assignee Thirunarayanan Balathandayuthapani [ thiru ] Marko Mäkelä [ marko ]
            Status Stalled [ 10000 ] In Review [ 10002 ]
            marko Marko Mäkelä made changes -
            Assignee Marko Mäkelä [ marko ] Thirunarayanan Balathandayuthapani [ thiru ]
            Status In Review [ 10002 ] Stalled [ 10000 ]
            thiru Thirunarayanan Balathandayuthapani made changes -
            Assignee Thirunarayanan Balathandayuthapani [ thiru ] Marko Mäkelä [ marko ]
            Status Stalled [ 10000 ] In Review [ 10002 ]
            marko Marko Mäkelä made changes -
            Assignee Marko Mäkelä [ marko ] Thirunarayanan Balathandayuthapani [ thiru ]
            Status In Review [ 10002 ] Stalled [ 10000 ]
            thiru Thirunarayanan Balathandayuthapani made changes -
            Assignee Thirunarayanan Balathandayuthapani [ thiru ] Sergei Golubchik [ serg ]
            Status Stalled [ 10000 ] In Review [ 10002 ]
            serg Sergei Golubchik made changes -
            Assignee Sergei Golubchik [ serg ] Thirunarayanan Balathandayuthapani [ thiru ]
            Status In Review [ 10002 ] Stalled [ 10000 ]
            thiru Thirunarayanan Balathandayuthapani made changes -
            Component/s Storage Engine - InnoDB [ 10129 ]
            Fix Version/s 10.4.31 [ 29010 ]
            Fix Version/s 10.5.22 [ 29011 ]
            Fix Version/s 10.6.15 [ 29013 ]
            Fix Version/s 10.9.8 [ 29015 ]
            Fix Version/s 10.10.6 [ 29017 ]
            Fix Version/s 10.11.5 [ 29019 ]
            Fix Version/s 11.0.3 [ 28920 ]
            Fix Version/s 10.4 [ 22408 ]
            Fix Version/s 10.5 [ 23123 ]
            Fix Version/s 10.6 [ 24028 ]
            Resolution Fixed [ 1 ]
            Status Stalled [ 10000 ] Closed [ 6 ]
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            danblack Daniel Black made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            thiru Thirunarayanan Balathandayuthapani made changes -
            marko Marko Mäkelä made changes -
            thiru Thirunarayanan Balathandayuthapani made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            thiru Thirunarayanan Balathandayuthapani made changes -
            marko Marko Mäkelä made changes -
            mariadb-jira-automation Jira Automation (IT) made changes -
            Zendesk Related Tickets 113397

            People

              thiru Thirunarayanan Balathandayuthapani
              hholzgra Hartmut Holzgraefe
              Votes:
              2 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.