Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-9823

LOAD DATA INFILE silently truncates incomplete byte sequences

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 5.5, 10.0, 10.1, 10.2
    • Fix Version/s: 10.2.0
    • Component/s: Character Sets
    • Labels:
      None

      Description

      If I insert an incomplete multi-byte character into a table:

      DROP TABLE IF EXISTS t1;
      CREATE TABLE t1 (a VARCHAR(10) CHARACTER SET ujis);
      INSERT INTO t1 VALUES (0x8FA1);
      SHOW WARNINGS;
      SELECT HEX(a) FROM t1;
      

      it correctly returns a warning:

      +---------+------+------------------------------------------------------------+
      | Level   | Code | Message                                                    |
      +---------+------+------------------------------------------------------------+
      | Warning | 1366 | Incorrect string value: '\x8F\xA1' for column 'a' at row 1 |
      +---------+------+------------------------------------------------------------+
      

      and translates every byte in the incomplete sequence to QUESTION MARK:

      +--------+
      | HEX(a) |
      +--------+
      | 3F3F   |
      +--------+
      

      If I put the same sequence into a file:

      printf "\x8F\xA1"> /tmp/test.txt
      

      and load it:

      DELETE FROM t1;
      LOAD DATA INFILE '/tmp/test.txt' INTO TABLE t1 CHARACTER SET ujis;
      SELECT HEX(a) FROM t1;
      

      it returns no warnings and truncates the incomplete sequence:

      +--------+
      | HEX(a) |
      +--------+
      |        |
      +--------+
      

      LOAD should be fixed to work consistently with INSERT.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                bar Alexander Barkov
                Reporter:
                bar Alexander Barkov
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: