[MDEV-9823] LOAD DATA INFILE silently truncates incomplete byte sequences Created: 2016-03-29  Updated: 2016-04-06  Resolved: 2016-04-06

Status: Closed
Project: MariaDB Server
Component/s: Character Sets
Affects Version/s: 5.5, 10.0, 10.1, 10.2
Fix Version/s: 10.2.0

Type: Bug Priority: Major
Reporter: Alexander Barkov Assignee: Alexander Barkov
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Blocks
blocks MDEV-6353 my_ismbchar() and my_mbcharlen() refa... Closed

 Description   

If I insert an incomplete multi-byte character into a table:

DROP TABLE IF EXISTS t1;
 
CREATE TABLE t1 (a VARCHAR(10) CHARACTER SET ujis);
 
INSERT INTO t1 VALUES (0x8FA1);
 
SHOW WARNINGS;
 
SELECT HEX(a) FROM t1;

it correctly returns a warning:

+---------+------+------------------------------------------------------------+
 
| Level   | Code | Message                                                    |
 
+---------+------+------------------------------------------------------------+
 
| Warning | 1366 | Incorrect string value: '\x8F\xA1' for column 'a' at row 1 |
 
+---------+------+------------------------------------------------------------+

and translates every byte in the incomplete sequence to QUESTION MARK:

+--------+
 
| HEX(a) |
 
+--------+
 
| 3F3F   |
 
+--------+

If I put the same sequence into a file:

printf "\x8F\xA1"> /tmp/test.txt

and load it:

DELETE FROM t1;
 
LOAD DATA INFILE '/tmp/test.txt' INTO TABLE t1 CHARACTER SET ujis;
 
SELECT HEX(a) FROM t1;

it returns no warnings and truncates the incomplete sequence:

+--------+
 
| HEX(a) |
 
+--------+
 
|        |
 
+--------+

LOAD should be fixed to work consistently with INSERT.
Generated at Thu Feb 08 07:37:36 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.