[MDEV-15938] TINYTEXT CHARACTER SET utf8 COMPRESSED truncates data - Jira

Details

Type: Bug
Status: Closed (View Workflow)
Priority: Major
Resolution: Fixed
Affects Version/s: 10.3(EOL)
Fix Version/s: 10.3.7
Component/s: Server
Labels:
None

Description

SET sql_mode='';

CREATE OR REPLACE TABLE t1(

  a TINYTEXT CHARACTER SET utf8 COMPRESSED,

  b TINYTEXT CHARACTER SET utf8

);

INSERT INTO t1 VALUES (REPEAT(_latin1'a',250), REPEAT(_latin1'a',250));

SELECT CHAR_LENGTH(a), CHAR_LENGTH(b) FROM t1;

+----------------+----------------+

| CHAR_LENGTH(a) | CHAR_LENGTH(b) |

+----------------+----------------+

|             84 |            250 |

+----------------+----------------+

Looks wrong. The expected behavior would be to write all 250 characters into both columns.

Attachments

Activity

Ascending order - Click to sort in descending order

Alexander Barkov created issue - 2018-04-20 09:08

Alexander Barkov made changes - 2018-04-20 09:10

Field	Original Value	New Value
Description	I insert a string consisting of 250 ASCII characters in two TINYTEXT column (compressed and non-compressed). The compressed column truncates data. The non-compressed column write the entire value: {code:sql} SET sql_mode=''; CREATE OR REPLACE TABLE t1( a TINYTEXT CHARACTER SET utf8 COMPRESSED, b TINYTEXT CHARACTER SET utf8 ); INSERT INTO t1 VALUES (REPEAT(_latin1'a',250), REPEAT(_latin1'a',250)); SELECT CHAR_LENGTH(a), CHAR_LENGTH(b) FROM t1; {code} {noformat} +----------------+----------------+ \| CHAR_LENGTH(a) \| CHAR_LENGTH(b) \| +----------------+----------------+ \| 84 \| 250 \| +----------------+----------------+ {noformat} Looks wrong. The expected behavior would be to write all {{250}} characters into both columns.	I insert an latin1 string consisting of 250 ASCII characters in two TINYTEXT columns (compressed and non-compressed) with CHARACTER SET utf8. The compressed column truncates data. The non-compressed column write the entire value: {code:sql} SET sql_mode=''; CREATE OR REPLACE TABLE t1( a TINYTEXT CHARACTER SET utf8 COMPRESSED, b TINYTEXT CHARACTER SET utf8 ); INSERT INTO t1 VALUES (REPEAT(_latin1'a',250), REPEAT(_latin1'a',250)); SELECT CHAR_LENGTH(a), CHAR_LENGTH(b) FROM t1; {code} {noformat} +----------------+----------------+ \| CHAR_LENGTH(a) \| CHAR_LENGTH(b) \| +----------------+----------------+ \| 84 \| 250 \| +----------------+----------------+ {noformat} Looks wrong. The expected behavior would be to write all {{250}} characters into both columns.

Alexander Barkov made changes - 2018-04-20 09:11

Description

I insert an latin1 string consisting of 250 ASCII characters in two TINYTEXT columns (compressed and non-compressed) with CHARACTER SET utf8. The compressed column truncates data. The non-compressed column write the entire value:

{code:sql}
SET sql_mode='';
CREATE OR REPLACE TABLE t1(
a TINYTEXT CHARACTER SET utf8 COMPRESSED,
b TINYTEXT CHARACTER SET utf8
);
INSERT INTO t1 VALUES (REPEAT(_latin1'a',250), REPEAT(_latin1'a',250));
SELECT CHAR_LENGTH(a), CHAR_LENGTH(b) FROM t1;
{code}
{noformat}
+----------------+----------------+
| CHAR_LENGTH(a) | CHAR_LENGTH(b) |
+----------------+----------------+
| 84 | 250 |
+----------------+----------------+
{noformat}

Looks wrong. The expected behavior would be to write all {{250}} characters into both columns.

I insert an latin1 string consisting of 250 ASCII characters in two TINYTEXT columns (compressed and non-compressed) with CHARACTER SET utf8. The compressed column truncates data. The non-compressed column writes the entire value:

{code:sql}
SET sql_mode='';
CREATE OR REPLACE TABLE t1(
a TINYTEXT CHARACTER SET utf8 COMPRESSED,
b TINYTEXT CHARACTER SET utf8
);
INSERT INTO t1 VALUES (REPEAT(_latin1'a',250), REPEAT(_latin1'a',250));
SELECT CHAR_LENGTH(a), CHAR_LENGTH(b) FROM t1;
{code}
{noformat}
+----------------+----------------+
| CHAR_LENGTH(a) | CHAR_LENGTH(b) |
+----------------+----------------+
| 84 | 250 |
+----------------+----------------+
{noformat}

Looks wrong. The expected behavior would be to write all {{250}} characters into both columns.

Alexander Barkov made changes - 2018-04-20 09:11

Description

I insert an latin1 string consisting of 250 ASCII characters in two TINYTEXT columns (compressed and non-compressed) with CHARACTER SET utf8. The compressed column truncates data. The non-compressed column writes the entire value:

{code:sql}
SET sql_mode='';
CREATE OR REPLACE TABLE t1(
a TINYTEXT CHARACTER SET utf8 COMPRESSED,
b TINYTEXT CHARACTER SET utf8
);
INSERT INTO t1 VALUES (REPEAT(_latin1'a',250), REPEAT(_latin1'a',250));
SELECT CHAR_LENGTH(a), CHAR_LENGTH(b) FROM t1;
{code}
{noformat}
+----------------+----------------+
| CHAR_LENGTH(a) | CHAR_LENGTH(b) |
+----------------+----------------+
| 84 | 250 |
+----------------+----------------+
{noformat}

Looks wrong. The expected behavior would be to write all {{250}} characters into both columns.

I insert an latin1 string consisting of 250 ASCII characters in two TINYTEXT columns (compressed and non-compressed) with CHARACTER SET utf8. The compressed column truncates data. The non-compressed column writes the entire value, as exprected:

{code:sql}
SET sql_mode='';
CREATE OR REPLACE TABLE t1(
a TINYTEXT CHARACTER SET utf8 COMPRESSED,
b TINYTEXT CHARACTER SET utf8
);
INSERT INTO t1 VALUES (REPEAT(_latin1'a',250), REPEAT(_latin1'a',250));
SELECT CHAR_LENGTH(a), CHAR_LENGTH(b) FROM t1;
{code}
{noformat}
+----------------+----------------+
| CHAR_LENGTH(a) | CHAR_LENGTH(b) |
+----------------+----------------+
| 84 | 250 |
+----------------+----------------+
{noformat}

Looks wrong. The expected behavior would be to write all {{250}} characters into both columns.

Alexander Barkov made changes - 2018-04-20 09:11

Description

I insert an latin1 string consisting of 250 ASCII characters in two TINYTEXT columns (compressed and non-compressed) with CHARACTER SET utf8. The compressed column truncates data. The non-compressed column writes the entire value, as exprected:

{code:sql}
SET sql_mode='';
CREATE OR REPLACE TABLE t1(
a TINYTEXT CHARACTER SET utf8 COMPRESSED,
b TINYTEXT CHARACTER SET utf8
);
INSERT INTO t1 VALUES (REPEAT(_latin1'a',250), REPEAT(_latin1'a',250));
SELECT CHAR_LENGTH(a), CHAR_LENGTH(b) FROM t1;
{code}
{noformat}
+----------------+----------------+
| CHAR_LENGTH(a) | CHAR_LENGTH(b) |
+----------------+----------------+
| 84 | 250 |
+----------------+----------------+
{noformat}

Looks wrong. The expected behavior would be to write all {{250}} characters into both columns.

I insert an latin1 string consisting of 250 ASCII characters in two TINYTEXT columns (compressed and non-compressed) with CHARACTER SET utf8. The compressed column truncates data. The non-compressed column writes the entire value, as expected:

{code:sql}
SET sql_mode='';
CREATE OR REPLACE TABLE t1(
a TINYTEXT CHARACTER SET utf8 COMPRESSED,
b TINYTEXT CHARACTER SET utf8
);
INSERT INTO t1 VALUES (REPEAT(_latin1'a',250), REPEAT(_latin1'a',250));
SELECT CHAR_LENGTH(a), CHAR_LENGTH(b) FROM t1;
{code}
{noformat}
+----------------+----------------+
| CHAR_LENGTH(a) | CHAR_LENGTH(b) |
+----------------+----------------+
| 84 | 250 |
+----------------+----------------+
{noformat}

Looks wrong. The expected behavior would be to write all {{250}} characters into both columns.

Alexander Barkov added a comment - 2018-04-20 09:13 - edited

The same problem is repeatable with utf16:

SET sql_mode='';

CREATE OR REPLACE TABLE t1(

  a TINYTEXT CHARACTER SET utf16 COMPRESSED,

  b TINYTEXT CHARACTER SET utf16

);

INSERT INTO t1 VALUES (REPEAT(_latin1'a',100), REPEAT(_latin1'a',100));

SELECT CHAR_LENGTH(a), CHAR_LENGTH(b) FROM t1;

+----------------+----------------+

| CHAR_LENGTH(a) | CHAR_LENGTH(b) |

+----------------+----------------+

|             63 |            100 |

+----------------+----------------+

It should be repeatable if:

There is a conversion
The column character set has a variable encoding (mbminlen < mbmaxlen): euckr, gb2312, gbk, utf8, utf8mb4, utf16, utf16le, eucjpms

Alexander Barkov added a comment - 2018-04-20 09:13 - edited The same problem is repeatable with utf16: SET sql_mode= '' ; CREATE OR REPLACE TABLE t1( a TINYTEXT CHARACTER SET utf16 COMPRESSED, b TINYTEXT CHARACTER SET utf16 ); INSERT INTO t1 VALUES (REPEAT(_latin1 'a' ,100), REPEAT(_latin1 'a' ,100)); SELECT CHAR_LENGTH(a), CHAR_LENGTH(b) FROM t1; +----------------+----------------+ | CHAR_LENGTH(a) | CHAR_LENGTH(b) | +----------------+----------------+ | 63 | 100 | +----------------+----------------+ It should be repeatable if: There is a conversion The column character set has a variable encoding (mbminlen < mbmaxlen): euckr, gb2312, gbk, utf8, utf8mb4, utf16, utf16le, eucjpms

Sergey Vojtovich made changes - 2018-04-25 10:50

Status

Open [ 1 ]

Confirmed [ 10101 ]

Sergey Vojtovich added a comment - 2018-04-25 10:50

bar, please review fix for this bug
https://github.com/MariaDB/server/commit/9d8d3b848142a684a6fdcddf49eccab9c905fb78

Sergey Vojtovich added a comment - 2018-04-25 10:50 bar , please review fix for this bug https://github.com/MariaDB/server/commit/9d8d3b848142a684a6fdcddf49eccab9c905fb78

Sergey Vojtovich made changes - 2018-04-25 10:50

Assignee	Sergey Vojtovich [ svoj ]	Alexander Barkov [ bar ]
Status	Confirmed [ 10101 ]	In Review [ 10002 ]

Alexander Barkov added a comment - 2018-04-27 13:33

svoj, the patch is fine.
Please move the utf16 part to a separate test file with:

-- source include/have_utf16.inc

Probably it's a good idea to make a new file ctype_utf16_compressed.test for this.

Alexander Barkov added a comment - 2018-04-27 13:33 svoj , the patch is fine. Please move the utf16 part to a separate test file with: -- source include/have_utf16.inc Probably it's a good idea to make a new file ctype_utf16_compressed.test for this.

Alexander Barkov made changes - 2018-04-27 13:33

Status

In Review [ 10002 ]

Stalled [ 10000 ]

Alexander Barkov made changes - 2018-04-27 13:33

Assignee

Alexander Barkov [ bar ]

Sergey Vojtovich [ svoj ]

Sergey Vojtovich made changes - 2018-04-30 15:35

issue.field.resolutiondate

2018-04-30 15:35:04.0

2018-04-30 15:35:04.471

Sergey Vojtovich made changes - 2018-04-30 15:35

Component/s		Server [ 13907 ]
Fix Version/s		10.3.7 [ 23005 ]
Fix Version/s	10.3 [ 22126 ]
Resolution		Fixed [ 1 ]
Status	Stalled [ 10000 ]	Closed [ 6 ]

Sergei Golubchik made changes - 2021-12-06 21:47

Workflow

MariaDB v3 [ 86665 ]

MariaDB v4 [ 154194 ]

People

Assignee:: Sergey Vojtovich

Reporter:: Alexander Barkov

Votes:: 0 Vote for this issue

Watchers:: 3 Start watching this issue

Dates

Created:: 2018-04-20 09:08

Updated:: 2018-04-30 15:35

Resolved:: 2018-04-30 15:35

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.

MariaDB Server

Details

Description

Attachments

Activity

People

Dates

Git Integration