[MDEV-7533] COLUMN_JSON() doesn't escape control characters in string values Created: 2015-01-31  Updated: 2018-05-16  Resolved: 2018-01-23

Status: Closed
Project: MariaDB Server
Component/s: Dynamic Columns
Affects Version/s: 10.0.15
Fix Version/s: 10.0.34, 10.1.31, 10.2.13, 10.3.5

Type: Bug Priority: Major
Reporter: Tom Assignee: Oleksandr Byelkin
Resolution: Fixed Votes: 1
Labels: None
Environment:

Darwin 13.4.0 Darwin Kernel Version 13.4.0: Sun Aug 17 19:50:11 PDT 2014; root:xnu-2422.115.4~1/RELEASE_X86_64 x86_64


Issue Links:
Duplicate
duplicates MDEV-12947 COLUMN_JSON does not properly encode ... Closed
Sprint: 10.0.34

 Description   

According to RFC 4627 http://www.ietf.org/rfc/rfc4627.txt

"All Unicode characters may be placed within the quotation marks except for the characters that must be escaped: quotation mark, reverse solidus, and the control characters (U+0000 through U+001F)."

If you store a string that contains a control character in a dynamic column and fetch it back with COLUMN_JSON() then the control character is not escaped in the JSON encoded value returned.

All the JSON decoders I have tried it with fail on such strings.



 Comments   
Comment by Tom [ 2015-02-01 ]

There is a related question.

Dynamic columns allow BINARY values. JSON cannot in general represent binary data. At present, if a dynamic column has a BINARY value then COLUMN_JSON just puts it in quotation marks and sends it. If the document is utf8 then this usually means that the document returned is not valid JSON. Most JSON encoders reject inputs that cannot be encoded with an error but Maria instead returns an invalid document that cannot be decoded using normal JSON libraries.

However, I don't know what the right behavior should be.

Comment by Claudio Galdiolo [ 2016-04-07 ]

Server version: 10.1.13-MariaDB - Homebrew

-- this returns a valid json string
SELECT COLUMN_JSON(COLUMN_CREATE('test','Text "escape"')) AS json;

+----------------------------+
| json                       |
+----------------------------+
| {"test":"Text \"escape\""} |
+----------------------------+

-- this does not return a valid json string
SELECT COLUMN_JSON(COLUMN_CREATE('test','First line\nSecond line')) AS json;

+-----------------------------------+
| json                              |
+-----------------------------------+
| {"test":"First line
Second line"} |
+-----------------------------------+

The valid json output for the last query should be:

+------------------------------------+
| json                               |
+------------------------------------+
| {"test":"First line\nSecond line"} |
+------------------------------------+

Comment by Oleksandr Byelkin [ 2018-01-22 ]

github branch: bb-10.0-MDEV-7533

revision-id: 16298d9e82eb507c5e0ca6d097e6a7016582ccbc (mariadb-10.0.33-47-g16298d9e82e)
parent(s): 5e87f49a9927bf3fd0bd9af789bbdd329e4b4416
author: Oleksandr Byelkin
committer: Oleksandr Byelkin
timestamp: 2018-01-22 16:53:10 +0100
message:

MDEV-7533: COLUMN_JSON() doesn't escape control characters in string values

escape all charecters less or equal 0x1F (control symbols)
(shorter sequence are not used to make code simple, long encoding is always legal according to the rfc4627)

Generated at Thu Feb 08 07:20:19 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.