Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-7533

COLUMN_JSON() doesn't escape control characters in string values

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • 10.0.15
    • 10.0.34, 10.1.31, 10.2.13, 10.3.5
    • Dynamic Columns
    • None
    • Darwin 13.4.0 Darwin Kernel Version 13.4.0: Sun Aug 17 19:50:11 PDT 2014; root:xnu-2422.115.4~1/RELEASE_X86_64 x86_64
    • 10.0.34

    Description

      According to RFC 4627 http://www.ietf.org/rfc/rfc4627.txt

      "All Unicode characters may be placed within the quotation marks except for the characters that must be escaped: quotation mark, reverse solidus, and the control characters (U+0000 through U+001F)."

      If you store a string that contains a control character in a dynamic column and fetch it back with COLUMN_JSON() then the control character is not escaped in the JSON encoded value returned.

      All the JSON decoders I have tried it with fail on such strings.

      Attachments

        Issue Links

          Activity

            thefsb Tom added a comment -

            There is a related question.

            Dynamic columns allow BINARY values. JSON cannot in general represent binary data. At present, if a dynamic column has a BINARY value then COLUMN_JSON just puts it in quotation marks and sends it. If the document is utf8 then this usually means that the document returned is not valid JSON. Most JSON encoders reject inputs that cannot be encoded with an error but Maria instead returns an invalid document that cannot be decoded using normal JSON libraries.

            However, I don't know what the right behavior should be.

            thefsb Tom added a comment - There is a related question. Dynamic columns allow BINARY values. JSON cannot in general represent binary data. At present, if a dynamic column has a BINARY value then COLUMN_JSON just puts it in quotation marks and sends it. If the document is utf8 then this usually means that the document returned is not valid JSON. Most JSON encoders reject inputs that cannot be encoded with an error but Maria instead returns an invalid document that cannot be decoded using normal JSON libraries. However, I don't know what the right behavior should be.
            galdiolo Claudio Galdiolo added a comment - - edited

            Server version: 10.1.13-MariaDB - Homebrew

            -- this returns a valid json string
            SELECT COLUMN_JSON(COLUMN_CREATE('test','Text "escape"')) AS json;
            

            +----------------------------+
            | json                       |
            +----------------------------+
            | {"test":"Text \"escape\""} |
            +----------------------------+
            

            -- this does not return a valid json string
            SELECT COLUMN_JSON(COLUMN_CREATE('test','First line\nSecond line')) AS json;
            

            +-----------------------------------+
            | json                              |
            +-----------------------------------+
            | {"test":"First line
            Second line"} |
            +-----------------------------------+
            

            The valid json output for the last query should be:

            +------------------------------------+
            | json                               |
            +------------------------------------+
            | {"test":"First line\nSecond line"} |
            +------------------------------------+
            

            galdiolo Claudio Galdiolo added a comment - - edited Server version: 10.1.13-MariaDB - Homebrew -- this returns a valid json string SELECT COLUMN_JSON(COLUMN_CREATE( 'test' , 'Text "escape"' )) AS json; +----------------------------+ | json | +----------------------------+ | {"test":"Text \"escape\""} | +----------------------------+ -- this does not return a valid json string SELECT COLUMN_JSON(COLUMN_CREATE( 'test' , 'First line\nSecond line' )) AS json; +-----------------------------------+ | json | +-----------------------------------+ | {"test":"First line Second line"} | +-----------------------------------+ The valid json output for the last query should be: +------------------------------------+ | json | +------------------------------------+ | {"test":"First line\nSecond line"} | +------------------------------------+

            github branch: bb-10.0-MDEV-7533

            revision-id: 16298d9e82eb507c5e0ca6d097e6a7016582ccbc (mariadb-10.0.33-47-g16298d9e82e)
            parent(s): 5e87f49a9927bf3fd0bd9af789bbdd329e4b4416
            author: Oleksandr Byelkin
            committer: Oleksandr Byelkin
            timestamp: 2018-01-22 16:53:10 +0100
            message:

            MDEV-7533: COLUMN_JSON() doesn't escape control characters in string values

            escape all charecters less or equal 0x1F (control symbols)
            (shorter sequence are not used to make code simple, long encoding is always legal according to the rfc4627)

            sanja Oleksandr Byelkin added a comment - github branch: bb-10.0- MDEV-7533 revision-id: 16298d9e82eb507c5e0ca6d097e6a7016582ccbc (mariadb-10.0.33-47-g16298d9e82e) parent(s): 5e87f49a9927bf3fd0bd9af789bbdd329e4b4416 author: Oleksandr Byelkin committer: Oleksandr Byelkin timestamp: 2018-01-22 16:53:10 +0100 message: MDEV-7533 : COLUMN_JSON() doesn't escape control characters in string values escape all charecters less or equal 0x1F (control symbols) (shorter sequence are not used to make code simple, long encoding is always legal according to the rfc4627) —

            People

              sanja Oleksandr Byelkin
              thefsb Tom
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.