Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-41784

JSON_SCHEMA_VALID rejects matching enum strings encoded as UCS2 or UTF16

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Minor
    • Resolution: Duplicate
    • 12.3.3
    • N/A
    • JSON
    • None
    • Linux/amd64
    • Not for Release Notes

    Description

      JSON_SCHEMA_VALID rejects the valid JSON string "x" against schema enum: ["x"] when the document uses UCS2 or UTF16 encoding. The same value is accepted as UTF8 and after an explicit conversion from UCS2 to UTF8MB4. This was observed with MariaDB 12.3.3 on Linux/amd64.

      Reproducer

      Run the following SQL:

      SELECT 'utf8_valid' AS phase,JSON_VALID('"x"') AS valid_json,JSON_SCHEMA_VALID('{"enum":["x"]}','"x"') AS actual,1 AS expected;
      SELECT 'ucs2_valid' AS phase,JSON_VALID(CONVERT('"x"' USING ucs2)) AS valid_json,JSON_SCHEMA_VALID('{"enum":["x"]}',CONVERT('"x"' USING ucs2)) AS actual,1 AS expected;
      SELECT 'ucs2_invalid' AS phase,JSON_VALID(CONVERT('"y"' USING ucs2)) AS valid_json,JSON_SCHEMA_VALID('{"enum":["x"]}',CONVERT('"y"' USING ucs2)) AS actual,0 AS expected;
      SELECT 'utf16_valid' AS phase,JSON_VALID(CONVERT('"x"' USING utf16)) AS valid_json,JSON_SCHEMA_VALID('{"enum":["x"]}',CONVERT('"x"' USING utf16)) AS actual,1 AS expected;
      SELECT 'roundtrip' AS phase,JSON_VALID(CONVERT(CONVERT('"x"' USING ucs2) USING utf8mb4)) AS valid_json,JSON_SCHEMA_VALID('{"enum":["x"]}',CONVERT(CONVERT('"x"' USING ucs2) USING utf8mb4)) AS actual,1 AS expected;
      SELECT 'cleanup' AS phase,0 AS created_objects;

      The client returns:

      phase	valid_json	actual	expected
      utf8_valid	1	1	1
      phase	valid_json	actual	expected
      ucs2_valid	1	0	1
      phase	valid_json	actual	expected
      ucs2_invalid	1	0	0
      phase	valid_json	actual	expected
      utf16_valid	1	0	1
      phase	valid_json	actual	expected
      roundtrip	1	1	1
      phase	created_objects
      cleanup	0

      All document inputs pass JSON_VALID, but JSON_SCHEMA_VALID rejects the matching UCS2 and UTF16 strings. The UTF8 value and explicit UTF8 roundtrip are accepted, while the nonmatching "y" control is rejected as expected.

      JSON Schema enum validation compares the JSON value with the enum member. The valid JSON string "x" should therefore match enum member "x" whether the document is represented as UTF8, UCS2, or UTF16; "y" should remain rejected.

      Source review suggests that document strings retain their input encoding while schema enum keys use the schema encoding, producing incompatible representations during lookup. In particular, UCS2 places a zero byte before the ASCII character. This is an inferred explanation for the observed result rather than a runtime-confirmed root cause.

      This occurs when JSON_SCHEMA_VALID compares a JSON string with an enum member while the document string uses UCS2 or UTF16 encoding.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              imchifan 刘启帆
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.