JSON_OBJECTAGG returns an unparseable document when the result character set is not ASCII compatible

JSON_OBJECTAGG() writes its opening brace as a single byte no matter how
wide a character of the result is. In ucs2, utf16 and utf32 the
closing brace is written at full width, so the two ends of the object do not
match and everything between them sits one byte out of step. The result is
not a document: JSON_VALID() answers 0, and reading the value back as
text produces a different set of characters entirely.

The value is wrong at rest, not merely on the way to the client. It is stored
that way, compared that way, and passed to other JSON functions that way.

How to repeat

SELECT JSON_VALID(JSON_OBJECTAGG(CONVERT(k USING ucs2),

                                 CONVERT(v USING ucs2))) AS valid

  FROM (SELECT 'a' AS k, 'b' AS v UNION ALL SELECT 'c', 'd') d;

valid

0

Warnings:

Note  4038  Syntax error in JSON text in argument 1 to function 'json_valid' at position 2

The bytes show it plainly - the opening brace is 7B, one byte, while the
closing brace is 007D, two:

SELECT HEX(JSON_OBJECTAGG(CONVERT(k USING ucs2),

                          CONVERT(v USING ucs2))) AS h

  FROM (SELECT 'a' AS k, 'b' AS v UNION ALL SELECT 'c', 'd') d;

7B 002200610022003A002200620022002C0020002200630022003A002200640022 007D

^^                                                                 ^^^^

one byte                                                       two bytes

Read back as text, the misalignment turns the whole value into unrelated
characters and the server rejects its own output:

SELECT CONVERT(JSON_OBJECTAGG(CONVERT(k USING ucs2),

                              CONVERT(v USING ucs2)) USING utf8mb4) AS readable

  FROM (SELECT 'a' AS k, 'b' AS v UNION ALL SELECT 'c', 'd') d;

readable

<a run of CJK ideographs, then a replacement character>

Warnings:

Warning  1300  Invalid ucs2 character string: '}'

Affected character sets

Every row below was measured on 10.11.

So the defect appears exactly where one character is more than one byte.
Where a character is one byte the two ends agree by coincidence and the
result is correct.

The pairs between the braces are written correctly in every case: the key
quotes, the colon and the ", " separator all come out at full width. Only
the opening brace is wrong.

Neither the number of pairs nor the shape of the query matters. One pair is
affected as much as several, each group of a GROUP BY is affected, and so
is the copy of the aggregate made for WITH ROLLUP.

Root cause

The opening brace is written while the expression is still being parsed, and
the character set of the result is not settled until fix_fields() runs,
which is later:

/* sql/item_jsonfunc.h - the constructor, at parse time */

Item_func_json_objectagg(THD *thd, Item *key, Item *value) :

  Item_sum(thd, key, value)

{

  quick_group= FALSE;

  result.append('{');        /* result is still my_charset_bin here */

}

/* sql/item_jsonfunc.cc - fix_fields(), later */

result.set_charset(collation.collation);

String::append(char) converts what it is given only when the string it is
appending to has mbminlen > 1. At construction time result carries the
default character set, my_charset_bin, whose mbminlen is 1, so the
brace goes down as one raw byte and stays that way.

The closing brace is written when the result is asked for, by which time
collation.collation has been applied to result, so that one is
converted normally. Hence the mismatch.

Two further places measure that brace rather than writing it, and both are
correct only where a character is one byte:

/* clear(), once per group - keeps byte 0 and rebuilds after it */

void Item_func_json_objectagg::clear()

{

  result.length(1);

  ...

}

/* add() - decides whether this pair needs a separator in front of it */

if (result.length() > 1 && result.append(STRING_WITH_LEN(", ")))

  ...

cleanup() does the same result.length(1).

Affected versions

JSON_OBJECTAGG was added by MDEV-16620 (commit ba8e5e689c8,
2019-10-14), first released in 10.5.0. All four of the places above were
present in that commit:

/* sql/item_jsonfunc.h @ ba8e5e689c8 */

Item_func_json_objectagg(THD *thd, Item *key, Item *value) :

  Item_sum(thd, key, value)

{

  result.append("{");

}

/* sql/item_jsonfunc.cc @ ba8e5e689c8 */

result.length(1);          /* cleanup() */

result.length(1);          /* clear()   */

if (result.length() > 1)   /* add()     */

Every release from 10.5.0 onward is therefore expected to be affected.

Verified on 10.11 only (10.11.19-MariaDB-debug). Other branches were not
tested.

Related, but a separate defect

JSON_ARRAYAGG() produces a similar looking result in the same character
sets - one byte wide brackets around full width elements - but for an
unrelated reason. Its brackets are built in a scratch String
which is then exchanged with the result, and that scratch string is left at
its default character set; the exchange carries the character set across with
the bytes. The same mislabelling additionally stops the result being
converted to character_set_results on its way to the client, which
JSON_OBJECTAGG does not suffer from.

That is reported separately as MDEV-40606. The two want separate fixes, and
neither is a consequence of the other.

Note

Found while testing an unrelated change to the JSON aggregates; the affected
character sets are not covered by any existing test.