JSON_MERGE and JSON_MERGE_PATCH silently accept a document argument with trailing text that every other JSON function rejects

JSON_MERGE() and JSON_MERGE_PATCH() read a document argument only as far
as its first value. Text standing after that value is neither read nor reported.
The functions return an answer composed from the first value alone, with no
diagnostic of any kind.

Every other JSON function rejects the very same text. JSON_VALID returns 0
about it, and JSON_INSERT, JSON_REMOVE, JSON_EXTRACT and
JSON_TYPE all return NULL with Warning 4038. So the same column value is
a document to two functions and not a document to the rest, and nothing tells
the caller which of the two answers they are getting.

How to repeat

SELECT JSON_MERGE_PATCH('{"a":1} rubbish', '{"b":2}') AS obj_lhs;

SELECT JSON_MERGE_PATCH('{"a":1}', '{"b":2} rubbish') AS obj_rhs;

SELECT JSON_MERGE('[1,2] rubbish', '{"a":1}')        AS merge_lhs;

obj_lhs

{"a": 1, "b": 2}

obj_rhs

{"a": 1, "b": 2}

merge_lhs

[1, 2, {"a": 1}]

No warning, no note, in any of the three.

The same text put to any other JSON function:

SELECT JSON_VALID('[1,2] rubbish')                AS valid;

SELECT JSON_INSERT('{"a":1} rubbish', '$.b', 2)   AS ins;

SELECT JSON_REMOVE('{"a":1,"b":2} rubbish', '$.b') AS rem;

SELECT JSON_EXTRACT('{"a":1} rubbish', '$.a')     AS ext;

SELECT JSON_TYPE('[1,2] rubbish')                 AS typ;

valid

0

Warnings:

Note  4038  Syntax error in JSON text in argument 1 to function 'json_valid' at position 7

ins

NULL

Warnings:

Warning  4038  Syntax error in JSON text in argument 1 to function 'json_insert' at position 16

rem

NULL

Warnings:

Warning  4038  Syntax error in JSON text in argument 1 to function 'json_remove' at position 9

ext

NULL

Warnings:

Warning  4038  Syntax error in JSON text in argument 1 to function 'json_extract' at position 9

typ

NULL

Warnings:

Warning  4038  Syntax error in JSON text in argument 1 to function 'json_type' at position 7

What stands after the value can be a whole document

It is not only unparseable rubbish that goes unreported. A second complete
document standing after the first is discarded in silence:

SELECT JSON_MERGE_PATCH('{"a":1} {"zzz":9}', '{"b":2}') AS second_doc;

second_doc

{"a": 1, "b": 2}

The key zzz is nowhere in the answer and nothing was said about it.

Out of a table

The case this is met in practice is a column whose check constraint was off
when the row went in, so the bytes a JSON column holds are not necessarily
a document:

CREATE TABLE t (id INT, j JSON);

SET SESSION check_constraint_checks = OFF;

INSERT INTO t VALUES (1, '{"a":1} trailing'), (2, '{"a":1,'), (3, '{"a":1}');

SET SESSION check_constraint_checks = ON;

SELECT id, JSON_MERGE_PATCH(j, '{"z":1}') AS v FROM t ORDER BY id;

id  v

1   {"a": 1, "z": 1}

2   NULL

3   {"a": 1, "z": 1}

Warnings:

Warning  4037  Unexpected end of JSON text in argument 1 to function 'json_merge_patch'

Row 2 is refused and reported. Row 1 is just as much not a document, and it is
neither refused nor reported: it is indistinguishable in the result from row 3,
which is a document.

Root cause

Both functions compose their answer out of what they read, and they read only
the first value of each argument.

Item_func_json_merge_patch::val_str() scans each argument and hands the two
engines to do_merge_patch():

/* sql/item_jsonfunc.cc */

json_scan_start(&je1, js1->charset(), (const uchar *) js1->ptr(),

                (const uchar *) js1->ptr() + js1->length());

...

if (do_merge_patch(str, &je1, &je2, &empty_result))

  goto error_return;

do_merge_patch() stops at the end of the value it was given. Where the two
values are not both objects it skips the left one outright:

else

{

  if (!json_value_scalar(je1) && json_skip_level(je1))

    return 1;

  *empty_result= je2->value_type == JSON_VALUE_NULL;

  if (!(*empty_result) && copy_value_patch(str, je2))

    return 1;

}

json_skip_level() stops at the close of that value and never reaches what
follows it. Neither engine is ever carried on to the end of its argument, so
s.error is never set and there is nothing for the caller's

error_return:

  if (je1.s.error)

    report_json_error(js1, &je1, 0);

  if (je2.s.error)

    report_json_error(js2, &je2, n_arg);

to report. Item_func_json_merge::val_str() and do_merge() have the same
shape.

The result the function does compose is then read back by

if (json_nice(&je1, str, Item_func_json_format::LOOSE))

  goto error_return;

but that reads the COMPOSED text, which holds only what was copied out of the
first values and never holds the trailing text. So that reading cannot catch it
either.

By contrast the other mutators copy the untouched remainder of their document
argument into the result:

/* Item_func_json_insert::val_str() */

append_simple(str, v_to, js->end() - v_to)

The trailing text lands in the composed result, the reading back above then
meets it, and that is why those functions report it. The two merge functions
never copy it, so it is never met.

The one place it is reported

JSON_MERGE_PATCH does report it in a single case: when every argument before
one is SQL NULL and that argument is not an object, the whole of it is taken over
as the answer rather than merged into, so the trailing text ends up in the
composed result after all and the reading back meets it:

SELECT JSON_MERGE_PATCH(NULL, '[1,2] rubbish') AS adopt_final;

adopt_final

NULL

Warnings:

Warning  4038  Syntax error in JSON text in argument 1 to function 'json_merge_patch' at position 7

Add one more argument and the taken-over document is merged over, so it leaves
the composed result again and the report goes with it:

SELECT JSON_MERGE_PATCH(NULL, '[1,2] rubbish', '{"a":1}') AS adopt_merged;

adopt_merged

{"a": 1}

Note also that the warning in the first of these names argument 1, which in
that statement is the four characters NULL and holds no JSON text. The
position is an offset into the server's own composed text, not into any
argument. That misattribution is a separate matter and is not what this report
is about.

Affected versions

JSON_MERGE was added by 27025221fe2 (2016-10-19, "MDEV-9143 JSON_xxx
functions"), first released in 10.2.0. JSON_MERGE_PATCH was added by
cd16d6d5187 (2019-05-17, "MDEV-13992 Implement JSON_MERGE_PATCH"), first
released in 10.2.25.

The reading stops at the first value at the tips of 10.5, 10.6, 10.11, 11.4,
11.8, 12.0, 12.1, 12.2, 12.3 and main.

Verified on 10.11 only (10.11.19-MariaDB-debug). Other branches were read but
not run.

Note

Trailing whitespace is not this: {{JSON_MERGE_PATCH('

', '

')}}
is a well formed document with space after it and nothing is wrong with it.
What is described here is text that is not whitespace.

Found while auditing the JSON functions for an unrelated change. Nothing in the
test suite covers it.