JSON_MERGE and JSON_MERGE_PATCH compose an answer out of an object argument that is not a JSON document, silently dropping whatever stood past the break

JSON_MERGE is documented as raising an error when an argument is not a
valid JSON document. For an object argument that breaks before its
first value is complete, it does not: it composes an answer out of as
much of the argument as it managed to read, and says nothing.

SELECT JSON_MERGE('{}', '{"a":1 "b":2}') AS v;

v

{"a": 1}

{"a":1 "b":2}

has no comma between its two members, so it is not a JSON
document. What comes back is an object no argument held: composed,
well formed, and missing whatever stood past the break. No error and no
warning is raised.

Because the answer is well formed, nothing downstream has any reason to
reject it:

SELECT JSON_VALID(JSON_MERGE('{}', '{"a":1 "b":2}')) AS v;

v

1

SELECT JSON_LENGTH(JSON_MERGE('{}', '{"a":1 "b":2}')) AS v;

v

1

The same function refuses the same argument next door

Whether an argument is refused or composed from is decided by something
that has nothing to do with that argument: whether the OTHER argument
has any keys.

SELECT JSON_MERGE('{}',      '{"a":1 "b":2}') AS v;

v

{"a": 1}

SELECT JSON_MERGE('{"b":9}', '{"a":1 "b":2}') AS v;

v

NULL

Warnings:

Warning	4038	Syntax error in JSON text in argument 2 to function

'json_merge_preserve' at position 8

Both, in one statement, over one column:

CREATE TABLE t (id INT, a VARCHAR(64), b VARCHAR(64));

INSERT INTO t VALUES (1, '{}', '{"a":1 "b":2}'),

                     (2, '{"b":9}', '{"a":1 "b":2}');

SELECT id, JSON_MERGE(a, b) AS v FROM t ORDER BY id;

id	v

1	{"a": 1}

2	NULL

How to repeat

Run on 10.11.19-MariaDB-debug at 1dab253482d. The output shown is what
the server produced.

-- argument 2 breaks, argument 1 being an empty object

SELECT JSON_MERGE('{}', '{"a":1 "b":2}') AS v;

v

{"a": 1}

SELECT JSON_MERGE('{}', '{"a":1,') AS v;

v

{"a": 1}

-- an argument that is nothing but an opening brace

SELECT JSON_MERGE('{}', '{') AS v;

v

{}

-- argument 1 breaks, argument 2 being an empty object

SELECT JSON_MERGE('{"a":1 "b":2}', '{}') AS v;

v

{"a": 1}

-- three arguments, so a composed document is on the left by the time

-- the broken one is reached

SELECT JSON_MERGE('{}', '{"a":1 "b":2}', '{"c":3}') AS v;

v

{"a": 1, "c": 3}

-- JSON_MERGE_PATCH walks the same two loops

SELECT JSON_MERGE_PATCH('{}', '{"a":1 "b":2}') AS v;

v

{"a": 1}

SELECT JSON_MERGE_PATCH('{"a":1 "b":2}', '{}') AS v;

v

{"a": 1}

-- and argument 1 not being an object at all, which is a different walk

SELECT JSON_MERGE_PATCH('[]', '{"a":1 "b":2}') AS v;

v

{"a": 1}

SELECT JSON_MERGE_PATCH('1', '{"a":1 "b":2}') AS v;

v

{"a": 1}

SELECT JSON_MERGE_PATCH('true', '{') AS v;

v

{}

-- the break below the top level, down the same road

SELECT JSON_MERGE_PATCH('[]', '{"a":{"p":1 "q":2}}') AS v;

v

{"a": {"p": 1}}

The shape of the argument on the other side decides it here too, and one
statement over one column shows both:

CREATE TABLE tm (id INT, a VARCHAR(64), b VARCHAR(64));

INSERT INTO tm VALUES (1, '[]', '{"a":1 "b":2}'),

                      (2, '{}', '{"a":1 "b":2}');

SELECT id, JSON_MERGE_PATCH(a, b) AS v FROM tm ORDER BY id;

id	v

1	{"a": 1}

2	NULL

What every other JSON function says about the same characters:

SELECT JSON_EXTRACT('{"a":1 "b":2}', '$') AS v;

v

NULL

Warnings:

Warning	4038	Syntax error in JSON text in argument 1 to function

'json_extract' at position 8

Not this bug: text standing after a complete value

An argument whose first value IS complete, with text after it, is a
different case. There the merging has a whole value to work from, what
stands after it never reaches the answer, and the answer has always
been given:

SELECT JSON_MERGE('{}', '{"a":1} rubbish') AS v;

v

{"a": 1}

That is not what this report is about. It is MDEV-40628.

Relationship to MDEV-40628

MDEV-40628 reports that trailing text after a complete value is accepted in
silence, and it draws its contrast with a document that breaks before its
value is complete, which it says is refused and reported. Its table:

CREATE TABLE t (id INT, j JSON);

SET SESSION check_constraint_checks = OFF;

INSERT INTO t VALUES (1, '{"a":1} trailing'), (2, '{"a":1,'), (3, '{"a":1}');

SET SESSION check_constraint_checks = ON;

SELECT id, JSON_MERGE_PATCH(j, '{"z":1}') AS v FROM t ORDER BY id;

id	v

1	{"a": 1, "z": 1}

2	NULL

3	{"a": 1, "z": 1}

Warnings:

Warning	4037	Unexpected end of JSON text in argument 1 to function

'json_merge_patch'

Row 2 is refused there because the other argument,

, has a key. Change
that one argument to an object with no keys and nothing is refused and nothing
is reported. A truncated document, a document with text after it, and a
genuine document all answer alike:

SELECT id, JSON_MERGE_PATCH(j, '{}') AS v FROM t ORDER BY id;

id	v

1	{"a": 1}

2	{"a": 1}

3	{"a": 1}

So the behaviour MDEV-40628 relies on as the correct one is not the behaviour
in every case, and this report is the other half of the same subject rather
than a restatement of it. MDEV-40628 asks for a document that is not one to
be reported. This asks for a document that is not one to stop being merged
from.

Root cause

Both functions walk an object argument's keys with a loop whose
condition asks only whether the scanner moved:

/* sql/item_jsonfunc.cc - do_merge() */

    while (json_scan_next(je1) == 0 &&

           je1->state != JST_OBJ_END)

    {

      ...

      if (unlikely(je1->s.error))

        return 1;

      ...

    }

    *je2= sav_je2;

The loop therefore ends on a refusal exactly as it ends on the object's
end, and the two are not told apart afterwards. The check inside the
loop only covers a failure met while reading a key name; a failure met
by the json_scan_next() in the loop condition itself ends the loop and
is never looked at.

There are five such loop ends. Four are in the merging walks - two in
do_merge() and two in do_merge_patch(), one per argument in each. The
fifth is in copy_value_patch(), which is the road JSON_MERGE_PATCH takes
when argument 1 is not an object: there is nothing to merge with, so
argument 2 is copied whole, and that copy walks the keys with the same
loop. Its caller does not look at je2->s.error after it returns either.

/* sql/item_jsonfunc.cc - copy_value_patch() */

    while (json_scan_next(je) == 0 &&

           je->state != JST_OBJ_END)

    {

      ...

    }

    if (str->append('}'))

      return 1;

That loop is entered for every object nested inside the value as well as
for the value itself, so a break below the top level goes the same way.

This is also why the other argument's shape decides the outcome. When
argument 1 has keys, the outer loop turns at least once and reaches an
inner loop over argument 2 that does check:

      while (json_scan_next(je2) == 0 &&

          je2->state != JST_OBJ_END)

      {

        ...

      }

      if (unlikely(je2->s.error))

        return 2;

With an empty object on the left the outer loop never turns, that inner
loop never runs, and the refusal is lost.

Affected versions

JSON_MERGE was added in 10.2.0 and JSON_MERGE_PATCH in 10.2.25, and the
loops have had this shape throughout.

Verified on 10.11 only (10.11.19-MariaDB-debug, commit 1dab253482d).
Other branches were not tested.

Note

Found while auditing the JSON functions against their documented
contracts. No existing test merges an object argument that breaks
before its first value is complete against an argument with no keys,
which is why it has not shown up.