Details
-
Bug
-
Status: Open (View Workflow)
-
Minor
-
Resolution: Unresolved
-
10.6, 10.11, 11.4, 11.8, 12.3, 13.0
-
None
-
None
Description
JSON_ARRAYAGG is documented in the same words as GROUP_CONCAT, which it
derives from: "The maximum returned length in bytes is determined by the
group_concat_max_len server system variable." GROUP_CONCAT ends on that
limit exactly. JSON_ARRAYAGG ends two bytes past it.
CREATE TABLE t1 (s VARCHAR(20)); |
INSERT INTO t1 VALUES ('aaaaaaaaaa'), ('bbbbbbbbbb'); |
|
|
SET SESSION group_concat_max_len = 25; |
SELECT JSON_ARRAYAGG(s) AS v, LENGTH(JSON_ARRAYAGG(s)) AS len FROM t1; |
v len
|
["aaaaaaaaaa","bbbbbbbbbb"] 27 |
The limit is 25 and 27 bytes come back. No warning is raised: the
elements ended exactly on the limit, so nothing was cut, and the two
brackets went on afterwards.
Where a cut does happen the answer is still over the limit, and there
the warning is raised - for the overshoot that is left, not for the one
the warning is about:
SET SESSION group_concat_max_len = 12; |
SELECT JSON_ARRAYAGG(s) AS v, LENGTH(JSON_ARRAYAGG(s)) AS len FROM t1; |
v len
|
["aaaaaaaaaa"] 14 |
Warnings:
|
Warning 1260 Row 2 was cut by JSON_ARRAYAGG() |
GROUP_CONCAT over the same rows and the same limit ends on 12:
SET SESSION group_concat_max_len = 12; |
SELECT GROUP_CONCAT(s) AS v, LENGTH(GROUP_CONCAT(s)) AS len FROM t1; |
v len
|
aaaaaaaaaa,b 12
|
Warnings:
|
Warning 1260 Row 2 was cut by GROUP_CONCAT() |
How to repeat
CREATE TABLE t1 (s VARCHAR(20)); |
INSERT INTO t1 VALUES ('aaaaaaaaaa'), ('bbbbbbbbbb'); |
|
|
SET SESSION group_concat_max_len = 25; |
SELECT LENGTH(JSON_ARRAYAGG(s)) AS len, @@group_concat_max_len AS cap |
FROM t1; |
len cap
|
27 25
|
|
|
SET SESSION group_concat_max_len = 12; |
SELECT LENGTH(JSON_ARRAYAGG(s)) AS len, @@group_concat_max_len AS cap |
FROM t1; |
len cap
|
14 12
|
|
|
DROP TABLE t1; |
In a character set that writes no character in one byte the overshoot
is wider, the brackets being written in the set of the result:
CREATE TABLE t2 (a VARCHAR(20)) CHARACTER SET ucs2; |
INSERT INTO t2 VALUES ('aaaa'), ('bbbb'); |
|
|
SET SESSION group_concat_max_len = 26; |
SELECT LENGTH(JSON_ARRAYAGG(a ORDER BY a)) AS len, |
@@group_concat_max_len AS cap FROM t2; |
len cap
|
30 26
|
|
|
DROP TABLE t2; |
Four bytes over, and again with no warning.
Root cause
Item_func_group_concat::dump_leaf_key() tests the accumulated result
once per row, in sql/item_sum.cc:
uint max_length= table->in_use->gconcat_max_len();
|
...
|
/* stop if length of result more than max_length */ |
if (result->length() > max_length) |
{
|
...
|
item->cut_max_length(result, old_length, max_length);
|
For GROUP_CONCAT what has been accumulated is the whole answer, so that
test is exact. JSON_ARRAYAGG's brackets are not in that buffer: they
are written afterwards, in Item_func_json_arrayagg::val_str(), when the
group is asked for. So the row test is made against a body that is not
yet the answer, and the answer is longer than the body it was tested by
- by one bracket at each end, each as wide as the narrowest character of
the result's character set.
Where the body ends exactly on the limit the test never fires, so there
is no cut and no warning, and the answer is over the limit in silence.
Relationship to MDEV-39817
MDEV-39817 (commit 59141cbd2c4, 2026-06-01) fixed a different fault in
the same cut and added a test whose recorded answer is over the limit:
set session group_concat_max_len = 9; |
select json_arrayagg(c order by c) from t; |
json_arrayagg(c order by c) |
["ab","cd"] |
Eleven bytes against a limit of nine. That commit also widened the
width the item declares, in Item_func_json_arrayagg::fix_fields():
/* account for opening and closing brackets */ |
max_length= MY_MIN(max_length + 2*collation.collation->mbminlen,
|
UINT_MAX32);
|
which is visible to a caller:
SET SESSION group_concat_max_len = 9; |
CREATE TABLE x AS SELECT JSON_ARRAYAGG(c) AS v FROM t; |
SHOW CREATE TABLE x; |
`v` varchar(11) |
|
|
CREATE TABLE y AS SELECT GROUP_CONCAT(c) AS v FROM t; |
SHOW CREATE TABLE y; |
`v` varchar(9) |
So the declared width already allows for the answer running past the
limit. Whether that was meant to settle what the limit covers, or only
to keep a materialised column from truncating what the function was
already returning, the commit does not say; its message is about the
off-by-one it fixes and says nothing about the limit.
One case where the limit cannot be kept
group_concat_max_len may be set as low as 4, and an empty array is as
short as this function goes. In a character set of four bytes to the
character that is eight bytes:
CREATE TABLE t3 (a VARCHAR(20)) CHARACTER SET utf32; |
INSERT INTO t3 VALUES ('aa'), ('bb'); |
SET SESSION group_concat_max_len = 4; |
SELECT LENGTH(JSON_ARRAYAGG(a)) AS len FROM t3; |
len
|
8
|
DROP TABLE t3; |
There is nothing left to cut, so any fix has this floor: what comes back
is the limit or the two brackets, whichever is longer. The declared
width quoted above already covers it.
Affected versions
JSON_ARRAYAGG was added in 10.5.0. The limit began to be applied to it
in 10.5.4 (MDEV-22844), and the row test has been made against the
bracket-free body ever since.
Verified on 10.11 only (10.11.19-MariaDB-debug, commit 1dab253482d,
which contains MDEV-39817). Other branches were not tested.
Note
Found while auditing the JSON functions against their documented
contracts. No test in the suite compares LENGTH() of either JSON
aggregate against @@group_concat_max_len, which is why it has not shown
up: func_json_agg_limits records the bytes at many caps but never
asserts them against the cap.
JSON_OBJECTAGG has the same shape one byte wide, its closing brace being
written in val_str() as well.
Attachments
Issue Links
- split from
-
MDEV-40642 Project "JSON Phoenix" (placeholder)
-
- Open
-