[MDEV-32854] Make JSON_DEPTH_LIMIT configurable Created: 2023-11-21  Updated: 2024-02-07

Status: Open
Project: MariaDB Server
Component/s: JSON
Fix Version/s: 11.5

Type: New Feature Priority: Critical
Reporter: Edward Stoever Assignee: Rucha Deodhar
Resolution: Unresolved Votes: 2
Labels: Compatibility


 Description   

The current JSON_DEPTH_LIMIT is 32 as defined in include/json_lib.h
Source code:
https://github.com/MariaDB/server/blob/22ffe4dd227451ebd0ef842850ef57b81aa8f930/include/json_lib.h#L10

This limit is not documented.

Customer requests to make this limit configurable with a global variable.



 Comments   
Comment by Rucha Deodhar [ 2024-01-15 ]

ralf.gebhardt sanja looks like there is hard coded limit of 100
And AFAIK the standards don't mention the limit explicitly.
I think it is possible to have user change the limit (like adding a server variable ) but we would still like there to be a upper limit right ?

Comment by Miguel Angel Carnero [ 2024-01-24 ]

A doubt: Is that hard coded limit of 100 https://github.com/mysql/mysql-server/blob/87307d4ddd88405117e3f1e51323836d57ab1f57/sql-common/json_syntax_check.cc#L82 the new limit? or is it a different limit? The limit we saw and was limiting our product is limit of 32: https://github.com/MariaDB/server/blob/8ad1e26b1bafa4ed9928306efc10c047f2274108/include/json_lib.h#L10

Thanks!

Comment by Rucha Deodhar [ 2024-01-24 ]

macarnero Hi ! My comment was a reply to an internal comment about MySQL, confirming the limit of 100 for MySQL, 32 for MariaDB.

Comment by Miguel Angel Carnero [ 2024-02-06 ]

We need to keep using MariaDB. Changing this to Critical means this request is under study? Thanks

Comment by Oleksandr Byelkin [ 2024-02-06 ]

The result of discussion on runtime meeting:

  1. make dynamic array based on mem_root (to be discussed how exactly)
  2. replace

    uint array_counters[JSON_DEPTH_LIMIT]
    

    with JSON function class member of above dynamic array

  3. Initialize the array during prepare and cleanup (if needed, I think it will not be needed) on cleanup()
  4. Make debug binarias test which emulate EOM (limit in debugging size of the dynamic array y debugging time flag) to be sure that nice error returned (like no enough memory to process so deep JSON (better on JSON of 1000+ of depth)

so advaltages:

  • No need to setup the depth (automatic)
  • No much memory taken from the thread stack
  • No memory overhead for them who do not use JSON
Comment by Oleksandr Byelkin [ 2024-02-06 ]

For early versions we can think of increasing limit to 100 or 200 and moving the array to mem_root (should be easy and safe change)

Comment by Miguel Angel Carnero [ 2024-02-06 ]

Could you please confirm against which version are you gonna apply this enhancement? e.g. 10.6? Thanks.

Comment by Ralf Gebhardt [ 2024-02-06 ]

Hi macarnero, an enhancement to MariaDB Community Server only can be done for new release series. This enhancement might be in the 11.5 preview, if the enhancement is finished in time for this preview release. The enhancement will then be added to 11.5 RC, if internal and community testing does not show issues.

For MariaDB Enterprise Server we have the option to backport changes. You can contact me directly if you have more questions.

Generated at Thu Feb 08 10:34:32 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.