[MDEV-12035] DBMS always crashes calling json_get_item() with valid JSON Created: 2017-02-09  Updated: 2021-01-04  Resolved: 2021-01-02

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - Connect
Affects Version/s: 10.1.13, 10.1.21
Fix Version/s: N/A

Type: Bug Priority: Minor
Reporter: Andy Cooper Assignee: Olivier Bertrand
Resolution: Incomplete Votes: 0
Labels: need_feedback
Environment:

Windows 10



 Description   

Every time I invoke the SQL below, the DBMS crashes on Windows Seems to be an issue with the JSON parser, possibly related to nested arrays:

SQL to crash the server:

select json_get_item('{"yearPeriods":[{"startDay":1,"startMonth":1,"weekSchedule":[{"periods":[{"startMinutes":450,"profileId":1},{"startMinutes":1290,"profileId":2}],"day":"MONDAY"},{"periods":[{"startMinutes":450,"profileId":1},{"startMinutes":1290,"profileId":2}],"day":"TUESDAY"},{"periods":[{"startMinutes":450,"profileId":1},{"startMinutes":1290,"profileId":2}],"day":"WEDNESDAY"},{"periods":[{"startMinutes":450,"profileId":1},{"startMinutes":1290,"profileId":2}],"day":"THURSDAY"},{"periods":[{"startMinutes":450,"profileId":1},{"startMinutes":1290,"profileId":2}],"day":"FRIDAY"},{"periods":[{"startMinutes":450,"profileId":1},{"startMinutes":1290,"profileId":2}],"day":"SATURDAY"},{"periods":[{"startMinutes":450,"profileId":1},{"startMinutes":1290,"profileId":2}],"day":"SUNDAY"}]}]}',
							'yearPeriods:[0]:weekSchedule:[0]');

If I remove some of the internal day objects (e.g. FRIDAY through MONDAY) the call completes successfully. Also if I refactor out the internal periods array into attributes the call completes successfully; all other modifications (e.g. adding extra attributes) fails.

Initially seens on MariaDB 10.1.13, but currently producing on 10.1.21 as well.
Windows Crash dump below (from a variant of the above JSON):

Version: '10.1.21-MariaDB'  socket: ''  port: 3306  mariadb.org binary distribution
170209 11:52:40 [ERROR] mysqld got exception 0xc0000005 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
 
Server version: 10.1.21-MariaDB
key_buffer_size=134217728
read_buffer_size=131072
max_used_connections=1
max_threads=1001
thread_count=1
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 2311984 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x23471a95768
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...

ha_connect.dll!longjmp()[longjmp.asm:98]
ha_connect.dll!PlugSubAlloc()[plugutil.c:521]
ha_connect.dll!JSNX::ParseJpath()[jsonudf.cpp:252]
ha_connect.dll!json_get_item()[jsonudf.cpp:2832]
mysqld.exe!udf_handler::val_str()[item_func.cc:3689]
mysqld.exe!Item_func_udf_str::val_str()[item_func.cc:3876]
mysqld.exe!Item::save_in_field()[item.cc:6027]
mysqld.exe!sp_eval_expr()[sp_head.cc:454]
mysqld.exe!sp_rcontext::set_variable()[sp_rcontext.cc:378]
mysqld.exe!my_var_sp::set()[sql_class.cc:3895]
mysqld.exe!select_dumpvar::send_data()[sql_class.cc:3918]
mysqld.exe!JOIN::exec_inner()[sql_select.cc:2589]
mysqld.exe!JOIN::exec()[sql_select.cc:2511]
mysqld.exe!mysql_select()[sql_select.cc:3446]
mysqld.exe!handle_select()[sql_select.cc:384]
mysqld.exe!execute_sqlcom_select()[sql_parse.cc:5896]
mysqld.exe!mysql_execute_command()[sql_parse.cc:2971]
mysqld.exe!sp_instr_stmt::exec_core()[sp_head.cc:3213]
mysqld.exe!sp_lex_keeper::reset_lex_and_exec_core()[sp_head.cc:2977]
mysqld.exe!sp_instr_stmt::execute()[sp_head.cc:3127]
mysqld.exe!sp_head::execute()[sp_head.cc:1315]
mysqld.exe!sp_head::execute_procedure()[sp_head.cc:2102]
mysqld.exe!do_execute_sp()[sql_parse.cc:2407]
mysqld.exe!mysql_execute_command()[sql_parse.cc:5258]
mysqld.exe!mysql_parse()[sql_parse.cc:7324]
mysqld.exe!dispatch_command()[sql_parse.cc:1491]
mysqld.exe!do_command()[sql_parse.cc:1109]
mysqld.exe!threadpool_process_request()[threadpool_common.cc:271]
mysqld.exe!io_completion_callback()[threadpool_win.cc:568]
KERNEL32.DLL!TermsrvOpenRegEntry()
ntdll.dll!RtlReleasePebLock()
ntdll.dll!RtlReleaseSRWLockExclusive()
KERNEL32.DLL!BaseThreadInitThunk()
ntdll.dll!RtlUserThreadStart()

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x234a6a49860): select json_get_item( NAME_CONST('v_scheduleJson',_utf8'{\"yearPeriods\":[{\"startDay\":1,\"startMonth\":1,\"weekSchedule\":[{\"day\":\"MONDAY\",\"periods\":[{\"startMinutes\":450,\"profileId\":1},{\"startMinutes\":1290,\"profileId\":2}],\"DUMMY\":1},{\"day\":\"TUESDAY\",\"periods\":[{\"startMinutes\":450,\"profileId\":1},{\"startMinutes\":1290,\"profileId\":2}],\"DUMMY\":1},{\"day\":\"WEDNESDAY\",\"periods\":[{\"startMinutes\":450,\"profileId\":1},{\"startMinutes\":1290,\"profileId\":2}],\"DUMMY\":1},{\"day\":\"THURSDAY\",\"periods\":[{\"startMinutes\":450,\"profileId\":1},{\"startMinutes\":1290,\"profileId\":2}],\"DUMMY\":1},{\"day\":\"FRIDAY\",\"periods\":[{\"startMinutes\":450,\"profileId\":1},{\"startMinutes\":1290,\"profileId\":2}],\"DUMMY\":1},{\"day\":\"SATURDAY\",\"periods\":[{\"startMinutes\":450,\"profileId\":1},{\"startMinutes\":1290,\"profileId\":2}],\"DUMMY\":1},{\"day\":\"SUNDAY\",\"periods\":[{\"startMinutes\":450,\"profileId\":1},{\"startMinutes\":1290,\"profileId\":2}],\"DUMMY\":1}]}]}' COLLATE 'utf8_general_ci'), CONCAT('yearPeriods:[0]:weekSchedule:[', NAME_CONST('v_dayScheduleIndex',1),']')) into v_dayProfileSchedule
Connection ID (thread ID): 2
Status: NOT_KILLED
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=off
 
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.



 Comments   
Comment by Olivier Bertrand [ 2017-02-10 ]

Are you sure this is exactly the query that crashes the server? This is because on my machine the same query:

select json_get_item('{"yearPeriods":[{"startDay":1,"startMonth":1,"weekSchedule":[{"periods":[{"startMinutes":450,"profileId":1},{"startMinutes":1290,"profileId":2}],"day":"MONDAY"},{"periods":[{"startMinutes":450,"profileId":1},{"startMinutes":1290,"profileId":2}],"day":"TUESDAY"},{"periods":[{"startMinutes":450,"profileId":1},{"startMinutes":1290,"profileId":2}],"day":"WEDNESDAY"},{"periods":[{"startMinutes":450,"profileId":1},{"startMinutes":1290,"profileId":2}],"day":"THURSDAY"},{"periods":[{"startMinutes":450,"profileId":1},{"startMinutes":1290,"profileId":2}],"day":"FRIDAY"},{"periods":[{"startMinutes":450,"profileId":1},{"startMinutes":1290,"profileId":2}],"day":"SATURDAY"},{"periods":[{"startMinutes":450,"profileId":1},{"startMinutes":1290,"profileId":2}],"day":"SUNDAY"}]}]}','yearPeriods:[0]:weekSchedule:[0]') as Result;

is executed successfully returning:

Result
{"periods":[{"startMinutes":450,"profileId":1},{"startMinutes":1290,"profileId":2}],"day":"MONDAY"}

Looking into the code, this error seems to be caused by a memory sub-allocation failure not correctly handled by CONNECT. This will be fixed. However it is strange that this does not occur on my machine. The calculation of the required memory size should be the same unless there is a difference for 32bits/64bits machines (mine is 32bits)

Meanwhile you may have a more acurate needed memory calculation by saying that the string you pass as the first argument of the function is a JSON object. Instead of saying:

select json_get_item(1st_arg, 2nd_arg);

add the json_ alias for the first string argument:

select json_get_item(1st_arg json_, 2nd_arg);

Comment by Andy Cooper [ 2017-02-10 ]

Absolutely yes, that query crashed my server. I am running with a 64-bit Windows OS (Build 14393.693) and 64-bit MariaDB, though.

I can confirm that your json_ alias workaround does appear to work, which is good news thanks; it is nicer than my own workaround which involved using casting to and from jbin_ objects using json_object() and jbin_object() in order to call jbin_get_item().

Comment by Olivier Bertrand [ 2017-05-06 ]

The crash can also come from the longjmp which, I don't know why, crashes effectively once upon a time only in this specific function.
This is why, in the last distributed versions of MariaDB, I have removed all longjmp and replaced them by try/catch's.
Could you check this solves your problem so I can close this?
Thanks.

Comment by Olivier Bertrand [ 2021-01-04 ]

This is now outdated. With newer versions of Connect the final path should be specified as:

'$.yearPeriods[0].weekSchedule[0]'

... and it works on my Windows machine.

Generated at Thu Feb 08 07:54:36 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.