[MDEV-12420] pcre stack overflow on match Created: 2017-03-31  Updated: 2017-07-20  Due: 2017-05-05  Resolved: 2017-05-16

Status: Closed
Project: MariaDB Server
Component/s: Server
Affects Version/s: 10.0.30, 10.1.22, 10.2.5
Fix Version/s: 10.1.24, 10.0.31, 10.2.7

Type: Bug Priority: Critical
Reporter: Don Lindsay Assignee: Sergei Golubchik
Resolution: Fixed Votes: 0
Labels: contribution, foundation, patch, security
Environment:

Fedora 25 x86_64

Attachments: File stackoverflow_query.sql    
Issue Links:
Duplicate
duplicates MDEV-13356 10.0.29 crashes on REGEXP with "(.|x)... Closed

 Description   

The attached table-free query causes a seg fault in 10.2.4, 10.1.21, and 10.0.21 (and presumably all the others). Setting max_recursive_iterations to various values didn't turn that into a polite error, but according to the documenation, it should have.

NOTE: setting 'thread_stack=800k' fixes this specific test, so it's definitely a stack issue. Examining a dumpfile showed many recursions of pcre 'match'.

 Comments   
Comment by Don Lindsay [ 2017-04-08 ]

Also segfaults MariaDB-server-10.2.5 (regardless of the setting of max_recursive_iterations, ie the same bug)

Comment by Daniel Black [ 2017-04-08 ]

max_recursive_iterations is for CTE - its not a setting that affects regexes

Ubuntu-17.04: libpcre3:amd64 2:8.39-3

....
 
#536 0x00007fad28da9aee in match (eptr=<optimized out>, ecode=0x7facbc0160a2 "y", 
    mstart=0x7facbc0160c8 "Alpha,Bravo,Charlie,Delta,Echo,Foxtrot,StrataCentral,Golf,Hotel,India,Juliet,Kilo,Lima,Mike,StrataL3,November,Oscar,StrataL2,Sand,P3,P4SwitchTest,Arsys,Poppa,ExtensionMgr,Arp,Quebec,Romeo,StrataApiV2,"..., offset_top=2, md=0x7fad251ef4c0, eptrb=0x0, 
    rdepth=<optimized out>) at pcre_exec.c:2061
 
#537 0x00007fad28d9cdd6 in match (eptr=<optimized out>, ecode=0x7facbc01604c "\222\205", 
    mstart=0x7facbc0160c8 "Alpha,Bravo,Charlie,Delta,Echo,Foxtrot,StrataCentral,Golf,Hotel,India,Juliet,Kilo,Lima,Mike,StrataL3,November,Oscar,StrataL2,Sand,P3,P4SwitchTest,Arsys,Poppa,ExtensionMgr,Arp,Quebec,Romeo,StrataApiV2,"..., offset_top=2, md=0x7fad251ef4c0, eptrb=0x0, 
    rdepth=<optimized out>) at pcre_exec.c:1878
 
#538 0x00007fad28dad6a1 in pcre_exec (argument_re=0x7facbc016008, extra_data=<optimized out>, subject=<optimized out>, length=1288, 
    start_offset=0, options=0, offsets=0x7facbc0103d4, offsetcount=0) at pcre_exec.c:6936
 
---Type <return> to continue, or q <return> to quit---
 
#539 0x000000000078f269 in Regexp_processor_pcre::pcre_exec_with_warn (code=<optimized out>, extra=<optimized out>, 
    subject=0x7facbc0160c8 "Alpha,Bravo,Charlie,Delta,Echo,Foxtrot,StrataCentral,Golf,Hotel,India,Juliet,Kilo,Lima,Mike,StrataL3,November,Oscar,StrataL2,Sand,P3,P4SwitchTest,Arsys,Poppa,ExtensionMgr,Arp,Quebec,Romeo,StrataApiV2,"..., length=2, startoffset=0, options=0, 
    ovector=0x7facbc0103d4, ovecsize=0, this=<optimized out>) at /home/build/sql/item_cmpfunc.cc:5486
 
#540 Regexp_processor_pcre::exec (this=0x7facbc010390, str=<optimized out>, offset=0, n_result_offsets_to_convert=0)
 
    at /home/build/sql/item_cmpfunc.cc:5508
 
#541 0x000000000078f3f5 in Regexp_processor_pcre::exec (this=0x7facbc010390, item=0x7facbc010010, offset=0, n_result_offsets_to_convert=0)
 
    at /home/build/sql/item_cmpfunc.cc:5538
 
#542 0x000000000078f686 in Item_func_regex::val_int (this=0x7facbc0102d0) at /home/build/sql/item_cmpfunc.cc:5580
 
#543 0x00000000007ad85a in eval_const_cond (cond=0x7facbc0162df) at /home/build/sql/item_func.cc:79
 
#544 0x000000000061bc34 in Item::remove_eq_conds (this=0x7facbc0102d0, thd=<optimized out>, cond_value=0x7facbc010878, 
    top_level_arg=<optimized out>) at /home/build/sql/sql_select.cc:15532
 
#545 0x00000000005fd5e7 in optimize_cond (join=0x7facbc010570, conds=0x7facbc0102d0, join_list=<optimized out>, 
    ignore_on_conds=<error reading variable: access outside bounds of object referenced via synthetic pointer>, cond_value=0x7facbc010878, 
    cond_equal=<optimized out>, flags=1) at /home/build/sql/sql_select.cc:15088
 
#546 JOIN::optimize_inner (this=0x7facbc010570) at /home/build/sql/sql_select.cc:1309
 
#547 0x00000000005fcdff in JOIN::optimize (this=0x7facbc010570) at /home/build/sql/sql_select.cc:1084
 
#548 0x00000000005faed5 in mysql_select (thd=<optimized out>, tables=<optimized out>, wild_num=<optimized out>, fields=..., 
    conds=<optimized out>, og_num=<optimized out>, order=<optimized out>, group=<optimized out>, having=<optimized out>, 
    proc_param=<optimized out>, select_options=<optimized out>, result=0x7fad251ef4c0, unit=<optimized out>, select_lex=0x7facbc004a78)
 
    at /home/build/sql/sql_select.cc:3644
 
 
 
#549 0x00000000005fad0f in handle_select (thd=0x7facbc0009a8, lex=0x7facbc004280, result=0x7facbc010550, 
    setup_tables_done_option=<optimized out>) at /home/build/sql/sql_select.cc:361
 
#550 0x00000000005dd810 in execute_sqlcom_select (thd=0x7facbc0009a8, all_tables=<optimized out>) at /home/build/sql/sql_parse.cc:6431
 
#551 0x00000000005d68ad in mysql_execute_command (thd=<optimized out>) at /home/build/sql/sql_parse.cc:3448
 
#552 0x00000000005d44d8 in mysql_parse (thd=0x7facbc0009a8, 
    rawbuf=0x7facbc00ef30 "SELECT 1 FROM dual WHERE ('Alpha,Bravo,Charlie,Delta,Echo,Foxtrot,StrataCentral,Golf,Hotel,India,Juliet,Kilo,Lima,Mike,StrataL3,November,Oscar,StrataL2,Sand,P3,P4SwitchTest,Arsys,Poppa,ExtensionMgr,Ar"..., length=<optimized out>, parser_state=0x7fad251f1400, 
---Type <return> to continue, or q <return> to quit---
 
    is_com_multi=<optimized out>, is_next_command=<error reading variable: access outside bounds of object referenced via synthetic pointer>)
 
    at /home/build/sql/sql_parse.cc:7874
 
#553 0x00000000005d23c7 in dispatch_command (command=<optimized out>, thd=0x7facbc0009a8, 
    packet=0x7facbc00ef30 "SELECT 1 FROM dual WHERE ('Alpha,Bravo,Charlie,Delta,Echo,Foxtrot,StrataCentral,Golf,Hotel,India,Juliet,Kilo,Lima,Mike,StrataL3,November,Oscar,StrataL2,Sand,P3,P4SwitchTest,Arsys,Poppa,ExtensionMgr,Ar"..., packet_length=<optimized out>, 
    is_com_multi=<optimized out>, is_next_command=<optimized out>) at /home/build/sql/sql_parse.cc:1811
 
#554 0x00000000005d3471 in do_command (thd=0x7facbc0009a8) at /home/build/sql/sql_parse.cc:1361
 
#555 0x000000000069e75e in do_handle_one_connection (connect=<optimized out>) at /home/build/sql/sql_connect.cc:1354
 
#556 0x000000000069e3d3 in handle_one_connection (arg=0x3ea5a58) at /home/build/sql/sql_connect.cc:1260
 
#557 0x00007fad29b566da in start_thread (arg=0x7fad251f2700) at pthread_create.c:456
 
#558 0x00007fad27a3f17f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:105

Comment by Daniel Black [ 2017-04-08 ]

proof of concept fix in PR with extended warnings

100 for depth?

If we where to make depth dynamic in Item_func_regex::val_int and use (like check_stack_overrun) the concepts below:

m_pcre_extra.match_limit_recursion= (my_thread_stack_size - margin - used_stack(thd->thread_stack,(char*) &stack_used)) ) / 656;

656 could be determined programmatic (from pcretest) and perhaps an extra margin.

Comment by Daniel Black [ 2017-04-11 ]

serg do you want to go down the path of dynamically determining the pcre recursion limit based on available stack?

Comment by Sergei Golubchik [ 2017-04-17 ]

We've added stack size check to pcre (first to our copy, then the patch was merged upstream). May be pcre needs to call it in more places? I'll take a look.

Dynamically determining the pcre recursion limit is an interesting idea, thanks! I'll give it a try.

Comment by Daniel Black [ 2017-04-17 ]

Had a bit of a look. How far of using pcre2 (version 10.x). Has a more featured api

Comment by Daniel Black [ 2017-04-18 ]

> May be pcre needs to call it in more places?

I think you are right. pcre_stack_guard is only called from pcre_compile currently and not from match. There isn't an upstream bug for it. The version 2 api is more explicit calling it pcre2_set_compile_recursion_guard (which has a recursion value as an argument) however there isn't a match callback.

Added https://github.com/MariaDB/server/pull/356/commits/91908884347e69fe8cbc8a9a78c0f77bf5b4e1bc, the pcre_exec function wasn't returning the right value (returned ~20) however the non-exposed match function was. So its in an incomplete state now.

Generated at Thu Feb 08 07:57:35 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.