Details

    Description

      The attached table-free query causes a seg fault in 10.2.4, 10.1.21, and 10.0.21 (and presumably all the others). Setting max_recursive_iterations to various values didn't turn that into a polite error, but according to the documenation, it should have.

      NOTE: setting 'thread_stack=800k' fixes this specific test, so it's definitely a stack issue. Examining a dumpfile showed many recursions of pcre 'match'.

      Attachments

        Issue Links

          Activity

            donlindsay Don Lindsay added a comment -

            Also segfaults MariaDB-server-10.2.5 (regardless of the setting of max_recursive_iterations, ie the same bug)

            donlindsay Don Lindsay added a comment - Also segfaults MariaDB-server-10.2.5 (regardless of the setting of max_recursive_iterations, ie the same bug)
            danblack Daniel Black added a comment -

            max_recursive_iterations is for CTE - its not a setting that affects regexes

            Ubuntu-17.04: libpcre3:amd64 2:8.39-3

            ....
            #536 0x00007fad28da9aee in match (eptr=<optimized out>, ecode=0x7facbc0160a2 "y", 
                mstart=0x7facbc0160c8 "Alpha,Bravo,Charlie,Delta,Echo,Foxtrot,StrataCentral,Golf,Hotel,India,Juliet,Kilo,Lima,Mike,StrataL3,November,Oscar,StrataL2,Sand,P3,P4SwitchTest,Arsys,Poppa,ExtensionMgr,Arp,Quebec,Romeo,StrataApiV2,"..., offset_top=2, md=0x7fad251ef4c0, eptrb=0x0, 
                rdepth=<optimized out>) at pcre_exec.c:2061
            #537 0x00007fad28d9cdd6 in match (eptr=<optimized out>, ecode=0x7facbc01604c "\222\205", 
                mstart=0x7facbc0160c8 "Alpha,Bravo,Charlie,Delta,Echo,Foxtrot,StrataCentral,Golf,Hotel,India,Juliet,Kilo,Lima,Mike,StrataL3,November,Oscar,StrataL2,Sand,P3,P4SwitchTest,Arsys,Poppa,ExtensionMgr,Arp,Quebec,Romeo,StrataApiV2,"..., offset_top=2, md=0x7fad251ef4c0, eptrb=0x0, 
                rdepth=<optimized out>) at pcre_exec.c:1878
            #538 0x00007fad28dad6a1 in pcre_exec (argument_re=0x7facbc016008, extra_data=<optimized out>, subject=<optimized out>, length=1288, 
                start_offset=0, options=0, offsets=0x7facbc0103d4, offsetcount=0) at pcre_exec.c:6936
            ---Type <return> to continue, or q <return> to quit---
            #539 0x000000000078f269 in Regexp_processor_pcre::pcre_exec_with_warn (code=<optimized out>, extra=<optimized out>, 
                subject=0x7facbc0160c8 "Alpha,Bravo,Charlie,Delta,Echo,Foxtrot,StrataCentral,Golf,Hotel,India,Juliet,Kilo,Lima,Mike,StrataL3,November,Oscar,StrataL2,Sand,P3,P4SwitchTest,Arsys,Poppa,ExtensionMgr,Arp,Quebec,Romeo,StrataApiV2,"..., length=2, startoffset=0, options=0, 
                ovector=0x7facbc0103d4, ovecsize=0, this=<optimized out>) at /home/build/sql/item_cmpfunc.cc:5486
            #540 Regexp_processor_pcre::exec (this=0x7facbc010390, str=<optimized out>, offset=0, n_result_offsets_to_convert=0)
                at /home/build/sql/item_cmpfunc.cc:5508
            #541 0x000000000078f3f5 in Regexp_processor_pcre::exec (this=0x7facbc010390, item=0x7facbc010010, offset=0, n_result_offsets_to_convert=0)
                at /home/build/sql/item_cmpfunc.cc:5538
            #542 0x000000000078f686 in Item_func_regex::val_int (this=0x7facbc0102d0) at /home/build/sql/item_cmpfunc.cc:5580
            #543 0x00000000007ad85a in eval_const_cond (cond=0x7facbc0162df) at /home/build/sql/item_func.cc:79
            #544 0x000000000061bc34 in Item::remove_eq_conds (this=0x7facbc0102d0, thd=<optimized out>, cond_value=0x7facbc010878, 
                top_level_arg=<optimized out>) at /home/build/sql/sql_select.cc:15532
            #545 0x00000000005fd5e7 in optimize_cond (join=0x7facbc010570, conds=0x7facbc0102d0, join_list=<optimized out>, 
                ignore_on_conds=<error reading variable: access outside bounds of object referenced via synthetic pointer>, cond_value=0x7facbc010878, 
                cond_equal=<optimized out>, flags=1) at /home/build/sql/sql_select.cc:15088
            #546 JOIN::optimize_inner (this=0x7facbc010570) at /home/build/sql/sql_select.cc:1309
            #547 0x00000000005fcdff in JOIN::optimize (this=0x7facbc010570) at /home/build/sql/sql_select.cc:1084
            #548 0x00000000005faed5 in mysql_select (thd=<optimized out>, tables=<optimized out>, wild_num=<optimized out>, fields=..., 
                conds=<optimized out>, og_num=<optimized out>, order=<optimized out>, group=<optimized out>, having=<optimized out>, 
                proc_param=<optimized out>, select_options=<optimized out>, result=0x7fad251ef4c0, unit=<optimized out>, select_lex=0x7facbc004a78)
                at /home/build/sql/sql_select.cc:3644
             
            #549 0x00000000005fad0f in handle_select (thd=0x7facbc0009a8, lex=0x7facbc004280, result=0x7facbc010550, 
                setup_tables_done_option=<optimized out>) at /home/build/sql/sql_select.cc:361
            #550 0x00000000005dd810 in execute_sqlcom_select (thd=0x7facbc0009a8, all_tables=<optimized out>) at /home/build/sql/sql_parse.cc:6431
            #551 0x00000000005d68ad in mysql_execute_command (thd=<optimized out>) at /home/build/sql/sql_parse.cc:3448
            #552 0x00000000005d44d8 in mysql_parse (thd=0x7facbc0009a8, 
                rawbuf=0x7facbc00ef30 "SELECT 1 FROM dual WHERE ('Alpha,Bravo,Charlie,Delta,Echo,Foxtrot,StrataCentral,Golf,Hotel,India,Juliet,Kilo,Lima,Mike,StrataL3,November,Oscar,StrataL2,Sand,P3,P4SwitchTest,Arsys,Poppa,ExtensionMgr,Ar"..., length=<optimized out>, parser_state=0x7fad251f1400, 
            ---Type <return> to continue, or q <return> to quit---
                is_com_multi=<optimized out>, is_next_command=<error reading variable: access outside bounds of object referenced via synthetic pointer>)
                at /home/build/sql/sql_parse.cc:7874
            #553 0x00000000005d23c7 in dispatch_command (command=<optimized out>, thd=0x7facbc0009a8, 
                packet=0x7facbc00ef30 "SELECT 1 FROM dual WHERE ('Alpha,Bravo,Charlie,Delta,Echo,Foxtrot,StrataCentral,Golf,Hotel,India,Juliet,Kilo,Lima,Mike,StrataL3,November,Oscar,StrataL2,Sand,P3,P4SwitchTest,Arsys,Poppa,ExtensionMgr,Ar"..., packet_length=<optimized out>, 
                is_com_multi=<optimized out>, is_next_command=<optimized out>) at /home/build/sql/sql_parse.cc:1811
            #554 0x00000000005d3471 in do_command (thd=0x7facbc0009a8) at /home/build/sql/sql_parse.cc:1361
            #555 0x000000000069e75e in do_handle_one_connection (connect=<optimized out>) at /home/build/sql/sql_connect.cc:1354
            #556 0x000000000069e3d3 in handle_one_connection (arg=0x3ea5a58) at /home/build/sql/sql_connect.cc:1260
            #557 0x00007fad29b566da in start_thread (arg=0x7fad251f2700) at pthread_create.c:456
            #558 0x00007fad27a3f17f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:105
            

            danblack Daniel Black added a comment - max_recursive_iterations is for CTE - its not a setting that affects regexes Ubuntu-17.04: libpcre3:amd64 2:8.39-3 .... #536 0x00007fad28da9aee in match (eptr=<optimized out>, ecode=0x7facbc0160a2 "y", mstart=0x7facbc0160c8 "Alpha,Bravo,Charlie,Delta,Echo,Foxtrot,StrataCentral,Golf,Hotel,India,Juliet,Kilo,Lima,Mike,StrataL3,November,Oscar,StrataL2,Sand,P3,P4SwitchTest,Arsys,Poppa,ExtensionMgr,Arp,Quebec,Romeo,StrataApiV2,"..., offset_top=2, md=0x7fad251ef4c0, eptrb=0x0, rdepth=<optimized out>) at pcre_exec.c:2061 #537 0x00007fad28d9cdd6 in match (eptr=<optimized out>, ecode=0x7facbc01604c "\222\205", mstart=0x7facbc0160c8 "Alpha,Bravo,Charlie,Delta,Echo,Foxtrot,StrataCentral,Golf,Hotel,India,Juliet,Kilo,Lima,Mike,StrataL3,November,Oscar,StrataL2,Sand,P3,P4SwitchTest,Arsys,Poppa,ExtensionMgr,Arp,Quebec,Romeo,StrataApiV2,"..., offset_top=2, md=0x7fad251ef4c0, eptrb=0x0, rdepth=<optimized out>) at pcre_exec.c:1878 #538 0x00007fad28dad6a1 in pcre_exec (argument_re=0x7facbc016008, extra_data=<optimized out>, subject=<optimized out>, length=1288, start_offset=0, options=0, offsets=0x7facbc0103d4, offsetcount=0) at pcre_exec.c:6936 ---Type <return> to continue, or q <return> to quit--- #539 0x000000000078f269 in Regexp_processor_pcre::pcre_exec_with_warn (code=<optimized out>, extra=<optimized out>, subject=0x7facbc0160c8 "Alpha,Bravo,Charlie,Delta,Echo,Foxtrot,StrataCentral,Golf,Hotel,India,Juliet,Kilo,Lima,Mike,StrataL3,November,Oscar,StrataL2,Sand,P3,P4SwitchTest,Arsys,Poppa,ExtensionMgr,Arp,Quebec,Romeo,StrataApiV2,"..., length=2, startoffset=0, options=0, ovector=0x7facbc0103d4, ovecsize=0, this=<optimized out>) at /home/build/sql/item_cmpfunc.cc:5486 #540 Regexp_processor_pcre::exec (this=0x7facbc010390, str=<optimized out>, offset=0, n_result_offsets_to_convert=0) at /home/build/sql/item_cmpfunc.cc:5508 #541 0x000000000078f3f5 in Regexp_processor_pcre::exec (this=0x7facbc010390, item=0x7facbc010010, offset=0, n_result_offsets_to_convert=0) at /home/build/sql/item_cmpfunc.cc:5538 #542 0x000000000078f686 in Item_func_regex::val_int (this=0x7facbc0102d0) at /home/build/sql/item_cmpfunc.cc:5580 #543 0x00000000007ad85a in eval_const_cond (cond=0x7facbc0162df) at /home/build/sql/item_func.cc:79 #544 0x000000000061bc34 in Item::remove_eq_conds (this=0x7facbc0102d0, thd=<optimized out>, cond_value=0x7facbc010878, top_level_arg=<optimized out>) at /home/build/sql/sql_select.cc:15532 #545 0x00000000005fd5e7 in optimize_cond (join=0x7facbc010570, conds=0x7facbc0102d0, join_list=<optimized out>, ignore_on_conds=<error reading variable: access outside bounds of object referenced via synthetic pointer>, cond_value=0x7facbc010878, cond_equal=<optimized out>, flags=1) at /home/build/sql/sql_select.cc:15088 #546 JOIN::optimize_inner (this=0x7facbc010570) at /home/build/sql/sql_select.cc:1309 #547 0x00000000005fcdff in JOIN::optimize (this=0x7facbc010570) at /home/build/sql/sql_select.cc:1084 #548 0x00000000005faed5 in mysql_select (thd=<optimized out>, tables=<optimized out>, wild_num=<optimized out>, fields=..., conds=<optimized out>, og_num=<optimized out>, order=<optimized out>, group=<optimized out>, having=<optimized out>, proc_param=<optimized out>, select_options=<optimized out>, result=0x7fad251ef4c0, unit=<optimized out>, select_lex=0x7facbc004a78) at /home/build/sql/sql_select.cc:3644   #549 0x00000000005fad0f in handle_select (thd=0x7facbc0009a8, lex=0x7facbc004280, result=0x7facbc010550, setup_tables_done_option=<optimized out>) at /home/build/sql/sql_select.cc:361 #550 0x00000000005dd810 in execute_sqlcom_select (thd=0x7facbc0009a8, all_tables=<optimized out>) at /home/build/sql/sql_parse.cc:6431 #551 0x00000000005d68ad in mysql_execute_command (thd=<optimized out>) at /home/build/sql/sql_parse.cc:3448 #552 0x00000000005d44d8 in mysql_parse (thd=0x7facbc0009a8, rawbuf=0x7facbc00ef30 "SELECT 1 FROM dual WHERE ('Alpha,Bravo,Charlie,Delta,Echo,Foxtrot,StrataCentral,Golf,Hotel,India,Juliet,Kilo,Lima,Mike,StrataL3,November,Oscar,StrataL2,Sand,P3,P4SwitchTest,Arsys,Poppa,ExtensionMgr,Ar"..., length=<optimized out>, parser_state=0x7fad251f1400, ---Type <return> to continue, or q <return> to quit--- is_com_multi=<optimized out>, is_next_command=<error reading variable: access outside bounds of object referenced via synthetic pointer>) at /home/build/sql/sql_parse.cc:7874 #553 0x00000000005d23c7 in dispatch_command (command=<optimized out>, thd=0x7facbc0009a8, packet=0x7facbc00ef30 "SELECT 1 FROM dual WHERE ('Alpha,Bravo,Charlie,Delta,Echo,Foxtrot,StrataCentral,Golf,Hotel,India,Juliet,Kilo,Lima,Mike,StrataL3,November,Oscar,StrataL2,Sand,P3,P4SwitchTest,Arsys,Poppa,ExtensionMgr,Ar"..., packet_length=<optimized out>, is_com_multi=<optimized out>, is_next_command=<optimized out>) at /home/build/sql/sql_parse.cc:1811 #554 0x00000000005d3471 in do_command (thd=0x7facbc0009a8) at /home/build/sql/sql_parse.cc:1361 #555 0x000000000069e75e in do_handle_one_connection (connect=<optimized out>) at /home/build/sql/sql_connect.cc:1354 #556 0x000000000069e3d3 in handle_one_connection (arg=0x3ea5a58) at /home/build/sql/sql_connect.cc:1260 #557 0x00007fad29b566da in start_thread (arg=0x7fad251f2700) at pthread_create.c:456 #558 0x00007fad27a3f17f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:105
            danblack Daniel Black added a comment - - edited

            proof of concept fix in PR with extended warnings

            100 for depth?

            If we where to make depth dynamic in Item_func_regex::val_int and use (like check_stack_overrun) the concepts below:

            m_pcre_extra.match_limit_recursion= (my_thread_stack_size - margin - used_stack(thd->thread_stack,(char*) &stack_used)) ) / 656;
            

            656 could be determined programmatic (from pcretest) and perhaps an extra margin.

            danblack Daniel Black added a comment - - edited proof of concept fix in PR with extended warnings 100 for depth? Above test case shows 537 match functions before the stack overflowed. http://pcre.org/original/doc/html/pcrestack.html shows about 500-656 bytes per stack frame https://mariadb.com/kb/en/mariadb/server-system-variables/#thread_stack a default of 297984 seems to correspond to about 600 frames. The rest of the server needs a few stack frames too. If we where to make depth dynamic in Item_func_regex::val_int and use (like check_stack_overrun) the concepts below: m_pcre_extra.match_limit_recursion= (my_thread_stack_size - margin - used_stack(thd->thread_stack,( char *) &stack_used)) ) / 656; 656 could be determined programmatic (from pcretest) and perhaps an extra margin.
            danblack Daniel Black added a comment -

            serg do you want to go down the path of dynamically determining the pcre recursion limit based on available stack?

            danblack Daniel Black added a comment - serg do you want to go down the path of dynamically determining the pcre recursion limit based on available stack?

            We've added stack size check to pcre (first to our copy, then the patch was merged upstream). May be pcre needs to call it in more places? I'll take a look.

            Dynamically determining the pcre recursion limit is an interesting idea, thanks! I'll give it a try.

            serg Sergei Golubchik added a comment - We've added stack size check to pcre (first to our copy, then the patch was merged upstream). May be pcre needs to call it in more places? I'll take a look. Dynamically determining the pcre recursion limit is an interesting idea, thanks! I'll give it a try.
            danblack Daniel Black added a comment -

            Had a bit of a look. How far of using pcre2 (version 10.x). Has a more featured api

            danblack Daniel Black added a comment - Had a bit of a look. How far of using pcre2 (version 10.x). Has a more featured api
            danblack Daniel Black added a comment -

            > May be pcre needs to call it in more places?

            I think you are right. pcre_stack_guard is only called from pcre_compile currently and not from match. There isn't an upstream bug for it. The version 2 api is more explicit calling it pcre2_set_compile_recursion_guard (which has a recursion value as an argument) however there isn't a match callback.

            Added https://github.com/MariaDB/server/pull/356/commits/91908884347e69fe8cbc8a9a78c0f77bf5b4e1bc, the pcre_exec function wasn't returning the right value (returned ~20) however the non-exposed match function was. So its in an incomplete state now.

            danblack Daniel Black added a comment - > May be pcre needs to call it in more places? I think you are right. pcre_stack_guard is only called from pcre_compile currently and not from match. There isn't an upstream bug for it. The version 2 api is more explicit calling it pcre2_set_compile_recursion_guard (which has a recursion value as an argument) however there isn't a match callback. Added https://github.com/MariaDB/server/pull/356/commits/91908884347e69fe8cbc8a9a78c0f77bf5b4e1bc , the pcre_exec function wasn't returning the right value (returned ~20) however the non-exposed match function was. So its in an incomplete state now.

            People

              serg Sergei Golubchik
              donlindsay Don Lindsay
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.