Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-13173

An RLIKE that previously worked on 10.0 now returns "Got error 'pcre_exec: recursion limit of 100 exceeded' from regexp"

Details

    Description

      The PCRE recursion limit has apparently decreased in 10.1. There is no way to increase this limit through configuration.

      eg:

      MariaDB [10_1_sandbox]> SELECT CONCAT(REPEAT('100,',98),'101') RLIKE '^(([1-9][0-9]*),)*[1-9][0-9]*$' TEST;
      		 
      +------+
      		 
      | TEST |
      		 
      +------+
      		 
      |    1 |
      		 
      +------+
      		 
      1 row in set (0.02 sec)
      		 
       
      		 
      MariaDB [10_1_sandbox]> SELECT CONCAT(REPEAT('100,',99),'101') RLIKE '^(([1-9][0-9]*),)*[1-9][0-9]*$' TEST;
      		 
      +------+
      		 
      | TEST |
      		 
      +------+
      		 
      |    0 |
      		 
      +------+
      		 
      1 row in set, 1 warning (0.02 sec)
      		 
       
      		 
      MariaDB [sandbox]> show warnings;
      		 
      +---------+------+--------------------------------------------------------------------+
      		 
      | Level   | Code | Message                                                            |
      		 
      +---------+------+--------------------------------------------------------------------+
      		 
      | Warning | 1139 | Got error 'pcre_exec: recursion limit of 100 exceeded' from regexp |
      		 
      +---------+------+--------------------------------------------------------------------+
      		 
      1 row in set (0.15 sec)

      In 10.0:

      MariaDB [10_0_sandbox]> SELECT CONCAT(REPEAT('100,',99),'101') RLIKE '^(([1-9][0-9]*),)*[1-9][0-9]*$' TEST;
      		 
      +------+
      		 
      | TEST |
      		 
      +------+
      		 
      |    1 |
      		 
      +------+
      		 
      1 row in set (0.14 sec)

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-13356 10.0.29 crashes on REGEXP with "(.|x)*" on certain conditions

          • Major - Major loss of function.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            Rich Rich Theobald created issue -
            elenst Elena Stepanova added a comment -

            The difference in behavior is reproducible, it was introduced by this revision:

            commit d672f88ef73e3fc566a382600968c3e51249de1a
            		 
            Author: Daniel Black <grooverdan@users.sourceforge.net>
            		 
            Date:   Sat Apr 8 22:47:56 2017 +1000
            		 
             
            		 
                MDEV-12420: PCRE stack overflow

            I will leave it to danblack and serg to decide whether this particular effect was desirable.

            elenst Elena Stepanova added a comment - The difference in behavior is reproducible, it was introduced by this revision: commit d672f88ef73e3fc566a382600968c3e51249de1a Author: Daniel Black <grooverdan@users.sourceforge.net> Date: Sat Apr 8 22:47:56 2017 +1000   MDEV-12420: PCRE stack overflow I will leave it to danblack and serg to decide whether this particular effect was desirable.
            elenst Elena Stepanova made changes -
            Field Original Value New Value
            Component/s Server [ 13907 ]
            Fix Version/s 10.0 [ 16000 ]
            Fix Version/s 10.1 [ 16100 ]
            Affects Version/s 10.0.31 [ 22501 ]
            Assignee Sergei Golubchik [ serg ]
            elenst Elena Stepanova made changes -
            Status Open [ 1 ] Confirmed [ 10101 ]
            serg Sergei Golubchik added a comment -

            Not desirable, it's a bug.

            serg Sergei Golubchik added a comment - Not desirable, it's a bug.
            danblack Daniel Black added a comment -

            So Regexp_processor_pcre::set_recursion_limit(THD *) was added however never called and hence the default 100L being used. Would this need to be called by fix_fields (which has a THD arg)?

            Few commits are relevant commits d672f88ef73e3fc566a382600968c3e51249de1a..52aa200919b1fd9357c05bcdfc66a42e51f242b3

            danblack Daniel Black added a comment - So Regexp_processor_pcre::set_recursion_limit(THD *) was added however never called and hence the default 100L being used. Would this need to be called by fix_fields (which has a THD arg)? Few commits are relevant commits d672f88ef73e3fc566a382600968c3e51249de1a..52aa200919b1fd9357c05bcdfc66a42e51f242b3
            danblack Daniel Black added a comment - - edited

            https://github.com/grooverdan/mariadb-server/tree/10.0-MDEV-13173-pcre_exec-100-limit

            danblack Daniel Black added a comment - - edited https://github.com/grooverdan/mariadb-server/tree/10.0-MDEV-13173-pcre_exec-100-limit
            svoj Sergey Vojtovich made changes -
            Labels contribution foundation
            serg Sergei Golubchik made changes -
            Status Confirmed [ 10101 ] In Progress [ 3 ]
            serg Sergei Golubchik made changes -
            Status In Progress [ 3 ] Stalled [ 10000 ]
            serg Sergei Golubchik made changes -
            serg Sergei Golubchik made changes -
            Fix Version/s 10.0.32 [ 22504 ]
            Fix Version/s 10.1.26 [ 22553 ]
            Fix Version/s 10.2.8 [ 22544 ]
            Fix Version/s 10.0 [ 16000 ]
            Fix Version/s 10.1 [ 16100 ]
            Resolution Fixed [ 1 ]
            Status Stalled [ 10000 ] Closed [ 6 ]
            Rich Rich Theobald added a comment -

            It looks like the limit was not increased enough.

            The warning on 10.1.26 is now:

            MariaDB [test]> SELECT CONCAT(REPEAT('0,1,21,',99),'101') RLIKE '^([0-9]+,)*[0-9]+$' TEST;
            		 
            +------+
            		 
            | TEST |
            		 
            +------+
            		 
            |    0 |
            		 
            +------+
            		 
            1 row in set, 1 warning (0.01 sec)
            		 
             
            		 
            MariaDB [test]> show warnings;
            		 
            +---------+------+--------------------------------------------------------------------+
            		 
            | Level   | Code | Message                                                            |
            		 
            +---------+------+--------------------------------------------------------------------+
            		 
            | Warning | 1139 | Got error 'pcre_exec: recursion limit of 519 exceeded' from regexp |
            		 
            +---------+------+--------------------------------------------------------------------+
            		 
            1 row in set (0.00 sec)

            This works on 10.0.28

            MariaDB [staging]> SELECT CONCAT(REPEAT('0,1,21,',99),'101') RLIKE '^([0-9]+,)*[0-9]+$' TEST;
            		 
            +------+
            		 
            | TEST |
            		 
            +------+
            		 
            |    1 |
            		 
            +------+
            		 
            1 row in set (0.18 sec)

            Is this configurable?

            Rich Rich Theobald added a comment - It looks like the limit was not increased enough. The warning on 10.1.26 is now: MariaDB [test]> SELECT CONCAT(REPEAT('0,1,21,',99),'101') RLIKE '^([0-9]+,)*[0-9]+$' TEST; +------+ | TEST | +------+ | 0 | +------+ 1 row in set, 1 warning (0.01 sec)   MariaDB [test]> show warnings; +---------+------+--------------------------------------------------------------------+ | Level | Code | Message | +---------+------+--------------------------------------------------------------------+ | Warning | 1139 | Got error 'pcre_exec: recursion limit of 519 exceeded' from regexp | +---------+------+--------------------------------------------------------------------+ 1 row in set (0.00 sec) This works on 10.0.28 MariaDB [staging]> SELECT CONCAT(REPEAT('0,1,21,',99),'101') RLIKE '^([0-9]+,)*[0-9]+$' TEST; +------+ | TEST | +------+ | 1 | +------+ 1 row in set (0.18 sec) Is this configurable?
            serg Sergei Golubchik added a comment -

            Yes, it's automatically detected based on your stack size (thread stack size divided by prce frame size, minus some safety margin), few more levels of recursion would've crashed your server.

            Increase thread stack — that should help. See the --thread-stack option.

            serg Sergei Golubchik added a comment - Yes, it's automatically detected based on your stack size (thread stack size divided by prce frame size, minus some safety margin), few more levels of recursion would've crashed your server. Increase thread stack — that should help. See the --thread-stack option.
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 81371 ] MariaDB v4 [ 152381 ]

            People

              serg Sergei Golubchik
              Rich Rich Theobald
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.