Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-34099

AddressSanitizer running out of memory regardless of stack_thread size

Details

    Description

      Regardless of how large we set the thread_stack variable ASAN seems to complain about running out of memory at the same point for numerous tests in the main suite. This was found on 10.5 using clang16 with -DWITH_ASAN=YES. The error messages suggest that this seem to be related to the test runner itself as opposed to the actual engine. This does not occur when compiling in debug mode.

      Example output for stack_thread=50M and stack_thread=30M

      CURRENT_TEST: main.type_timestamp_hires12910Could not execute 'check-testcase' before testcase 'main.type_timestamp_hires' (res: 1):12911mysqltest: Logging to '/usr/share/mysql-test/var/2/tmp/check-mysqld_1.log'.12912mysqltest: Results saved in '/usr/share/mysql-test/var/2/tmp/check-mysqld_1.result'.12913mysqltest: Connecting to server localhost:16000 (socket /usr/share/mysql-test/var/tmp/2/mysqld.1.sock) as 'root', connection 'default', attempt 0 ...12914mysqltest: ... Connected.12915mysqltest: Start processing test commands from './include/check-testcase.test' ...12916mysqltest: At line 19: query 'let $tmp= `SELECT '$tmp' = 'No such row'`' failed: 1436: Thread stack overrun:  63936096 bytes used of a 52428800 byte stack, and 32000 bytes needed. Consider increasing the thread_stack system variable
      		 
       
      		 
      CURRENT_TEST: main.type_timestamp_hires12878Could not execute 'check-testcase' before testcase 'main.type_timestamp_hires' (res: 1):12879mysqltest: Logging to '/usr/share/mysql-test/var/6/tmp/check-mysqld_1.log'.12880mysqltest: Results saved in '/usr/share/mysql-test/var/6/tmp/check-mysqld_1.result'.12881mysqltest: Connecting to server localhost:16060 (socket /usr/share/mysql-test/var/tmp/6/mysqld.1.sock) as 'root', connection 'default', attempt 0 ...12882mysqltest: ... Connected.12883mysqltest: Start processing test commands from './include/check-testcase.test' ...12884mysqltest: At line 19: query 'let $tmp= `SELECT '$tmp' = 'No such row'`' failed: 1436: Thread stack overrun:  42964576 bytes used of a 31457280 byte stack, and 32000 bytes needed. Consider increasing the thread_stack system variable.

      Attachments

        Issue Links

          blocks

          Bug - A problem which impairs or prevents the functions of the product. MDBF-740 bump clang-14 asan builder to use clang 20 (and do ubsan too)

          • Major - Major loss of function.
          • Open
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-31605 cmake/stack_direction.c does not work correctly on clang 16

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-16660 Inadequate DEFAULT_THREAD_STACK size for AddressSanitizer

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-33210 Several performance_schema tests fail due to explicit (too small for ASAN) thread_stack

          • Major - Major loss of function.
          • Confirmed

          Activity

            Ascending order - Click to sort in descending order
            anson Anson created issue -
            anson Anson made changes -
            Field Original Value New Value
            anson Anson made changes -
            anson Anson made changes -
            Description Regardless of how large we set the thread_stack variable ASAN seems to complain about running out of memory at the same point for numerous tests in the main suite. This was found on clang16 and with {{-DWITH_ASAN=YES}}. The error messages suggest that this seem to be related to the test runner itself as opposed to the actual engine.

            Example output for stack_thread=50M and stack_thread=30M

            {code:java}
            CURRENT_TEST: main.type_timestamp_hires12910Could not execute 'check-testcase' before testcase 'main.type_timestamp_hires' (res: 1):12911mysqltest: Logging to '/usr/share/mysql-test/var/2/tmp/check-mysqld_1.log'.12912mysqltest: Results saved in '/usr/share/mysql-test/var/2/tmp/check-mysqld_1.result'.12913mysqltest: Connecting to server localhost:16000 (socket /usr/share/mysql-test/var/tmp/2/mysqld.1.sock) as 'root', connection 'default', attempt 0 ...12914mysqltest: ... Connected.12915mysqltest: Start processing test commands from './include/check-testcase.test' ...12916mysqltest: At line 19: query 'let $tmp= `SELECT '$tmp' = 'No such row'`' failed: 1436: Thread stack overrun: 63936096 bytes used of a 52428800 byte stack, and 32000 bytes needed. Consider increasing the thread_stack system variable

            CURRENT_TEST: main.type_timestamp_hires12878Could not execute 'check-testcase' before testcase 'main.type_timestamp_hires' (res: 1):12879mysqltest: Logging to '/usr/share/mysql-test/var/6/tmp/check-mysqld_1.log'.12880mysqltest: Results saved in '/usr/share/mysql-test/var/6/tmp/check-mysqld_1.result'.12881mysqltest: Connecting to server localhost:16060 (socket /usr/share/mysql-test/var/tmp/6/mysqld.1.sock) as 'root', connection 'default', attempt 0 ...12882mysqltest: ... Connected.12883mysqltest: Start processing test commands from './include/check-testcase.test' ...12884mysqltest: At line 19: query 'let $tmp= `SELECT '$tmp' = 'No such row'`' failed: 1436: Thread stack overrun: 42964576 bytes used of a 31457280 byte stack, and 32000 bytes needed. Consider increasing the thread_stack system variable.
            {code}
            		Regardless of how large we set the thread_stack variable ASAN seems to complain about running out of memory at the same point for numerous tests in the main suite. This was found on 10.5 using clang16 with {{-DWITH_ASAN=YES}}. The error messages suggest that this seem to be related to the test runner itself as opposed to the actual engine.

            Example output for stack_thread=50M and stack_thread=30M

            {code:java}
            CURRENT_TEST: main.type_timestamp_hires12910Could not execute 'check-testcase' before testcase 'main.type_timestamp_hires' (res: 1):12911mysqltest: Logging to '/usr/share/mysql-test/var/2/tmp/check-mysqld_1.log'.12912mysqltest: Results saved in '/usr/share/mysql-test/var/2/tmp/check-mysqld_1.result'.12913mysqltest: Connecting to server localhost:16000 (socket /usr/share/mysql-test/var/tmp/2/mysqld.1.sock) as 'root', connection 'default', attempt 0 ...12914mysqltest: ... Connected.12915mysqltest: Start processing test commands from './include/check-testcase.test' ...12916mysqltest: At line 19: query 'let $tmp= `SELECT '$tmp' = 'No such row'`' failed: 1436: Thread stack overrun: 63936096 bytes used of a 52428800 byte stack, and 32000 bytes needed. Consider increasing the thread_stack system variable

            CURRENT_TEST: main.type_timestamp_hires12878Could not execute 'check-testcase' before testcase 'main.type_timestamp_hires' (res: 1):12879mysqltest: Logging to '/usr/share/mysql-test/var/6/tmp/check-mysqld_1.log'.12880mysqltest: Results saved in '/usr/share/mysql-test/var/6/tmp/check-mysqld_1.result'.12881mysqltest: Connecting to server localhost:16060 (socket /usr/share/mysql-test/var/tmp/6/mysqld.1.sock) as 'root', connection 'default', attempt 0 ...12882mysqltest: ... Connected.12883mysqltest: Start processing test commands from './include/check-testcase.test' ...12884mysqltest: At line 19: query 'let $tmp= `SELECT '$tmp' = 'No such row'`' failed: 1436: Thread stack overrun: 42964576 bytes used of a 31457280 byte stack, and 32000 bytes needed. Consider increasing the thread_stack system variable.
            {code}
            anson Anson made changes -
            Description Regardless of how large we set the thread_stack variable ASAN seems to complain about running out of memory at the same point for numerous tests in the main suite. This was found on 10.5 using clang16 with {{-DWITH_ASAN=YES}}. The error messages suggest that this seem to be related to the test runner itself as opposed to the actual engine.

            Example output for stack_thread=50M and stack_thread=30M

            {code:java}
            CURRENT_TEST: main.type_timestamp_hires12910Could not execute 'check-testcase' before testcase 'main.type_timestamp_hires' (res: 1):12911mysqltest: Logging to '/usr/share/mysql-test/var/2/tmp/check-mysqld_1.log'.12912mysqltest: Results saved in '/usr/share/mysql-test/var/2/tmp/check-mysqld_1.result'.12913mysqltest: Connecting to server localhost:16000 (socket /usr/share/mysql-test/var/tmp/2/mysqld.1.sock) as 'root', connection 'default', attempt 0 ...12914mysqltest: ... Connected.12915mysqltest: Start processing test commands from './include/check-testcase.test' ...12916mysqltest: At line 19: query 'let $tmp= `SELECT '$tmp' = 'No such row'`' failed: 1436: Thread stack overrun: 63936096 bytes used of a 52428800 byte stack, and 32000 bytes needed. Consider increasing the thread_stack system variable

            CURRENT_TEST: main.type_timestamp_hires12878Could not execute 'check-testcase' before testcase 'main.type_timestamp_hires' (res: 1):12879mysqltest: Logging to '/usr/share/mysql-test/var/6/tmp/check-mysqld_1.log'.12880mysqltest: Results saved in '/usr/share/mysql-test/var/6/tmp/check-mysqld_1.result'.12881mysqltest: Connecting to server localhost:16060 (socket /usr/share/mysql-test/var/tmp/6/mysqld.1.sock) as 'root', connection 'default', attempt 0 ...12882mysqltest: ... Connected.12883mysqltest: Start processing test commands from './include/check-testcase.test' ...12884mysqltest: At line 19: query 'let $tmp= `SELECT '$tmp' = 'No such row'`' failed: 1436: Thread stack overrun: 42964576 bytes used of a 31457280 byte stack, and 32000 bytes needed. Consider increasing the thread_stack system variable.
            {code}
            		Regardless of how large we set the thread_stack variable ASAN seems to complain about running out of memory at the same point for numerous tests in the main suite. This was found on 10.5 using clang16 with {{-DWITH_ASAN=YES}}. The error messages suggest that this seem to be related to the test runner itself as opposed to the actual engine. This does not occur when compiling in debug mode.

            Example output for stack_thread=50M and stack_thread=30M

            {code:java}
            CURRENT_TEST: main.type_timestamp_hires12910Could not execute 'check-testcase' before testcase 'main.type_timestamp_hires' (res: 1):12911mysqltest: Logging to '/usr/share/mysql-test/var/2/tmp/check-mysqld_1.log'.12912mysqltest: Results saved in '/usr/share/mysql-test/var/2/tmp/check-mysqld_1.result'.12913mysqltest: Connecting to server localhost:16000 (socket /usr/share/mysql-test/var/tmp/2/mysqld.1.sock) as 'root', connection 'default', attempt 0 ...12914mysqltest: ... Connected.12915mysqltest: Start processing test commands from './include/check-testcase.test' ...12916mysqltest: At line 19: query 'let $tmp= `SELECT '$tmp' = 'No such row'`' failed: 1436: Thread stack overrun: 63936096 bytes used of a 52428800 byte stack, and 32000 bytes needed. Consider increasing the thread_stack system variable

            CURRENT_TEST: main.type_timestamp_hires12878Could not execute 'check-testcase' before testcase 'main.type_timestamp_hires' (res: 1):12879mysqltest: Logging to '/usr/share/mysql-test/var/6/tmp/check-mysqld_1.log'.12880mysqltest: Results saved in '/usr/share/mysql-test/var/6/tmp/check-mysqld_1.result'.12881mysqltest: Connecting to server localhost:16060 (socket /usr/share/mysql-test/var/tmp/6/mysqld.1.sock) as 'root', connection 'default', attempt 0 ...12882mysqltest: ... Connected.12883mysqltest: Start processing test commands from './include/check-testcase.test' ...12884mysqltest: At line 19: query 'let $tmp= `SELECT '$tmp' = 'No such row'`' failed: 1436: Thread stack overrun: 42964576 bytes used of a 31457280 byte stack, and 32000 bytes needed. Consider increasing the thread_stack system variable.
            {code}
            danblack Daniel Black made changes -
            danblack Daniel Black made changes -
            danblack Daniel Black made changes -
            Component/s Server [ 13907 ]
            danblack Daniel Black made changes -
            Labels ASAN
            danblack Daniel Black made changes -
            Priority Minor [ 4 ] Critical [ 2 ]
            danblack Daniel Black made changes -
            Assignee Daniel Black [ danblack ]
            danblack Daniel Black made changes -
            Fix Version/s 10.5 [ 23123 ]
            danblack Daniel Black made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            danblack Daniel Black made changes -
            Assignee Daniel Black [ danblack ] Oleksandr Byelkin [ sanja ]
            Status In Progress [ 3 ] In Review [ 10002 ]
            sanja Oleksandr Byelkin made changes -
            Assignee Oleksandr Byelkin [ sanja ] Daniel Black [ danblack ]
            Status In Review [ 10002 ] Stalled [ 10000 ]
            danblack Daniel Black made changes -
            Fix Version/s 10.5.26 [ 29832 ]
            Fix Version/s 10.5 [ 23123 ]
            Resolution Fixed [ 1 ]
            Status Stalled [ 10000 ] Closed [ 6 ]
            JIraAutomate JiraAutomate made changes -
            Fix Version/s 10.6.19 [ 29833 ]
            Fix Version/s 10.11.9 [ 29834 ]
            Fix Version/s 11.1.6 [ 29835 ]
            Fix Version/s 11.2.5 [ 29836 ]
            Fix Version/s 11.4.3 [ 29837 ]

            People

              danblack Daniel Black
              anson Anson
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.