[#MDEV-26451] ASAN: global-buffer-overflow in init_re

When the server/clients are built with system PCRE and clang-11, at least on RHEL-8 it causes ASAN errors.
Quite possibly it's not a MariaDB problem at all, or if it's a MariaDB problem, maybe it only affects mysqltest and thus is not to worry about much. Still, given that we release pre-10.4 packages built with the bundled PCRE, and 10.5+ packages linked with the system PCRE, and both are potentially affected, it's probably worth checking.

10.5 a6621867e
ASAN_OPTIONS="disable_coredump=0:abort_on_error=1" perl ./mtr main.1st

Could not execute 'check-testcase' before testcase 'main.1st' (res: 1):
mysqltest: Logging to '/10.5/mysql-test/var/tmp/check-mysqld_1.log'.
mysqltest: Results saved in '/10.5/mysql-test/var/tmp/check-mysqld_1.result'.
=================================================================
==11978==ERROR: AddressSanitizer: global-buffer-overflow on address 0x000001849cb0 at pc 0x000000467805 bp 0x7fff931d7230 sp 0x7fff931d69d8
WRITE of size 64 at 0x000001849cb0 thread T0
#0 0x467804 (/10.5/client/mariadb-test+0x467804)
#1 0x55ade5 (/10.5/client/mariadb-test+0x55ade5)
#2 0x560f55 (/10.5/client/mariadb-test+0x560f55)
#3 0x55ca10 (/10.5/client/mariadb-test+0x55ca10)
#4 0x7fa11c457812 (/lib64/libc.so.6+0x23812)
#5 0x43b41d (/10.5/client/mariadb-test+0x43b41d)

0x000001849cb0 is located 48 bytes to the left of global variable 'epbuf' defined in '/home/buildbot/10.5/client/mysqltest.cc:8771:15' (0x1849ce0) of size 100
0x000001849cb0 is located 0 bytes to the right of global variable 'ps_re' defined in '/home/buildbot/10.5/client/mysqltest.cc:264:16' (0x1849c80) of size 48
SUMMARY: AddressSanitizer: global-buffer-overflow (/10.5/client/mariadb-test+0x467804)
Shadow bytes around the buggy address:
0x000080301340: 01 f9 f9 f9 f9 f9 f9 f9 01 f9 f9 f9 f9 f9 f9 f9
0x000080301350: 01 f9 f9 f9 f9 f9 f9 f9 01 f9 f9 f9 f9 f9 f9 f9
0x000080301360: 01 f9 f9 f9 f9 f9 f9 f9 00 00 00 00 00 00 f9 f9
0x000080301370: f9 f9 f9 f9 01 f9 f9 f9 f9 f9 f9 f9 00 00 00 00
0x000080301380: 00 00 f9 f9 f9 f9 f9 f9 01 f9 f9 f9 f9 f9 f9 f9
=>0x000080301390: 00 00 00 00 00 00[f9]f9 f9 f9 f9 f9 00 00 00 00
0x0000803013a0: 00 00 00 00 00 00 00 00 04 f9 f9 f9 f9 f9 f9 f9
0x0000803013b0: 00 f9 f9 f9 f9 f9 f9 f9 01 f9 f9 f9 f9 f9 f9 f9
0x0000803013c0: 01 f9 f9 f9 f9 f9 f9 f9 01 f9 f9 f9 f9 f9 f9 f9
0x0000803013d0: 01 f9 f9 f9 f9 f9 f9 f9 01 f9 f9 f9 f9 f9 f9 f9
0x0000803013e0: 01 f9 f9 f9 f9 f9 f9 f9 00 f9 f9 f9 f9 f9 f9 f9
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==11978==ABORTING
mysqltest got signal 6
read_command_buf (0x631000014888): ��

Attempting backtrace...
stack_bottom = 0x0 thread_stack 0x3c000
/10.5/client/mysqltest[0x45ec7c]
/10.5/client/mysqltest(my_print_stacktrace+0x139)[0x6c3069]
/10.5/client/mysqltest[0x56fd7c]
/10.5/client/mysqltest[0x56fa72]
sigaction.c:0(__restore_rt)[0x7fa11db4ed80]
:0(__GI_raise)[0x7fa11c46b93f]
:0(__GI_abort)[0x7fa11c455c95]
/10.5/client/mysqltest[0x502252]
/10.5/client/mysqltest[0x50041c]
/10.5/client/mysqltest[0x4e354e]
/10.5/client/mysqltest[0x467827]
/10.5/client/mysqltest(_Z12init_re_compP7regex_tPKc+0x106)[0x55ade6]
/10.5/client/mysqltest[0x560f56]
/10.5/client/mysqltest(main+0xcb1)[0x55ca11]
??:0(__libc_start_main)[0x7fa11c457813]
/10.5/client/mysqltest(_start+0x2e)[0x43b41e]
Writing a core file...
mysqltest failed but provided no output

After some resolving

:0(__GI_abort)[0x7f8064cdbc95]

/10.5/client/mysqltest[0x502252 __sanitizer::Abort() + 50]

/10.5/client/mysqltest[0x50041c __sanitizer::Die() + 76]

/10.5/client/mysqltest[0x4e354e __asan::ReportGenericError(unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long, unsigned int, bool) + 654]

/10.5/client/mysqltest[0x467827 __interceptor_regcomp.part.289 + 519]

/10.5/client/mysqltest(init_re_comp(regex_t*, char const*)+0x106 __lsan::disable_counter + 142)[0x55ade6 init_re_comp(regex_t*, char const*) + 262]

/10.5/client/mysqltest[0x560f56 init_re() + 54]

/10.5/client/mysqltest(main+0xcb1 __lsan::disable_counter + 3129)[0x55ca11 main + 3249]

The server is built with

cmake .. -DMYSQL_MAINTAINER_MODE=OFF -DCMAKE_C_COMPILER=clang-11 -DCMAKE_CXX_COMPILER=clang++-11 -DWITH_ASAN=YES -DCMAKE_BUILD_TYPE=Debug -DWITH_SSL=bundled -DWITH_PCRE=system

As I understand, PCRE is supposed to be relevant here, so

$ ldd ../client/mariadb-test

...

	libpcre2-posix.so.2 => /lib64/libpcre2-posix.so.2 (0x00007f303c423000)

	libpcre2-8.so.0 => /lib64/libpcre2-8.so.0 (0x00007f303c19f000)

...

However, the same happens also on 10.4, built the same way, only it links with pcre, not pcre2:

10.4 1002703baa1 with system PCRE
main.1st [ fail ]
Test ended at 2021-08-20 17:16:21

CURRENT_TEST: main.1st


Could not execute 'check-testcase' before testcase 'main.1st' (res: 1):
mysqltest: Logging to '/home/buildbot/10.4/build_with_system_pcre_clang/mysql-test/var/tmp/check-mysqld_1.log'.
mysqltest: Results saved in '/home/buildbot/10.4/build_with_system_pcre_clang/mysql-test/var/tmp/check-mysqld_1.result'.
=================================================================
==12074==ERROR: AddressSanitizer: global-buffer-overflow on address 0x000001819d58 at pc 0x000000464d75 bp 0x7ffe0c1e3f90 sp 0x7ffe0c1e3738
WRITE of size 64 at 0x000001819d58 thread T0
#0 0x464d74 (/home/buildbot/10.4/build_with_system_pcre_clang/client/mysqltest+0x464d74)
#1 0x557b05 (/home/buildbot/10.4/build_with_system_pcre_clang/client/mysqltest+0x557b05)
#2 0x55db95 (/home/buildbot/10.4/build_with_system_pcre_clang/client/mysqltest+0x55db95)
#3 0x559717 (/home/buildbot/10.4/build_with_system_pcre_clang/client/mysqltest+0x559717)
#4 0x7ff1d6241812 (/lib64/libc.so.6+0x23812)
#5 0x43898d (/home/buildbot/10.4/build_with_system_pcre_clang/client/mysqltest+0x43898d)

0x000001819d58 is located 40 bytes to the left of global variable 'epbuf' defined in '/home/buildbot/10.4/client/mysqltest.cc:8719:15' (0x1819d80) of size 100
0x000001819d58 is located 0 bytes to the right of global variable 'ps_re' defined in '/home/buildbot/10.4/client/mysqltest.cc:263:16' (0x1819d40) of size 24
SUMMARY: AddressSanitizer: global-buffer-overflow (/home/buildbot/10.4/build_with_system_pcre_clang/client/mysqltest+0x464d74)
...

...
$ ldd ../client/mysqltest
libpcreposix.so.0 => /lib64/libpcreposix.so.0 (0x00007f31ac20d000)
libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f31abf9c000)
...

The same happens on 10.4 if it's built with bundled PCRE, but I've been told before that the bundled PCRE before 10.5 is bad, so maybe it's expected

{noformat:title=10.4 1002703baa1 with bundled PCRE}

main.1st                                 [ fail ]

        Test ended at 2021-08-20 17:17:47

CURRENT_TEST: main.1st

Could not execute 'check-testcase' before testcase 'main.1st' (res: 1):

mysqltest: Logging to '/home/buildbot/10.4/build_with_bundled_pcre_clang/mysql-test/var/tmp/check-mysqld_1.log'.

mysqltest: Results saved in '/home/buildbot/10.4/build_with_bundled_pcre_clang/mysql-test/var/tmp/check-mysqld_1.result'.

=================================================================

==12162==ERROR: AddressSanitizer: global-buffer-overflow on address 0x000001819d58 at pc 0x000000464d55 bp 0x7ffec2d3b330 sp 0x7ffec2d3aad8

WRITE of size 64 at 0x000001819d58 thread T0

    #0 0x464d54  (/home/buildbot/10.4/build_with_bundled_pcre_clang/client/mysqltest+0x464d54)

    #1 0x557ae5  (/home/buildbot/10.4/build_with_bundled_pcre_clang/client/mysqltest+0x557ae5)

    #2 0x55db75  (/home/buildbot/10.4/build_with_bundled_pcre_clang/client/mysqltest+0x55db75)

    #3 0x5596f7  (/home/buildbot/10.4/build_with_bundled_pcre_clang/client/mysqltest+0x5596f7)

    #4 0x7f4054085812  (/lib64/libc.so.6+0x23812)

    #5 0x43896d  (/home/buildbot/10.4/build_with_bundled_pcre_clang/client/mysqltest+0x43896d)

0x000001819d58 is located 40 bytes to the left of global variable 'epbuf' defined in '/home/buildbot/10.4/client/mysqltest.cc:8719:15' (0x1819d80) of size 100

0x000001819d58 is located 0 bytes to the right of global variable 'ps_re' defined in '/home/buildbot/10.4/client/mysqltest.cc:263:16' (0x1819d40) of size 24

SUMMARY: AddressSanitizer: global-buffer-overflow (/home/buildbot/10.4/build_with_bundled_pcre_clang/client/mysqltest+0x464d54)

$ ldd ../client/mysqltest

	linux-vdso.so.1 (0x00007ffc2b3ce000)

	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f82f93aa000)

	libgnutls.so.30 => /lib64/libgnutls.so.30 (0x00007f82f8fb9000)

	libm.so.6 => /lib64/libm.so.6 (0x00007f82f8c37000)

	libdl.so.2 => /lib64/libdl.so.2 (0x00007f82f8a33000)

	libz.so.1 => /lib64/libz.so.1 (0x00007f82f881c000)

	libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f82f8487000)

	librt.so.1 => /lib64/librt.so.1 (0x00007f82f827e000)

	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f82f8066000)

	libc.so.6 => /lib64/libc.so.6 (0x00007f82f7ca2000)

	/lib64/ld-linux-x86-64.so.2 (0x00007f82f95ca000)

	libp11-kit.so.0 => /lib64/libp11-kit.so.0 (0x00007f82f796f000)

	libidn2.so.0 => /lib64/libidn2.so.0 (0x00007f82f7752000)

	libunistring.so.2 => /lib64/libunistring.so.2 (0x00007f82f73d1000)

	libtasn1.so.6 => /lib64/libtasn1.so.6 (0x00007f82f71be000)

	libnettle.so.6 => /lib64/libnettle.so.6 (0x00007f82f6f85000)

	libhogweed.so.4 => /lib64/libhogweed.so.4 (0x00007f82f6d55000)

	libgmp.so.10 => /lib64/libgmp.so.10 (0x00007f82f6abf000)

	libffi.so.6 => /lib64/libffi.so.6 (0x00007f82f68b6000)

It doesn't happen with the bundled PCRE on 10.5.
It doesn't happen when the server is built with gcc-8 (be it 10.4 or 10.5, system or bundled PCRE).
Apparently 10.5+ behaves as 10.5 in this regard, and 10.4- behaves as 10.4, although I didn't check all permutations.

To summarize,

with clang-11

server \ pcre	bundled	system
10.5+
10.4-

with gcc-8

server \ pcre	bundled	system
10.5+
10.4-

Legend: here means that there is no ASAN errors. Maybe there is still an undetected problem.

[MDEV-26451] ASAN: global-buffer-overflow in init_re_comp on RHEL-8 build with clang-11 Created: 2021-08-20 Updated: 2023-11-17
Status:	Open
Project:	MariaDB Server
Component/s:	Platform RedHat, Scripts & Clients
Affects Version/s:	10.2, 10.3, 10.4, 10.5, 10.6
Fix Version/s:	10.4, 10.5, 10.6

[MDEV-26451] ASAN: global-buffer-overflow in init_re_comp on RHEL-8 build with clang-11 Created: 2021-08-20 Updated: 2023-11-17