Uploaded image for project: 'MariaDB Foundation Development'
  1. MariaDB Foundation Development
  2. MDBF-741

Remove the gcc UBSAN builder to use the clang based UBSAN

Details

    • Bug
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Buildbot
    • Sprint 6 (24.03.2025)

    Description

      Evidently, that the clang UBSAN is significantly more capable than the gcc based UBSAN.

      There are some significant amount of UBSAN errors that will occur in even the most basic of MTR tests.

      Attachments

        Issue Links

          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. CONC-711 UBSAN: client_mpvio_write_packet through pointer to incorrect function

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDBF-740 bump clang-14 asan builder to use clang 20 (and do ubsan too)

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22940 perfschema, clang, UBSAN - fails to build - libperfschema.a(pfs_instr.cc.o):(.data+0x63b8): undefined reference to `typeinfo for THD'

          • Major - Major loss of function.
          • Open

          Task - A task that needs to be done. MDEV-25454 Make MariaDB server UBSAN safe

          • Major - Major loss of function.
          • Confirmed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-34507 UBSAN: mysys/hash.c:197:20: runtime error: call to function get_sys_var_length

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-34508 UBSAN: mf_keycache.c:5950:11: runtime error: call to function init_simple_key_cache

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-34509 UBSAN: call to function option_cmp(my_option*, my_option*) through pointer to incorrect function type

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-34510 UBSAN: crc32 x86 - integer overflow

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-34511 UBSAN: qsort based function argument mismatch

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-36347 UBSAN: plugins.auth_v0100 - runtime error: call to function do_auth_0x0100 through pointer to incorrect function type

          • Major - Major loss of function.
          • Stalled
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-33478 Tests massively fail with clang-18 -fsanitize=memory

          • Major - Major loss of function.
          • Closed
          links to

          Web Link kernel infra precondition

          Activity

            Ascending order - Click to sort in descending order
            danblack Daniel Black added a comment -

            resolving MDBF-740 immediately facilitates the removal of gcc ubsan.

            rational is that the big contributor to ubsan is google which is pretty exclusively clang, so any gcc capabilities will fall behind clang.

            danblack Daniel Black added a comment - resolving MDBF-740 immediately facilitates the removal of gcc ubsan. rational is that the big contributor to ubsan is google which is pretty exclusively clang, so any gcc capabilities will fall behind clang.
            danblack Daniel Black added a comment - - edited

            UBSAN_OPTIONS=print_stacktrace=1

            for full options.

            Maintained suppression list https://github.com/mariadb-corporation/mariadb-qa/blob/master/UBSAN.filter

            disable:

            • PERFSCHEMA - MDEV-22940 (compile error on clang unit test)

            disable or update suppressions:

            danblack Daniel Black added a comment - - edited UBSAN_OPTIONS=print_stacktrace=1 for full options. Maintained suppression list https://github.com/mariadb-corporation/mariadb-qa/blob/master/UBSAN.filter disable: PERFSCHEMA - MDEV-22940 (compile error on clang unit test) disable or update suppressions: CONNECT - MDEV-36337 - in review - affects a few replication ones that use sql_example UDF too. MROONGA - MDEV-20544 - WIP draft - hopefully resolved with submodule update. FEDERATED/FEDERATEDX - MDEV-36341 core - MDEV-36343 + MDEV-36344 , MDEV-36346 - currently suppressed plugins.auth_v0100 - or MDEV-36347
            danblack Daniel Black added a comment - 

            export UBSAN_OPTIONS=suppressions=${HOME}/mariadb-qa/UBSAN.filter:print_stacktrace=1:report_error_type=1

            danblack Daniel Black added a comment - export UBSAN_OPTIONS=suppressions=${HOME}/mariadb-qa/UBSAN.filter:print_stacktrace=1:report_error_type=1
            danblack Daniel Black added a comment -

            from: Roel

            Good news. Setting hard/soft stack to 16000000 in /etc/security/limits.conf (rather than unlimited) resolves the issue, and the vm.mmap_rnd_bits=28 setting is no longer required.

            https://github.com/google/sanitizers/issues/856#issuecomment-2746537788

            danblack Daniel Black added a comment - from: Roel Good news. Setting hard/soft stack to 16000000 in /etc/security/limits.conf (rather than unlimited) resolves the issue, and the vm.mmap_rnd_bits=28 setting is no longer required. https://github.com/google/sanitizers/issues/856#issuecomment-2746537788
            danblack Daniel Black added a comment -

            test plugins.multiauth can be enabled after deployment. (needs USER=buildbot in env like other tests - MDBF coming).

            still mask sys_vars/t/thread_stack_basic - differing result stack sizes.

            unknown - main/mysqld--help - requires perfschema - blocked on MDEV-22940

            danblack Daniel Black added a comment - test plugins.multiauth can be enabled after deployment. (needs USER=buildbot in env like other tests - MDBF coming). still mask sys_vars/t/thread_stack_basic - differing result stack sizes. unknown - main/mysqld--help - requires perfschema - blocked on MDEV-22940
            danblack Daniel Black added a comment -

            working on getting server clean:

            so far mroonga upstream https://github.com/mroonga/mroonga/pull/902
            our UDF functions - https://github.com/MariaDB/server/pull/3906
            MDEV-36347 in review
            MDEV-35715 masked https://github.com/mariadb-corporation/mariadb-qa/blob/master/UBSAN.filter#L30
            MDEV-36341 federatedx still to fix:

            WIP - probably temp structure insufficiently populated.

            		 
            federated.federated_transactions 'X'     [ pass ]    159
            		 
            ***Warnings generated in error logs during shutdown after running tests: federated.federated_transactions
            		 
             
            		 
            /source/storage/federatedx/ha_federatedx.cc:1514:17: runtime error: applying non-zero offset 138217089612088 to null pointer
            		 
            SUMMARY: UndefinedBehaviorSanitizer: nullptr-with-nonzero-offset /source/storage/federatedx/ha_federatedx.cc:1514:17 
             
            		 
               mysql-test/mtr --manual-gdb='b fill_server;b  __ubsan::ScopedReport::~ScopedReport; r'  --mem federated.federated_transactions

            Test:

            diff --git a/include/my_global.h b/include/my_global.h
            		 
            index fd774bdac1f..ebef24cf1f0 100644
            		 
            --- a/include/my_global.h
            		 
            +++ b/include/my_global.h
            		 
            @@ -440,10 +440,7 @@ extern "C" int madvise(void *addr, size_t len, int behav);
            		 
             /*
            		 
                Suppress uninitialized variable warning without generating code.
            		 
             */
            		 
            -#if defined(__GNUC__)
            		 
            -/* GCC specific self-initialization which inhibits the warning. */
            		 
            -#define UNINIT_VAR(x) x= x
            		 
            -#elif defined(_lint) || defined(FORCE_INIT_OF_VARS)
            		 
            +#if defined(_lint) || defined(FORCE_INIT_OF_VARS)
            		 
             #define UNINIT_VAR(x) x= 0

            its a hack. There was a USan abouve a UNINIT var of bool that was being assigned a 0/1 value. Will find again.

            danblack Daniel Black added a comment - working on getting server clean: so far mroonga upstream https://github.com/mroonga/mroonga/pull/902 our UDF functions - https://github.com/MariaDB/server/pull/3906 MDEV-36347 in review MDEV-35715 masked https://github.com/mariadb-corporation/mariadb-qa/blob/master/UBSAN.filter#L30 MDEV-36341 federatedx still to fix: WIP - probably temp structure insufficiently populated. federated.federated_transactions 'X' [ pass ] 159 ***Warnings generated in error logs during shutdown after running tests: federated.federated_transactions   /source/storage/federatedx/ha_federatedx.cc:1514:17: runtime error: applying non-zero offset 138217089612088 to null pointer SUMMARY: UndefinedBehaviorSanitizer: nullptr-with-nonzero-offset /source/storage/federatedx/ha_federatedx.cc:1514:17   mysql-test/mtr --manual-gdb='b fill_server;b __ubsan::ScopedReport::~ScopedReport; r' --mem federated.federated_transactions Test: diff --git a/include/my_global.h b/include/my_global.h index fd774bdac1f..ebef24cf1f0 100644 --- a/include/my_global.h +++ b/include/my_global.h @@ -440,10 +440,7 @@ extern "C" int madvise(void *addr, size_t len, int behav); /* Suppress uninitialized variable warning without generating code. */ -#if defined(__GNUC__) -/* GCC specific self-initialization which inhibits the warning. */ -#define UNINIT_VAR(x) x= x -#elif defined(_lint) || defined(FORCE_INIT_OF_VARS) +#if defined(_lint) || defined(FORCE_INIT_OF_VARS) #define UNINIT_VAR(x) x= 0 its a hack. There was a USan abouve a UNINIT var of bool that was being assigned a 0/1 value. Will find again.

            People

              danblack Daniel Black
              danblack Daniel Black
              Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - 5d Original Estimate - 5d
                  5d
                  Remaining:
                  Time Spent - 1d 7h Remaining Estimate - 3d 3h
                  3d 3h
                  Logged:
                  Time Spent - 1d 7h Remaining Estimate - 3d 3h
                  1d 7h