UBSAN: runtime error: -nan is outside the range of representable values of type 'unsigned long long' in Index_statistics::set_avg_frequency on INSERT

--source include/have_innodb.inc

CREATE TABLE t (c1 VARCHAR(10),c2 VARCHAR(10),PRIMARY KEY(c1,c2),FULLTEXT KEY k (c2)) ENGINE=InnoDB;

INSERT INTO t VALUES ('a','b');

/test/11.8_opt_san/sql/sql_statistics.h:620:35: runtime error: -nan is outside the range of representable values of type 'unsigned long long'

    #0 0x55eecfb6c61a in Index_statistics::set_avg_frequency(unsigned int, double) /test/11.8_opt_san/sql/sql_statistics.h:620:35

    #1 0x55eecfb6c61a in read_statistics_for_table(THD*, TABLE*, TABLE_LIST*, bool, bool) /test/11.8_opt_san/sql/sql_statistics.cc:3148:24

    #2 0x55eecfb6c61a in read_statistics_for_tables(THD*, TABLE_LIST*, bool) /test/11.8_opt_san/sql/sql_statistics.cc:3354:23

    #3 0x55eecf3f65af in open_and_lock_tables(THD*, DDL_options_st const&, TABLE_LIST*, bool, unsigned int, Prelocking_strategy*) /test/11.8_opt_san/sql/sql_base.cc:5647:12

    #4 0x55eecf563147 in open_and_lock_tables(THD*, TABLE_LIST*, bool, unsigned int) /test/11.8_opt_san/sql/sql_base.h:532:10

    #5 0x55eecf563147 in mysql_insert(THD*, TABLE_LIST*, List<Item>&, List<List<Item>>&, List<Item>&, List<Item>&, enum_duplicates, bool, select_result*) /test/11.8_opt_san/sql/sql_insert.cc:784:9

    #6 0x55eecf73ba2a in mysql_execute_command(THD*, bool) /test/11.8_opt_san/sql/sql_parse.cc:4471:10

    #7 0x55eecf6fbc92 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /test/11.8_opt_san/sql/sql_parse.cc:7901:18

    #8 0x55eecf6f0b9e in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool) /test/11.8_opt_san/sql/sql_parse.cc:1903:7

    #9 0x55eecf6fea6e in do_command(THD*, bool) /test/11.8_opt_san/sql/sql_parse.cc:1416:17

    #10 0x55eecfedfe38 in do_handle_one_connection(CONNECT*, bool) /test/11.8_opt_san/sql/sql_connect.cc:1415:11

    #11 0x55eecfedf280 in handle_one_connection /test/11.8_opt_san/sql/sql_connect.cc:1327:5

    #12 0x55eecf027b0c in asan_thread_start(void*) asan_interceptors.cpp.o

    #13 0x1455fa89ca93 in start_thread nptl/pthread_create.c:447:8

    #14 0x1455fa929c3b in clone3 misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

SUMMARY: UndefinedBehaviorSanitizer: float-cast-overflow /test/11.8_opt_san/sql/sql_statistics.h:620:35 

/test/11.8_dbg_san/sql/sql_statistics.h:620:35: runtime error: -nan is outside the range of representable values of type 'unsigned long long'

    #0 0x5599fced8418 in Index_statistics::set_avg_frequency(unsigned int, double) /test/11.8_dbg_san/sql/sql_statistics.h:620:35

    #1 0x5599fcea478a in read_statistics_for_table(THD*, TABLE*, TABLE_LIST*, bool, bool) /test/11.8_dbg_san/sql/sql_statistics.cc:3148:24

    #2 0x5599fce9ecbf in read_statistics_for_tables(THD*, TABLE_LIST*, bool) /test/11.8_dbg_san/sql/sql_statistics.cc:3354:23

    #3 0x5599fce9bacc in read_statistics_for_tables_if_needed(THD*, TABLE_LIST*) /test/11.8_dbg_san/sql/sql_statistics.cc:3213:12

    #4 0x5599fbeb432a in open_and_lock_tables(THD*, DDL_options_st const&, TABLE_LIST*, bool, unsigned int, Prelocking_strategy*) /test/11.8_dbg_san/sql/sql_base.cc:5647:12

    #5 0x5599fbf17866 in open_and_lock_tables(THD*, TABLE_LIST*, bool, unsigned int) /test/11.8_dbg_san/sql/sql_base.h:532:10

    #6 0x5599fc1b56e9 in mysql_insert(THD*, TABLE_LIST*, List<Item>&, List<List<Item>>&, List<Item>&, List<Item>&, enum_duplicates, bool, select_result*) /test/11.8_dbg_san/sql/sql_insert.cc:784:9

    #7 0x5599fc537907 in mysql_execute_command(THD*, bool) /test/11.8_dbg_san/sql/sql_parse.cc:4471:10

    #8 0x5599fc4cdef9 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /test/11.8_dbg_san/sql/sql_parse.cc:7901:18

    #9 0x5599fc4aedb8 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool) /test/11.8_dbg_san/sql/sql_parse.cc:1903:7

    #10 0x5599fc4d7e56 in do_command(THD*, bool) /test/11.8_dbg_san/sql/sql_parse.cc:1416:17

    #11 0x5599fd63a556 in do_handle_one_connection(CONNECT*, bool) /test/11.8_dbg_san/sql/sql_connect.cc:1415:11

    #12 0x5599fd638d19 in handle_one_connection /test/11.8_dbg_san/sql/sql_connect.cc:1327:5

    #13 0x5599fb6815fc in asan_thread_start(void*) asan_interceptors.cpp.o

    #14 0x14ebf7c9ca93 in start_thread nptl/pthread_create.c:447:8

    #15 0x14ebf7d29c3b in clone3 misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

SUMMARY: UndefinedBehaviorSanitizer: float-cast-overflow /test/11.8_dbg_san/sql/sql_statistics.h:620:35 

Compiled with a recent version of Clang (I used Clang 18.1.3) with LLVM 18. Ubuntu instructions:

     # Note: llvm-17-linker-tools installs /usr/lib/llvm-17/lib/LLVMgold.so, which is needed for compilation, and LLVMgold.so is no longer included in LLVM 18

     sudo apt install clang llvm-18 llvm-18-linker-tools llvm-18-runtime llvm-18-tools llvm-18-dev libstdc++-14-dev llvm-dev llvm-17-linker-tools

     sudo ln -s /usr/lib/llvm-17/lib/LLVMgold.so /usr/lib/llvm-18/lib/LLVMgold.so

Compiled with: '-DCMAKE_C_COMPILER=/usr/bin/clang -DCMAKE_CXX_COMPILER=/usr/bin/clang++' and:

    -DWITH_ASAN=ON -DWITH_ASAN_SCOPE=ON -DWITH_UBSAN=ON -DWSREP_LIB_WITH_ASAN=ON

Set before execution:

    export UBSAN_OPTIONS=print_stacktrace=1:report_error_type=1   # And you may also want to supress UBSAN startup issues using 'suppressions=UBSAN.filter'. For an example of UBSAN.filter, which includes current startup issues see: https://github.com/mariadb-corporation/mariadb-qa/blob/master/UBSAN.filter