[MDEV-28882] Assertion `tmp >= 0' failed in best_access_path Created: 2022-06-17  Updated: 2023-12-07

Status: Stalled
Project: MariaDB Server
Component/s: Optimizer
Affects Version/s: N/A
Fix Version/s: 10.11

Type: Bug Priority: Critical
Reporter: Elena Stepanova Assignee: Sergei Petrunia
Resolution: Unresolved Votes: 0
Labels: affects-tests

Issue Links:
Relates
relates to MDEV-28852 Improve optimization of joins with ma... Closed

 Description   

We've seen this assertion failure before, but this time it seems to be related to

commit b254844ed197cfe0b305ff37442b03a4573ee931
Author: Monty
Date:   Tue May 31 17:36:32 2022 +0300
 
    Improve pruning in greedy_search by sorting tables during search
    
    MDEV-28073 Slow query performance in MariaDB when using many tables

SET histogram_type= JSON_HB;
 
CREATE TABLE t1 (a varchar(1)) ENGINE=MyISAM;
 
INSERT INTO t1 VALUES ('o'),('s'),('j'),('s'),('y'),('s'),('l'),
  ('q'),('x'),('m'),('t'),('d'),('v'),('j'),('p'),('t'),('b'),('q');
 
CREATE TABLE t2 (b varchar(1), c int, KEY(b), KEY(c)) ENGINE=MyISAM;
INSERT INTO t2 VALUES ('r',NULL),('a',NULL),('r',1);
 
ANALYZE TABLE t1, t2 PERSISTENT FOR ALL;
SELECT * FROM t1 JOIN t2 ON a = b WHERE a > 'y' AND c != 9;
 
DROP TABLE t1, t2;

preview-10.10-MDEV-28852-optimizer a81e3e159

sql/sql_select.cc:8324: void best_access_path(JOIN*, JOIN_TAB*, table_map, const POSITION*, uint, bool, double, POSITION*, POSITION*): Assertion `tmp >= 0' failed.
220617 21:53:29 [ERROR] mysqld got signal 6 ;
 
#7  0x00007f3d62e4d662 in __GI___assert_fail (assertion=0x56272d6d1b2f "tmp >= 0", file=0x56272d6d00c0 "/data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_select.cc", line=8324, function=0x56272d6d1a28 "void best_access_path(JOIN*, JOIN_TAB*, table_map, const POSITION*, uint, bool, double, POSITION*, POSITION*)") at assert.c:101
No locals.
#8  0x000056272bf2dca3 in best_access_path (join=0x7f3d4c0187e0, s=0x7f3d4c064700, remaining_tables=2, join_positions=0x7f3d4c064cd8, idx=1, disable_jbuf=false, record_count=1.7976931348623153e+308, pos=0x7f3d4c0653d8, loose_scan_pos=0x7f3d4c0654d8) at /data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_select.cc:8324
        rows = inf
        access_cost_factor = 0
        found_part = 1
        key = 0
        keyinfo = 0x7f3d4c05a060
        key_parts = 1
        notnull_part = 1
        found_ref = 1
        ft_key = false
        ref_or_null_part = 0
        key_flags = 64
        const_part = 0
        all_parts = 1
        trace_access_idx = {<Json_writer_struct> = {_vptr.Json_writer_struct = 0x56272e01c308 <vtable for Json_writer_object+16>, my_writer = 0x0, context = {writer = 0x0}, closed = false}, <No data fields>}
        keyuse = 0x7f3d4c065ce8
        max_key_part = 4294967295
        start_key = 0x7f3d4c065c90
        table = 0x7f3d4c203538
        best_records = 1.7976931348623157e+308
        thd = 0x7f3d4c000db8
        use_cond_selectivity = 4
        best_key = 0x0
        best_max_key_part = 0
        found_constraint = 1 '\001'
        best = 1.7976931348623157e+308
        best_time = 1.7976931348623157e+308
        records = 1.5
        best_ref_depends_map = 0
        key_dependent = 0
        best_filter = 0x0
        tmp = -nan(0x8000000000000)
        rec = 10
        best_uses_jbuf = false
        eq_join_set = 0x7f3d4c2036c8
        hj_start_key = 0x0
        spl_plan = 0x0
        filter = 0x7f3d4c066628
        cause = 0x0
        best_type = JT_UNKNOWN
        type = JT_REF
        loose_scan_opt = {try_loosescan = false, bound_sj_equalities = 0, handled_sj_equalities = 0, loose_scan_keyparts = 0, max_loose_keypart = 0, part1_conds_met = false, quick_uses_applicable_index = 0, quick_max_loose_keypart = 0, best_loose_scan_key = 0, best_loose_scan_cost = 1.7976931348623157e+308, best_loose_scan_records = 0, best_loose_scan_start_key = 0x0, best_max_loose_keypart = 0, best_ref_depend_map = 0}
        _db_stack_frame_ = {func = 0x56272d6d1e22 "get_plans_for_tables", file = 0x56272d6d00c0 "/data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_select.cc", level = 2147483662, line = -1, prev = 0x7f3d5d7f5d40}
        trace_wrapper = {<Json_writer_struct> = {_vptr.Json_writer_struct = 0x56272e01c308 <vtable for Json_writer_object+16>, my_writer = 0x0, context = {writer = 0x0}, closed = false}, <No data fields>}
        trace_paths = {<Json_writer_struct> = {_vptr.Json_writer_struct = 0x56272e01c2e8 <vtable for Json_writer_array+16>, my_writer = 0x0, context = {writer = 0x0}, closed = false}, <No data fields>}
        __PRETTY_FUNCTION__ = "void best_access_path(JOIN*, JOIN_TAB*, table_map, const POSITION*, uint, bool, double, POSITION*, POSITION*)"
        trace_access_scan = {<Json_writer_struct> = {_vptr.Json_writer_struct = 0x56272e01c308 <vtable for Json_writer_object+16>, my_writer = 0x0, context = {writer = 0x0}, closed = false}, <No data fields>}
#9  0x000056272bf354d3 in get_costs_for_tables (join=0x7f3d4c0187e0, remaining_tables=2, idx=1, record_count=1.7976931348623153e+308, trace_one_table=0x7f3d5d7f5ef0, pos=0x7f3d4c064ca8, store_position=0x7f3d5d7f5e28, allowed_tables=0x7f3d5d7f5e38, stop_on_eq_ref=false) at /data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_select.cc:10036
        wrapper = {<Json_writer_struct> = {_vptr.Json_writer_struct = 0x56272e01c308 <vtable for Json_writer_object+16>, my_writer = 0x0, context = {writer = 0x0}, closed = false}, <No data fields>}
        real_table_bit = 2
        thd = 0x7f3d4c000db8
        sort_position = 0x7f3d4c0653d8
        sort_end = 0x7f3d5d7f5db0
        s = 0x7f3d4c064700
        found_tables = 0
        found_eq_ref = false
        disable_jbuf = false
        _db_stack_frame_ = {func = 0x56272d6d1f70 "best_extension_by_limited_search", file = 0x56272d6d00c0 "/data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_select.cc", level = 2147483661, line = -1, prev = 0x7f3d5d7f5ed0}
        __PRETTY_FUNCTION__ = "bool get_costs_for_tables(JOIN*, table_map, uint, double, Json_writer_object*, JOIN_TAB**, SORT_POSITION**, table_map*, bool)"
#10 0x000056272bf36387 in best_extension_by_limited_search (join=0x7f3d4c0187e0, remaining_tables=2, idx=1, record_count=1.7976931348623153e+308, read_time=3.5953862697246305e+307, search_depth=61, prune_level=2, use_cond_selectivity=4, processed_eq_ref_tables=0x7f3d5d7f5ff0) at /data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_select.cc:10299
        table_map = 2
        trace_one_table = {<Json_writer_struct> = {_vptr.Json_writer_struct = 0x56272e01c308 <vtable for Json_writer_object+16>, my_writer = 0x0, context = {writer = 0x0}, closed = false}, <No data fields>}
        best_ref = 0x7f3d4c064ca8
        arr = {<Json_writer_struct> = {_vptr.Json_writer_struct = 0x56272e01c2e8 <vtable for Json_writer_array+16>, my_writer = 0x0, context = {writer = 0x0}, closed = false}, <No data fields>}
        thd = 0x7f3d4c000db8
        s = 0x4c064cb0
        best_record_count = 1.7976931348623157e+308
        best_read_time = 1.7976931348623157e+308
        best_res = SEARCH_OK
        tables_left = 1
        found_tables = 0
        accepted_tables = 32512
        found_eq_ref_tables = 0
        used_eq_ref_table = 0
        allowed_tables = 2
        allowed_current_tables = 2
        sort = 0x7f3d5d7f5db0
        sort_end = 0x7f3d5d7f5db0
        _db_stack_frame_ = {func = 0x56272d6d1f70 "best_extension_by_limited_search", file = 0x56272d6d00c0 "/data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_select.cc", level = 2147483660, line = -1, prev = 0x7f3d5d7f60a0}
        __PRETTY_FUNCTION__ = "enum_best_search best_extension_by_limited_search(JOIN*, table_map, uint, double, double, uint, uint, uint, table_map*)"
#11 0x000056272bf3751f in best_extension_by_limited_search (join=0x7f3d4c0187e0, remaining_tables=3, idx=0, record_count=1, read_time=0, search_depth=62, prune_level=2, use_cond_selectivity=4, processed_eq_ref_tables=0x7f3d5d7f6178) at /data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_select.cc:10446
        trace_rest = {<Json_writer_struct> = {_vptr.Json_writer_struct = 0x56272e01c2e8 <vtable for Json_writer_array+16>, my_writer = 0x0, context = {writer = 0x0}, closed = false}, <No data fields>}
        trace_one_table = {<Json_writer_struct> = {_vptr.Json_writer_struct = 0x56272e01c308 <vtable for Json_writer_object+16>, my_writer = 0x0, context = {writer = 0x0}, closed = false}, <No data fields>}
        filter_cmp_gain = 0
        pushdown_cond_selectivity = 1
        real_table_bit = 1
        current_record_count = 1.7976931348623153e+308
        current_read_time = 3.5953862697246305e+307
        partial_join_cardinality = 1.7976931348623153e+308
        position = 0x7f3d4c064cd8
        loose_scan_pos = 0x7f3d4c0652d8
        pos = 0x7f3d5d7f5f70
        thd = 0x7f3d4c000db8
        s = 0x7f3d4c064348
        best_record_count = 1.7976931348623153e+308
        best_read_time = 3.5953862697246305e+307
        best_res = 1275071712
        tables_left = 2
        found_tables = 2
        accepted_tables = 1
        found_eq_ref_tables = 0
        used_eq_ref_table = 0
        allowed_tables = 3
        allowed_current_tables = 3
        sort = 0x7f3d5d7f5f70
        sort_end = 0x7f3d5d7f5f90
        _db_stack_frame_ = {func = 0x56272d6d1d33 "greedy_search", file = 0x56272d6d00c0 "/data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_select.cc", level = 2147483659, line = -1, prev = 0x7f3d5d7f61b0}
        __PRETTY_FUNCTION__ = "enum_best_search best_extension_by_limited_search(JOIN*, table_map, uint, double, double, uint, uint, uint, table_map*)"
#12 0x000056272bf322c9 in greedy_search (join=0x7f3d4c0187e0, remaining_tables=3, search_depth=62, prune_level=2, use_cond_selectivity=4) at /data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_select.cc:9282
        is_interleave_error = false
        pos = 0x7f3d5d7f61c0
        record_count = 1
        read_time = 0
        idx = 0
        best_idx = 0
        size_remain = 2
        usable_tables = 3
        eq_ref_tables = 0
        best_pos = {table = 0x0, records_read = 0, cond_selectivity = 0, read_time = 0, prefix_record_count = 0, prefix_cost = 2.7519649000858304e-314, key = 0x0, spl_plan = 0x0, range_rowid_filter_info = 0x0, ref_depend_map = 0, key_dependent = 0, dups_producing_tables = 0, inner_tables_handled_with_other_sjs = 0, dups_weedout_picker = {<Semi_join_strategy_picker> = {_vptr.Semi_join_strategy_picker = 0x56272e03ad80 <vtable for Duplicate_weedout_picker+16>}, first_dupsweedout_table = 61, dupsweedout_tables = 0, is_used = false}, firstmatch_picker = {<Semi_join_strategy_picker> = {_vptr.Semi_join_strategy_picker = 0x56272e03adc0 <vtable for Firstmatch_picker+16>}, first_firstmatch_table = 61, first_firstmatch_rtbl = 139901538362208, firstmatch_need_tables = 139901538362032, is_used = false}, loosescan_picker = {<Semi_join_strategy_picker> = {_vptr.Semi_join_strategy_picker = 0x56272e03ae00 <vtable for LooseScan_picker+16>}, first_loosescan_table = 61, loosescan_need_tables = 0, loosescan_key = 1568629600, loosescan_parts = 32573, is_used = false}, sjmat_picker = {<Semi_join_strategy_picker> = {_vptr.Semi_join_strategy_picker = 0x56272e03ae40 <vtable for Sj_materialization_picker+16>}, is_used = false, sjm_scan_last_inner = 0, sjm_scan_need_tables = 0}, sj_strategy = SJ_OPT_NONE, type = JT_UNKNOWN, n_sj_tables = 0, use_join_buffer = false}
        best_table = 0x56272d73ce4f
        n_tables = 2
        _db_stack_frame_ = {func = 0x56272d6d1c2f "choose_plan", file = 0x56272d6d00c0 "/data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_select.cc", level = 2147483658, line = -1, prev = 0x7f3d5d7f6320}
        __PRETTY_FUNCTION__ = "bool greedy_search(JOIN*, table_map, uint, uint, uint)"
#13 0x000056272bf30ea5 in choose_plan (join=0x7f3d4c0187e0, join_tables=3) at /data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_select.cc:8847
        search_depth = 62
        prune_level = 2
        use_cond_selectivity = 4
        straight_join = false
        thd = 0x7f3d4c000db8
        _db_stack_frame_ = {func = 0x56272d6d137b "make_join_statistics", file = 0x56272d6d00c0 "/data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_select.cc", level = 2147483657, line = -1, prev = 0x7f3d5d7f6570}
        jtab_sort_func = 0x56272bf3130f <join_tab_cmp(void const*, void const*, void const*)>
        wrapper = {<Json_writer_struct> = {_vptr.Json_writer_struct = 0x56272e01c308 <vtable for Json_writer_object+16>, my_writer = 0x0, context = {writer = 0x0}, closed = false}, <No data fields>}
        trace_plan = {<Json_writer_struct> = {_vptr.Json_writer_struct = 0x56272e01c2e8 <vtable for Json_writer_array+16>, my_writer = 0x0, context = {writer = 0x0}, closed = false}, <No data fields>}
        __PRETTY_FUNCTION__ = "bool choose_plan(JOIN*, table_map)"
#14 0x000056272bf2213b in make_join_statistics (join=0x7f3d4c0187e0, tables_list=@0x7f3d4c015750: {<base_list> = {<Sql_alloc> = {<No data fields>}, first = 0x7f3d4c018db8, last = 0x7f3d4c018dc8, elements = 2}, <No data fields>}, keyuse_array=0x7f3d4c018b20) at /data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_select.cc:5968
        records = 1
        unit = 0x7f3d4c0051e8
        error = 0
        table = 0x7f3d4c203538
        i = 2
        table_count = 2
        const_count = 0
        key = 0
        sort_space = 6
        found_const_table_map = 0
        all_table_map = 3
        const_ref = {buffer = {0}}
        eq_part = {buffer = {1}}
        has_expensive_keyparts = false
        table_vector = 0x7f3d4c064cb8
        stat = 0x7f3d4c064348
        stat_end = 0x7f3d4c064ab8
        s = 0x7f3d4c064ab8
        stat_ref = 0x7f3d4c064ab8
        stat_vector = 0x7f3d4c064ca0
        keyuse = 0x7f3d4c065ce8
        start_keyuse = 0x7f3d4c065c90
        outer_join = 0
        no_rows_const_tables = 0
        sargables = 0x7f3d4c065c80
        ti = {<base_list_iterator> = {list = 0x7f3d4c015750, el = 0x56272ed103e0 <end_of_list>, prev = 0x7f3d4c018dc8, current = 0x56272ed103e0 <end_of_list>}, <No data fields>}
        tables = 0x0
        thd = 0x7f3d4c000db8
        _db_stack_frame_ = {func = 0x56272d6d085d "JOIN::optimize_inner", file = 0x56272d6d00c0 "/data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_select.cc", level = 2147483656, line = -1, prev = 0x7f3d5d7f66c0}
        ref_changed = 0
        __PRETTY_FUNCTION__ = "bool make_join_statistics(JOIN*, List<TABLE_LIST>&, DYNAMIC_ARRAY*)"
#15 0x000056272bf0b1cc in JOIN::optimize_inner (this=0x7f3d4c0187e0) at /data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_select.cc:2511
        _db_stack_frame_ = {func = 0x56272d6d12ff "mysql_select", file = 0x56272d6d00c0 "/data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_select.cc", level = 2147483655, line = -1, prev = 0x7f3d5d7f6820}
        trace_wrapper = {<Json_writer_struct> = {_vptr.Json_writer_struct = 0x56272e01c308 <vtable for Json_writer_object+16>, my_writer = 0x0, context = {writer = 0x0}, closed = false}, <No data fields>}
        trace_prepare = {<Json_writer_struct> = {_vptr.Json_writer_struct = 0x56272e01c308 <vtable for Json_writer_object+16>, my_writer = 0x0, context = {writer = 0x0}, closed = false}, <No data fields>}
        trace_steps = {<Json_writer_struct> = {_vptr.Json_writer_struct = 0x56272e01c2e8 <vtable for Json_writer_array+16>, my_writer = 0x0, context = {writer = 0x0}, closed = false}, <No data fields>}
        sel = 0x7f3d4c015538
        eq_list = {<base_list> = {<Sql_alloc> = {<No data fields>}, first = 0x56272ed103e0 <end_of_list>, last = 0x7f3d5d7f66a0, elements = 0}, <No data fields>}
        ignore_on_expr = false
        __PRETTY_FUNCTION__ = "int JOIN::optimize_inner()"
#16 0x000056272bf068f9 in JOIN::optimize (this=0x7f3d4c0187e0) at /data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_select.cc:1850
        res = 0
        init_state = JOIN::NOT_OPTIMIZED
#17 0x000056272bf1c20b in mysql_select (thd=0x7f3d4c000db8, tables=0x7f3d4c015b10, fields=@0x7f3d4c0157d8: {<base_list> = {<Sql_alloc> = {<No data fields>}, first = 0x7f3d4c015ac8, last = 0x7f3d4c019178, elements = 3}, <No data fields>}, conds=0x7f3d4c017d28, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147748608, result=0x7f3d4c0187b8, unit=0x7f3d4c0051e8, select_lex=0x7f3d4c015538) at /data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_select.cc:5038
        err = 0
        free_join = true
        _db_stack_frame_ = {func = 0x56272d6d01d0 "handle_select", file = 0x56272d6d00c0 "/data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_select.cc", level = 2147483654, line = -1, prev = 0x7f3d5d7f68e0}
        join = 0x7f3d4c0187e0
#18 0x000056272befcedf in handle_select (thd=0x7f3d4c000db8, lex=0x7f3d4c005110, result=0x7f3d4c0187b8, setup_tables_done_option=0) at /data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_select.cc:583
        unit = 0x7f3d4c0051e8
        res = false
        select_lex = 0x7f3d4c015538
        _db_stack_frame_ = {func = 0x56272d6bb8a8 "mysql_execute_command", file = 0x56272d6bab40 "/data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_parse.cc", level = 2147483653, line = -1, prev = 0x7f3d5d7f6f30}
#19 0x000056272be5384a in execute_sqlcom_select (thd=0x7f3d4c000db8, all_tables=0x7f3d4c015b10) at /data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_parse.cc:6260
        save_protocol = 0x0
        lex = 0x7f3d4c005110
        result = 0x7f3d4c0187b8
        res = false
        __PRETTY_FUNCTION__ = "bool execute_sqlcom_select(THD*, TABLE_LIST*)"
#20 0x000056272be4029f in mysql_execute_command (thd=0x7f3d4c000db8, is_called_from_prepared_stmt=false) at /data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_parse.cc:3944
        privileges_requested = SELECT_ACL
        res = 0
        up_result = 0
        lex = 0x7f3d4c005110
        select_lex = 0x7f3d4c015538
        first_table = 0x7f3d4c015b10
        all_tables = 0x7f3d4c015b10
        unit = 0x7f3d4c0051e8
        have_table_map_for_update = false
        rpl_filter = 0x56272e021310 <vtable for Internal_error_handler+16>
        _db_stack_frame_ = {func = 0x56272d6bcc3f "mysql_parse", file = 0x56272d6bab40 "/data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_parse.cc", level = 2147483652, line = -1, prev = 0x7f3d5d7f7380}
        __PRETTY_FUNCTION__ = "int mysql_execute_command(THD*, bool)"
        ots = {ctx = 0x7f3d4c004da8, traceable = false}
        orig_binlog_format = BINLOG_FORMAT_MIXED
        orig_current_stmt_binlog_format = BINLOG_FORMAT_STMT
#21 0x000056272be5cc04 in mysql_parse (thd=0x7f3d4c000db8, rawbuf=0x7f3d4c015460 "SELECT * FROM t1 JOIN t2 ON a = b WHERE a > 'y' AND c != 9", length=58, parser_state=0x7f3d5d7f7500) at /data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_parse.cc:8036
        found_semicolon = 0x0
        error = 32573
        lex = 0x7f3d4c005110
        err = false
        _db_stack_frame_ = {func = 0x56272d6bb0e9 "dispatch_command", file = 0x56272d6bab40 "/data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_parse.cc", level = 2147483651, line = -1, prev = 0x7f3d5d7f74e0}
        __PRETTY_FUNCTION__ = "void mysql_parse(THD*, char*, uint, Parser_state*)"
#22 0x000056272be33f05 in dispatch_command (command=COM_QUERY, thd=0x7f3d4c000db8, packet=0x7f3d4c00b9e9 "SELECT * FROM t1 JOIN t2 ON a = b WHERE a > 'y' AND c != 9", packet_length=58, blocking=true) at /data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_parse.cc:1894
        packet_end = 0x7f3d4c01549a ""
        parser_state = {m_lip = {lookahead_token = -1, lookahead_yylval = 0x0, m_thd = 0x7f3d4c000db8, m_ptr = 0x7f3d4c01549b "\004", m_tok_start = 0x7f3d4c01549b "\004", m_tok_end = 0x7f3d4c01549b "\004", m_end_of_query = 0x7f3d4c01549a "", m_tok_start_prev = 0x7f3d4c01549a "", m_buf = 0x7f3d4c015460 "SELECT * FROM t1 JOIN t2 ON a = b WHERE a > 'y' AND c != 9", m_buf_length = 58, m_echo = true, m_echo_saved = false, m_cpp_buf = 0x7f3d4c0154f8 "SELECT * FROM t1 JOIN t2 ON a = b WHERE a > 'y' AND c != 9", m_cpp_ptr = 0x7f3d4c015532 "", m_cpp_tok_start = 0x7f3d4c015532 "", m_cpp_tok_start_prev = 0x7f3d4c015532 "", m_cpp_tok_end = 0x7f3d4c015532 "", m_body_utf8 = 0x0, m_body_utf8_ptr = 0x14c000cb8 <error: Cannot access memory at address 0x14c000cb8>, m_cpp_utf8_processed_ptr = 0x0, next_state = MY_LEX_END, found_semicolon = 0x0, ignore_space = false, stmt_prepare_mode = false, multi_statements = true, yylineno = 1, m_digest = 0x0, in_comment = NO_COMMENT, in_comment_saved = (unknown: 0x10000), m_cpp_text_start = 0x7f3d4c015531 "9", m_cpp_text_end = 0x7f3d4c015532 "", m_underscore_cs = 0x0}, m_yacc = {yacc_yyss = 0x0, yacc_yyvs = 0x0, m_set_signal_info = {m_item = {0x0 <repeats 13 times>}}, m_lock_type = TL_READ_DEFAULT, m_mdl_type = MDL_SHARED_READ}, m_digest_psi = 0x7f3d4c004b38}
        net = 0x7f3d4c0010f0
        error = false
        do_end_of_statement = true
        _db_stack_frame_ = {func = 0x56272d6bad6d "do_command", file = 0x56272d6bab40 "/data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_parse.cc", level = 2147483650, line = -1, prev = 0x7f3d5d7f7dc0}
        drop_more_results = false
        __PRETTY_FUNCTION__ = "dispatch_command_return dispatch_command(enum_server_command, THD*, char*, uint, bool)"
        __FUNCTION__ = "dispatch_command"
        res = <optimized out>
#23 0x000056272be310de in do_command (thd=0x7f3d4c000db8, blocking=true) at /data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_parse.cc:1407
        return_value = DISPATCH_COMMAND_SUCCESS
        packet = 0x7f3d4c00b9e8 "\003SELECT * FROM t1 JOIN t2 ON a = b WHERE a > 'y' AND c != 9"
        packet_length = 59
        net = 0x7f3d4c0010f0
        command = COM_QUERY
        _db_stack_frame_ = {func = 0x56272db40749 "?func", file = 0x56272db4074f "?file", level = 2147483649, line = -1, prev = 0x0}
        __PRETTY_FUNCTION__ = "dispatch_command_return do_command(THD*, bool)"
        __FUNCTION__ = "do_command"
#24 0x000056272c19b9f7 in do_handle_one_connection (connect=0x562731f08f78, put_in_cache=true) at /data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_connect.cc:1418
        create_user = true
        thr_create_utime = 1914121414503
        thd = 0x7f3d4c000db8
        __PRETTY_FUNCTION__ = "void do_handle_one_connection(CONNECT*, bool)"
#25 0x000056272c19b1fe in handle_one_connection (arg=0x562731f0b958) at /data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/sql/sql_connect.cc:1312
        connect = 0x562731f0b958
#26 0x000056272caa9fe7 in pfs_spawn_thread (arg=0x562731f08ae8) at /data/src/preview-10.10-MDEV-28852-optimizer-gcov-Jun17/storage/perfschema/pfs.cc:2201
        typed_arg = 0x562731f08ae8
        user_arg = 0x562731f0b958
        user_start_routine = 0x56272c19b149 <handle_one_connection(void*)>
        pfs = 0x7f3d627415c0
        klass = 0x562731941540
#27 0x00007f3d63317ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477
        ret = <optimized out>
        pd = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139901538371328, -2832153553564932155, 140736454476238, 140736454476239, 139901538369408, 311296, 2794643718716975045, 2794744409914927045}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = 0
#28 0x00007f3d62f16def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95



 Comments   
Comment by Sergei Petrunia [ 2022-06-21 ]

The first odd thing I was able to observe:

  3562	          rows= records_in_column_ranges(&param, idx, key);
  3563	          if (rows != DBL_MAX)
  3564	          {
  3565	            key->field->cond_selectivity= rows/table_records;
  3566	            selectivity_for_column.add("selectivity_from_histogram",
  3567	                                       key->field->cond_selectivity);

(gdb) p rows
  $123 = -7.1999997963700935e-08
(gdb) p table->alias.Ptr
  $124 = 0x7fffe0046c48 "t1"

(didn't check if this is the cause of the issue, yet)

Comment by Sergei Petrunia [ 2022-06-21 ]

The problem of bad estimate is the JSON_HB histogram.
Histogram_json_hb::range_selectivity() is invoked for this range:

t1.a > 'y'

and it produces negative selectivity. This happens, because max>min.:

(gdb) print min
  $154 = 1.0000000039999999
(gdb) print max
  $153 = 1

Comment by Sergei Petrunia [ 2022-06-21 ]

Histogram (pretty-printed):

{
  "target_histogram_size": 254,
  "collected_at": "2022-06-21 19:25:09",
  "collected_by": "10.10.0-MariaDB-debug-log",
  "histogram_hb": [
    { "start": "b",             "size": 0.055555556, "ndv": 1 },
    { "start": "d",             "size": 0.055555556, "ndv": 1 },
    { "start": "j",             "size": 0.111111111, "ndv": 1 },
    { "start": "l",             "size": 0.055555556, "ndv": 1 },
    { "start": "m",             "size": 0.055555556, "ndv": 1 },
    { "start": "o",             "size": 0.055555556, "ndv": 1 },
    { "start": "p",             "size": 0.055555556, "ndv": 1 },
    { "start": "q",             "size": 0.111111111, "ndv": 1 },
    { "start": "s",             "size": 0.166666667, "ndv": 1 },
    { "start": "t",             "size": 0.111111111, "ndv": 1 },
    { "start": "v",             "size": 0.055555556, "ndv": 1 },
    { "start": "x",             "size": 0.055555556, "ndv": 1 },
    { "start": "y", "end": "y", "size": 0.055555556, "ndv": 1}
  ]
}

Manually re-computing the values:

MariaDB [test]> with T as (select distinct a from t1) select T.a, count(*) / @cnt from T join t1 on T.a=t1.a  group by t1.a;
+------+-----------------+
| a    | count(*) / @cnt |
+------+-----------------+
| b    |          0.0556 |
| d    |          0.0556 |
| j    |          0.1111 |
| l    |          0.0556 |
| m    |          0.0556 |
| o    |          0.0556 |
| p    |          0.0556 |
| q    |          0.1111 |
| s    |          0.1667 |
| t    |          0.1111 |
| v    |          0.0556 |
| x    |          0.0556 |
| y    |          0.0556 |
+------+-----------------+
13 rows in set (0.009 sec)

Comment by Sergei Petrunia [ 2022-06-21 ]

MariaDB [test]> set @hist='[
    '>     { "start": "b",             "size": 0.055555556, "ndv": 1 },
    '>     { "start": "d",             "size": 0.055555556, "ndv": 1 },
    '>     { "start": "j",             "size": 0.111111111, "ndv": 1 },
    '>     { "start": "l",             "size": 0.055555556, "ndv": 1 },
    '>     { "start": "m",             "size": 0.055555556, "ndv": 1 },
    '>     { "start": "o",             "size": 0.055555556, "ndv": 1 },
    '>     { "start": "p",             "size": 0.055555556, "ndv": 1 },
    '>     { "start": "q",             "size": 0.111111111, "ndv": 1 },
    '>     { "start": "s",             "size": 0.166666667, "ndv": 1 },
    '>     { "start": "t",             "size": 0.111111111, "ndv": 1 },
    '>     { "start": "v",             "size": 0.055555556, "ndv": 1 },
    '>     { "start": "x",             "size": 0.055555556, "ndv": 1 },
    '>     { "start": "y", "end": "y", "size": 0.055555556, "ndv": 1 }
    '>   ]';

MariaDB [test]> select sum(size) from json_table(@hist, '$[*]' columns (size decimal(16,15) path '$.size')) TBL;
+-------------------+
| sum(size)         |
+-------------------+
| 1.000000004000000 |
+-------------------+

Comment by Sergei Petrunia [ 2022-06-21 ]

Looks like a rounding error to me.

Comment by Sergei Petrunia [ 2022-06-21 ]

A simpler testcase:

CREATE TABLE t1 (a varchar(1));
 
INSERT INTO t1 VALUES ('o'),('s'),('j'),('s'),('y'),('s'),('l'),
  ('q'),('x'),('m'),('t'),('d'),('v'),('j'),('p'),('t'),('b'),('q');
 
set histogram_type=json_hb;
analyze table t1 persistent for all;
explain format=json select * from t1 where a > 'y';

shows

{
  "query_block": {
    "select_id": 1,
    "nested_loop": [
      {
        "table": {
          "table_name": "t1",
          "access_type": "ALL",
          "rows": 18,
          "filtered": -4e-7,
          "attached_condition": "t1.a > 'y'"
        }
      }
    ]
  }

Comment by Sergei Petrunia [ 2022-06-22 ]

10.8 part of the fix:

commit c76c97ddee531d0f08f36d45ec264473784383a8 (HEAD -> bb-10.8-mdev28882, origin/bb-10.8-mdev28882)
Author: Sergei Petrunia <sergey@mariadb.com>
Date:   Wed Jun 22 11:39:53 2022 +0300
 
    MDEV-28882: Assertion `tmp >= 0' failed in best_access_path
    
    Histogram_json_hb::range_selectivity() may return small negative
    numbers due to rounding errors in the histogram.
    
    Make sure the returned value is non-negative.
    Add an assert to catch negative values that are not small.

Comment by Sergei Petrunia [ 2022-06-22 ]

Got approval for this from oleg.smirnov, pushed it into 10.8

MDEV-28882: Assertion `tmp >= 0' failed in best_access_path
Histogram_json_hb::range_selectivity() may return small negative
numbers due to rounding errors in the histogram.
 
Make sure the returned value is non-negative.
Add an assert to catch negative values that are not small.
 
(attempt #2)

Comment by Sergei Petrunia [ 2022-06-22 ]

But why are the consequences of negative #rows so bad?

In both 10.10 and 10.8, negative estimate numbers cause very high estimates here:

      current_record_count= COST_MULT(record_count, position->records_read);

we have

(gdb) p record_count
  $45 = 1
(gdb) p position->records_read
  $46 = -7.1999997963700935e-08

And we get:

(gdb) print current_record_count
  $48 = 1.7976931348623153e+308

This is because COST_MULT doesn't work for negative numbers:

#define COST_MULT(c,f) (COST_MAX / (f) > (c) ? (c) * (f) : COST_MAX)

Put f=-7e-08, c=1

  COST_MAX / (f=negative_number) = negative_number.

this is less than c=1, any positive number. So we get COST_MAX as result.

Comment by Sergei Petrunia [ 2022-06-22 ]

On a difference between 10.8 and preview-10.10

In 10.8, the join orders are considered in this order:

t2
t2 t1
t1  
// Here we get COST_MAX as prefix cost and don't investigate further.

in 10.10, the join prefixes are considered in this order:

t2
t1
// t1 is cheaper.
Put t1 into the join prefix. 
Try to consider: t1, t2
Crash.

I do not see any problem in the code that's new to 10.10.

There are some possible faults in the older code.

Comment by Sergei Petrunia [ 2023-09-14 ]

elenst, there was another similar testcase presented somewhere that wasn't covered by this patch. Somehow I dont' see if it got its own MDEV. Let me find it and act on this MDEV then.

Generated at Thu Feb 08 10:04:11 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.