The assert was added by
marko.makela@oracle.com-20111107113719-geae9ea2tpy1li52
Mon 2011-11-07 13:37:19 +0200

Add debug assertions to catch Bug#13345378 earlier.

In all callers of row_sel_convert_mysql_key_to_innobase(), assert
that the converted key is empty or nonempty when it should be.

MySQL-5.1 does that also:

#4 0x00000000008b541c in ha_innobase::records_in_range (this=0x7fffe0014468, keynr=0, min_key=0x0, max_key=0x0) at handler/ha_innodb.cc:6174
#5 0x0000000000794ef4 in check_quick_keys (param=0x7ffff40fa340, idx=0, key_tree=0x7fffe0015a48, min_key=0x7ffff40fa6b0 "D\253\017\364\377\177", min_key_flag=0, min_keypart=-1, max_key=0x7ffff40fb5ae "", max_key_flag=0, max_keypart=-1) at opt_range.cc:7690
#6 0x0000000000794413 in check_quick_select (param=0x7ffff40fa340, idx=0, tree=0x7fffe0015a48, update_tbl_stats=true) at opt_range.cc:7465
#7 0x000000000078d3fa in get_key_scans_params (param=0x7ffff40fa340, tree=0x7fffe0015290, index_read_must_be_used=false, update_tbl_stats=true, read_time=3.3000000000000003) at opt_range.cc:4865
#8 0x0000000000787d3f in SQL_SELECT::test_quick_select (this=0x7fffe0006038, thd=0x175d6d8, keys_to_use=..., prev_tables=0, limit=18446744073709551615, force_quick_range=false) at opt_range.cc:2372
#9 0x00000000006e8792 in get_quick_record_count (thd=0x175d6d8, select=0x7fffe0006038, table=0x7fffe0001318, keys=0x7fffe0005c08, limit=18446744073709551615) at sql_select.cc:2600
#10 0x00000000006e9e67 in make_join_statistics (join=0x7fffe0002568, tables_arg=0x7fffe0004ce8, conds=0x7fffe0005610, keyuse_array=0x7fffe0003b28) at sql_select.cc:3042
#11 0x00000000006e2631 in JOIN::optimize (this=0x7fffe0002568) at sql_select.cc:1025
#12 0x00000000006e8479 in mysql_select (thd=0x175d6d8, rref_pointer_array=0x175f7b8, tables=0x7fffe0004ce8, wild_num=0, fields=..., conds=0x7fffe0005610, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147764736, result=0x7fffe00057b8, unit=0x175f1c0, select_lex=0x175f5e8) at sql_select.cc:2539
#13 0x00000000006e0016 in handle_select (thd=0x175d6d8, lex=0x175f120, result=0x7fffe00057b8, setup_tables_done_option=0) at sql_select.cc:269
#14 0x000000000068017d in execute_sqlcom_select (thd=0x175d6d8, all_tables=0x7fffe0004ce8) at sql_parse.cc:5206
#15 0x00000000006773da in mysql_execute_command (thd=0x175d6d8) at sql_parse.cc:2336
#16 0x000000000068250b in mysql_parse (thd=0x175d6d8, rawbuf=0x7fffe0004b48 "select 1 from t1 where b <= 1 and a <> ''", length=41, found_semicolon=0x7ffff40fec70)

It will also crash.

Bazaar history looks weird: there is a changeset that adds an assert for while-domain ranges, but there is no changeset with an actual fix.

[psergey@pylon-fedora15 mysql-5.6-new]$ bzr log --include-merges | grep -A2 -B5 13345378
revno: 2661.809.52
committer: Marko Mäkelä <marko.makela@oracle.com>
branch nick: mysql-5.1
timestamp: Mon 2011-11-07 13:37:19 +0200
message:
Add debug assertions to catch Bug#13345378 earlier.

In all callers of row_sel_convert_mysql_key_to_innobase(), assert
[psergey@pylon-fedora15 mysql-5.6-new]$

The problem can be replicated as follows:

create table t1 (a char(2) charset utf8,b double, primary key (a(1)),key (b));
insert into t1 values ('',1);
select 1 from t1 where b <= 1 and a <> '';

When executing the select, the condition a<>'' causes construction of two ranges,
(-inf, '') and ('', +inf).
Then key_or() combines them into a (-inf, +inf) range.

... which is incorrect because the constant of '' (empty string) was not included in either range. This is not going to cause any visible problems, though: the only effect will be that range access will read some un-necessary rows (which will be then discarded when re-checking the WHERE clause).

Decided to remove the assert, after all.