Re: Crash in JOIN_CACHE::get_offset with join_cache_level=6 in maria 5.3-dsmrr-cpk
Test case:

--source include/have_innodb.inc

SET SESSION join_cache_level=6;

CREATE TABLE `CC` (
`pk` int(11) NOT NULL AUTO_INCREMENT,
`col_int_key` int(11) DEFAULT NULL,
`col_varchar_key` varchar(1) DEFAULT NULL,
PRIMARY KEY (`pk`),
KEY `col_varchar_key` (`col_varchar_key`,`col_int_key`)
) ENGINE=InnoDB AUTO_INCREMENT=30 DEFAULT CHARSET=latin1;
INSERT INTO `CC` VALUES (29,4,'c');
INSERT INTO `CC` VALUES (17,0,'z');
CREATE TABLE `C` (
`pk` int(11) NOT NULL AUTO_INCREMENT,
`col_int_key` int(11) DEFAULT NULL,
`col_varchar_key` varchar(1) DEFAULT NULL,
PRIMARY KEY (`pk`),
KEY `col_varchar_key` (`col_varchar_key`,`col_int_key`)
) ENGINE=InnoDB AUTO_INCREMENT=21 DEFAULT CHARSET=latin1;
INSERT INTO `C` VALUES (11,0,NULL);
INSERT INTO `C` VALUES (18,5,'y');
CREATE TABLE `B` (
`pk` int(11) NOT NULL AUTO_INCREMENT,
`col_int_key` int(11) DEFAULT NULL,
`col_varchar_key` varchar(1) DEFAULT NULL,
PRIMARY KEY (`pk`),
KEY `col_varchar_key` (`col_varchar_key`,`col_int_key`)
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=latin1;
INSERT INTO `B` VALUES (1,7,'f');

SELECT table2 .`col_varchar_key`
FROM C STRAIGHT_JOIN ( B table2 JOIN CC table3 ON table2 .`pk` ) ON table3 .`col_varchar_key` AND table3 .`pk` = table2 .`col_int_key` ;

Re: Crash in JOIN_CACHE::get_offset with join_cache_level=6 in maria 5.3-dsmrr-cpk
Valgrind:

==12613== Thread 23:
==12613== Invalid read of size 1
==12613== at 0x8458D1E: JOIN_CACHE::get_offset(unsigned int, unsigned char*) (sql_select.h:455)
==12613== by 0x8458F85: JOIN_CACHE::get_rec_length(unsigned char*) (sql_select.h:634)
==12613== by 0x8453032: JOIN_CACHE::read_referenced_field(st_cache_field*, unsigned char*, unsigned int*) (sql_join_cache.cc:1511)
==12613== by 0x8455FD2: JOIN_CACHE_BKA::get_next_key(unsigned char**) (sql_join_cache.cc:2512)
==12613== by 0x8455077: bka_range_seq_next(void*, st_key_multi_range*) (sql_join_cache.cc:2180)
==12613== by 0x87F9BC7: DsMrr_impl::dsmrr_fill_key_buffer() (multi_range_read.cc:859)
==12613== by 0x87F78E8: DsMrr_impl::dsmrr_init(handler*, st_range_seq_if*, void*, unsigned int, unsigned int, st_handler_buffer*) (multi_range_read.cc:458)
==12613== by 0x8B45354: ha_innobase::multi_range_read_init(st_range_seq_if*, void*, unsigned int, unsigned int, st_handler_buffer*) (ha_innodb.cc:11213)
==12613== by 0x8455ADD: JOIN_CACHE_BKA::init_join_matching_records(st_range_seq_if*, unsigned int) (sql_join_cache.cc:2396)
==12613== by 0x8455528: JOIN_CACHE_BKA::join_matching_records(bool) (sql_join_cache.cc:2303)
==12613== by 0x845359F: JOIN_CACHE::join_records(bool) (sql_join_cache.cc:1639)
==12613== by 0x84E1D47: sub_select_cache(JOIN*, st_join_table*, bool) (sql_select.cc:12898)
==12613== by 0x845466B: JOIN_CACHE::generate_full_extensions(unsigned char*) (sql_join_cache.cc:1944)
==12613== by 0x84541A9: JOIN_CACHE_BNL::join_matching_records(bool) (sql_join_cache.cc:1836)
==12613== by 0x845359F: JOIN_CACHE::join_records(bool) (sql_join_cache.cc:1639)
==12613== by 0x84E1A6F: sub_select_cache(JOIN*, st_join_table*, bool) (sql_select.cc:12878)

Re: Crash in JOIN_CACHE::get_offset with join_cache_level=6 in maria 5.3-dsmrr-cpk
EXPLAIN of the offending query:

-----------------------------------------------------------------------------------------------------------------------+

-----------------------------------------------------------------------------------------------------------------------+

-----------------------------------------------------------------------------------------------------------------------+

Re: Crash in JOIN_CACHE::get_offset with join_cache_level=6 in maria 5.3-dsmrr-cpk
The problem can be observed in mainline 5.3 code, as well. One needs to modify the query so that table3 is accessed via non-primary key:

MariaDB [dsm1]> alter table CC add key(pk);

and force the table2,C,table3 join order in the query:

MariaDB [dsm1]> explain SELECT table2 .`col_varchar_key` FROM B table2 STRAIGHT_JOIN C STRAIGHT_JOIN CC table3 force index(pk) WHERE table2 .`pk` AND table3 .`col_varchar_key` AND table3 .`pk` = table2 .`col_int_key`;
----------------------------------------------------------------------------------------------------------------------+

----------------------------------------------------------------------------------------------------------------------+

----------------------------------------------------------------------------------------------------------------------+

Then, put a breakpoint at bka_range_seq_next():

Breakpoint 1, bka_range_seq_next (rseq=0xb1246f0, range=0xb124858) at sql_join_cache.cc:2173

And run the SELECT:

MariaDB [dsm1]> SELECT table2 .`col_varchar_key` FROM B table2 STRAIGHT_JOIN C STRAIGHT_JOIN CC table3 force index(pk) WHERE table2 .`pk` AND table3 .`col_varchar_key` AND table3 .`pk` = table2 .`col_int_key`;

When the breakpoint is hit, we can observe that join buffer has two records:

JOIN_CACHE_BKA::get_next_key (this=0xb1246f0, key=0xb124858) at sql_join_cache.cc:2464
(gdb) p this->records
$1 = 2

Continue execution two times, so that we're stopped in the 3rd invocation of bka_range_seq_next().

(gdb) c
Continuing.

Breakpoint 1, bka_range_seq_next (rseq=0xb1246f0, range=0xb124858) at sql_join_cache.cc:2173
(gdb) c
Continuing.

Breakpoint 1, bka_range_seq_next (rseq=0xb1246f0, range=0xb124858) at sql_join_cache.cc:2173

At this point, one would expect that bka_range_seq_next() returns EOF, since all records in the buffer have already been enumerated.

(gdb) next
(gdb) next
(gdb) next
(gdb) next
(gdb) step
JOIN_CACHE_BKA::get_next_key (this=0xb1246f0, key=0xb124858) at sql_join_cache.cc:2464
(gdb) p last_rec_pos - pos
$6 = 0
(gdb) up
#1 0x082f8257 in bka_range_seq_next (rseq=0xb1246f0, range=0xb124858) at sql_join_cache.cc:2177
(gdb) up
#2 0x084d7936 in handler::multi_range_read_next (this=0xb1247b8, range_info=0x9e8765ec) at multi_range_read.cc:259
(gdb) up
#3 0x084d8476 in DsMrr_impl::dsmrr_fill_buffer (this=0xb0ea340) at multi_range_read.cc:491
(gdb) p rowids_buf
$7 = (uchar *) 0xb1459d2 '¥' <repeats 200 times>...

Remember this pointer ^^^^^^

(gdb) down
#2 0x084d7936 in handler::multi_range_read_next (this=0xb1247b8, range_info=0x9e8765ec) at multi_range_read.cc:259
(gdb) down
#1 0x082f8257 in bka_range_seq_next (rseq=0xb1246f0, range=0xb124858) at sql_join_cache.cc:2177
(gdb) down
#0 JOIN_CACHE_BKA::get_next_key (this=0xb1246f0, key=0xb124858) at sql_join_cache.cc:2464
(gdb) p pos
$8 = (uchar *) 0xb1459d2 '¥' <repeats 200 times>...

Now, we see that JOIN_CACHE_BKA::pos points to BKA's rowids buffer (which contains uninitialized data because we haven't written any rowids to it yet).

Yet, if we continue execution we see attempts to read the uninitialized data:

(gdb) next
(gdb) next
(gdb) step
JOIN_CACHE::get_rec_length (this=0xb1246f0, ptr=0xb1459d2 '¥' <repeats 200 times>...) at sql_select.h:634
(gdb) print ptr
$10 = (uchar *) 0xb1459d2 '¥' <repeats 200 times>...
Run till exit from #0 JOIN_CACHE::get_rec_length (this=0xb1246f0, ptr=0xb1459d2 '¥' <repeats 200 times>...) at sql_select.h:634
0x082f996a in JOIN_CACHE_BKA::get_next_key (this=0xb1246f0, key=0xb124858) at sql_join_cache.cc:2471
Value returned is $11 = 165

At this point we're processing apparent garbage. We happen to avoid crash in 5.3-main but crash in 5.3-dsmrr-cpk (because it writes key values at the start of the buffer, which causes join cache code to read different garbage values).

Re: Crash in JOIN_CACHE::get_offset with join_cache_level=6 in maria 5.3-dsmrr-cpk
Igor, could you please take a look from BKA side?

Re: Crash in JOIN_CACHE::get_offset with join_cache_level=6 in maria 5.3-dsmrr-cpk
Ok point taken, I will re-verify bugs using the main tree + valgrind to avoid situations where crashes only happen on the feature tree due to some particular arrangement of memory accesses.

Launchpad bug id: 623209