[MDEV-18366] Crash on SELECT on a table that contains indexed virtual columns Created: 2019-01-24  Updated: 2020-10-06  Resolved: 2020-09-01

Status: Closed
Project: MariaDB Server
Component/s: Server, Virtual Columns
Affects Version/s: 10.2, 10.3, 10.4, 10.5
Fix Version/s: 10.2.35, 10.3.26, 10.4.16, 10.5.7

Type: Bug Priority: Major
Reporter: Marko Mäkelä Assignee: Nikita Malyavin
Resolution: Fixed Votes: 1
Labels: ASAN, crash, valgrind, virtual_columns

Issue Links:
Relates
relates to MDEV-18486 Database crash on a table with indexe... Closed
relates to MDEV-23167 Server crashes upon HANDLER READ from... Confirmed
relates to MDEV-18147 [Draft] Assertion `templ->mysql_col_l... Closed
relates to MDEV-18217 [Draft] [ERROR] InnoDB: Apparent cor... Closed
relates to MDEV-18338 Merge new release of InnoDB 5.7.25 to... Closed

 Description   

The test that I added to gcol.innodb_virtual_index.test frequently (but not always) crashes on the SELECT * FROM t1 after ROLLBACK:

10.2 b572814baa1746efeb65bf55b4c7a7d535e283a1

 
CURRENT_TEST: gcol.innodb_virtual_index
 
mysqltest: At line 281: query 'SELECT * FROM t1' failed: 2013: Lost connection to MySQL server during query

Here is one of many occrurences from buildbot.

ASAN is not saying anything helpful before the SIGSEGV. Valgrind on a WITH_VALGRIND build tells that UNIV_MEM_ASSERT_W(dest, templ->mysql_col_len) in row_sel_field_store_in_mysql_format_func() reports unaddressable bytes.

In build_template_field(), I see an insane value for this:

templ->mysql_col_offset = (ulint) get_field_offset(table, field);

For column a, we have field->ptr-table.record[0] = 27369. I guess that this can be any garbage, because I think that the code crashes on column a, and on one of the runs where it crashed, we had mysql_col_offset = 49481. Perhaps some MEM_UNDEFINED instrumentation is missing somewhere, because Valgrind did not catch anything. monitor get_vbits is reporting all bits of both field->ptr and table->record[0] as valid.

Stack trace to make it more findable:

10.2 b572814baa1

 
#3  <signal handler called>
 
#4  0x0000558ddaee9449 in row_sel_field_store_in_mysql_format_func (dest=0x7f68340c2a79 <error: Cannot access memory at address 0x7f68340c2a79>, templ=0x7f67340e02c0, index=0x7f673405ef88, field_no=0, data=0x7f677fd2807e "\200", len=4) at /data/src/10.2/storage/innobase/row/row0sel.cc:2829
 
#5  0x0000558ddaee9eb6 in row_sel_store_mysql_field_func (mysql_rec=0x7f67340e6840 "\377", prebuilt=0x7f67340cb978, rec=0x7f677fd2807e "\200", index=0x7f673405ef88, offsets=0x7f677c4f5380, field_no=0, templ=0x7f67340e02c0) at /data/src/10.2/storage/innobase/row/row0sel.cc:3116
 
#6  0x0000558ddaeea34e in row_sel_store_mysql_rec (mysql_rec=0x7f67340e6840 "\377", prebuilt=0x7f67340cb978, rec=0x7f677fd2807e "\200", vrow=0x0, rec_clust=false, index=0x7f673405ef88, offsets=0x7f677c4f5380) at /data/src/10.2/storage/innobase/row/row0sel.cc:3246
 
#7  0x0000558ddaeef658 in row_search_mvcc (buf=0x7f67340e6840 "\377", mode=PAGE_CUR_G, prebuilt=0x7f67340cb978, match_mode=0, direction=0) at /data/src/10.2/storage/innobase/row/row0sel.cc:5532
 
#8  0x0000558ddad7f1bf in ha_innobase::index_read (this=0x7f67340cd718, buf=0x7f67340e6840 "\377", key_ptr=0x0, key_len=0, find_flag=HA_READ_AFTER_KEY) at /data/src/10.2/storage/innobase/handler/ha_innodb.cc:9435
 
#9  0x0000558ddad7ffa0 in ha_innobase::index_first (this=0x7f67340cd718, buf=0x7f67340e6840 "\377") at /data/src/10.2/storage/innobase/handler/ha_innodb.cc:9812
 
#10 0x0000558ddad801ab in ha_innobase::rnd_next (this=0x7f67340cd718, buf=0x7f67340e6840 "\377") at /data/src/10.2/storage/innobase/handler/ha_innodb.cc:9905
 
#11 0x0000558ddaa69aa1 in handler::ha_rnd_next (this=0x7f67340cd718, buf=0x7f67340e6840 "\377") at /data/src/10.2/sql/handler.cc:2579
 
#12 0x0000558ddabd6a6e in rr_sequential (info=0x7f67340142d0) at /data/src/10.2/sql/records.cc:485
 
#13 0x0000558dda85cadc in join_init_read_record (tab=0x7f6734014208) at /data/src/10.2/sql/sql_select.cc:19690
 
#14 0x0000558dda85a8ae in sub_select (join=0x7f6734012d20, join_tab=0x7f6734014208, end_of_records=false) at /data/src/10.2/sql/sql_select.cc:18761
 
#15 0x0000558dda859ea9 in do_select (join=0x7f6734012d20, procedure=0x0) at /data/src/10.2/sql/sql_select.cc:18308
 
#16 0x0000558dda83448f in JOIN::exec_inner (this=0x7f6734012d20) at /data/src/10.2/sql/sql_select.cc:3623
 
#17 0x0000558dda83393c in JOIN::exec (this=0x7f6734012d20) at /data/src/10.2/sql/sql_select.cc:3418
 
#18 0x0000558dda834b07 in mysql_select (thd=0x7f6734000b00, tables=0x7f6734012620, wild_num=1, fields=..., conds=0x0, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147748608, result=0x7f6734012d00, unit=0x7f67340046e0, select_lex=0x7f6734004e18) at /data/src/10.2/sql/sql_select.cc:3818
 
#19 0x0000558dda828d17 in handle_select (thd=0x7f6734000b00, lex=0x7f6734004618, result=0x7f6734012d00, setup_tables_done_option=0) at /data/src/10.2/sql/sql_select.cc:376
 
#20 0x0000558dda7f41c7 in execute_sqlcom_select (thd=0x7f6734000b00, all_tables=0x7f6734012620) at /data/src/10.2/sql/sql_parse.cc:6481
 
#21 0x0000558dda7ea148 in mysql_execute_command (thd=0x7f6734000b00) at /data/src/10.2/sql/sql_parse.cc:3487
 
#22 0x0000558dda7f80a1 in mysql_parse (thd=0x7f6734000b00, rawbuf=0x7f6734012448 "SELECT * FROM t1", length=16, parser_state=0x7f677c4f7200, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:8015
 
#23 0x0000558dda7e59db in dispatch_command (command=COM_QUERY, thd=0x7f6734000b00, packet=0x7f67340967b1 "", packet_length=16, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:1826
 
#24 0x0000558dda7e4332 in do_command (thd=0x7f6734000b00) at /data/src/10.2/sql/sql_parse.cc:1379
 
#25 0x0000558dda9371c3 in do_handle_one_connection (connect=0x558dddd1ece0) at /data/src/10.2/sql/sql_connect.cc:1335
 
#26 0x0000558dda936f50 in handle_one_connection (arg=0x558dddd1ece0) at /data/src/10.2/sql/sql_connect.cc:1241
 
#27 0x0000558ddad5ce84 in pfs_spawn_thread (arg=0x558dddd24af0) at /data/src/10.2/storage/perfschema/pfs.cc:1862
 
#28 0x00007f678bd88494 in start_thread (arg=0x7f677c4f8700) at pthread_create.c:333
 
#29 0x00007f678a16e93f in clone () from /lib/x86_64-linux-gnu/libc.so.6



 Comments   
Comment by Marko Mäkelä [ 2019-01-24 ]

When fixing this, please revert my temporary patch to the test.

Comment by Alice Sherepa [ 2020-07-29 ]

also on 10.5:

200729  8:56:42 [ERROR] mysqld got signal 11 ;
 
 
 
mysys/stacktrace.c:306(my_print_stacktrace)[0x558aff529bd6]
 
??:0(__restore_rt)[0x7f7852a39730]
 
row/row0sel.cc:2831(row_sel_field_store_in_mysql_format_func(unsigned char*, mysql_row_templ_t const*, dict_index_t const*, unsigned long, unsigned char const*, unsigned long))[0x558b004321f6]
 
row/row0sel.cc:2983(row_sel_store_mysql_field(unsigned char*, row_prebuilt_t*, unsigned char const*, dict_index_t const*, unsigned short const*, unsigned long, mysql_row_templ_t const*))[0x558b00433415]
 
row/row0sel.cc:3130(row_sel_store_mysql_rec(unsigned char*, row_prebuilt_t*, unsigned char const*, dtuple_t const*, bool, dict_index_t const*, unsigned short const*))[0x558b004422c2]
 
row/row0sel.cc:5435(row_search_mvcc(unsigned char*, page_cur_mode_t, row_prebuilt_t*, unsigned long, unsigned long))[0x558b0002ae84]
 
handler/ha_innodb.cc:8953(ha_innobase::index_read(unsigned char*, unsigned char const*, unsigned int, ha_rkey_function))[0x558b0002e088]
 
handler/ha_innodb.cc:9327(ha_innobase::index_first(unsigned char*))[0x558b0002e5a0]
 
handler/ha_innodb.cc:9420(ha_innobase::rnd_next(unsigned char*))[0x558aff5427a2]
 
sql/handler.cc:3060(handler::ha_rnd_next(unsigned char*))[0x558aff97c24c]
 
sql/records.cc:511(rr_sequential(READ_RECORD*))[0x558afebb577a]
 
sql/records.h:79(READ_RECORD::read_record())[0x558afeea7b62]
 
sql/sql_select.cc:21586(join_init_read_record(st_join_table*))[0x558afeea5ef7]
 
sql/sql_select.cc:20638(sub_select(JOIN*, st_join_table*, bool))[0x558afee33d24]
 
sql/sql_select.cc:20175(do_select(JOIN*, Procedure*))[0x558afee31372]
 
sql/sql_select.cc:4450(JOIN::exec_inner())[0x558afee3528b]
 
sql/sql_select.cc:4232(JOIN::exec())[0x558afee07a26]
 
sql/sql_select.cc:417(handle_select(THD*, LEX*, select_result*, unsigned long))[0x558afed79d80]
 
sql/sql_parse.cc:6209(execute_sqlcom_select(THD*, TABLE_LIST*))[0x558afed6903e]
 
sql/sql_parse.cc:3931(mysql_execute_command(THD*))[0x558afed84c7c]
 
sql/sql_parse.cc:7993(mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool))[0x558afed5bb95]
 
sql/sql_parse.cc:1869(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool))[0x558afed584ad]
 
sql/sql_parse.cc:1347(do_command(THD*))[0x558aff17c6c6]
 
sql/sql_connect.cc:1410(do_handle_one_connection(CONNECT*, bool))[0x558aff17c023]
 
sql/sql_connect.cc:1314(handle_one_connection)[0x558affe31886]
 
nptl/pthread_create.c:487(start_thread)[0x7f7852a2efa3]
 
x86_64/clone.S:97(clone)[0x7f78520354cf]
 
 
 
Query (0x62b0000a12a8): SELECT * FROM t1

from innodb_virtual_index.test:

--source include/have_innodb.inc
 
 
CREATE TABLE t1(a INT PRIMARY KEY, b INT, vb DATE AS(b) VIRTUAL, KEY(vb))
 
ENGINE=InnoDB;
 
INSERT IGNORE INTO t1 (a,b) VALUES(1,20190132);
 
BEGIN;
 
DELETE FROM t1;
 
--error ER_TRUNCATED_WRONG_VALUE
 
INSERT INTO t1 (a,b) VALUES(1,20190123);
 
ROLLBACK;
 
SELECT * FROM t1;

Comment by Marko Mäkelä [ 2020-08-07 ]

I sent some review comments. I like the idea of using RAII to prevent this class of problems.

Can you please describe the exact flow of events in a failure scenario?

Generated at Thu Feb 08 08:43:33 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.