[MDEV-16940] Server crashes in unsafe_key_update upon attempt to update view through 2nd execution of SP Created: 2018-08-11  Updated: 2021-05-02  Resolved: 2021-01-20

Status: Closed
Project: MariaDB Server
Component/s: Stored routines, Views
Affects Version/s: 5.5, 10.0, 10.1, 10.2, 10.3, 10.4, 10.5
Fix Version/s: 10.2.37, 10.3.28, 10.4.18, 10.5.9

Type: Bug Priority: Critical
Reporter: Elena Stepanova Assignee: Igor Babaev
Resolution: Fixed Votes: 1
Labels: affects-tests

Issue Links:
Relates
relates to MDEV-18624 ASAN heap-use-after-free in mysql_der... Closed
relates to MDEV-17435 Server crashes inst_select_lex::get_f... Closed
relates to MDEV-24823 Crash with invalid multi-table update... Closed

 Description   

CREATE TABLE t1 (a INT);
CREATE TABLE t2 (b INT);
 
CREATE VIEW v AS SELECT a, b FROM t1 JOIN t2;
CREATE PROCEDURE sp() UPDATE v SET a = 8, b = 9;
--error ER_VIEW_MULTIUPDATE
CALL sp;
--error ER_VIEW_MULTIUPDATE
CALL sp;
 
# Cleanup
DROP PROCEDURE sp;
DROP VIEW v;
DROP TABLE t1, t2;

5.5 ad577091ed

#3  <signal handler called>
#4  0x00000000006b6d07 in unsafe_key_update (leaves=..., tables_for_update=3) at /data/src/5.5/sql/sql_update.cc:1122
#5  0x00000000006b7412 in mysql_multi_update_prepare (thd=0x7f514a264060) at /data/src/5.5/sql/sql_update.cc:1278
#6  0x000000000060237c in mysql_execute_command (thd=0x7f514a264060) at /data/src/5.5/sql/sql_parse.cc:2858
#7  0x00000000008e22c0 in sp_instr_stmt::exec_core (this=0x7f51491806e0, thd=0x7f514a264060, nextp=0x7f5150071c28) at /data/src/5.5/sql/sp_head.cc:3220
#8  0x00000000008e1a67 in sp_lex_keeper::reset_lex_and_exec_core (this=0x7f5149180720, thd=0x7f514a264060, nextp=0x7f5150071c28, open_tables=false, instr=0x7f51491806e0) at /data/src/5.5/sql/sp_head.cc:2998
#9  0x00000000008e1fe8 in sp_instr_stmt::execute (this=0x7f51491806e0, thd=0x7f514a264060, nextp=0x7f5150071c28) at /data/src/5.5/sql/sp_head.cc:3144
#10 0x00000000008dde51 in sp_head::execute (this=0x7f5149198078, thd=0x7f514a264060, merge_da_on_success=true) at /data/src/5.5/sql/sp_head.cc:1431
#11 0x00000000008dfbb7 in sp_head::execute_procedure (this=0x7f5149198078, thd=0x7f514a264060, args=0x7f514a268020) at /data/src/5.5/sql/sp_head.cc:2197
#12 0x0000000000605edf in mysql_execute_command (thd=0x7f514a264060) at /data/src/5.5/sql/sql_parse.cc:4115
#13 0x000000000060a44e in mysql_parse (thd=0x7f514a264060, rawbuf=0x7f5149148078 "CALL sp", length=7, parser_state=0x7f5150072640) at /data/src/5.5/sql/sql_parse.cc:5923
#14 0x00000000005fe163 in dispatch_command (command=COM_QUERY, thd=0x7f514a264060, packet=0x7f514df5b061 "CALL sp", packet_length=7) at /data/src/5.5/sql/sql_parse.cc:1066
#15 0x00000000005fd355 in do_command (thd=0x7f514a264060) at /data/src/5.5/sql/sql_parse.cc:793
#16 0x00000000007008e3 in do_handle_one_connection (thd_arg=0x7f514a264060) at /data/src/5.5/sql/sql_connect.cc:1268
#17 0x0000000000700670 in handle_one_connection (arg=0x7f514a264060) at /data/src/5.5/sql/sql_connect.cc:1184
#18 0x0000000000a0fce9 in pfs_spawn_thread (arg=0x7f514af76fc0) at /data/src/5.5/storage/perfschema/pfs.cc:1015
#19 0x00007f514fca5494 in start_thread (arg=0x7f5150073700) at pthread_create.c:333
#20 0x00007f514e6bb93f in clone () from /lib/x86_64-linux-gnu/libc.so.6

All of 5.5-10.3 crash in the same fashion, both debug and non-debug.
Didn't get the crash on MySQL 5.5, 5.6, 5.7.



 Comments   
Comment by Hartmut Holzgraefe [ 2020-06-30 ]

A customer has now run into crashes that look very similar, although not 100% the same. First crash was:

200608 23:20:21 [ERROR] mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
 
Server version: 10.1.34-MariaDB
key_buffer_size=33554432
read_buffer_size=262144
max_used_connections=223
max_threads=2002
thread_count=143
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1611965 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x7f04238b5008
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7ee8a243c110 thread_stack 0x48400
/data/mysql/bin/mysqld(my_print_stacktrace+0x2b)[0x7f1bc54c400b]
/data/mysql/bin/mysqld(handle_fatal_signal+0x4d5)[0x7f1bc501fe85]
/lib64/libpthread.so.0(+0x3206e0f710)[0x7f1bc4619710]
/data/mysql/bin/mysqld(_Z26mysql_multi_update_prepareP3THD+0x238)[0x7f1bc4f408b8]
/data/mysql/bin/mysqld(_Z21mysql_execute_commandP3THD+0x1e07)[0x7f1bc4ea6cc7]
/data/mysql/bin/mysqld(_ZN13sp_instr_stmt9exec_coreEP3THDPj+0x1e)[0x7f1bc511ef0e]
/data/mysql/bin/mysqld(_ZN13sp_lex_keeper23reset_lex_and_exec_coreEP3THDPjbP8sp_instr+0x8e)[0x7f1bc51249fe]
/data/mysql/bin/mysqld(_ZN13sp_instr_stmt7executeEP3THDPj+0x155)[0x7f1bc5124ef5]
/data/mysql/bin/mysqld(_ZN7sp_head7executeEP3THDb+0x73c)[0x7f1bc5121e9c]
/data/mysql/bin/mysqld(_ZN7sp_head17execute_procedureEP3THDP4ListI4ItemE+0x616)[0x7f1bc5123246]
/data/mysql/bin/mysqld(_Z21mysql_execute_commandP3THD+0x6271)[0x7f1bc4eab131]
/data/mysql/bin/mysqld(_ZN13sp_instr_stmt9exec_coreEP3THDPj+0x1e)[0x7f1bc511ef0e]
/data/mysql/bin/mysqld(_ZN13sp_lex_keeper23reset_lex_and_exec_coreEP3THDPjbP8sp_instr+0x8e)[0x7f1bc51249fe]
/data/mysql/bin/mysqld(_ZN13sp_instr_stmt7executeEP3THDPj+0x155)[0x7f1bc5124ef5]
/data/mysql/bin/mysqld(_ZN7sp_head7executeEP3THDb+0x73c)[0x7f1bc5121e9c]
/data/mysql/bin/mysqld(_ZN7sp_head17execute_procedureEP3THDP4ListI4ItemE+0x616)[0x7f1bc5123246]
/data/mysql/bin/mysqld(_Z21mysql_execute_commandP3THD+0x6271)[0x7f1bc4eab131]
/data/mysql/bin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x285)[0x7f1bc4eadf45]
/data/mysql/bin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x225e)[0x7f1bc4eb0a4e]
/data/mysql/bin/mysqld(_Z10do_commandP3THD+0x177)[0x7f1bc4eb0ff7]
/data/mysql/bin/mysqld(_Z24do_handle_one_connectionP3THD+0x17f)[0x7f1bc4f722af]
/data/mysql/bin/mysqld(handle_one_connection+0x47)[0x7f1bc4f723e7]
/data/mysql/bin/mysqld(+0x748ecd)[0x7f1bc5191ecd]
/lib64/libpthread.so.0(+0x3206e079d1)[0x7f1bc46119d1]
/lib64/libc.so.6(clone+0x6d)[0x7f1bc2af68fd]
 
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x7ef046757020): UPDATE `Orders` a         INNER join tempdb._AllOrder_wktbl ta on (ta.OrdUUId = a.OrdUUId )     SET a.TO_DATE=@CURR_DATE ,        a.EzlastUpdate=a.EzlastUpdate      WHERE a.TO_DATE = @MAX_DATE
Connection ID (thread ID): 1670811
Status: NOT_KILLED

So it happened on a multi table UPDATE inside a stored procedure, but already crashed on mysql_multi_update_prepare with unsafe_key_update not seen in this backtrace

Whether one of the two tables involved in the UPDATE was actually a VIEW is not known.

The second crash indeed happened in unsafe_key_update inside a stored procedure call, but in this case it seems to have been a direct call
from mysql_execute_command to unsafe_key_update, with
mysql_multi_update_prepare missing. This time the actual query was not logged to the error log. Also strange that on this incident the backtraces was already C++-demangled ...

200629 22:58:54 [ERROR] mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
 
Server version: 10.1.34-MariaDB
key_buffer_size=33554432
read_buffer_size=262144
max_used_connections=204
max_threads=2002
thread_count=30
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1611965 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x7f860a5c8008
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7f706e83c110 thread_stack 0x48400
mysys/stacktrace.c:268(my_print_stacktrace)[0x7f8690bf000b]
sql/signal_handler.cc:168(handle_fatal_signal)[0x7f869074be85]
/lib64/libpthread.so.0(+0x3206e0f710)[0x7f868fd45710]
sql/sql_update.cc:1213(unsafe_key_update)[0x7f869066c8b8]
sql/sql_parse.cc:3809(mysql_execute_command(THD*))[0x7f86905d2cc7]
sql/sp_head.cc:3220(sp_instr_stmt::exec_core(THD*, unsigned int*))[0x7f869084af0e]
sql/sp_head.cc:2984(sp_lex_keeper::reset_lex_and_exec_core(THD*, unsigned int*, bool, sp_instr*))[0x7f86908509fe]
sql/sp_head.cc:3135(sp_instr_stmt::execute(THD*, unsigned int*))[0x7f8690850ef5]
sql/sp_head.cc:1317(sp_head::execute(THD*, bool))[0x7f869084de9c]
sql/sp_head.cc:2102(sp_head::execute_procedure(THD*, List<Item>*))[0x7f869084f246]
sql/sql_parse.cc:2425(do_execute_sp)[0x7f86905d7131]
sql/sp_head.cc:3220(sp_instr_stmt::exec_core(THD*, unsigned int*))[0x7f869084af0e]
sql/sp_head.cc:2984(sp_lex_keeper::reset_lex_and_exec_core(THD*, unsigned int*, bool, sp_instr*))[0x7f86908509fe]
sql/sp_head.cc:3135(sp_instr_stmt::execute(THD*, unsigned int*))[0x7f8690850ef5]
sql/sp_head.cc:1317(sp_head::execute(THD*, bool))[0x7f869084de9c]
sql/sp_head.cc:2102(sp_head::execute_procedure(THD*, List<Item>*))[0x7f869084f246]
sql/sql_parse.cc:2425(do_execute_sp)[0x7f86905d7131]
sql/sql_parse.cc:7449(mysql_parse(THD*, char*, unsigned int, Parser_state*))[0x7f86905d9f45]
sql/sql_parse.cc:1492(dispatch_command(enum_server_command, THD*, char*, unsigned int))[0x7f86905dca4e]
sql/sql_parse.cc:1123(do_command(THD*))[0x7f86905dcff7]
sql/sql_connect.cc:1330(do_handle_one_connection(THD*))[0x7f869069e2af]
sql/sql_connect.cc:1244(handle_one_connection)[0x7f869069e3e7]
perfschema/pfs.cc:1864(pfs_spawn_thread)[0x7f86908bdecd]

There was a third crash shortly after, but that did not even have a backtrace ...

All three crashes happened with 10.1.34.

Comment by Oleksandr Byelkin [ 2021-01-15 ]

It is not problem for prepared statmemns, because they do not allow to prepare statement with such error.

Comment by Oleksandr Byelkin [ 2021-01-15 ]

The real problem is that the VIEW got in leaf tables list because "full_list" is not requested by Multiupdate_prelocking_strategy::handle_end

Comment by Oleksandr Byelkin [ 2021-01-15 ]

the above change was done by d37465a9cc458ab215105de22875ce0a64c0efc2

And I do not understand the patch (why it changed full_list to FALSE, and how derived supposed to be processed if TABLE shoud be present for more checks in leas check:

Comment by Igor Babaev [ 2021-01-19 ]

Here's my analysis of the bug.

The first call of SP sp():
When executing mysql_multi_update_prepare() the function Multiupdate_prelocking_strategy::handle_end()
is invoked.
In this function after the call of setup_tables_and_check_access() the list select_lex->leaf_tables contains only one table v. Yet after the code

  if (select_lex->handle_derived(thd->lex, DT_MERGE))
    DBUG_RETURN(1);

has been executed this list contains t1,t2.
Then the call of unsafe_key_update() detects an error ER_VIEW_MULTIUPDATE. As a result the function Multiupdate_prelocking_strategy::handle_end is quit immediately with return code TRUE and after this
mysql_multi_update_prepare() is quit immediately without call of lex->save_prep_leaf_tables(). At this moment select_lex->prep_leaf_list_state remains equal to SELECT_LEX::READY as it was set by the call of setup_tables_and_check_access().

The second call of SP sp()
At the second call of sp() we come to the call of setup_tables_and_check_access() with

select_lex->first_cond_optimization == TRUE
select_lex->select_lex->prep_leaf_list_state == SELECT_LEX::READY

The following code is executed in setup_tables()

  if (select_lex->first_cond_optimization)
  {
    leaves.empty();
    if (select_lex->prep_leaf_list_state != SELECT_LEX::SAVED)
    {
      make_leaves_list(thd, leaves, tables, full_table_list, first_select_table);
      select_lex->prep_leaf_list_state= SELECT_LEX::READY;
      select_lex->leaf_tables_exec.empty();
    }
    else
    {
      List_iterator_fast <TABLE_LIST> ti(select_lex->leaf_tables_prep);
      while ((table_list= ti++))
        leaves.push_back(table_list, thd->mem_root);
    }
   ...

This code put into select_lex->leaf_tables only v, as the 'then' branch of the conditional statement is chosen.
As the select of the view v has been already merged the code

  if (select_lex->handle_derived(thd->lex, DT_MERGE))
    DBUG_RETURN(1);

does nothing and select_lex->leaf_tables still contains only v when we come to the call of
unsafe_key_update(). Meanwhile the code of this function assumes that we have only base tables in
select_lex->leaf_tables.

Comment by Oleksandr Byelkin [ 2021-01-19 ]

OK to push! Thank you for investigating and fixing it!

Comment by Igor Babaev [ 2021-01-20 ]

A fix for this bug was pushed into 10.2

Generated at Thu Feb 08 08:32:42 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.