[MDEV-29855] Crash with SPIDER_DIRECT_SQL and spider_udf_ds_use_real_table=1 Created: 2022-10-22  Updated: 2022-11-24  Resolved: 2022-11-24

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - Spider
Affects Version/s: 10.3, 10.4, 10.9.3, 10.5, 10.6, 10.7, 10.8, 10.10
Fix Version/s: 10.11.2, 10.3.38, 10.4.28, 10.5.19, 10.6.12, 10.7.8, 10.8.7, 10.9.5, 10.10.3

Type: Bug Priority: Critical
Reporter: markus makela Assignee: Nayuta Yanagisawa (Inactive)
Resolution: Fixed Votes: 3
Labels: None


 Description   

Here's what I did:

MariaDB [test]> SET @connection_string = 'wrapper "mysql", user "maxuser", password "maxpwd", host "127.0.0.1", port "3001", database "test"';
 
Query OK, 0 rows affected (0.000 sec)
 
 
 
MariaDB [test]> CREATE OR REPLACE TABLE t1(id INT);
 
Query OK, 0 rows affected (0.001 sec)
 
 
 
MariaDB [test]> SET spider_udf_ds_use_real_table=1;
 
Query OK, 0 rows affected (0.000 sec)
 
 
 
MariaDB [test]> SELECT SPIDER_DIRECT_SQL('select 1 as 1', 't1', @connection_string);
 
ERROR 2013 (HY000): Lost connection to server during query

Stacktrace:

221022 12:09:56 [ERROR] mysqld got signal 11 ;
 
This could be because you hit a bug. It is also possible that this binary
 
or one of the libraries it was linked against is corrupt, improperly built,
 
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
 
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
 
Server version: 10.9.3-MariaDB-1:10.9.3+maria~ubu2204-log
 
key_buffer_size=134217728
 
read_buffer_size=131072
 
max_used_connections=3
 
max_threads=10002
 
thread_count=23
 
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 22159044 K  bytes of memory
 
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x7ff7d4002888
 
Attempting backtrace. You can use the following information to find out
 
where mysqld died. If you see no messages after this, something went
 
terribly wrong...
 
stack_bottom = 0x7ff80407cc78 thread_stack 0x49000
 
Printing to addr2line failed
 
mysqld(my_print_stacktrace+0x32)[0x55931fed1342]
 
mysqld(handle_fatal_signal+0x478)[0x55931f9a28e8]
 
/lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7ff81ea46520]
 
mysqld(strmake+0x10)[0x55931ff32020]
 
mysqld(_ZN11MDL_request16init_with_sourceEN7MDL_key18enum_mdl_namespaceEPKcS3_13enum_mdl_type17enum_mdl_durationS3_j+0x41)[0x55931f87e3e1]
 
/usr/lib/mysql/plugin/ha_spider.so(_Z22spider_direct_sql_bodyP11st_udf_initP11st_udf_argsPcS3_c+0x49a)[0x7ff7f857033a]
 
mysqld(_ZN17Item_func_udf_int7val_intEv+0x4e)[0x55931fa1b36e]
 
mysqld(_ZNK12Type_handler18Item_send_longlongEP4ItemP8ProtocolP8st_value+0x1d)[0x55931f90250d]
 
mysqld(_ZN8Protocol19send_result_set_rowEP4ListI4ItemE+0xea)[0x55931f67810a]
 
mysqld(_ZN11select_send9send_dataER4ListI4ItemE+0x37)[0x55931f6f5607]
 
mysqld(_ZN4JOIN10exec_innerEv+0xcd0)[0x55931f7ca200]
 
mysqld(_ZN4JOIN4execEv+0x39)[0x55931f7ca939]
 
mysqld(_Z12mysql_selectP3THDP10TABLE_LISTR4ListI4ItemEPS4_jP8st_orderS9_S7_S9_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x121)[0x55931f7c89c1]
 
mysqld(_Z13handle_selectP3THDP3LEXP13select_resultm+0x154)[0x55931f7c9174]
 
mysqld(+0x7ebce5)[0x55931f742ce5]
 
mysqld(_Z21mysql_execute_commandP3THDb+0x47d2)[0x55931f751ff2]
 
mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x1e7)[0x55931f753697]
 
mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjb+0x14d5)[0x55931f755e35]
 
mysqld(_Z10do_commandP3THDb+0x138)[0x55931f757b38]
 
mysqld(_Z24do_handle_one_connectionP7CONNECTb+0x3bf)[0x55931f87641f]
 
mysqld(handle_one_connection+0x5d)[0x55931f87676d]
 
mysqld(+0xc76de6)[0x55931fbcdde6]
 
/lib/x86_64-linux-gnu/libc.so.6(+0x94b43)[0x7ff81ea98b43]
 
/lib/x86_64-linux-gnu/libc.so.6(clone+0x44)[0x7ff81eb29bb4]
 
Trying to get some variables.
 
Some pointers may be invalid and cause the dump to abort.
 
Query (0x7ff7d4011290): SELECT SPIDER_DIRECT_SQL('select 1 as 1', 't1', @connection_string)
 
Connection ID (thread ID): 29
 
Status: NOT_KILLED
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off
 
The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
 
information that should help you find out what is causing the crash.
 
Writing a core file...
 
Working directory at /var/lib/mysql
 
Resource Limits:
 
Limit                     Soft Limit           Hard Limit           Units     
Max cpu time              unlimited            unlimited            seconds   
Max file size             unlimited            unlimited            bytes     
Max data size             unlimited            unlimited            bytes     
Max stack size            8388608              unlimited            bytes     
Max core file size        unlimited            unlimited            bytes     
Max resident set          unlimited            unlimited            bytes     
Max processes             unlimited            unlimited            processes 
Max open files            60000                60000                files     
Max locked memory         8388608              8388608              bytes     
Max address space         unlimited            unlimited            bytes     
Max file locks            unlimited            unlimited            locks     
Max pending signals       127100               127100               signals   
Max msgqueue size         819200               819200               bytes     
Max nice priority         0                    0                    
Max realtime priority     0                    0                    
Max realtime timeout      unlimited            unlimited            us        
Core pattern: |/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h
 
Kernel version: Linux version 5.19.16-200.fc36.x86_64 (mockbuild@bkernel01.iad2.fedoraproject.org) (gcc (GCC) 12.2.1 20220819 (Red Hat 12.2.1-2), GNU ld version 2.37-36.fc36) #1 SMP PREEMPT_DYNAMIC Sun Oct 16 22:50:04 UTC 2022

This also happens with the ODBC wrapper. Using a temporary table prevents this.

 Comments   
Comment by Nayuta Yanagisawa (Inactive) [ 2022-10-23 ]

MTR test case:

 
 
--disable_query_log
 
--disable_result_log
 
--source ../../t/test_init.inc
 
--enable_result_log
 
--enable_query_log
 
 
--connection child2_1
 
CREATE DATABASE auto_test_remote;
 
USE auto_test_remote;
 
eval CREATE TABLE tbl_a (
 
    a INT
 
) $CHILD2_1_ENGINE $CHILD2_1_CHARSET;
 
 
--connection master_1
 
CREATE DATABASE auto_test_local;
 
USE auto_test_local;
 
eval CREATE TABLE tbl_a (
 
    a INT
 
) $MASTER_1_ENGINE $MASTER_1_CHARSET COMMENT='table "tbl_a", srv "s_2_1"';
 
 
 
SET spider_udf_ds_use_real_table=1;
 
SELECT SPIDER_DIRECT_SQL('select 1 as 1', 'tbl_a', 'srv "s_2_1"');
 
 
 
--connection master_1
 
DROP DATABASE IF EXISTS auto_test_local;
 
 
--connection child2_1
 
DROP DATABASE IF EXISTS auto_test_remote;
 
 
--disable_query_log
 
--disable_result_log
 
--source ../t/test_deinit.inc
 
--enable_query_log
 
--enable_result_log

Comment by Nayuta Yanagisawa (Inactive) [ 2022-10-23 ]

The stack trace produced by the above test case:

10.9 d86ad1f127fdc71e888e2e168b99f561f111a0b2

 
Thread 2 received signal SIGSEGV, Segmentation fault.
 
[Switching to Thread 97701.97743]
 
0x00007f93b4a5197d in ?? () from /lib/x86_64-linux-gnu/libc.so.6
 
(rr) bt
 
#0  0x00007f93b4a5197d in ?? () from /lib/x86_64-linux-gnu/libc.so.6
 
#1  0x000056118894e1ff in MDL_key::mdl_key_init (this=0x7f939d1cd7c8, mdl_namespace_arg=MDL_key::TABLE, db=0x0, name_arg=0x0)
 
    at /home/nayuta_mariadb/repo/mariadb-server/10.9/sql/mdl.h:429
 
#2  0x00005611889fd22f in MDL_request::init_with_source (this=0x7f939d1cd7a8, mdl_namespace=MDL_key::TABLE, db_arg=0x0, name_arg=0x0, mdl_type_arg=MDL_SHARED_WRITE, 
    mdl_duration_arg=MDL_TRANSACTION, src_file=0x7f939ce26ce0 "/home/nayuta_mariadb/repo/mariadb-server/10.9/sql/table.h", src_line=2246)
 
    at /home/nayuta_mariadb/repo/mariadb-server/10.9/sql/mdl.cc:1010
 
#3  0x00007f939ccc6946 in TABLE_LIST::init_one_table (this=0x7f939d1cd320, db_arg=0x7f939d1cd338, table_name_arg=0x7f939d1cd348, alias_arg=0x0, lock_type_arg=TL_WRITE)
 
    at /home/nayuta_mariadb/repo/mariadb-server/10.9/sql/table.h:2246
 
#4  0x00007f939cd67c58 in spider_direct_sql_body (initid=0x7f93900150d8, args=0x7f9390015098, is_null=0x7f9390015109 "", error=0x7f9390015108 "", bg=0 '\000')
 
    at /home/nayuta_mariadb/repo/mariadb-server/10.9/storage/spider/spd_direct_sql.cc:1640
 
#5  0x00007f939cd69073 in spider_direct_sql (initid=0x7f93900150d8, args=0x7f9390015098, is_null=0x7f9390015109 "", error=0x7f9390015108 "")
 
    at /home/nayuta_mariadb/repo/mariadb-server/10.9/storage/spider/spd_udf.cc:29
 
#6  0x0000561188c9baf2 in udf_handler::val_int (this=0x7f9390015088, null_value=0x7f939d1cdad7 "") at /home/nayuta_mariadb/repo/mariadb-server/10.9/sql/sql_udf.h:108
 
#7  0x0000561188c8e789 in Item_func_udf_int::val_int (this=0x7f9390014fe0) at /home/nayuta_mariadb/repo/mariadb-server/10.9/sql/item_func.cc:3800
 
#8  0x0000561188acba6f in Type_handler::Item_send_longlong (this=0x56118a254de0 <type_handler_slonglong>, item=0x7f9390014fe0, protocol=0x7f9390001a78, 
    buf=0x7f939d1cdc20) at /home/nayuta_mariadb/repo/mariadb-server/10.9/sql/sql_type.cc:7496
 
#9  0x0000561188ad9d0e in Type_handler_longlong::Item_send (this=0x56118a254de0 <type_handler_slonglong>, item=0x7f9390014fe0, protocol=0x7f9390001a78, 
    buf=0x7f939d1cdc20) at /home/nayuta_mariadb/repo/mariadb-server/10.9/sql/sql_type.h:5743
 
#10 0x0000561188679610 in Item::send (this=0x7f9390014fe0, protocol=0x7f9390001a78, buffer=0x7f939d1cdc20)
 
    at /home/nayuta_mariadb/repo/mariadb-server/10.9/sql/item.h:1232
 
#11 0x00005611886c35cb in Protocol::send_result_set_row (this=0x7f9390001a78, row_items=0x7f9390014b60)
 
    at /home/nayuta_mariadb/repo/mariadb-server/10.9/sql/protocol.cc:1328
 
#12 0x0000561188783dbd in select_send::send_data (this=0x7f9390015ad0, items=...) at /home/nayuta_mariadb/repo/mariadb-server/10.9/sql/sql_class.cc:3105
 
#13 0x00005611888ceab9 in select_result_sink::send_data_with_check (this=0x7f9390015ad0, items=..., u=0x7f9390005890, sent=0)
 
    at /home/nayuta_mariadb/repo/mariadb-server/10.9/sql/sql_class.h:5721
 
#14 0x0000561188881fda in JOIN::exec_inner (this=0x7f9390015af8) at /home/nayuta_mariadb/repo/mariadb-server/10.9/sql/sql_select.cc:4673
 
#15 0x0000561188881813 in JOIN::exec (this=0x7f9390015af8) at /home/nayuta_mariadb/repo/mariadb-server/10.9/sql/sql_select.cc:4585
 
#16 0x00005611888831d5 in mysql_select (thd=0x7f9390001468, tables=0x0, fields=..., conds=0x0, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, 
    select_options=2147486464, result=0x7f9390015ad0, unit=0x7f9390005890, select_lex=0x7f93900148c0)
 
    at /home/nayuta_mariadb/repo/mariadb-server/10.9/sql/sql_select.cc:5065
 
#17 0x0000561188871d5c in handle_select (thd=0x7f9390001468, lex=0x7f93900057b8, result=0x7f9390015ad0, setup_tables_done_option=0)
 
    at /home/nayuta_mariadb/repo/mariadb-server/10.9/sql/sql_select.cc:579
 
#18 0x0000561188816b87 in execute_sqlcom_select (thd=0x7f9390001468, all_tables=0x0) at /home/nayuta_mariadb/repo/mariadb-server/10.9/sql/sql_parse.cc:6261
 
#19 0x000056118880dabf in mysql_execute_command (thd=0x7f9390001468, is_called_from_prepared_stmt=false)
 
    at /home/nayuta_mariadb/repo/mariadb-server/10.9/sql/sql_parse.cc:3945
 
#20 0x000056118881bb67 in mysql_parse (thd=0x7f9390001468, rawbuf=0x7f93900147d0 "SELECT SPIDER_DIRECT_SQL('select 1 as 1', 'tbl_a', 'srv \"s_2_1\"')", length=65, 
    parser_state=0x7f939d1cf1f0) at /home/nayuta_mariadb/repo/mariadb-server/10.9/sql/sql_parse.cc:8023
 
#21 0x000056118880784a in dispatch_command (command=COM_QUERY, thd=0x7f9390001468, packet=0x7f939000c1d9 "", packet_length=65, blocking=true)
 
    at /home/nayuta_mariadb/repo/mariadb-server/10.9/sql/sql_parse.cc:1894
 
#22 0x000056118880619e in do_command (thd=0x7f9390001468, blocking=true) at /home/nayuta_mariadb/repo/mariadb-server/10.9/sql/sql_parse.cc:1407
 
#23 0x00005611889ed7c5 in do_handle_one_connection (connect=0x56118babe8d8, put_in_cache=true) at /home/nayuta_mariadb/repo/mariadb-server/10.9/sql/sql_connect.cc:1416
 
#24 0x00005611889ed52e in handle_one_connection (arg=0x56118babe8d8) at /home/nayuta_mariadb/repo/mariadb-server/10.9/sql/sql_connect.cc:1318
 
#25 0x0000561188f225be in pfs_spawn_thread (arg=0x56118ba84468) at /home/nayuta_mariadb/repo/mariadb-server/10.9/storage/perfschema/pfs.cc:2201
 
#26 0x00007f93b4948b43 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
 
#27 0x00007f93b49d9bb4 in clone () from /lib/x86_64-linux-gnu/libc.so.6

Comment by Nayuta Yanagisawa (Inactive) [ 2022-10-23 ]

One should not pass table_list.db and table_list.table_name to the function because it update the very members internally. 

      table_list.init_one_table(
 
        &table_list.db, &table_list.table_name, 0, TL_WRITE);

The TABLE_LIST::init_one_table() is called previously, and there seems to be no need to call it again. So, simply removing the call will resolve the problem.

Comment by Nayuta Yanagisawa (Inactive) [ 2022-10-23 ]

https://github.com/MariaDB/server/commit/cd60dcf849b7a0866a679a6be842a6ffff54b662

Generated at Thu Feb 08 10:11:48 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.