[MDEV-29640] FederatedX does not properly handle pushdown in case of difference in local and remote table names Created: 2022-09-26  Updated: 2022-10-26  Resolved: 2022-10-26

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - Federated
Affects Version/s: 10.4, 10.5, 10.6, 10.7, 10.8, 10.9, 10.10
Fix Version/s: 10.4.27, 10.5.18, 10.6.11, 10.7.7, 10.8.6, 10.9.4, 10.10.2, 10.11.1

Type: Bug Priority: Major
Reporter: Alice Sherepa Assignee: Sergei Petrunia
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Relates
relates to MDEV-25080 Allow pushdown of queries involving U... Closed

 Description    

MariaDB [test]> create or replace table t1 (i int);
 
Query OK, 0 rows affected (0,046 sec)
 
 
 
MariaDB [test]> insert into t1 values (1),(2);
 
Query OK, 2 rows affected (0,014 sec)
 
Records: 2  Duplicates: 0  Warnings: 0
 
 
 
MariaDB [test]> create table t ENGINE=FEDERATED    CONNECTION="mysql://root@127.0.0.1:3315/test/t1";
 
Query OK, 0 rows affected (0,025 sec)
 
 
 
MariaDB [test]> set global federated_pushdown=1;
 
Query OK, 0 rows affected (0,000 sec)
 
 
 
MariaDB [test]> select * from t;
 
ERROR 1030 (HY000): Got error 10000 "Unknown error 10000" from storage engine FEDERATED
 
 
 
MariaDB [test]> set global federated_pushdown=0;
 
Query OK, 0 rows affected (0,000 sec)
 
 
 
MariaDB [test]> select * from t;
 
+------+
 
| i    |
 
+------+
 
|    1 |
 
|    2 |
 
+------+
 
2 rows in set (0,003 sec)
 



--source have_federatedx.inc
 
--source include/federated.inc
 
 
 
connection slave;
 
 
 
create table t1(i int);
 
  insert into t1 values (1),(2);
 
 
 
--replace_result $SLAVE_MYPORT SLAVE_PORT
 
eval
 
CREATE TABLE t1_fed ENGINE="FEDERATED" CONNECTION='mysql://root@127.0.0.1:$SLAVE_MYPORT/test/t1';
 
 
 
set global federated_pushdown=0;
 
select * from t1_fed;
 
 
 
set global federated_pushdown=1;
 
select * from t1_fed;
 
 
 
source include/federated_cleanup.inc;



 Comments   
Comment by Elena Stepanova [ 2022-09-29 ]

I'll leave fix version as alice set it, as I don't know if it's feasible (or practical) to fix in previous versions. Adjust as needed.

Comment by Oleg Smirnov [ 2022-10-20 ]

The bug is caused by different names of tables on the remote and local servers. It's not a problem while the pushdown is disabled 'cause the CONNECTION strings are being parsed for every table during the execution. But in case of pushdown the whole query is passed down to the engine without any translation, so the remote server may try to select data from a non-existing table ("t" in the example above).

Comment by Alice Sherepa [ 2022-10-25 ]

MDEV-29863 in bb-10.4-MDEV-29640 branch does not crash, but returns "Got error 10000 "Unknown error 10000" from storage engine FEDERATED"

Comment by Alice Sherepa [ 2022-10-25 ] 

INSTALL SONAME 'ha_federatedx';
 
 
 
eval CREATE SERVER fedlink FOREIGN DATA WRAPPER mysql OPTIONS (USER 'root', HOST '127.0.0.1', DATABASE 'test1', PORT $MASTER_MYPORT);
 
 
create database test1;
 
create table test1.t1 (a int, b int);
 
create table test1.t2 (a int, b int);
 
 
 
CREATE TABLE t1 ENGINE=FEDERATED  connection='fedlink/t1';
 
CREATE TABLE t2 ENGINE=FEDERATED  connection='fedlink/t2';
 
 
 
set global federated_pushdown=1;
 
 
 
insert into t1 values (1,1),(2,2);
 
insert into t2 values (1,1),(2,2),(3,3);
 
 
 
update t1,t2 set t1.a=5 where t1.a=t2.a;
 
 
 
 
 
DROP TABLE t1,t2,test1.t1,test1.t2;
 
UNINSTALL SONAME 'ha_federatedx';

bb-10.4-MDEV-29640 e78a948b9b27d9e6a8d1967cc12ff1

 
mysqltest: At line 17: query 'update t1,t2 set t1.a=5 where t1.a=t2.a' failed: 1030: Got error 10000 "Unknown error 10000" from storage engine FEDERATED

Comment by Alice Sherepa [ 2022-10-25 ]

the previos test also ends up with ASAN heap-use-after-free on shutdown, probably the same as MDEV-29178

=================================================================
 
==210307==ERROR: AddressSanitizer: heap-use-after-free on address 0x62b00005c588 at pc 0x55586a8cd3b5 bp 0x7ffe7eae8360 sp 0x7ffe7eae8350
 
READ of size 8 at 0x62b00005c588 thread T0
 
    #0 0x55586a8cd3b4 in thd_increment_bytes_sent /10.4/sql/sql_class.cc:4443
 
    #1 0x55586b53b5a2 in net_real_write /10.4/sql/net_serv.cc:745
 
    #2 0x55586b539187 in net_flush /10.4/sql/net_serv.cc:384
 
    #3 0x55586b53a056 in net_write_command /10.4/sql/net_serv.cc:535
 
    #4 0x55586b0c1d54 in cli_advanced_command /10.4/sql-common/client.c:468
 
    #5 0x55586b0d5399 in mysql_close_slow_part /10.4/sql-common/client.c:3460
 
    #6 0x55586b0d54b2 in mysql_close /10.4/sql-common/client.c:3472
 
    #7 0x7f2f048601dc in federatedx_io_mysql::~federatedx_io_mysql() /10.4/storage/federatedx/federatedx_io_mysql.cc:153
 
    #8 0x7f2f048602ef in federatedx_io_mysql::~federatedx_io_mysql() /10.4/storage/federatedx/federatedx_io_mysql.cc:157
 
    #9 0x7f2f0485aaed in federatedx_txn::close(st_fedrated_server*) /10.4/storage/federatedx/federatedx_txn.cc:88
 
    #10 0x7f2f0483e1a7 in free_server /10.4/storage/federatedx/ha_federatedx.cc:1686
 
    #11 0x7f2f0483e8ee in free_share /10.4/storage/federatedx/ha_federatedx.cc:1725
 
    #12 0x7f2f0483f9a0 in ha_federatedx::close() /10.4/storage/federatedx/ha_federatedx.cc:1855
 
    #13 0x55586b1994f2 in handler::ha_close() /10.4/sql/handler.cc:2877
 
    #14 0x55586acfd280 in closefrm(TABLE*) /10.4/sql/table.cc:4249
 
    #15 0x55586afc548b in intern_close_table /10.4/sql/table_cache.cc:221
 
    #16 0x55586afc5b92 in tc_purge(bool) /10.4/sql/table_cache.cc:334
 
    #17 0x55586a839fbb in purge_tables(bool) /10.4/sql/sql_base.cc:335
 
    #18 0x55586afc8949 in tdc_start_shutdown() /10.4/sql/table_cache.cc:657
 
    #19 0x55586a6d50eb in clean_up /10.4/sql/mysqld.cc:2000
 
    #20 0x55586a6e461d in mysqld_main(int, char**) /10.4/sql/mysqld.cc:5978
 
    #21 0x55586a6cae2c in main /10.4/sql/main.cc:25
 
    #22 0x7f2f0e22a082 in __libc_start_main ../csu/libc-start.c:308
 
    #23 0x55586a6cad4d in _start (/10.4/sql/mysqld+0xfb6d4d)
 
 
 
0x62b00005c588 is located 5000 bytes inside of 26268-byte region [0x62b00005b200,0x62b00006189c)
 
freed by thread T5 here:
 
    #0 0x7f2f0ed5240f in __interceptor_free ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:122
 
    #1 0x55586c5e85ed in free_memory /10.4/mysys/safemalloc.c:279
 
    #2 0x55586c5e7ba9 in sf_free /10.4/mysys/safemalloc.c:197
 
    #3 0x55586c5b65a7 in my_free /10.4/mysys/my_malloc.c:222
 
    #4 0x55586a6f0eb5 in ilink::operator delete(void*, unsigned long) /10.4/sql/sql_list.h:684
 
    #5 0x55586a8b4a80 in THD::~THD() /10.4/sql/sql_class.cc:1729
 
    #6 0x55586a6d90ed in one_thread_per_connection_end(THD*, bool) /10.4/sql/mysqld.cc:2788
 
    #7 0x55586adcd679 in do_handle_one_connection(CONNECT*) /10.4/sql/sql_connect.cc:1431
 
    #8 0x55586adccd83 in handle_one_connection /10.4/sql/sql_connect.cc:1324
 
    #9 0x55586ba582d2 in pfs_spawn_thread /10.4/storage/perfschema/pfs.cc:1869
 
    #10 0x7f2f0e754608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477
 
 
 
previously allocated by thread T5 here:
 
    #0 0x7f2f0ed52808 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:144
 
    #1 0x55586c5e755d in sf_malloc /10.4/mysys/safemalloc.c:118
 
    #2 0x55586c5b5ab0 in my_malloc /10.4/mysys/my_malloc.c:101
 
    #3 0x55586a6f0e62 in ilink::operator new(unsigned long) /10.4/sql/sql_list.h:680
 
    #4 0x55586adcdebf in CONNECT::create_thd(THD*) /10.4/sql/sql_connect.cc:1517
 
    #5 0x55586adcd05c in do_handle_one_connection(CONNECT*) /10.4/sql/sql_connect.cc:1363
 
    #6 0x55586adccd83 in handle_one_connection /10.4/sql/sql_connect.cc:1324
 
    #7 0x55586ba582d2 in pfs_spawn_thread /10.4/storage/perfschema/pfs.cc:1869
 
    #8 0x7f2f0e754608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477
 
 
 
Thread T5 created by T0 here:
 
    #0 0x7f2f0ec7f815 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cc:208
 
    #1 0x55586ba586c3 in spawn_thread_v1 /10.4/storage/perfschema/pfs.cc:1919
 
    #2 0x55586a6ccbdf in inline_mysql_thread_create /10.4/include/mysql/psi/mysql_thread.h:1275
 
    #3 0x55586a6e4d57 in create_thread_to_handle_connection(CONNECT*) /10.4/sql/mysqld.cc:6295
 
    #4 0x55586a6e54f2 in create_new_thread(CONNECT*) /10.4/sql/mysqld.cc:6365
 
    #5 0x55586a6e59d8 in handle_accepted_socket(st_mysql_socket, st_mysql_socket) /10.4/sql/mysqld.cc:6463
 
    #6 0x55586a6e6894 in handle_connections_sockets() /10.4/sql/mysqld.cc:6621
 
    #7 0x55586a6e445c in mysqld_main(int, char**) /10.4/sql/mysqld.cc:5953
 
    #8 0x55586a6cae2c in main /10.4/sql/main.cc:25
 
    #9 0x7f2f0e22a082 in __libc_start_main ../csu/libc-start.c:308
 
 
 
SUMMARY: AddressSanitizer: heap-use-after-free /10.4/sql/sql_class.cc:4443 in thd_increment_bytes_sent
 
Shadow bytes around the buggy address:
 
  0x0c5680003860: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
 
  0x0c5680003870: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
 
  0x0c5680003880: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
 
  0x0c5680003890: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
 
  0x0c56800038a0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
 
=>0x0c56800038b0: fd[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd
 
  0x0c56800038c0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
 
  0x0c56800038d0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
 
  0x0c56800038e0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
 
  0x0c56800038f0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
 
  0x0c5680003900: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
 
Shadow byte legend (one shadow byte represents 8 application bytes):
 
  Addressable:           00
 
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
 
  Freed heap region:       fd
 
  Stack left redzone:      f1
 
  Stack mid redzone:       f2
 
  Stack right redzone:     f3
 
  Stack after return:      f5
 
  Stack use after scope:   f8
 
  Global redzone:          f9
 
  Global init order:       f6
 
  Poisoned by user:        f7
 
  Container overflow:      fc
 
  Array cookie:            ac
 
  Intra object redzone:    bb
 
  ASan internal:           fe
 
  Left alloca redzone:     ca
 
  Right alloca redzone:    cb
 
  Shadow gap:              cc
 
==210307==ABORTING

without federated_pushdown=1:

=================================================================
 
==211806==ERROR: LeakSanitizer: detected memory leaks
 
 
 
Direct leak of 32 byte(s) in 1 object(s) allocated from:
 
    #0 0x7f86c2417587 in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cc:104
 
    #1 0x7f86b7efeccf  (<unknown module>)
 
    #2 0x7f86b7eff0ff  (<unknown module>)
 
    #3 0x55710d1634b1 in handler::ha_open(TABLE*, char const*, int, unsigned int, st_mem_root*, List<String>*) /10.4/sql/handler.cc:2811
 
    #4 0x55710ccc7143 in open_table_from_share(THD*, TABLE_SHARE*, st_mysql_const_lex_string const*, unsigned int, unsigned int, unsigned int, TABLE*, bool, List<String>*) /10.4/sql/table.cc:4145
 
    #5 0x55710c80e8a8 in open_table(THD*, TABLE_LIST*, Open_table_context*) /10.4/sql/sql_base.cc:2109
 
    #6 0x55710c8180da in open_and_process_table /10.4/sql/sql_base.cc:3909
 
    #7 0x55710c81acca in open_tables(THD*, DDL_options_st const&, TABLE_LIST**, unsigned int*, unsigned int, Prelocking_strategy*) /10.4/sql/sql_base.cc:4390
 
    #8 0x55710c81fe6b in open_and_lock_tables(THD*, DDL_options_st const&, TABLE_LIST*, bool, unsigned int, Prelocking_strategy*) /10.4/sql/sql_base.cc:5337
 
    #9 0x55710c778e1f in open_and_lock_tables(THD*, TABLE_LIST*, bool, unsigned int) /10.4/sql/sql_base.h:503
 
    #10 0x55710c8ebd51 in mysql_insert(THD*, TABLE_LIST*, List<Item>&, List<List<Item> >&, List<Item>&, List<Item>&, enum_duplicates, bool) /10.4/sql/sql_insert.cc:764
 
    #11 0x55710c9ac38c in mysql_execute_command(THD*) /10.4/sql/sql_parse.cc:4599
 
    #12 0x55710c9c41de in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /10.4/sql/sql_parse.cc:7982
 
    #13 0x55710c99aab8 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /10.4/sql/sql_parse.cc:1857
 
    #14 0x55710c9975b8 in do_command(THD*) /10.4/sql/sql_parse.cc:1378
 
    #15 0x55710cd984df in do_handle_one_connection(CONNECT*) /10.4/sql/sql_connect.cc:1420
 
    #16 0x55710cd97d83 in handle_one_connection /10.4/sql/sql_connect.cc:1324
 
    #17 0x55710da232d2 in pfs_spawn_thread /10.4/storage/perfschema/pfs.cc:1869
 
    #18 0x7f86c1e17608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477
 
 
 
SUMMARY: AddressSanitizer: 32 byte(s) leaked in 1 allocation(s).

Comment by Oleg Smirnov [ 2022-10-25 ]

alice, can you run your tests against the recently pushed bb-10.4-MDEV-29640-MDEV-29863?

Comment by Alice Sherepa [ 2022-10-25 ]

oleg.smirnov the error after UPDATE/DELETE is still repeatable in bb-10.4-MDEV-29640-MDEV-29863

Comment by Oleg Smirnov [ 2022-10-26 ]

The UPDATE issue doesn't seem related to this task topic so we decided to file it as a separate case: https://jira.mariadb.org/browse/MDEV-29874.

Comment by Oleg Smirnov [ 2022-10-26 ]

Th bugfix is pushed to 10.4.

Generated at Thu Feb 08 10:10:10 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.