2022-09-21 11:16:50 25 [Warning] You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'dummy' at line 1
2022-09-21 11:16:50 26 [Warning] You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'dummy' at line 1
2022-09-21 11:16:50 27 [Warning] You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'dummy' at line 1
2022-09-21 11:16:50 28 [Warning] You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'dummy' at line 1
2022-09-21 11:16:50 29 [Warning] You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'dummy' at line 1
2022-09-21 11:16:50 30 [Warning] You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'dummy' at line 1
2022-09-21 11:16:50 4 [Warning] WSREP: handlerton rollback failed, thd 4 226 conf 0 SQL HANDLER t READ FIRST
While the eventual outcome looks understandable (init_connect command failed) the in-between outcomes seem to require some work;
1. The Remote MariaDB server has gone away is odd given the server has not gone away (and is reachable). This is likely due to #2, ref next item.
2. Self-referencing should not be allowed? (Spider t table referencing to itself)
3. The various 'Aborted connections'. This too is likely due to #2? Also see #6 in this regard.
4. 'init_connect' should likely not be allowed for the spider connection?
5. The final errors (mysql_ha_read/ WSREP) are likely due to #4.
6. It is odd that there are 5 attempts? If this is a hardcoded number of retries within Spider it may be best to improve this code to a single attempt with appropriate return values if it fails etc.?
7. More as a question to jplindst - why is WSREP involved in this HANDLER trx?
Attachments
Issue Links
relates to
MDEV-6268SPIDER table with no COMMENT clause causes queries to wait forever
Closed
MDEV-30012Triple thread hang in states 'NULL', 'NULL' and 'closing tables' on INSERT
Open
MDEV-30576Include spider error codes in extra/perror, especially 12719, 12701
Open
MDEV-29605SIGSEGV in spider_db_ping, ASAN heap-use-after-free in spider_db_ping and UBSAN dynamic-type-mismatch in spider_db_ping on CREATE TABLE
Closed
MDEV-29606Assertion `transaction->implicit_xid.is_null()' failed in THD::init_for_queries and Assertion `transaction.is_empty()' failed in prepare_new_connection_state
Confirmed
MDEV-29676Dual thread hang in 'closing tables' and 'Waiting for table metadata lock' on Spider CREATE OR REPLACE TABLE
Closed
MDEV-29800ERROR 1062 (23000): Duplicate entry 'test-t-0' for key 'PRIMARY' on CoR for Spider table, DDL_LOG: Got error 1032 or 12524 when trying to execute action for entry 1/2/3/4/5/6/7 of type 'rename table'
Closed
MDEV-29854SIGSEGV in spider_string::length on LOCK TABLES
Closed
MDEV-30011SIGSEGV in spider_db_mbase::fin_loop_check on I_S SELECT
Open
split to
MDEV-30578Multiple connections in spider when executing invalid init_command on spider table
Open
MDEV-30579Confusing WSREP warning on HANDLER READ FIRST for spider table (possibly caused by zero-length self-referencing)
Open
MDEV-30580Investigate whether self-/multi- referencing spider tables cause problems
Another thing to consider disallowing is dual-referenced/multi-referenced/cross-referenced tables. i.e. one InnoDB table referenced by two or more Spider tables, or Spider tables cross referenced, or a Spider table referenced by two or more other Spider tables etc.
Roel Van de Paar
added a comment - Another thing to consider disallowing is dual-referenced/multi-referenced/cross-referenced tables. i.e. one InnoDB table referenced by two or more Spider tables, or Spider tables cross referenced, or a Spider table referenced by two or more other Spider tables etc.
nayuta-yanagisawa, Spider can detect self-referencing (and, generally, circular referencing) tables, it was implemented in MDEV-6268. Strange, that it didn't work in this case.
Sergei Golubchik
added a comment - nayuta-yanagisawa , Spider can detect self-referencing (and, generally, circular referencing) tables, it was implemented in MDEV-6268 . Strange, that it didn't work in this case.
Nayuta Yanagisawa (Inactive)
added a comment - - edited serg Thank you for the information. I'm wondering why the patch for MDEV-6268 has been pushed only to 10.5+. https://github.com/MariaDB/server/commit/23c8adda74935211ca8f8a50676cf4f94e9215fb
Roel Van de Paar
added a comment - - edited An additional longer testcase which also produces the exact same outcomes as the one in this comment (infinite loop, error 12719).
INSTALL PLUGIN Spider SONAME 'ha_spider.so' ;
CREATE USER Spider@localhost IDENTIFIED BY 'PWD1' ;
CREATE SERVER srv FOREIGN DATA WRAPPER MYSQL OPTIONS (SOCKET '../socket.sock' , DATABASE 'test' , user 'Spider' , PASSWORD 'PWD1' );
CREATE TABLE t (c INT );
CREATE TABLE t1 (a DATE ) ENGINE=MEMORY;
CREATE TABLE t4 (c INT );
CREATE TABLE t2 (d INT , KEY (d)) ENGINE=Spider COMMENT= 'WRAPPER "mysql",srv "srv",TABLE "t"' ;
CREATE TABLE t5 (id_product INT ) ENGINE=Spider COMMENT= 'WRAPPER "mysql",srv "srv",TABLE "t"' ;
INSERT INTO t2 VALUES ();
CREATE TABLE t6 (a INT );
CREATE OR REPLACE TABLE t (a INT ,b INT ) ENGINE=Spider COMMENT= 'WRAPPER "mysql",srv "srv",TABLE "t"' ;
SELECT * FROM t5;
FLUSH TABLES WITH READ LOCK;
UNLOCK TABLES;
INSERT INTO t1 VALUES ( NULL ),( 'foo' ),( 'bar' ),( NULL );
SELECT * FROM (t1 JOIN t2) NATURAL LEFT JOIN (t6 NATURAL JOIN t4);
> Spider can detect self-referencing (and, generally, circular referencing) tables, it was implemented in MDEV-6268. Strange, that it didn't work in this case.
That is because, in the current implementation, the self-referencing check is done after Spider has successfully connected to a remote node. Due to the 'dummy' init_connect, Spider fails to connect to the data node, and the check is not executed.
Nayuta Yanagisawa (Inactive)
added a comment - > Spider can detect self-referencing (and, generally, circular referencing) tables, it was implemented in MDEV-6268 . Strange, that it didn't work in this case.
That is because, in the current implementation, the self-referencing check is done after Spider has successfully connected to a remote node. Due to the 'dummy' init_connect, Spider fails to connect to the data node, and the check is not executed.
One simple fix could be to do the self-referencing check in spider table creation rather than connection, and reject self-referencing create statements (error 12719). To that end, here's an mtr testcase (similar to self_reference.test but with the error placed earlier)
--echo #
--echo # MDEV-29583 Disallow Spider self/dual/multi/cross-referencing tables, consider not executing/skipping/disabling init_connect for Spider user
#--error 12719 - this is currently when the error occurs
#SELECT a FROM tbl_a;
--connection master_1
DROPDATABASEIF EXISTS auto_test_local;
--disable_query_log
--disable_result_log
--source ../t/test_deinit.inc
--enable_query_log
--enable_result_log
This approach is similar to MDEV-29562, but the self-referencing check ("loop check") implemented in commit 23c8adda749 is much more complicated than the charset check. BTW spider_db_mbase::set_loop_check() is not covered by any test, see the attachment in MDEV-30437.
Hopefully this will fix all the problems that occur with self-referencing spider tables, including MDEV-29676.
Will this fix the multi-referencing (e.g. t1 -> t <- t2) or cross-referencing (e.g. t1 -> t2 -> t1)? Depends on the implementation of loop check. But I think fixing just the self-referencing is sufficient for now.
Yuchen Pei
added a comment - - edited One simple fix could be to do the self-referencing check in spider table creation rather than connection, and reject self-referencing create statements (error 12719). To that end, here's an mtr testcase (similar to self_reference.test but with the error placed earlier)
--echo #
--echo # MDEV-29583 Disallow Spider self/dual/multi/cross-referencing tables, consider not executing/skipping/disabling init_connect for Spider user
--echo #
--disable_query_log
--disable_result_log
--source ../../t/test_init.inc
--enable_result_log
--enable_query_log
--connection master_1
CREATE DATABASE auto_test_local;
USE auto_test_local;
SET spider_same_server_link= on ;
#this should be when the error occurs
--error 12719
eval CREATE TABLE tbl_a (
a INT
) $MASTER_1_ENGINE COMMENT= 'table "tbl_a", srv "s_1"' ;
# --error 12719 - this is currently when the error occurs
# SELECT a FROM tbl_a;
--connection master_1
DROP DATABASE IF EXISTS auto_test_local;
--disable_query_log
--disable_result_log
--source ../t/test_deinit.inc
--enable_query_log
--enable_result_log
This approach is similar to MDEV-29562 , but the self-referencing check ("loop check") implemented in commit 23c8adda749 is much more complicated than the charset check. BTW spider_db_mbase::set_loop_check() is not covered by any test, see the attachment in MDEV-30437 .
Hopefully this will fix all the problems that occur with self-referencing spider tables, including MDEV-29676 .
Will this fix the multi-referencing (e.g. t1 -> t <- t2) or cross-referencing (e.g. t1 -> t2 -> t1)? Depends on the implementation of loop check. But I think fixing just the self-referencing is sufficient for now.
The problem in the ticket description is mainly caused by issuing an invalid init_command, causing the data node server to disconnect. The same error messages would occur if the spider table is not self-referencing.
OTOH, there are outstanding items raised in the ticket, and they have been splitted into their own tickets MDEV-30578, MDEV-30579 and MDEV-30580.
Yuchen Pei
added a comment - The problem in the ticket description is mainly caused by issuing an invalid init_command, causing the data node server to disconnect. The same error messages would occur if the spider table is not self-referencing.
OTOH, there are outstanding items raised in the ticket, and they have been splitted into their own tickets MDEV-30578 , MDEV-30579 and MDEV-30580 .
Also see
MDEV-29605SIGSEGV in spider_db_ping on INSERT