[MDEV-3616] LP:1046882 - sub-subselect sometimes looses reference to enclosing table Created: 2012-09-06  Updated: 2015-02-02  Resolved: 2012-10-04

Status: Closed
Project: MariaDB Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Daniel Heimann (Inactive) Assignee: Oleksandr Byelkin
Resolution: Fixed Votes: 0
Labels: Launchpad

Attachments: XML File LPexportBug1046882.xml    

 Description   

Trying to match a field with the out-most table from a EXISTS-sub-subquery sometimes silently fails.
There is a combination which seemingly "repairs" this, but then again reproduceably breaks it - see below.
We already tried to turn off all optimizer features and disabled query cache; both to no avail.

To easily reproduce this, please use the contained 'mysql' database with the following example queries.

the clients:

CLIENT-A: mysql-cli
Version: 15.1 Distrib 5.5.25-MariaDB, for debian-linux-gnu (x86_64) using readline 5.1)
Connected via: unix-socket

CLIENT-B: php Application (phpmyadmin o.e. to execute Queries will do)
Version: php5.3.3-7+squeeze14 (default debian package; tried both: mysql & mysqli)
Connected via: unix-socket

the queries:

"QUERY-GOOD":
SELECT a.* FROM mysql.user a WHERE ( EXISTS ( SELECT 1 FROM mysql.user b WHERE b.user = a.user LIMIT 1 ) );

"QUERY-BAD":
SELECT a.* FROM mysql.user a WHERE ( SELECT EXISTS ( SELECT 1 FROM mysql.user b WHERE b.user = a.user LIMIT 1 ) );

Notice the "SELECT EXISTS" vs. "EXISTS" only.

the bug:

QUERY-GOOD works on CLIENT-A and CLIENT-B.
QUERY-BAD works every time on MySQL 5.5.25a, MySQL 5.1.63 and MySQL 5.0.51a. Not so, in MariaDB 5.5.25 (5.5.25-MariaDB-mariadb1~squeeze-log for debian-linux-gnu on x86_64 (mariadb.org binary distribution)):

If CLIENT-B executes QUERY-BAD the result-set is empty. (should not and is not in MySQL-5.*)
If CLIENT-A afterwards executes QUERY-BAD the result-set is empty, too. (should not and is not in MySQL-5.*)
If CLIENT-A now executes QUERY-GOOD there are results returned. (always)
If CLIENT-A then executes QUERY-BAD again there are still results returned! (strange: initially the result was empty)
it stays like this for CLIENT-A until...
if CLIENT-B executes QUERY-BAD once: it gets the same results like CLIENT-A. ("healed")
if CLIENT-B executes QUERY-BAD one more time: the result-set is once more empty ("broken again")
if CLIENT-A now executes QUERY-BAD the result-set it is empty again and stays so, until QUERY-GOOD is re-executed.

I don't think MariaDB should return empty result-sets when MySQL does not.
I further assume queries originating from different clients, shouldn't interfere like this.
Sorry, if this was overly complex.



 Comments   
Comment by Elena Stepanova [ 2012-09-06 ]

Re: sub-subselect sometimes looses reference to enclosing table
Hi Daniel,

Does CLIENT-A ever return wrong results for QUERY-BAD if CLIENT-B was NOT involved in the flow before?

Thanks.

Comment by Daniel Heimann (Inactive) [ 2012-09-06 ]

Re: sub-subselect sometimes looses reference to enclosing table
hi Elena,

thanks for your immediate reply!

CLIENT-A is unaffected, when restarting the daemon and only accessing it with the mysql cli.
As soon as php-Clients (by the way: php5.2.13 compiled from source (with Client API library version => 5.0.51a, Client API header version => 5.0.32) also leads to the aforementioned behaviour) join in, the flow starts as described above.

How else, can I help?

Comment by Elena Stepanova [ 2012-09-06 ]

Re: sub-subselect sometimes looses reference to enclosing table
Hi Daniel,

I've installed phpmyadmin, but couldn't so far reproduce the problem, the results differ starting from the first step: instead of
"If CLIENT-B executes QUERY-BAD the result-set is empty" i'm getting a valid result set.

So, please

1) enable general log (set global general_log=1);

2) execute your scenario;

3) then execute it again, only before each SELECT also run EXPLAIN EXTENDED SELECT ...
(still run the SELECTs too, to make sure that results didn't change because of the EXPLAIN);

4) attach the resulting general log (it shouldn't be big, but you can compress it if it makes sense) and, importantly, your cnf file.

Thanks!

Comment by Daniel Heimann (Inactive) [ 2012-09-07 ]

Re: sub-subselect sometimes looses reference to enclosing table
hi Elena,

I went through the following steps (after turning general_log=1):

1. phpmyadmin -> bad-query (empty result-set)
2. cli -> bad-query (empty result-set)

3. cli -> good-query (non-empty result-set)
4. cli -> bad-query (non-empty result-set)

5. phpmyadmin -> bad (non-empty result-set)
6. phpmyadmin -> bad (empty result-set)

Then I reexecuted 1 - 6 with "EXPLAIN EXTENDED" in front of it.
Sorry, phpmyadmin generates some more queries in the log, to render its output.

general log is:

/usr/sbin/mysqld, Version: 5.5.25-MariaDB-mariadb1~squeeze (mariadb.org binary distribution). started with:
Tcp port: 3306 Unix socket: /var/run/mysqld/mysqld.sock
Time Id Command Argument
120907 11:30:13 13572 Connect root@localhost as anonymous on
13572 Query SET CHARACTER SET 'utf8'
13572 Query SET collation_connection = 'utf8_general_ci'
13572 Init DB mysql
13572 Query SHOW TABLES LIKE 'user'
13572 Init DB mysql
13572 Query SHOW TABLE STATUS FROM `mysql` LIKE 'user%'
13572 Query SHOW FULL COLUMNS FROM `mysql`.`user`
13572 Quit
120907 11:30:31 13573 Connect root@localhost as anonymous on
13573 Query SET CHARACTER SET 'utf8'
13573 Query SET collation_connection = 'utf8_general_ci'
13573 Init DB mysql
13573 Init DB mysql
13573 Query SELECT a.* FROM mysql.user a WHERE ( SELECT EXISTS ( SELECT 1 FROM mysql.user b WHERE b.user = a.user LIMIT 1 ) )
LIMIT 0, 30
13573 Query SELECT SQL_CALC_FOUND_ROWS a . * FROM mysql.user a WHERE ( SELECT EXISTS (
SELECT 1
FROM mysql.user b
WHERE b.user = a.user
LIMIT 1 ) ) LIMIT 1
13573 Query SELECT FOUND_ROWS()
13573 Init DB mysql
13573 Query SHOW TABLES LIKE 'user'
13573 Init DB mysql
13573 Query SHOW TABLE STATUS FROM `mysql` LIKE 'user%'
13573 Query SELECT COUNT FROM `mysql`.`user`
13573 Query SHOW FULL COLUMNS FROM `mysql`.`user`
13573 Quit
120907 11:33:59 13571 Query SELECT a.* FROM mysql.user a WHERE ( SELECT EXISTS ( SELECT 1 FROM mysql.user b WHERE b.user = a.user LIMIT 1 ) )
120907 11:34:11 13571 Query SELECT a.* FROM mysql.user a WHERE ( EXISTS ( SELECT 1 FROM mysql.user b WHERE b.user = a.user LIMIT 1 ) )
120907 11:34:16 13571 Query SELECT a.* FROM mysql.user a WHERE ( SELECT EXISTS ( SELECT 1 FROM mysql.user b WHERE b.user = a.user LIMIT 1 ) )
120907 11:34:59 13574 Connect root@localhost as anonymous on
13574 Query SET CHARACTER SET 'utf8'
13574 Query SET collation_connection = 'utf8_general_ci'
13574 Init DB mysql
13574 Init DB mysql
13574 Query SELECT a.* FROM mysql.user a WHERE ( SELECT EXISTS ( SELECT 1 FROM mysql.user b WHERE b.user = a.user LIMIT 1 ) )
LIMIT 0, 30
13574 Query SELECT SQL_CALC_FOUND_ROWS a . * FROM mysql.user a WHERE ( SELECT EXISTS (
SELECT 1
FROM mysql.user b
WHERE b.user = a.user
LIMIT 1 ) ) LIMIT 1
13574 Query SELECT FOUND_ROWS()
13574 Init DB mysql
13574 Query SHOW TABLES LIKE 'user'
13574 Init DB mysql
13574 Query SHOW TABLE STATUS FROM `mysql` LIKE 'user%'
13574 Query SELECT COUNT FROM `mysql`.`user`
13574 Query SHOW INDEX FROM `mysql`.`user`
13574 Query SHOW FULL COLUMNS
FROM `mysql`.`user`
13574 Quit
120907 11:35:05 13575 Connect root@localhost as anonymous on
13575 Query SET CHARACTER SET 'utf8'
13575 Query SET collation_connection = 'utf8_general_ci'
13575 Init DB mysql
13575 Init DB mysql
13575 Query SELECT a.* FROM mysql.user a WHERE ( SELECT EXISTS ( SELECT 1 FROM mysql.user b WHERE b.user = a.user LIMIT 1 ) )
LIMIT 0, 30
13575 Query SELECT SQL_CALC_FOUND_ROWS a . * FROM mysql.user a WHERE ( SELECT EXISTS (
SELECT 1
FROM mysql.user b
WHERE b.user = a.user
LIMIT 1 ) ) LIMIT 1
13575 Query SELECT FOUND_ROWS()
13575 Init DB mysql
13575 Query SHOW TABLES LIKE 'user'
13575 Init DB mysql
13575 Query SHOW TABLE STATUS FROM `mysql` LIKE 'user%'
13575 Query SELECT COUNT FROM `mysql`.`user`
13575 Query SHOW INDEX FROM `mysql`.`user`
13575 Query SHOW FULL FIELDS FROM `user`
13575 Query SHOW CREATE TABLE `mysql`.`user`
13575 Query SHOW FULL COLUMNS
FROM `mysql`.`user`
13575 Query SHOW TRIGGERS FROM `mysql` LIKE 'user'
13575 Quit
120907 11:37:51 13576 Connect root@localhost as anonymous on
13576 Query SET CHARACTER SET 'utf8'
13576 Query SET collation_connection = 'utf8_general_ci'
13576 Init DB mysql
13576 Query SHOW TABLES LIKE 'user'
13576 Init DB mysql
13576 Query SHOW TABLE STATUS FROM `mysql` LIKE 'user%'
13576 Query SHOW FULL COLUMNS FROM `mysql`.`user`
13576 Quit
120907 11:37:57 13577 Connect root@localhost as anonymous on
13577 Query SET CHARACTER SET 'utf8'
13577 Query SET collation_connection = 'utf8_general_ci'
13577 Init DB mysql
13577 Init DB mysql
13577 Query EXPLAIN EXTENDED SELECT a.* FROM mysql.user a WHERE ( SELECT EXISTS ( SELECT 1 FROM mysql.user b WHERE b.user = a.user LIMIT 1 ) )
13577 Init DB mysql
13577 Query SHOW TABLES LIKE 'user'
13577 Init DB mysql
13577 Query SHOW TABLE STATUS FROM `mysql` LIKE 'user%'
13577 Query SELECT COUNT FROM `mysql`.`user`
13577 Quit
120907 11:38:13 13571 Query EXPLAIN EXTENDED SELECT a.* FROM mysql.user a WHERE ( SELECT EXISTS ( SELECT 1 FROM mysql.user b WHERE b.user = a.user LIMIT 1 ) )
120907 11:38:24 13571 Query EXPLAIN EXTENDED SELECT a.* FROM mysql.user a WHERE ( EXISTS ( SELECT 1 FROM mysql.user b WHERE b.user = a.user LIMIT 1 ) )
120907 11:38:41 13571 Query EXPLAIN EXTENDED SELECT a.* FROM mysql.user a WHERE ( SELECT EXISTS ( SELECT 1 FROM mysql.user b WHERE b.user = a.user LIMIT 1 ) )
120907 11:39:02 13578 Connect root@localhost as anonymous on
13578 Query SET CHARACTER SET 'utf8'
13578 Query SET collation_connection = 'utf8_general_ci'
13578 Init DB mysql
13578 Query SHOW TABLES LIKE 'user'
13578 Init DB mysql
13578 Query SHOW TABLE STATUS FROM `mysql` LIKE 'user%'
13578 Query SHOW FULL COLUMNS FROM `mysql`.`user`
13578 Quit
120907 11:39:08 13579 Connect root@localhost as anonymous on
13579 Query SET CHARACTER SET 'utf8'
13579 Query SET collation_connection = 'utf8_general_ci'
13579 Init DB mysql
13579 Init DB mysql
13579 Query EXPLAIN EXTENDED SELECT a.* FROM mysql.user a WHERE ( SELECT EXISTS ( SELECT 1 FROM mysql.user b WHERE b.user = a.user LIMIT 1 ) )
13579 Init DB mysql
13579 Query SHOW TABLES LIKE 'user'
13579 Init DB mysql
13579 Query SHOW TABLE STATUS FROM `mysql` LIKE 'user%'
13579 Query SELECT COUNT FROM `mysql`.`user`
13579 Quit
120907 11:39:12 13580 Connect root@localhost as anonymous on
13580 Query SET CHARACTER SET 'utf8'
13580 Query SET collation_connection = 'utf8_general_ci'
13580 Init DB mysql
13580 Query SHOW TABLES LIKE 'user'
13580 Init DB mysql
13580 Query SHOW TABLE STATUS FROM `mysql` LIKE 'user%'
13580 Query SHOW FULL COLUMNS FROM `mysql`.`user`
13580 Quit
120907 11:39:15 13581 Connect root@localhost as anonymous on
13581 Query SET CHARACTER SET 'utf8'
13581 Query SET collation_connection = 'utf8_general_ci'
13581 Init DB mysql
13581 Init DB mysql
13581 Query EXPLAIN EXTENDED SELECT a.* FROM mysql.user a WHERE ( SELECT EXISTS ( SELECT 1 FROM mysql.user b WHERE b.user = a.user LIMIT 1 ) )
13581 Init DB mysql
13581 Query SHOW TABLES LIKE 'user'
13581 Init DB mysql
13581 Query SHOW TABLE STATUS FROM `mysql` LIKE 'user%'
13581 Query SELECT COUNT FROM `mysql`.`user`
13581 Quit
120907 11:39:40 13571 Query set global general_log=0

my.cnf (grep -vE "^(#.*|)$" my.cnf) is:

[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock
[mysqld_safe]
socket = /var/run/mysqld/mysqld.sock
nice = 0
mysqld = mysqld
syslog
[mysqld]
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
lc_messages_dir = /usr/share/mysql
lc_messages = en_US
skip-external-locking
skip-name-resolve
bind-address = 172.30.1.64
ft_min_word_len = 2
max_connections = 100
max_allowed_packet = 16M
thread_stack = 256K
thread_cache_size = 128
sort_buffer_size = 2M
bulk_insert_buffer_size = 8M
tmp_table_size = 256M
max_heap_table_size = 256M
join_buffer_size = 128K
connect_timeout = 10
wait_timeout = 600
table_open_cache = 20480
open_files_limit = 40960
myisam_recover = BACKUP
key_buffer_size = 128M
myisam_sort_buffer_size = 32M
concurrent_insert = AUTO
read_buffer_size = 2M
read_rnd_buffer_size = 1M
query_cache_limit = 1M
query_cache_size = 512M
query_cache_type = OFF
general_log_file = /var/log/mysql/mysql.log
general_log = 0
log_warnings = 2
slow_query_log_file = /var/log/mysql/mariadb-slow.log
long_query_time = 10
log_slow_verbosity = query_plan
expire_logs_days = 10
max_binlog_size = 100M
default_storage_engine = InnoDB
innodb_buffer_pool_size = 1024M
innodb_log_buffer_size = 8M
innodb_file_per_table = ON
innodb_open_files = 1280
innodb_io_capacity = 400
innodb_flush_method = O_DIRECT
innodb_read_io_threads = 8
innodb_write_io_threads = 8
innodb_flush_log_at_trx_commit = 2
blackhole = OFF
performance_schema = 0
archive = OFF
federated = OFF
feedback = OFF
[mysqldump]
quick
quote-names
max_allowed_packet = 16M
[mysql]
no-auto-rehash
[isamchk]
key_buffer = 16M
!includedir /etc/mysql/conf.d/

included stuff in /etc/mysql/conf.d/* is just:
[client]
default-character-set = latin1
[mysqld_safe]
syslog

Thank you very much, for guidance. What else, is needed?

Comment by Elena Stepanova [ 2012-09-07 ]

Re: sub-subselect sometimes looses reference to enclosing table
Hi Daniel,

Thank you, sorry if I wasn't clear in my previous comment – when you execute EXPLAIN EXTENDED, we need the actual result set. It will look somewhat like this:

-----------------------------------------------------------------------------------------------------

id select_type table type possible_keys key key_len ref rows filtered Extra

-----------------------------------------------------------------------------------------------------

1 PRIMARY a ALL NULL NULL NULL NULL 6 100.00 Using where
3 DEPENDENT SUBQUERY b index NULL PRIMARY 228 NULL 6 100.00 Using where; Using index

-----------------------------------------------------------------------------------------------------
2 rows in set, 3 warnings (0.00 sec)

(but probably a little bit different, at least for the wrong results, and this difference is what we are chasing now).

Even better if you additionally execute SHOW WARNINGS right after SHOW EXPLAIN, it will provide more information (will be verbose, considering the number of columns in the table, but it's okay).

No need to re-send the log and the config file, please just provide the output of your commands if possible.

Thank you.

Comment by Elena Stepanova [ 2012-09-07 ]

Re: sub-subselect sometimes looses reference to enclosing table
Sorry, of course I meant not "after SHOW EXPLAIN" but "after EXPLAIN EXTENDED".
(SHOW EXPLAIN is something we're working on now, so it's stuck in my head).

Comment by Daniel Heimann (Inactive) [ 2012-09-07 ]

Re: sub-subselect sometimes looses reference to enclosing table
hi Elena,

thank you for clarification. I'll leave out the result-sets (when not empty, because of privacy issues). After showing "explain extended" and "show warnings" results from executing GOOD and working-BAD query in CLI, I executed "QUERY-BAD" in phpmyadmin, which then yields QUERY-BAD to not work (return no results) in CLIENT-A (CLI), too.

I collected CLI output in the following order:

1. CLI (CLIENT-A) with QUERY-GOOD
2. CLI (CLIENT-A) with QUERY-BAD (but still returning same result-set, as QUERY-GOOD)
phpmyadmin (CLIENT-B)
3. CLI (CLIENT-A) with QUERY-BAD (returning empty result-set, after CLIENT-B executed QUERY-BAD)

Here we go:

1. CLI (CLIENT-A) with QUERY-GOOD

MariaDB [(none)]> EXPLAIN EXTENDED SELECT a.* FROM mysql.user a WHERE ( EXISTS ( SELECT 1 FROM mysql.user b WHERE b.user = a.user LIMIT 1 ) );
-----------------------------------------------------------------------------------------------------

id select_type table type possible_keys key key_len ref rows filtered Extra

-----------------------------------------------------------------------------------------------------

1 PRIMARY a ALL NULL NULL NULL NULL 351 100.00 Using where
2 DEPENDENT SUBQUERY b index NULL PRIMARY 228 NULL 351 100.00 Using where; Using index

-----------------------------------------------------------------------------------------------------
2 rows in set, 2 warnings (0.00 sec)

MariaDB [(none)]> SHOW WARNINGS;
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Level Code Message

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Note 1276 Field or reference 'mysql.a.User' of SELECT #2 was resolved in SELECT #1
Note 1003 select `mysql`.`a`.`Host` AS `Host`,`mysql`.`a`.`User` AS `User`,`mysql`.`a`.`Password` AS `Password`,`mysql`.`a`.`Select_priv` AS `Select_priv`,`mysql`.`a`.`Insert_priv` AS `Insert_priv`,`mysql`.`a`.`Update_priv` AS `Update_priv`,`mysql`.`a`.`Delete_priv` AS `Delete_priv`,`mysql`.`a`.`Create_priv` AS `Create_priv`,`mysql`.`a`.`Drop_priv` AS `Drop_priv`,`mysql`.`a`.`Reload_priv` AS `Reload_priv`,`mysql`.`a`.`Shutdown_priv` AS `Shutdown_priv`,`mysql`.`a`.`Process_priv` AS `Process_priv`,`mysql`.`a`.`File_priv` AS `File_priv`,`mysql`.`a`.`Grant_priv` AS `Grant_priv`,`mysql`.`a`.`References_priv` AS `References_priv`,`mysql`.`a`.`Index_priv` AS `Index_priv`,`mysql`.`a`.`Alter_priv` AS `Alter_priv`,`mysql`.`a`.`Show_db_priv` AS `Show_db_priv`,`mysql`.`a`.`Super_priv` AS `Super_priv`,`mysql`.`a`.`Create_tmp_table_priv` AS `Create_tmp_table_priv`,`mysql`.`a`.`Lock_tables_priv` AS `Lock_tables_priv`,`mysql`.`a`.`Execute_priv` AS `Execute_priv`,`mysql`.`a`.`Repl_slave_priv` AS `Repl_slave_priv`,`mysql`.`a`.`Repl_client_priv` AS `Repl_client_priv`,`mysql`.`a`.`Create_view_priv` AS `Create_view_priv`,`mysql`.`a`.`Show_view_priv` AS `Show_view_priv`,`mysql`.`a`.`Create_routine_priv` AS `Create_routine_priv`,`mysql`.`a`.`Alter_routine_priv` AS `Alter_routine_priv`,`mysql`.`a`.`Create_user_priv` AS `Create_user_priv`,`mysql`.`a`.`Event_priv` AS `Event_priv`,`mysql`.`a`.`Trigger_priv` AS `Trigger_priv`,`mysql`.`a`.`Create_tablespace_priv` AS `Create_tablespace_priv`,`mysql`.`a`.`ssl_type` AS `ssl_type`,`mysql`.`a`.`ssl_cipher` AS `ssl_cipher`,`mysql`.`a`.`x509_issuer` AS `x509_issuer`,`mysql`.`a`.`x509_subject` AS `x509_subject`,`mysql`.`a`.`max_questions` AS `max_questions`,`mysql`.`a`.`max_updates` AS `max_updates`,`mysql`.`a`.`max_connections` AS `max_connections`,`mysql`.`a`.`max_user_connections` AS `max_user_connections`,`mysql`.`a`.`plugin` AS `plugin`,`mysql`.`a`.`authentication_string` AS `authentication_string` from `mysql`.`user` `a` where <expr_cache><`mysql`.`a`.`User`>(exists(select 1 from `mysql`.`user` `b` where (`mysql`.`b`.`User` = `mysql`.`a`.`User`)))

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
2 rows in set (0.00 sec)

2. CLI (CLIENT-A) with QUERY-BAD (but still returning same result-set, as QUERY-GOOD)

MariaDB [(none)]> EXPLAIN EXTENDED SELECT a.* FROM mysql.user a WHERE ( SELECT EXISTS ( SELECT 1 FROM mysql.user b WHERE b.user = a.user LIMIT 1 ) );
-----------------------------------------------------------------------------------------------------

id select_type table type possible_keys key key_len ref rows filtered Extra

-----------------------------------------------------------------------------------------------------

1 PRIMARY a ALL NULL NULL NULL NULL 351 100.00 Using where
3 DEPENDENT SUBQUERY b index NULL PRIMARY 228 NULL 351 100.00 Using where; Using index

-----------------------------------------------------------------------------------------------------
2 rows in set, 3 warnings (0.00 sec)

MariaDB [(none)]> SHOW WARNINGS;
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Level Code Message

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Note 1276 Field or reference 'mysql.a.User' of SELECT #3 was resolved in SELECT #1
Note 1249 Select 2 was reduced during optimization
Note 1003 select `mysql`.`a`.`Host` AS `Host`,`mysql`.`a`.`User` AS `User`,`mysql`.`a`.`Password` AS `Password`,`mysql`.`a`.`Select_priv` AS `Select_priv`,`mysql`.`a`.`Insert_priv` AS `Insert_priv`,`mysql`.`a`.`Update_priv` AS `Update_priv`,`mysql`.`a`.`Delete_priv` AS `Delete_priv`,`mysql`.`a`.`Create_priv` AS `Create_priv`,`mysql`.`a`.`Drop_priv` AS `Drop_priv`,`mysql`.`a`.`Reload_priv` AS `Reload_priv`,`mysql`.`a`.`Shutdown_priv` AS `Shutdown_priv`,`mysql`.`a`.`Process_priv` AS `Process_priv`,`mysql`.`a`.`File_priv` AS `File_priv`,`mysql`.`a`.`Grant_priv` AS `Grant_priv`,`mysql`.`a`.`References_priv` AS `References_priv`,`mysql`.`a`.`Index_priv` AS `Index_priv`,`mysql`.`a`.`Alter_priv` AS `Alter_priv`,`mysql`.`a`.`Show_db_priv` AS `Show_db_priv`,`mysql`.`a`.`Super_priv` AS `Super_priv`,`mysql`.`a`.`Create_tmp_table_priv` AS `Create_tmp_table_priv`,`mysql`.`a`.`Lock_tables_priv` AS `Lock_tables_priv`,`mysql`.`a`.`Execute_priv` AS `Execute_priv`,`mysql`.`a`.`Repl_slave_priv` AS `Repl_slave_priv`,`mysql`.`a`.`Repl_client_priv` AS `Repl_client_priv`,`mysql`.`a`.`Create_view_priv` AS `Create_view_priv`,`mysql`.`a`.`Show_view_priv` AS `Show_view_priv`,`mysql`.`a`.`Create_routine_priv` AS `Create_routine_priv`,`mysql`.`a`.`Alter_routine_priv` AS `Alter_routine_priv`,`mysql`.`a`.`Create_user_priv` AS `Create_user_priv`,`mysql`.`a`.`Event_priv` AS `Event_priv`,`mysql`.`a`.`Trigger_priv` AS `Trigger_priv`,`mysql`.`a`.`Create_tablespace_priv` AS `Create_tablespace_priv`,`mysql`.`a`.`ssl_type` AS `ssl_type`,`mysql`.`a`.`ssl_cipher` AS `ssl_cipher`,`mysql`.`a`.`x509_issuer` AS `x509_issuer`,`mysql`.`a`.`x509_subject` AS `x509_subject`,`mysql`.`a`.`max_questions` AS `max_questions`,`mysql`.`a`.`max_updates` AS `max_updates`,`mysql`.`a`.`max_connections` AS `max_connections`,`mysql`.`a`.`max_user_connections` AS `max_user_connections`,`mysql`.`a`.`plugin` AS `plugin`,`mysql`.`a`.`authentication_string` AS `authentication_string` from `mysql`.`user` `a` where <expr_cache><`mysql`.`a`.`User`>(exists(select 1 from `mysql`.`user` `b` where (`mysql`.`b`.`User` = `mysql`.`a`.`User`)))

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
3 rows in set (0.00 sec)

3. CLI (CLIENT-A) with QUERY-BAD (returning empty result-set, after CLIENT-B executed QUERY-BAD two times)

MariaDB [(none)]> SELECT a.* FROM mysql.user a WHERE ( SELECT EXISTS ( SELECT 1 FROM mysql.user b WHERE b.user = a.user LIMIT 1 ) );
Empty set (0.00 sec)

MariaDB [(none)]> EXPLAIN EXTENDED SELECT a.* FROM mysql.user a WHERE ( SELECT EXISTS ( SELECT 1 FROM mysql.user b WHERE b.user = a.user LIMIT 1 ) );
-----------------------------------------------------------------------------------------------------

id select_type table type possible_keys key key_len ref rows filtered Extra

-----------------------------------------------------------------------------------------------------

1 PRIMARY a ALL NULL NULL NULL NULL 351 100.00 Using where
3 DEPENDENT SUBQUERY b index NULL PRIMARY 228 NULL 351 100.00 Using where; Using index

-----------------------------------------------------------------------------------------------------
2 rows in set, 3 warnings (0.00 sec)

MariaDB [(none)]> SHOW WARNINGS;
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Level Code Message

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Note 1276 Field or reference 'mysql.a.User' of SELECT #3 was resolved in SELECT #1
Note 1249 Select 2 was reduced during optimization
Note 1003 select `mysql`.`a`.`Host` AS `Host`,`mysql`.`a`.`User` AS `User`,`mysql`.`a`.`Password` AS `Password`,`mysql`.`a`.`Select_priv` AS `Select_priv`,`mysql`.`a`.`Insert_priv` AS `Insert_priv`,`mysql`.`a`.`Update_priv` AS `Update_priv`,`mysql`.`a`.`Delete_priv` AS `Delete_priv`,`mysql`.`a`.`Create_priv` AS `Create_priv`,`mysql`.`a`.`Drop_priv` AS `Drop_priv`,`mysql`.`a`.`Reload_priv` AS `Reload_priv`,`mysql`.`a`.`Shutdown_priv` AS `Shutdown_priv`,`mysql`.`a`.`Process_priv` AS `Process_priv`,`mysql`.`a`.`File_priv` AS `File_priv`,`mysql`.`a`.`Grant_priv` AS `Grant_priv`,`mysql`.`a`.`References_priv` AS `References_priv`,`mysql`.`a`.`Index_priv` AS `Index_priv`,`mysql`.`a`.`Alter_priv` AS `Alter_priv`,`mysql`.`a`.`Show_db_priv` AS `Show_db_priv`,`mysql`.`a`.`Super_priv` AS `Super_priv`,`mysql`.`a`.`Create_tmp_table_priv` AS `Create_tmp_table_priv`,`mysql`.`a`.`Lock_tables_priv` AS `Lock_tables_priv`,`mysql`.`a`.`Execute_priv` AS `Execute_priv`,`mysql`.`a`.`Repl_slave_priv` AS `Repl_slave_priv`,`mysql`.`a`.`Repl_client_priv` AS `Repl_client_priv`,`mysql`.`a`.`Create_view_priv` AS `Create_view_priv`,`mysql`.`a`.`Show_view_priv` AS `Show_view_priv`,`mysql`.`a`.`Create_routine_priv` AS `Create_routine_priv`,`mysql`.`a`.`Alter_routine_priv` AS `Alter_routine_priv`,`mysql`.`a`.`Create_user_priv` AS `Create_user_priv`,`mysql`.`a`.`Event_priv` AS `Event_priv`,`mysql`.`a`.`Trigger_priv` AS `Trigger_priv`,`mysql`.`a`.`Create_tablespace_priv` AS `Create_tablespace_priv`,`mysql`.`a`.`ssl_type` AS `ssl_type`,`mysql`.`a`.`ssl_cipher` AS `ssl_cipher`,`mysql`.`a`.`x509_issuer` AS `x509_issuer`,`mysql`.`a`.`x509_subject` AS `x509_subject`,`mysql`.`a`.`max_questions` AS `max_questions`,`mysql`.`a`.`max_updates` AS `max_updates`,`mysql`.`a`.`max_connections` AS `max_connections`,`mysql`.`a`.`max_user_connections` AS `max_user_connections`,`mysql`.`a`.`plugin` AS `plugin`,`mysql`.`a`.`authentication_string` AS `authentication_string` from `mysql`.`user` `a` where <expr_cache><`mysql`.`a`.`User`>(exists(select 1 from `mysql`.`user` `b` where (`mysql`.`b`.`User` = `mysql`.`a`.`User`)))

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
3 rows in set (0.00 sec)

I hope, I got you right this time. What else can I provide?

Comment by Elena Stepanova [ 2012-09-10 ]

Re: sub-subselect sometimes looses reference to enclosing table
Hi Daniel,

Thanks for all the data, unfortunately it still doesn't show the root of the problem (not because you collected it wrongly, but because the problem is apparently a tricky one).

Is it only reproducible on the `user` table with particular contents? You mentioned at the beginning that it could be used for convenience, and I agree that generally it's more convenient to use an already existing common table, but since it's not reproducible in our environment so far, and since your table contains private data, if possible, could you maybe give a full example, with a synthetic non-confidential table data and structure?

Thanks.

Comment by Oleksandr Byelkin [ 2012-09-10 ]

Re: sub-subselect sometimes looses reference to enclosing table
Could you please run such variant of that 3 SELECT "bad" query in 2 variants that give different result:
SELECT a.* FROM mysql.user a WHERE ( SELECT 0+EXISTS ( SELECT 1 FROM mysql.user b WHERE b.user = a.user LIMIT 1 ) );

Comment by Oleksandr Byelkin [ 2012-09-10 ]

Re: sub-subselect sometimes looses reference to enclosing table
ah and yet another one

SELECT a.* FROM mysql.user a WHERE 0+SELECT (EXISTS ( SELECT 1 FROM mysql.user b WHERE b.user = a.user LIMIT 1 ) );

Comment by Daniel Heimann (Inactive) [ 2012-09-10 ]

Re: sub-subselect sometimes looses reference to enclosing table
hi Elena,

yes of course; that was just for convenience. Initially we found the problem within another context (an e-commerce shop-system) and rewrote it to demonstrate the effect, with tables, everyone has available.

The described effect can be seen, by just executing (as db-root):

CREATE USER example IDENTIFIED BY "exexex";
CREATE DATABASE example;
GRANT ALL ON example.* TO example;

and then (as example):

CREATE TABLE `user` ( `User` char(2), PRIMARY KEY (`User`) );
INSERT INTO `user` values ('u1');
SELECT a.* FROM example.user a WHERE ( SELECT EXISTS ( SELECT 1 FROM example.user b WHERE b.user = a.user LIMIT 1 ) );

Nothing more is needed (thinking about it: sorry, for not coming up with that, from the very start). Especially the interaction between php and mysql-cli still kicks in as before.

While I put things together, I wanted to write some minimal php code to further reduce unknown variables (like phpmyadmin) in reproduceability. But neither a trivial function-based API approach...

$link = mysql_connect("$host", 'example', 'exexex');
$result = mysql_query('SELECT a.* FROM example.user a WHERE (SELECT EXISTS (SELECT 1 FROM example.user b WHERE b.user = a.user LIMIT 1))');
while ($row = mysql_fetch_assoc($result))

{ echo $row['User'] . "\n"; }


...nor mysqli-API alone...

$mysqli = new mysqli("$host", "example", "exexex", "example");
$result = $mysqli->query('SELECT a.* FROM example.user a WHERE (SELECT EXISTS (SELECT 1 FROM example.user b WHERE b.user = a.user LIMIT 1))');
while($row = $result->fetch_assoc()){ echo $row['User'] . "n"; }

...were able to fubar the query.

So I turned on general log once more and inspected all those queries, phpmyadmin put around "QUERY-BAD":

SET CHARACTER SET 'utf8'
SET collation_connection = 'utf8_general_ci'
SELECT a.* FROM example.user a WHERE ( SELECT EXISTS ( SELECT 1 FROM example.user b WHERE b.user = a.user LIMIT 1 ) ) LIMIT 0, 30
SELECT SQL_CALC_FOUND_ROWS a . * FROM example.user a WHERE ( SELECT EXISTS ( SELECT 1 FROM example.user b WHERE b.user = a.user LIMIT 1 ) ) LIMIT 1
SELECT FOUND_ROWS()
SHOW TABLES LIKE 'user'
SHOW TABLE STATUS FROM `example` LIKE 'user%'
SELECT COUNT FROM `example`.`user`
SHOW FULL COLUMNS FROM `example`.`user`

Trying all of these on CLI then finally showed, that after executing:

SHOW FULL COLUMNS FROM `example`.`user`

the result-set gets and stays empty for further QUERY-BAD until someone executes QUERY-GOOD.

I hope it helps, that we can rule out php.

Can you reproduce?

Comment by Daniel Heimann (Inactive) [ 2012-09-10 ]

Re: sub-subselect sometimes looses reference to enclosing table
hi,

Oleksandr "Sanja" Byelkin's query from comment #9 gets broken too, when someone does...

SHOW FULL COLUMNS FROM `example`.`user`;

and "magically repaired" by executing...

SELECT a.* FROM example.user a WHERE ( EXISTS ( SELECT 1 FROM example.user b WHERE b.user = a.user LIMIT 1 ) );

While the query from comment #10 can not be executed at all:
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'SELECT (EXISTS ( SELECT 1 FROM mysql.user b WHERE b.user = a.user LIMIT 1 ) )' at line 1

Comment by Elena Stepanova [ 2012-09-10 ]

Re: sub-subselect sometimes looses reference to enclosing table
Daniel,

Yes, I was able to reproduce it with 'SHOW FULL COLUMNS', as you described (I'm wondering why it didn't happen for me with phpmyadmin, possibly my hasty settings for phpmyadmin were such that this SHOW was not executed).

Thanks a lot for your analysis and thorough approach!

Comment by Daniel Heimann (Inactive) [ 2012-09-10 ]

Re: sub-subselect sometimes looses reference to enclosing table
hi Elena,

thank you for your very fast feedback and overall guidance in reporting this!
Reproducible bugs are much less worse, than (seemingly) unreproducible ones.

Happy tracing down to the roots/fixing.
Please tell me, if I can test or send in anything else to help.

Don't know if that matters, but I tested 5.5.27 (mysqld Ver 5.5.27-MariaDB-mariadb1~squeeze-log for debian-linux-gnu on x86_64 (mariadb.org binary distribution)) some minutes ago and the bug is still in there.

Comment by Elena Stepanova [ 2012-09-10 ]

Re: sub-subselect sometimes looses reference to enclosing table
Refiled in JIRA as https://mariadb.atlassian.net/browse/MDEV-521
The entry in JIRA contains a testcase which works both in MySQL client and MTR (basically the same as above, only generalized).

Hi Daniel,

If you're interested in the intermediate progress on the bug, you might want to subscribe to JIRA in general (https://mariadb.atlassian.net/secure/Dashboard.jspa) and to this bug in particular.
This LP entry will be updated when the bug is fixed, but otherwise probably won't receive many updates as we are gradually switching to JIRA for bug tracking.

Thanks again for your help.

Comment by Rasmus Johansson (Inactive) [ 2012-10-01 ]

Launchpad bug id: 1046882

Comment by Daniel Heimann (Inactive) [ 2012-10-01 ]

Re: sub-subselect sometimes looses reference to enclosing table
Thank you very much for all the hard work! Looking forward to 5.5.28.

Generated at Thu Feb 08 06:49:56 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.