[MDEV-10669] Crash in SELECT with window function used Created: 2016-08-26  Updated: 2016-10-27  Resolved: 2016-10-27

Status: Closed
Project: MariaDB Server
Component/s: Optimizer - Window functions
Affects Version/s: 10.2.1, 10.2.2
Fix Version/s: 10.2.2

Type: Bug Priority: Critical
Reporter: Valerii Kravchuk Assignee: Sergei Petrunia
Resolution: Fixed Votes: 0
Labels: None

Attachments: JPEG File window-func-split-sum-func.jpg    
Issue Links:
Relates
relates to MDEV-10815 Window Function Expressions Wrong Res... Closed

 Description   

Simple window function used together with IFNULL in the SELECT ... list leads to repeatable crash:

openxs@ao756:~/dbs/maria10.2$ bin/mysql -uroot test
 
Welcome to the MariaDB monitor.  Commands end with ; or \g.
 
Your MariaDB connection id is 2
 
Server version: 10.2.2-MariaDB MariaDB Server
 
 
 
Copyright (c) 2000, 2016, Oracle, MariaDB Corporation Ab and others.
 
 
 
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
 
 
 
MariaDB [test]> create table t(a decimal(35,10), b int);
 
Query OK, 0 rows affected (0.28 sec)
 
 
 
MariaDB [test]> insert into t(a,b) values(1,1);
 
Query OK, 1 row affected (0.05 sec)
 
 
 
MariaDB [test]> insert into t(a,b) values(2,1);
 
Query OK, 1 row affected (0.04 sec)
 
 
 
MariaDB [test]> insert into t(a,b) values(0,1);
 
Query OK, 1 row affected (0.04 sec)
 
 
 
MariaDB [test]> select * from t;
 
+--------------+------+
 
| a            | b    |
 
+--------------+------+
 
| 1.0000000000 |    1 |
 
| 2.0000000000 |    1 |
 
| 0.0000000000 |    1 |
 
+--------------+------+
 
3 rows in set (0.00 sec)
 
 
 
MariaDB [test]> SELECT ifnull(((t.a) / CASE WHEN sum(t.a) over(partition by t.b) =0 then null else null end) ,0) from t;
 
ERROR 2013 (HY000): Lost connection to MySQL server during query
 
MariaDB [test]> 160826 17:30:16 mysqld_safe Number of processes running now: 0
 
160826 17:30:16 mysqld_safe mysqld restarted
 
 
 
MariaDB [test]> exit
 
Bye
 
openxs@ao756:~/dbs/maria10.2$ tail -100 data/ao756.err
 
2016-08-26 17:26:59 139788388034496 [Note] InnoDB: Completed initialization of buffer pool
 
2016-08-26 17:26:59 139788388034496 [Note] InnoDB: Highest supported file format is Barracuda.
 
2016-08-26 17:26:59 139788388034496 [Note] InnoDB: 128 rollback segment(s) are active.
 
2016-08-26 17:26:59 139788388034496 [Note] InnoDB: Waiting for purge to start
 
2016-08-26 17:26:59 139788388034496 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.30-76.3 started; log sequence number 1616819
 
2016-08-26 17:26:59 139787830806272 [Note] InnoDB: Dumping buffer pool(s) not yet started
 
2016-08-26 17:26:59 139788388034496 [Note] Plugin 'FEEDBACK' is disabled.
 
2016-08-26 17:26:59 139788388034496 [Note] Server socket created on IP: '::'.
 
2016-08-26 17:26:59 139788388034496 [Note] /home/openxs/dbs/maria10.2/bin/mysqld: ready for connections.
 
Version: '10.2.2-MariaDB'  socket: '/tmp/mysql.sock'  port: 3306  MariaDB Server
 
160826 17:30:16 [ERROR] mysqld got signal 11 ;
 
This could be because you hit a bug. It is also possible that this binary
 
or one of the libraries it was linked against is corrupt, improperly built,
 
or misconfigured. This error can also be caused by malfunctioning hardware.
 
 
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
 
 
We will try our best to scrape up some info that will hopefully help
 
diagnose the problem, but since we have already crashed,
 
something is definitely wrong and this may fail.
 
 
 
Server version: 10.2.2-MariaDB
 
key_buffer_size=134217728
 
read_buffer_size=131072
 
max_used_connections=1
 
max_threads=153
 
thread_count=1
 
It is possible that mysqld could use up to
 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 467161 K  bytes of memory
 
Hope that's ok; if not, decrease some variables in the equation.
 
 
 
Thread pointer: 0x0x7f22d708a008
 
Attempting backtrace. You can use the following information to find out
 
where mysqld died. If you see no messages after this, something went
 
terribly wrong...
 
stack_bottom = 0x7f23052f69f0 thread_stack 0x48c00
 
/home/openxs/dbs/maria10.2/bin/mysqld(my_print_stacktrace+0x2e)[0x7f2304c0191e]
 
/home/openxs/dbs/maria10.2/bin/mysqld(handle_fatal_signal+0x38d)[0x7f230471013d]
 
/lib/x86_64-linux-gnu/libpthread.so.0(+0x10330)[0x7f2303088330]
 
/home/openxs/dbs/maria10.2/bin/mysqld(_ZN4Item13save_in_fieldEP5Fieldb+0x1e6)[0x7f230471fc66]
 
/home/openxs/dbs/maria10.2/bin/mysqld(_Z31compute_window_func_with_framesP16Item_window_funcP5TABLEP11READ_RECORD+0x1a6)[0x7f23046a25f6]
 
/home/openxs/dbs/maria10.2/bin/mysqld(_ZN18Window_func_runner4execEP5TABLEP9SORT_INFO+0x7b)[0x7f23046a27fb]
 
/home/openxs/dbs/maria10.2/bin/mysqld(_ZN17Window_funcs_sort4execEP4JOIN+0x5b)[0x7f23046a288b]
 
/home/openxs/dbs/maria10.2/bin/mysqld(_ZN24Window_funcs_computation4execEP4JOIN+0x18)[0x7f23046a2e78]
 
/home/openxs/dbs/maria10.2/bin/mysqld(_ZN7AGGR_OP8end_sendEv+0xd7)[0x7f23045da667]
 
/home/openxs/dbs/maria10.2/bin/mysqld(_Z24sub_select_postjoin_aggrP4JOINP13st_join_tableb+0x35)[0x7f23045da865]
 
/home/openxs/dbs/maria10.2/bin/mysqld(_ZN4JOIN10exec_innerEv+0x845)[0x7f23045db165]
 
/home/openxs/dbs/maria10.2/bin/mysqld(_ZN4JOIN4execEv+0x33)[0x7f23045db523]
 
/home/openxs/dbs/maria10.2/bin/mysqld(_Z12mysql_selectP3THDP10TABLE_LISTjR4ListI4ItemEPS4_jP8st_orderS9_S7_S9_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x112)[0x7f23045db672]
 
/home/openxs/dbs/maria10.2/bin/mysqld(_Z13handle_selectP3THDP3LEXP13select_resultm+0x23c)[0x7f23045dc12c]
 
/home/openxs/dbs/maria10.2/bin/mysqld(+0x46211e)[0x7f230457b11e]
 
mysys/stacktrace.c:247(my_print_stacktrace)[0x7f2304587dcc]
 
/home/openxs/dbs/maria10.2/bin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_statebb+0x2b5)[0x7f230458b515]
 
sql/item.cc:5956(Item::save_in_field(Field*, bool))[0x7f230458ed33]
 
sql/sql_window.cc:1753(compute_window_func_with_frames(Item_window_func*, TABLE*, READ_RECORD*))[0x7f230458f5ac]
 
/home/openxs/dbs/maria10.2/bin/mysqld(_Z24do_handle_one_connectionP7CONNECT+0x20a)[0x7f2304653a6a]
 
sql/sql_select.cc:26156(AGGR_OP::end_send())[0x7f2304653c2d]
 
/lib/x86_64-linux-gnu/libpthread.so.0(+0x8184)[0x7f2303080184]
 
sql/sql_parse.cc:6311(execute_sqlcom_select)[0x7f23027a337d]
 
 
 
Trying to get some variables.
 
Some pointers may be invalid and cause the dump to abort.
 
Query (0x7f22d70db020): SELECT ifnull(((t.a) / CASE WHEN sum(t.a) over(partition by t.b) =0 then null else null end) ,0) from t
 
Connection ID (thread ID): 2
 
Status: NOT_KILLED
 
 
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=off
 
...



 Comments   
Comment by Valerii Kravchuk [ 2016-08-26 ]

IFNULL does not really matter it seems:

Thread pointer: 0x0x7fad7cfb4008
 
Attempting backtrace. You can use the following information to find out
 
where mysqld died. If you see no messages after this, something went
 
terribly wrong...
 
stack_bottom = 0x7fad99a779f0 thread_stack 0x48c00
 
/home/openxs/dbs/maria10.2/bin/mysqld(my_print_stacktrace+0x2e)[0x7fad9938291e]
 
/home/openxs/dbs/maria10.2/bin/mysqld(handle_fatal_signal+0x38d)[0x7fad98e9113d]
 
/lib/x86_64-linux-gnu/libpthread.so.0(+0x10330)[0x7fad97809330]
 
/home/openxs/dbs/maria10.2/bin/mysqld(_ZN4Item13save_in_fieldEP5Fieldb+0x1e6)[0x7fad98ea0c66]
 
/home/openxs/dbs/maria10.2/bin/mysqld(_Z31compute_window_func_with_framesP16Item_window_funcP5TABLEP11READ_RECORD+0x1a6)[0x7fad98e235f6]
 
/home/openxs/dbs/maria10.2/bin/mysqld(_ZN18Window_func_runner4execEP5TABLEP9SORT_INFO+0x7b)[0x7fad98e237fb]
 
/home/openxs/dbs/maria10.2/bin/mysqld(_ZN17Window_funcs_sort4execEP4JOIN+0x5b)[0x7fad98e2388b]
 
/home/openxs/dbs/maria10.2/bin/mysqld(_ZN24Window_funcs_computation4execEP4JOIN+0x18)[0x7fad98e23e78]
 
/home/openxs/dbs/maria10.2/bin/mysqld(_ZN7AGGR_OP8end_sendEv+0xd7)[0x7fad98d5b667]
 
/home/openxs/dbs/maria10.2/bin/mysqld(_Z24sub_select_postjoin_aggrP4JOINP13st_join_tableb+0x35)[0x7fad98d5b865]
 
mysys/stacktrace.c:247(my_print_stacktrace)[0x7fad98d5c165]
 
sql/signal_handler.cc:159(handle_fatal_signal)[0x7fad98d5c523]
 
/home/openxs/dbs/maria10.2/bin/mysqld(_Z12mysql_selectP3THDP10TABLE_LISTjR4ListI4ItemEPS4_jP8st_orderS9_S7_S9_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x112)[0x7fad98d5c672]
 
sql/item.cc:5956(Item::save_in_field(Field*, bool))[0x7fad98d5d12c]
 
sql/sql_window.cc:1969(Window_funcs_computation::exec(JOIN*))[0x7fad98cfc11e]
 
/home/openxs/dbs/maria10.2/bin/mysqld(_Z21mysql_execute_commandP3THD+0x62dc)[0x7fad98d08dcc]
 
sql/sql_select.cc:26156(AGGR_OP::end_send())[0x7fad98d0c515]
 
sql/sql_parse.cc:6311(execute_sqlcom_select)[0x7fad98d0fd33]
 
sql/sql_parse.cc:1355(do_command(THD*))[0x7fad98d105ac]
 
sql/sql_connect.cc:1354(do_handle_one_connection(CONNECT*))[0x7fad98dd4a6a]
 
sql/sql_connect.cc:1262(handle_one_connection)[0x7fad98dd4c2d]
 
/lib/x86_64-linux-gnu/libpthread.so.0(+0x8184)[0x7fad97801184]
 
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fad96f2437d]
 
 
 
Trying to get some variables.
 
Some pointers may be invalid and cause the dump to abort.
 
Query (0x7fad7cc8b020): SELECT (CASE WHEN sum(t.a) over(partition by t.b) =0 then null else null end) as a from t
 
Connection ID (thread ID): 2
 
Status: NOT_KILLED

No crashing without OVER(PARTITION BY t.b):

MariaDB [test]> SELECT (CASE WHEN sum(t.a) =0 then null else null end) as a from t;
------

a

------

NULL

------
1 row in set (0.00 sec)

Comment by Vicențiu Ciorbaru [ 2016-09-15 ]

The problem lies within the CASE WHEN expression.
The simplified test case used is:

SELECT (CASE WHEN sum(t.a) over (partition by t.b)=0 THEN null ELSE null END) AS a FROM t;

When executing this query, we're creating a temporary table to store the window function results. The problem arises because our window function is hidden beneath an Item_func_case. When creating the temporary table fields, we only create one field, which represents the Item_func_case result value. Unfortunately, we do not create an extra field, specifically for the window function hidden inside.

Looking at Item_func_case implementation and based on how window functions are supposed to be computed, the only practical approach to implementing this that I see is:

1. Create extra fields for each "inner" window function.
2. Compute window functions in the separate fields.
3. Fill CASE values after all window function computations are performed.
4. Return requested field as part of the select loop.

This is the most straightforward and feels the most direct. The downside/tricky part is that we have to add another extra computation step, of resolving any items whose results are based on window function values. This however seems worth it as we could then allow for computing expressions with window functions.

Note, the following does not work either:

select -1 + sum(t.a) over (partition by t.b)
 
from t;

A fix for this kind of problem would enable expressions such as those to also work.

Comment by Igor Babaev [ 2016-09-15 ]

Vicentiu,
The same problem existed for expressions with aggregate functions. That problem was resolved by 'splitting' the expressions. Look at split_sum_func() and split_sum_func2(). You can use the same technique for expressions with window functions.

Comment by Vicențiu Ciorbaru [ 2016-09-15 ]

Igor, that is indeed a good idea, it is actually what Sergey Petrunia also pointed at. Also, take a look at MDEV-10815.

Here is however a peculiarity:

create table t(a decimal(35,10), b int);
 
insert into t(a,b) values(1,1);
 
insert into t(a,b) values(2,1);
 
insert into t(a,b) values(0,2);
 
 
 
SELECT CASE WHEN t.a = 0 then t.b else t.a
 
end as case_value_without_sum
 
FROM t;
 
 
 
case_value_without_sum
 
1.0000000000
 
2.0000000000
 
2.0000000000
 
 
 
-- This is a pretty tame result. For each row we either get t.a if t.a is non zero, or t.b otherwise.
 
 
 
-- However for this select statement, we only get 1 row. And it seems to be arbitrarily set at the value
 
-- of SUM(t.a) which is 3. 
SELECT CASE WHEN t.a = 0 then SUM(t.b) else SUM(t.a) end as case_value_with_sum
 
FROM t;
 
 
 
case_value_with_sum
 
3.0000000000

Comment by Vicențiu Ciorbaru [ 2016-09-15 ]

After discussion with Sergey Petrunia, the last query should be thought as function call with 3 expresssions:

SELECT case(t.a=0, SUM(t.b), SUM(t.a))
FROM t;

With such a use case we should perform aggregation. Given MySQL extension that we allow aggregates and non-aggregates without group by, we return the value for a random row within the group. In this case, the group is the whole table and we return the value of the case function for either the first or the second row. The MariaDB knowledge base is lacking in this regard, will update it.

Comment by Vicențiu Ciorbaru [ 2016-09-17 ]

Hi Sergey!

Please review the additional patch that adds on top of MDEV-10815's patch. This patch makes it so that item's who's nested arguments contain window functions work.

Ex: ifnull(1+sum(a) over (), 0).

This is the latest patch:
http://lists.askmonty.org/pipermail/commits/2016-September/009869.html

Comment by Sergei Petrunia [ 2016-09-17 ]

The patch works.
I'm concerned that the way currently split_sum_func works, it creates unnecessary indirections.

Let's take this query as an example (It crashes before this patch, it works after):

select a, b,(sum(t.a) over (partition by t.b order by a) + 1)*2 as sum_and_const from t;

The results of split_sum_func calls are as follows: . Item_aggregate between Item_func_plus and Item_func_mul is redundant.
I'm not 100% sure if this is harmless. If it is, it can be addressed outside of the scope of this MDEV.

Comment by Sergei Petrunia [ 2016-10-27 ]

The fix was pushed into MariaDB 10.2.2.
(the "concerns" voiced in previous comments are still there. They cause no visible harm, though)

Generated at Thu Feb 08 07:44:01 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.