Details
-
New Feature
-
Status: Open (View Workflow)
-
Major
-
Resolution: Unresolved
-
None
Description
Let's do the absolutely minimal streaming case here. Something like
select rank() over (order by id) from t1 limit 10; |
or even
select rank() over () from t1 limit 10; |
The goal is to read only 10 rows from the table for the above query.
Temporary table is enabled here:
|
sql_select.cc |
need_tmp= test_if_need_tmp_table();
|
and the decision is made here:
|
sql_select.h |
bool test_if_need_tmp_table() |
{
|
return ((const_tables != table_count && |
((select_distinct || !simple_order || !simple_group) ||
|
(group_list && order) ||
|
MY_TEST(select_options & OPTION_BUFFER_RESULT))) ||
|
(rollup.state != ROLLUP::STATE_NONE && select_distinct) ||
|
select_lex->have_window_funcs());
|
}
|
This should be extended like, for example
(select_lex->have_window_funcs() && !select_lex->have_streaming_window_funcs());
|
And the criteria could be for now:
- no global order by or group by
- window is the current row only
- window function from a small supported subset (e.g. RANK())
- no partition
- no filesort (no order by or order by via an index)
then we'll start relaxing these conditions. e.g. 4) and 5) should be easy to relax, 2) will likely stay forever. 1) will take a separate MDEV.
Solution ideas
1.
The code to do window function computation pass is in compute_window_func().
It will create cursors for current_row, start_bound, end_bound and then it has a pull-based loop:
while (true) |
{
|
if ((err= info.read_record())) |
break; // End of file. |
...
|
|
|
save_window_function_values(window_functions, tbl, rowid_buf);
|
|
|
}
|
If we are fine with "buffered" execution where we write rows into a temp.table and then read them back while computing window functions, this could still be used.
Otherwise, we could copy one loop iteration of compute_window_func() and compute on the fly.
Note that we would still need a dummy "temporary table" for computing expressions, like
SELECT 10 + row_number() over (ORDER BY key_to_use) FROM t1 ... ; |
We need to save row_number()'s return value $RET and then we can compute 10 +$RET.
2.
Window function expression is represented in the item tree by Item_window_func, the window function itself, for example, Item_sum_rank. The latter doesn't need a temporary table and can easily work for a stream of rows. Item_window_func::val_int() can invoke window_func()->val_int() directly without a temporary table. May be one can just force this code path somehow.
Attachments
Issue Links
- is part of
-
MDEV-35970 streaming window functions
-
- Open
-