[MCOL-894] make order by more scalable / faster Created: 2017-08-29  Updated: 2019-11-08  Resolved: 2019-11-08

Status: Closed
Project: MariaDB ColumnStore
Component/s: MariaDB Server
Affects Version/s: 1.0.10
Fix Version/s: 1.4.1

Type: New Feature Priority: Critical
Reporter: Richard Stracke Assignee: Daniel Lee (Inactive)
Resolution: Fixed Votes: 3
Labels: None
Environment:

all


Sub-Tasks:
Key
Summary
Type
Status
Assignee
MCOL-3581 Multi-threaded order-by for 1.2 Sub-Task Closed Daniel Lee  
Epic Link: ColumnStore Performance Improvements
Sprint: 2019-06

 Description   

Hello,

Order by processing from MariaDB is still single threaded.

The impression of columnstore suffer, if the query runs fast in the columnstore part,
but slow in total, because order by can be slowdown.



 Comments   
Comment by Andrew Hutchings (Inactive) [ 2017-08-29 ]

This is definitely something that has come up a lot recently. We have some ideas on how to solve this. It is too late to implement this in 1.1 but we could do this for a later version.

There is current a workaround if you are doing "order by X limit Y" and that is to wrap it in a subquery such as:

select * from (select * from t2 order by b limit 4) sq;

You need the "limit" though because "order by" will be optimised out without it.

Comment by Andrew Hutchings (Inactive) [ 2019-10-31 ]

This has been implemented in 1.2 for subqueries, etc... That effort is tracked in the subtask. It will be ported to 1.4 for all ColumnStore queries which will be tracked here.

Comment by Daniel Lee (Inactive) [ 2019-11-08 ]

Build verified: 1.4.1-1

[dlee@master centos7]$ cat gitversionInfo.txt
engine commit:
bd41ffd

1.4.0-1 beta and 1.4.1-1 is much faster than 1.2.5-1 and 1.2.6-1, using single or multiple thread.

In 1.4.0-1, using multiple also has a great performance gain.

1.4.1-1

MariaDB [tpch10]> set columnstore_orderby_threads=1;
Query OK, 0 rows affected (0.001 sec)

MariaDB [tpch10]> select l_orderkey from lineitem order by l_orderkey limit 10;
------------

l_orderkey

------------

1
1
1
1
1
1
2
3
3
3

------------
10 rows in set (4.232 sec)

MariaDB [tpch10]> select l_orderkey, l_comment from lineitem order by l_comment limit 10;
----------------------+

l_orderkey l_comment

----------------------+

753413 Tiresias
9714791 Tiresias
58301447 Tiresias
7299 Tiresias
85090 Tiresias
5277956 Tiresias
2570021 Tiresias
44041447 Tiresias
2475204 Tiresias
1141568 Tiresias

----------------------+
10 rows in set (35.206 sec)

MariaDB [tpch10]> set columnstore_orderby_threads=8;
Query OK, 0 rows affected (0.001 sec)

MariaDB [tpch10]> select l_orderkey from lineitem order by l_orderkey limit 10;
------------

l_orderkey

------------

1
1
1
1
1
1
2
3
3
3

------------
10 rows in set (2.524 sec)

MariaDB [tpch10]> set columnstore_orderby_threads=8;
Query OK, 0 rows affected (0.001 sec)

MariaDB [tpch10]> select l_orderkey, l_comment from lineitem order by l_comment limit 10;
----------------------+

l_orderkey l_comment

----------------------+

30202439 Tiresias
43662849 Tiresias
46411616 Tiresias
24971398 Tiresias
42581319 Tiresias
39185219 Tiresias
56893315 Tiresias
42593603 Tiresias
16359393 Tiresias
39959232 Tiresias

----------------------+
10 rows in set (16.405 sec)

Comment by Daniel Lee (Inactive) [ 2019-11-08 ]

closing ticket per test results for both this ticket and its subtask.

Generated at Thu Feb 08 02:24:36 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.