[MDEV-27286] Implement WITH FILL to GROUP BY Created: 2021-12-16  Updated: 2023-02-05

Status: Open
Project: MariaDB Server
Component/s: None
Fix Version/s: None

Type: Task Priority: Minor
Reporter: Antonio Fernandes Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: PNG File image-2021-12-16-19-13-21-584.png     PNG File screenshot-1.png    

 Description   

There are a lot of cases when GROUP BY does not handle "blanks" in (for example) dates.

There should be an option to fetch data with zeros in GROUP BY:

SELECT EventDate, count() FROM test.hits WHERE CounterID = 2841673 GROUP BY EventDate
ORDER BY EventDate WITH FILL STEP 1
 
┌──EventDate─┬─count()─┐
│ 2014-03-17 │       3 │
│ 2014-03-18 │       0 │
│ 2014-03-19 │       6 │
│ 2014-03-20 │       0 │
│ 2014-03-21 │       7 │
│ 2014-03-22 │       6 │
└────────────┴─────────┘

Source: https://clickhouse.com/blog/en/2021/clickhouse-v21.12-released/#support-for-interval-operator-inside-with-fill-modifier-for-order-by-clause
New links: https://clickhouse.com/docs/en/whats-new/changelog/2021/
And: https://github.com/ClickHouse/ClickHouse/pull/30927

This is a very common use case when aggregating results.



 Comments   
Comment by Sergei Golubchik [ 2021-12-16 ]

The SQL standard way would be to do a LEFT join with a table that has all dates in your interval. You can generate such a table on the fly with a recursive CTE or (in MariaDB specifically) with a sequence engine. Even if you don't know the interval in advance the above can be done, I think, with a set of CTEs, first one with your original query and then a recursive one to fill in blanks.

Comment by Antonio Fernandes [ 2021-12-16 ]

The approach we use is indeed a LEFT JOIN with a every-date table.

The approach used by Clickhouse is quite elegant and removes the need for additional tables (either materialized or generated).

Removing the dates table, speeds things quite a lot:

Just thought of it as very common use case

Generated at Thu Feb 08 09:51:46 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.