Details
-
Task
-
Status: Open (View Workflow)
-
Major
-
Resolution: Unresolved
-
23.02.24
-
None
-
None
Description
Summary: when a query's aggregation spills to disk, PrimProc logs nothing between the query's start and end. On a production 23.02 cluster, a daily aggregation query ran with no end, and PrimProc's journal was silent from its start until the node was restarted. The logs can't tell a disk-aggregation merge that is still progressing (possibly slowly, on a shared/NFS spill directory) from one that is stuck.
Proposal: log a few lines from disk-based aggregation (RowAggStorage), each with the session ID:
- spill start: spill directory, memory limit hit;
- each new generation: generation number, bytes and rows spilled so far;
- finalize start: generation count, total bytes on disk;
- finalize progress at most every N seconds (e.g. 60): generations merged / total, rows merged, bytes read back;
- finalize end: elapsed time.
Logging only: no behavior change. Throttled so a long merge produces at most one line per interval per aggregation thread group.
Status: not started.