Incorrect results when comparing timestamp columns in HAVING clause

I have encountered a following inconsistency with SELECT when using HAVING and timestamp columns:

HAVING Timestamp1 < Timestamp2

The result set will contain some correct rows, and some incorrect, like:

Timestamp1 = 2025-05-28 10:29:07

Timestamp2 = 2025-05-27 05:00:19

But if I add an ORDER BY, that instead of the default order, the order by is done by some specific column, the result set is then suddenly correct. Or by adding an unnecessary aggregator like MIN or MAX:

HAVING MIN(Timestamp1) < MIN(Timestamp2)

The SQL query is a bit complex, and also contain a left join to filter out rows where a specific criteria does not match before we go to the HAVING.

I do not yet have a dummy data where I could re-produce this one.

Maybe there has been similar issues before?