UNION query returns incorrect rows outside conditional evaluation

Please consider the following scenario:

drop table if exists union_bug;

create table union_bug (d datetime not null primary key);

insert into union_bug(d) values ('2016-06-01'),('2016-06-02'),('2016-06-03'),('2016-06-04');

select * from

(

	select * from union_bug where d between '2016-06-02' and '2016-06-05'

	union

	(select * from union_bug where d < '2016-06-05' order by d desc limit 1)

) onlyJun2toJun4

order by d

The above query should return 3 rows dated 02 through 04 June 2016. This can be born out by running the individual select statements.

However in practice this query returns 4 rows incorrectly dated 01 through 04 June 2016. It should NEVER NEVER NEVER return a row for 01 June, but it does.

I suspect this may be somehow related to the duplicate row from the lower query. If I change the 'union to a 'union all' then the query correctly returns the expected 4 rows, including the one duplicate row.