When geometry is not well constructed we get the NULL, as documentation said.

SELECT polygon(point(0,0),point(0,1),point(1,1),point(1,0),point(0,0))

NULL

In case of MySQL an descriptive error message is raised:

SELECT polygon(point(0,0),point(0,1),point(1,1),point(1,0),point(0,0))

ERROR 1210 (HY000) at line 1: Incorrect arguments to polygon

When we get in case if NULL is an argument, we get the NULL result what is ok.
See more example from here.