Enforcing innodb_open_files leads to fil_system->mutex problem

This was accidentally found by merging MDEV-13051 test to 10.2.
When InnoDB is trying to close data files to obey the innodb_open_files limit, it will release fil_system->mutex and fail to reacquire it. In a debug build, this will lead to a debug assertion failure.

mysqld: /mariadb/server/storage/innobase/fil/fil0fil.cc:823: bool fil_try_to_close_file_in_LRU(bool): Assertion `(&fil_system->mutex)->is_owned()' failed.

In a non-debug build, it could lead to a hang or crash (I did not test).

This was broken in a merge of 10.1 to 10.2 by me, already in MariaDB 10.2.4. The 10.1 code (changed by MDEV-11556) appears to be OK.