fil_space_t::create() may release fil_system.mutex

saahil reported a crash in the MDEV-14992 development branch. The reason for the crash is that fil_space_t::create() may release and reacquire fil_system.mutex and therefore make an invalid intermediate state (a tablespace with no data file) observable by other threads.

I believe that it is cleanest to remove the offending code from fil_space_t::create() and move it to the callers where it may be needed:

diff --git a/storage/innobase/fil/fil0fil.cc b/storage/innobase/fil/fil0fil.cc

index 2c598ec471c..16e41cf8d49 100644

--- a/storage/innobase/fil/fil0fil.cc

+++ b/storage/innobase/fil/fil0fil.cc

@@ -1005,13 +1005,6 @@ fil_space_t *fil_space_t::create(uint32_t id, uint32_t flags,

   {

     fil_system.default_encrypt_tables.push_back(*space);

     space->is_in_default_encrypt= true;

-

-    if (srv_n_fil_crypt_threads_started)

-    {

-      mysql_mutex_unlock(&fil_system.mutex);

-      fil_crypt_threads_signal();

-      mysql_mutex_lock(&fil_system.mutex);

-    }

   }

   return space;

@@ -1910,6 +1903,7 @@ fil_ibd_create(

 	fil_node_t* node = space->add(path, OS_FILE_CLOSED, size, false, true);

 	space->set_stopped();

 	mysql_mutex_unlock(&fil_system.mutex);

+	fil_crypt_threads_signal();

 	buf_block_t *header[2];

 	{

@@ -2251,6 +2245,7 @@ fil_space_t *fil_ibd_open(uint32_t id, uint32_t flags,

 		df_remote.is_open() ? df_remote.filepath() :

 		df_default.filepath(), OS_FILE_CLOSED, 0, false, true);

 	mysql_mutex_unlock(&fil_system.mutex);

+	fil_crypt_threads_signal();

 	if (must_validate && !srv_read_only_mode) {

 		df_remote.close();

Note: Before this patch, we were reading srv_n_fil_crypt_threads_started without holding fil_crypt_threads_mutex, which is protecting it. This patch just "blindly" calls a function that will acquire that mutex to signal fil_crypt_threads_cond, fixing that data race. There are a few more functions that invoke fil_space_t::create() during early startup, before fil_crypt_threads_init() is invoked at the end of srv_start(). It suffices that fil_crypt_threads_init() invokes fil_crypt_set_thread_cnt(), which will broadcast the condition variable.

There may be some places in the code where we check for tablespaces that have no data files. Thanks to this cleanup, any such checks can be removed as well. I will check this before filing a pull request.