We have a cluster of 3x mariadb/galera nodes handling data set of about 1 million tables and use mariabackup mechanism for state transferring.
Application using this dataset usually requires less then 300K tables open, thus no reason to set this value higher.
mariadb/mysqld has autodetection and autoset for open_files_limit variable that is calculated as
open_files_limit = MAX(max_connections*5, max_connections + table_open_cache*2)
system number of open files soft limit is set by mysqld and mariabackup according to this mentioned value.
As it was mentioned, table_open_cache can never get higher then 300K, max_connections never gets higher then 2000. For normal mysql performance this autodetection is good enough.
mariabackup opens all tables from all databases while replaying transaction log during SST. In same time, mariabackup that is called from running mysqld process inherits mysqld open_files_limit settings that is from above mentioned equation is far less then number of tables in dataset, thus SST fails as limit of open files is far less then really required.
Moreover, from what we found, if open_files_limit is set in both [mysqld] and [mariabackup] configuration file sections, value from [mysqld] section is used for SST. From other side, setting high values of open files limit slows down mysqld operations, and dramatically slows down operation on cluster.
It would be reasonable for mysqld to keep current autodetection mechanism and keep open_files_limit as low as possible for current connection table and cache configuration.
Hoewever, for state transfer (and thus for mariabackup) this autodetection should not be inherited and mariabackup should instead autodetect and set open files limit based on number of tables in dataset, not table cache size.
This affects only galera clusters when mariadackup is called from mysqld and inherits it's file limits that in fact should not happen.