SET GLOBAL cannot be atomic when inter-parameter constraints exist

According to serg, the SET statement should be atomic: if one of the parameters is deemed invalid, then none of the assignments will be performed. However, there exist some dependencies or constraints between parameters (see also MDEV-19895), and some parameters must be updated together or validated with respect to each other:

• innodb_log_file_disabled (MDEV-36301)
• innodb_log_file_size (MDEV-27812)
• innodb_log_group_home_dir (MDEV-36301)
  • could be changed within the same file system
  • could be changed if innodb_log_file_disabled was or will be set to ON

Changing the above parameters is special, because the operation involves writing out enough data pages so that the log checkpoint can be advanced sufficiently so that on checkpoint completion, the requested parameters will take over. If the operation is aborted before that, the parameters must be rolled back.

A change to the storage engine API seems to be necessary so that inter-parameter constraints can be validated and updated atomically together. In MDEV-36301, a proposal was made to pass some context via thread local storage. This has the following problems:

• Only one validation check per parameter is executed; there is no common validation step. If there was a constraint a<b and we initially had a=3, SET a=1,b=2 could be accepted but SET b=2,a=1 would be rejected.
• SET GLOBAL ...=DEFAULT will skip any validate function, so the suggested hack would completely ignore such assignments until MDEV-36770 is fixed.
• We are missing an error message saying that parameters X and Y cannot be updated together.
  • It does not seem feasible to allow concurrent updates of any parameters where LOCK_global_system_variables needs to be momentarily released.
  • Shrinking innodb_buffer_pool_size (MDEV-29445) may fail if the buffer pool is occupied by record locks, for example due to MDEV-24813. (Increasing innodb_buffer_pool_size should be fine.)
  • SET GLOBAL innodb_max_purge_lag_wait=0 (MDEV-16952) may block indefinitely if blocking read views or XA PREPARE transactions exist.
  • These parameters had better not specified together, nor together with any of the innodb_log_ parameters.
• The 'update' functions will be invoked one by one. If multiple parameters can be updated in one call, that call must make sure that subsequent (unnecessary) calls to update the remaining parameters will have no effect.
• The 'update' functions API does not allow any errors to be returned. For example, SET GLOBAL innodb_buffer_pool_size can fail if InnoDB is running out of buffer pool.
• If no 'validate' callback has been specified, integer parameters will silently be clamped to be between their minimum and maximum values and aligned to the block size.
• There is no clear interface for updates that can take several seconds or minutes to complete.

Currently, the implementation of the following non-debug SET GLOBAL innodb_ parameters includes a hack that will momentarily release LOCK_global_system_variables and wait until the request can be completed, or until a KILL or SHUTDOWN statement interrupts the operation:

• innodb_buffer_pool_size (MDEV-29445): will assign the value at the end, unless another SET GLOBAL innodb_buffer_pool_size was completed first
• innodb_log_checkpoint_now (MDEV-30000): not assigning the value anywhere
• innodb_log_file_size (MDEV-27812), innodb_log_file_disabled (MDEV-36301), innodb_log_group_home_dir (MDEV-36301): will assign the value at the end; a subsequent SET GLOBAL will be rejected if an operation is in progress
• innodb_max_purge_lag_wait (MDEV-16952): not assigning the value anywhere, but using it as a wait condition

The following non-debug SET GLOBAL innodb_ parameters are momentarily releasing LOCK_global_system_variables possibly for no valid reason (the original reason could have been the avoidance of potential lock order inversion), and we should consider retaining the mutex throughout the operation:

• innodb_adaptive_hash_index
• innodb_buffer_pool_dump_now
• innodb_buffer_pool_load_abort
• innodb_buffer_pool_load_now
• innodb_change_buffer_max_size
• innodb_cmp_per_index
• innodb_encrypt_tables
• innodb_encryption_rotate_key_age
• innodb_encryption_rotation_iops
• innodb_encryption_threads
• innodb_old_blocks_pct
• innodb_status_output