PURGE/FLUSH/SHOW BINARY LOGS hang while InnoDB binlog preallocation waits for disk space

With binlog_storage_engine=innodb, when the disk that holds the
binlog fills up, SHOW BINARY LOGS does not return until space is
freed outside the server. PURGE BINARY LOGS and
FLUSH BINARY LOGS wait on the same mutex, so the old binlog files
cannot be purged from SQL to free that space. The test case below
covers only SHOW BINARY LOGS.

The InnoDB binlog creates the next binlog file ahead of time in the
background thread. fsp_binlog_tablespace_create() creates the file
and extends it to its full size. When the extension fails, for example
with ENOSPC, the function deletes the partial file, sleeps 60 seconds
and tries again, with no limit on the number of tries.

The preallocation thread holds purge_binlog_mutex across the whole
call to fsp_binlog_tablespace_create(). The same mutex is taken by
(1) SHOW BINARY LOGS, (2) PURGE BINARY LOGS, and (3)
FLUSH BINARY LOGS. Each of these statements waits until a retry
succeeds.

Is this mutex needed while waiting for space to create & pre-allocate
the binlog? Can we limit the time holding the mutex to only when
updating the internal state to account for the newly allocated binlog?

To reproduce:

--source include/have_debug.inc

--source include/have_innodb_binlog.inc

call mtr.add_suppression('InnoDB: Unable to allocate file.* "No space left on device"');

--echo #

--echo # Simulate a full disk and make the preallocation thread start on

--echo # the next binlog file

--echo #

set @old_dbug= @@global.debug_dbug;

set @@global.debug_dbug= '+d,ib_alloc_file_disk_full';

FLUSH BINARY LOGS;

--echo # Waiting for the preallocation to fail with ENOSPC

--let ERRLOG= $MYSQLTEST_VARDIR/log/mysqld.1.err

perl;

  for (1..300)

  {

    open(F, '<', $ENV{ERRLOG}) or die "Cannot open $ENV{ERRLOG}: $!";

    my $n= grep(/InnoDB: Unable to allocate file/, <F>);

    close(F);

    exit(0) if $n > 0;

    select(undef, undef, undef, 0.1);

  }

  die "Timeout waiting for the binlog preallocation to fail";

EOF

--echo #

--echo # SHOW BINARY LOGS while the preallocation is retrying

--echo #

--connect (con1,localhost,root,,)

--send SHOW BINARY LOGS

--echo # Sleep 3s and check if SHOW BINARY LOGS is still running

--connection default

--sleep 3

select count(*) from information_schema.processlist where info = 'SHOW BINARY LOGS';

--connection con1

--reap