Data corruption after DROP COLUMN on InnoDB tables with char primary key

When an InnoDB table has a char field with a UTF8 collation which is set as primary key, dropping a column seems to cause data corruption.

The problem can be reproduced on a fresh database by executing the following queries:

CREATE TABLE example (

  id char(8) COLLATE utf8mb4_bin NOT NULL,

  extra_column int NOT NULL,

  PRIMARY KEY (id)

) ENGINE=InnoDB;

ALTER TABLE example DROP COLUMN extra_column;

The altered table keeps working fine until the MariaDB server is restarted. But after restart, attempts to access the table result in the following error:

#1932 - Table 'test.example' doesn't exist in engine

mysql_error.log excerpt:

2019-12-12 1:19:55 25 [ERROR] InnoDB: Table `test`.`example` contains unrecognizable instant ALTER metadata

The problem seems to occur only if the data type of the primary key column is char and only for certain collations (utf8mb4_bin, utf8_bin, utf8_unicode_ci, ...), but for any lengths. The data type of the column that is dropped does not seem to matter.

I presume this bug is related to the alter algorithm: It occurs with DEFAULT, INPLACE, NOCOPY and INSTANT, but not with COPY, i.e., the problem can be avoided by executing

SET SESSION alter_algorithm='COPY';

before running the ALTER query. If the table has already been altered but the MariaDB server has not yet been restarted, recreating the table via

OPTIMIZE TABLE example;

seems to prevent the mentioned data corruption.