long uniques don't work with unicode collations

set names utf8;

create table t1 (a text collate utf8_general_ci unique);

show create table t1;

insert t1 values ('a');

insert t1 values ('ä');

select * from t1;

select distinct * from t1;

select 'a' = 'ä';

drop table t1;

shows that one can insert two characters into a long unique index that compare equally.

It happens because calc_hash_for_unique() hashes string length in bytes together with the string content. So even if two utf8 strings hash to the same value, they can have different length in bytes and long unique hash will be different.

The effect of this bug is that if one has a table with
CREATE TABLE
...
xxx VARCHAR(255) COLLATE 'utf8mb3_general_ci',
...
UNIQUE INDEX `xxx` (column,...) USING HASH...

Then the following problems may occur:

• The table will accept duplicates for UTF8 strings that are different but should compare equal.
• If master and slave are of different versions where the bug is fixed on one of them or the hash value is different (see MDEV-32093) then the following can also happen:
• Insert works on master (which does not notice the duplicate key) but fails on slave (which notices the duplicate key)

Fix is to drop and add back the unique index or run ALTER TABLE xxx ENGINE=InnoDB (for InnoDB tables) to reconstruct the hash index.