Adding the first InnoDB FULLTEXT index loses matches for 64-bit FTS_DOC_ID values

Adding the first InnoDB FULLTEXT index to a table with user-supplied FTS_DOC_ID values above the 32-bit range leaves the rows intact, but Boolean full-text searches do not find them. I observed this with MariaDB 12.3.3.

This occurs when adding the first InnoDB FULLTEXT index to a table that supplies 64-bit FTS_DOC_ID values above the 32-bit range.

Reproducer

Save the following as case.sql:

CREATE DATABASE fts_test;

USE fts_test;

CREATE TABLE docs (

 id INT PRIMARY KEY,

 FTS_DOC_ID BIGINT UNSIGNED NOT NULL,

 body TEXT

) ENGINE=InnoDB;

INSERT INTO docs VALUES

 (1,4294967297,'marigold'),(2,4294967298,'saffron');

SELECT 'before', COUNT(*), MIN(FTS_DOC_ID), MAX(FTS_DOC_ID) FROM docs;

ALTER TABLE docs ADD FULLTEXT INDEX body_ft (body);

SELECT 'marigold', COALESCE(GROUP_CONCAT(id ORDER BY id),'NONE')

 FROM docs

 WHERE MATCH(body) AGAINST('+marigold' IN BOOLEAN MODE);

SELECT 'saffron', COALESCE(GROUP_CONCAT(id ORDER BY id),'NONE')

 FROM docs

 WHERE MATCH(body) AGAINST('+saffron' IN BOOLEAN MODE);

SELECT 'after', COUNT(*), MIN(FTS_DOC_ID), MAX(FTS_DOC_ID) FROM docs;

DROP DATABASE fts_test;

Run it with the MariaDB client:

mariadb < case.sql

The recorded execution produced these relevant rows (with the incidental example names shown above):

before  2  4294967297  4294967298

marigold  NONE

saffron   NONE

after   2  4294967297  4294967298

The table still contains both rows and their original document IDs after the index is built, but neither token matches its row. marigold should return primary key 1 and saffron should return primary key 2.

A recorded comparison isolates the behavior to the high-ID INPLACE path. With the same tokens, explicit ALGORITHM=COPY and ALGORITHM=INPLACE using small document IDs both returned one match for each token, while ALGORITHM=INPLACE with the large IDs returned zero matches:

inplace_high  2  0  0

copy_high     2  1  1

inplace_low   2  1  1

The source evidence suggests that adding the first FULLTEXT index can obtain a zero maximum document ID because the old table has no document-ID index, then select the compact sort representation and narrow document IDs to 32 bits. This is a supported explanation of the observed distinction rather than an independently proven root cause.