[MDEV-12676] InnoDB FTS duplicate key error Created: 2017-05-04 Updated: 2020-08-25 Resolved: 2017-10-16 |
|
| Status: | Closed |
| Project: | MariaDB Server |
| Component/s: | Full-text Search, Storage Engine - InnoDB, Storage Engine - XtraDB |
| Affects Version/s: | 10.0.29 |
| Fix Version/s: | 10.0.33, 10.1.29, 10.2.10, 10.3.3 |
| Type: | Bug | Priority: | Major |
| Reporter: | Will Fong | Assignee: | Marko Mäkelä |
| Resolution: | Fixed | Votes: | 2 |
| Labels: | upstream | ||
| Issue Links: |
|
||||||||||||||||
| Description |
|
We have a customer continuously getting this in their error log:
I haven't been able to reproduce this issue on my side. But looking through the source code, it doesn't seem like this type of error should be reported in the error log. Perhaps it could be better handled internally (silently ignored?). This is a problem because it uselessly fills up the error log. Thanks, 2017-08-10 - A careful code review would be needed to answer the question if this has any user-visible impact, or why this occurs in the first place. |
| Comments |
| Comment by Marko Mäkelä [ 2017-05-09 ] | ||||||||
|
As demonstrated in | ||||||||
| Comment by Will Fong [ 2017-05-09 ] | ||||||||
|
marko Thanks for the update. Can I get a confirmation that this doesn't affect query results / accuracy? It is just "error log noise"? Thanks! | ||||||||
| Comment by Marko Mäkelä [ 2017-08-10 ] | ||||||||
|
After analyzing both | ||||||||
| Comment by Dimitris Chatzis [ 2017-11-18 ] | ||||||||
|
Not fixed on latest 10.1.29 I am getting : InnoDB: Error (Duplicate key) writing word node to FTS auxiliary index table. when i use: journalctl -u mariadb --no-pager | ||||||||
| Comment by Marko Mäkelä [ 2017-11-20 ] | ||||||||
|
pamamolf, the fix of this particular error definitely is included in MariaDB 10.1.29. When you report an error, please include a minimal test case, preferrably based on SQL. If I understand correctly, the command "journalctl" is querying the logs via systemd. So, apparently you are seeing a message in the MariaDB error log. The message would seem to be generated by fts_sync_write_words():
This looks like a different duplicate key error. The one that was fixed was a race condition when assigning values for the hidden FTS_DOC_ID column in the user table. That is, the duplicate key error was reported for the hidden FTS_DOC_ID_INDEX(FTS_DOC_ID) in the user table. This reported duplicate key error is something different, in an "auxiliary table" that stores a part of the inverted index from tokenized words to sets of FTS_DOC_ID. The error would seem to be that instead of updating an existing word entry by adding more FTS_DOC_ID to the set, the logic is trying to insert a duplicate entry for the same tokenized word. Please file a separate issue for it, with a reduced SQL test case, so that we can repeat and fix it. I have a wild guess (which may be wrong): somewhere the code is using wrong collation when comparing words, and then instead of updating an existing word, it is trying to insert a duplicate one. Example: both the words 'bar' and 'bär' exist in the indexed column(s), and the collation threats these two words as equivalent, but some code inside InnoDB FTS is wrongly treating them as distinct words. | ||||||||
| Comment by Dimitris Chatzis [ 2017-11-20 ] | ||||||||
|
I am not expert at all and don't know how to post a test case But it sounds that the issue probably is what you guess: "somewhere the code is using wrong collation when comparing words, and then instead of updating an existing word, it is trying to insert a duplicate one. Example: both the words 'bar' and 'bär' exist in the indexed column(s), and the collation threats these two words as equivalent, but some code inside InnoDB FTS is wrongly treating them as distinct words." | ||||||||
| Comment by Massimo [ 2018-06-01 ] | ||||||||
|
Not fixed on latest 10.1.30-MariaDB-1~xenial after many errors report as " InnoDB: Error (Duplicate key) writing word node to FTS auxiliary index table." and not table mention, we fix the issue in rebuilds the tables with fulltext on innodb tables based on STATISTIC tables on information schema. after that the error did not show up anymore. | ||||||||
| Comment by Marko Mäkelä [ 2018-06-04 ] | ||||||||
|
For the record, I closed this bug on October 16, 2017 because I had applied a fix (with a test case) for the duplicate key error for the hidden unique FTS_DOC_ID column. The remaining duplicate key error was reported as |