Details

    Description

      We have a customer continuously getting this in their error log:

      InnoDB: Error (Duplicate key) writing word node to FTS auxiliary index table.
      

      I haven't been able to reproduce this issue on my side. But looking through the source code, it doesn't seem like this type of error should be reported in the error log. Perhaps it could be better handled internally (silently ignored?).

      This is a problem because it uselessly fills up the error log.

      Thanks,
      -will

      2017-08-10 - A careful code review would be needed to answer the question if this has any user-visible impact, or why this occurs in the first place.

      Attachments

        Issue Links

          Activity

            As demonstrated in MDEV-11649 and MDEV-12636, it is possible that FTS_DOC_ID sometimes is uninitialized garbage, causing this error.

            marko Marko Mäkelä added a comment - As demonstrated in MDEV-11649 and MDEV-12636 , it is possible that FTS_DOC_ID sometimes is uninitialized garbage, causing this error.
            wfong Will Fong added a comment -

            marko Thanks for the update. Can I get a confirmation that this doesn't affect query results / accuracy? It is just "error log noise"? Thanks!

            wfong Will Fong added a comment - marko Thanks for the update. Can I get a confirmation that this doesn't affect query results / accuracy? It is just "error log noise"? Thanks!

            After analyzing both MDEV-11649 and MDEV-12636, I think that this bug is unrelated to them.
            A careful code review would be needed to answer the question if this has any user-visible impact, or why this occurs in the first place.
            A repeatable test case would greatly help.

            marko Marko Mäkelä added a comment - After analyzing both MDEV-11649 and MDEV-12636 , I think that this bug is unrelated to them. A careful code review would be needed to answer the question if this has any user-visible impact, or why this occurs in the first place. A repeatable test case would greatly help.

            Not fixed on latest 10.1.29

            I am getting :

            InnoDB: Error (Duplicate key) writing word node to FTS auxiliary index table.

            when i use:

            journalctl -u mariadb --no-pager

            pamamolf Dimitris Chatzis added a comment - Not fixed on latest 10.1.29 I am getting : InnoDB: Error (Duplicate key) writing word node to FTS auxiliary index table. when i use: journalctl -u mariadb --no-pager

            pamamolf, the fix of this particular error definitely is included in MariaDB 10.1.29.

            When you report an error, please include a minimal test case, preferrably based on SQL. If I understand correctly, the command "journalctl" is querying the logs via systemd. So, apparently you are seeing a message in the MariaDB error log. The message would seem to be generated by fts_sync_write_words():

            		if (error != DB_SUCCESS && !print_error) {
            			ut_print_timestamp(stderr);
            			fprintf(stderr, "  InnoDB: Error (%s) writing "
            				"word node to FTS auxiliary index "
            				"table.\n", ut_strerr(error));
             
            			print_error = TRUE;
            		}
            

            This looks like a different duplicate key error. The one that was fixed was a race condition when assigning values for the hidden FTS_DOC_ID column in the user table. That is, the duplicate key error was reported for the hidden FTS_DOC_ID_INDEX(FTS_DOC_ID) in the user table. This reported duplicate key error is something different, in an "auxiliary table" that stores a part of the inverted index from tokenized words to sets of FTS_DOC_ID.

            The error would seem to be that instead of updating an existing word entry by adding more FTS_DOC_ID to the set, the logic is trying to insert a duplicate entry for the same tokenized word. Please file a separate issue for it, with a reduced SQL test case, so that we can repeat and fix it.

            I have a wild guess (which may be wrong): somewhere the code is using wrong collation when comparing words, and then instead of updating an existing word, it is trying to insert a duplicate one. Example: both the words 'bar' and 'bär' exist in the indexed column(s), and the collation threats these two words as equivalent, but some code inside InnoDB FTS is wrongly treating them as distinct words.

            marko Marko Mäkelä added a comment - pamamolf , the fix of this particular error definitely is included in MariaDB 10.1.29. When you report an error, please include a minimal test case, preferrably based on SQL. If I understand correctly, the command "journalctl" is querying the logs via systemd. So, apparently you are seeing a message in the MariaDB error log. The message would seem to be generated by fts_sync_write_words(): if (error != DB_SUCCESS && !print_error) { ut_print_timestamp(stderr); fprintf (stderr, " InnoDB: Error (%s) writing " "word node to FTS auxiliary index " "table.\n" , ut_strerr(error));   print_error = TRUE; } This looks like a different duplicate key error. The one that was fixed was a race condition when assigning values for the hidden FTS_DOC_ID column in the user table. That is, the duplicate key error was reported for the hidden FTS_DOC_ID_INDEX(FTS_DOC_ID) in the user table. This reported duplicate key error is something different, in an "auxiliary table" that stores a part of the inverted index from tokenized words to sets of FTS_DOC_ID. The error would seem to be that instead of updating an existing word entry by adding more FTS_DOC_ID to the set, the logic is trying to insert a duplicate entry for the same tokenized word. Please file a separate issue for it, with a reduced SQL test case, so that we can repeat and fix it. I have a wild guess (which may be wrong): somewhere the code is using wrong collation when comparing words, and then instead of updating an existing word, it is trying to insert a duplicate one. Example: both the words 'bar' and 'bär' exist in the indexed column(s), and the collation threats these two words as equivalent, but some code inside InnoDB FTS is wrongly treating them as distinct words.

            I am not expert at all and don't know how to post a test case Sorry.....

            But it sounds that the issue probably is what you guess:

            "somewhere the code is using wrong collation when comparing words, and then instead of updating an existing word, it is trying to insert a duplicate one. Example: both the words 'bar' and 'bär' exist in the indexed column(s), and the collation threats these two words as equivalent, but some code inside InnoDB FTS is wrongly treating them as distinct words."

            pamamolf Dimitris Chatzis added a comment - I am not expert at all and don't know how to post a test case Sorry..... But it sounds that the issue probably is what you guess: " somewhere the code is using wrong collation when comparing words, and then instead of updating an existing word, it is trying to insert a duplicate one. Example: both the words 'bar' and 'bär' exist in the indexed column(s), and the collation threats these two words as equivalent, but some code inside InnoDB FTS is wrongly treating them as distinct words. "
            massimo.disaro Massimo added a comment -

            Not fixed on latest 10.1.30-MariaDB-1~xenial

            after many errors report as " InnoDB: Error (Duplicate key) writing word node to FTS auxiliary index table." and not table mention, we fix the issue in rebuilds the tables with fulltext on innodb tables based on STATISTIC tables on information schema.

            after that the error did not show up anymore.

            massimo.disaro Massimo added a comment - Not fixed on latest 10.1.30-MariaDB-1~xenial after many errors report as " InnoDB: Error (Duplicate key) writing word node to FTS auxiliary index table." and not table mention, we fix the issue in rebuilds the tables with fulltext on innodb tables based on STATISTIC tables on information schema. after that the error did not show up anymore.

            For the record, I closed this bug on October 16, 2017 because I had applied a fix (with a test case) for the duplicate key error for the hidden unique FTS_DOC_ID column.

            The remaining duplicate key error was reported as MDEV-15237. If you have any information that could help reproduce or fix the problem, please post it there, not here.

            marko Marko Mäkelä added a comment - For the record, I closed this bug on October 16, 2017 because I had applied a fix (with a test case) for the duplicate key error for the hidden unique FTS_DOC_ID  column . The remaining duplicate key error was reported as MDEV-15237 . If you have any information that could help reproduce or fix the problem, please post it there, not here.

            People

              marko Marko Mäkelä
              wfong Will Fong
              Votes:
              2 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.