Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-41759

MyISAM and Aria: an excluded phrase in boolean mode excludes all rows that contain its words, in any order

    XMLWordPrintable

Details

    • Unexpected results
    • A boolean mode fulltext search with an excluded phrase (-"...") on a MyISAM or Aria table no longer excludes rows that contain the words of the phrase in another order or not next to each other.

    Description

      CREATE TABLE t (id INT PRIMARY KEY, c TEXT, FULLTEXT(c)) ENGINE=MyISAM;
      INSERT INTO t VALUES (1, 'xxxx alpha beta'), (2, 'xxxx beta alpha'),
        (3, 'xxxx alpha gamma beta'), (4, 'xxxx');
      SELECT id FROM t WHERE MATCH(c) AGAINST('+xxxx -"alpha beta"' IN BOOLEAN MODE);
      

      Expected: 2, 3, 4 (InnoDB returns these). Actual with MyISAM and Aria on 11.4 and main: 4. Rows 2 and 3 contain both words, but not the phrase "alpha beta". '+xxxx +"alpha beta"' correctly returns only row 1.

      Only the search through the fulltext index is affected: with IGNORE INDEX (c), MyISAM and Aria return 2, 3, 4, and MATCH() in the select list is > 0 for rows 2, 3 and 4.

      Cause: during the index search, _ftb_climb_the_tree() (storage/myisam/ft_boolean_search.c, the same in storage/maria/ma_ft_boolean_search.c) takes a phrase as matched as soon as all its words are in the row. It checks the phrase itself only when called from ft_boolean_find_relevance() (if (mode && ftbe->phrase)). For +"..." this is fine, because ft_boolean_read_next() rechecks every found row with ft_boolean_find_relevance(). For -"..." the row has already been rejected before that recheck.

      MySQL has the same code, and the bug is reported there as https://bugs.mysql.com/bug.php?id=84036 (earlier https://bugs.mysql.com/bug.php?id=31755). The test cases there fail on MariaDB, too.

      We ran into this with the ngram parser proposed in MDEV-10267, which turns each word of a boolean mode query into a phrase of its n-grams: '-abc' means -"ab bc", so it also excludes a row with 'bcab', which contains both n-grams but not abc.

      Attachments

        Issue Links

          Activity

            People

              sanja Oleksandr Byelkin
              Futureweb Andreas Schnederle-Wagner
              Oleksandr Byelkin Oleksandr Byelkin
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.