Summary of relevant failures from innodb_fts suite:

CREATE TABLE t1 (a VARCHAR(200), b TEXT, FULLTEXT (a,b)) ENGINE = InnoDB;
INSERT INTO t1 VALUES('MySQL has now support', 'for full-text search'), ('Full-text indexes', 'are called collections'), ('Only MyISAM tables','support collections'), ('Function MATCH ... AGAINST()','is used to do a search'), ('Full-text search in MySQL', 'implements vector space model');
select * from t1 where MATCH a,b AGAINST ('+collections -supp* -foobar*' IN BOOLEAN MODE);
------------------------------------------+

------------------------------------------+

------------------------------------------+
2 rows in set (0.01 sec)

(the 2nd row shouldn't be there)

Upd: fixed in 10.0-monty (revno 3637) after merge with InnoDB of 5.6.10

====================================================

CREATE TABLE t1 (a VARCHAR(30), FULLTEXT(a)) ENGINE = InnoDB;
INSERT INTO t1 VALUES('bbbbbbbbbbbbbbbbbbbbbbbbbbbbbb');
INSERT INTO t1 VALUES('testword\'\'');
SELECT a FROM t1 WHERE MATCH a AGAINST('testword\'\'' IN BOOLEAN MODE);
Empty set (0.01 sec)

(should be a row)

Upd: still fails in 10.0-monty (revno 3637) after merge with InnoDB of 5.6.10. It also fails in MySQL 5.6.10 (but works in 5.6.12), so the behavior is consistent with MySQL 5.6.10 (GA).

====================================================

SET NAMES utf8;
CREATE TABLE t1(a VARCHAR(255), FULLTEXT(a)) ENGINE = INNODB DEFAULT CHARSET=utf8;
INSERT INTO t1 VALUES('„MySQL“');
SELECT a FROM t1 WHERE MATCH a AGAINST('“MySQL„' IN BOOLEAN MODE);
Empty set (0.01 sec)

(should be a row)

Upd: still fails in 10.0-monty (revno 3637) after merge with InnoDB of 5.6.10. It also fails in MySQL 5.6.10 (but works in 5.6.12).

====================================================

CREATE TABLE articles (
id INT UNSIGNED AUTO_INCREMENT NOT NULL PRIMARY KEY,
title VARCHAR(200),
body TEXT,
FULLTEXT (title,body)
) ENGINE=InnoDB;
INSERT INTO articles (title,body) VALUES
('MySQL Tutorial','DBMS stands for DataBase ...') ,
('How To Use MySQL Well','After you went through a ...'),
('Optimizing MySQL','In this tutorial we will show ...'),
('1001 MySQL Tricks','1. Never run mysqld as root. 2. ...'),
('MySQL vs. YourSQL','In the following database comparison ...'),
('MySQL Security','When configured properly, MySQL ...');
ANALYZE TABLE articles;

SELECT * FROM articles WHERE MATCH (title,body) AGAINST ('mysql - Security&DBMS' IN BOOLEAN MODE);

-----------------------------------------------------------------

-----------------------------------------------------------------

-----------------------------------------------------------------
6 rows in set (0.01 sec)

According to comments in the text, it should work same way as

SELECT * FROM articles WHERE MATCH (title,body) AGAINST ('mysql - (Security DBMS)' IN BOOLEAN MODE);

but it does not:
SELECT * FROM articles WHERE MATCH (title,body) AGAINST ('mysql - (Security DBMS)' IN BOOLEAN MODE);
-----------------------------------------------------------------

-----------------------------------------------------------------

-----------------------------------------------------------------
4 rows in set (0.00 sec)

For a note, there seems to be a bug with the latter query on MyISAM tables

Upd: still fails in 10.0-monty (revno 3637) after merge with InnoDB of 5.6.10. It also fails in MySQL 5.6.10 (but works in 5.6.12).

====================================================

CREATE TABLE articles (
id INT UNSIGNED AUTO_INCREMENT NOT NULL PRIMARY KEY,
title VARCHAR(200),
body TEXT,
FULLTEXT (title,body)
) ENGINE=InnoDB;
INSERT INTO articles (title,body) VALUES
('MySQL Tutorial','DBMS stands for DataBase ...') ,
('How To Use MySQL Well','After you went through a ...'),
('Optimizing MySQL','In this tutorial we will show ...'),
('1001 MySQL Tricks','1. Never run mysqld as root. 2. ...'),
('MySQL vs. YourSQL','In the following database comparison ...'),
('MySQL Security','When configured properly, MySQL ...');
ANALYZE TABLE articles;
SELECT * FROM articles WHERE MATCH (title,body) AGAINST ('"following comparison"@3' IN BOOLEAN MODE);
Empty set (0.01 sec)

(should be a row)
It's just one example, basically proximity doesn't work at all.

Upd: fixed in 10.0-monty (revno 3637) after merge with InnoDB of 5.6.10

==============================================

Assertion failure

130703 21:54:11 InnoDB: Assertion failure in thread 139830781155072 in file fts0fts.cc line 4781
InnoDB: Failing assertion: savepoint->name == NULL

#5 0x00007f2ce8241b8b in __GI_abort () at abort.c:91
#6 0x0000000000d33d3c in fts_trx_free (fts_trx=0x7f2cb0025ad8) at 10.0/storage/innobase/fts/fts0fts.cc:4781
#7 0x0000000000ca8436 in trx_finalize_for_fts (trx=0x7f2cb00f4398, is_commit=0) at 10.0/storage/innobase/trx/trx0trx.cc:970
#8 0x0000000000ca8f4f in trx_commit (trx=0x7f2cb00f4398) at 10.0/storage/innobase/trx/trx0trx.cc:1170
#9 0x0000000000ca1246 in trx_rollback_finish (trx=0x7f2cb00f4398) at 10.0/storage/innobase/trx/trx0roll.cc:1323
#10 0x0000000000c9e94b in trx_rollback_to_savepoint_low (trx=0x7f2cb00f4398, savept=0x0) at 10.0/storage/innobase/trx/trx0roll.cc:117
#11 0x0000000000c9ebd4 in trx_rollback_for_mysql_low (trx=0x7f2cb00f4398) at 10.0/storage/innobase/trx/trx0roll.cc:181
#12 0x0000000000c9ef3d in trx_rollback_for_mysql (trx=0x7f2cb00f4398) at 10.0/storage/innobase/trx/trx0roll.cc:214
#13 0x0000000000bdb556 in innobase_rollback (hton=0x26b5738, thd=0x2c8e8d8, rollback_trx=true) at 10.0/storage/innobase/handler/ha_innodb.cc:3466
#14 0x000000000080609d in ha_rollback_trans (thd=0x2c8e8d8, all=true) at 10.0/sql/handler.cc:1561
#15 0x0000000000750700 in trans_rollback (thd=0x2c8e8d8) at 10.0/sql/transaction.cc:297
#16 0x00000000005e7277 in THD::cleanup (this=0x2c8e8d8) at 10.0/sql/sql_class.cc:1471
#17 0x000000000056af1f in thd_cleanup (thd=0x2c8e8d8) at 10.0/sql/mysqld.cc:2599
#18 0x000000000056b0b9 in unlink_thd (thd=0x2c8e8d8) at 10.0/sql/mysqld.cc:2655
#19 0x000000000056b504 in one_thread_per_connection_end (thd=0x2c8e8d8, put_in_cache=true) at 10.0/sql/mysqld.cc:2783
#20 0x000000000073d8c6 in do_handle_one_connection (thd_arg=0x2c8e8d8) at 10.0/sql/sql_connect.cc:1278
#21 0x000000000073d573 in handle_one_connection (arg=0x2c8e8d8) at 10.0/sql/sql_connect.cc:1181
#22 0x00000000009d32b4 in pfs_spawn_thread (arg=0x2602ca8) at 10.0/storage/perfschema/pfs.cc:1800
#23 0x00007f2ce921de9a in start_thread (arg=0x7f2ce40a1700) at pthread_create.c:308
#24 0x00007f2ce82fbcbd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112

create table `fts_test`(`a` text,fulltext key(`a`))engine=innodb;
set session autocommit=0;
insert into `fts_test` values ('');
savepoint `b`;
savepoint `b`;

Upd: fixed in 10.0-monty (revno 3637) after merge with InnoDB of 5.6.10

==============================================

The test below has a comment that claims

#

1. Test that filesort is not used if ordering on same match expression
2. as where clause
   #

and checks 'Sort%' status variables for that. Ours get populated, so we don't meet the requirement.
Strangely though, EXPLAIN on MySQL still shows filesort.

CREATE TABLE wp(
FTS_DOC_ID BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,
title VARCHAR(255) NOT NULL DEFAULT '',
text MEDIUMTEXT NOT NULL,
dummy INTEGER,
PRIMARY KEY (FTS_DOC_ID),
UNIQUE KEY FTS_DOC_ID_INDEX (FTS_DOC_ID),
FULLTEXT KEY idx (title,text)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

INSERT INTO wp (title, text) VALUES
('MySQL Tutorial','DBMS stands for MySQL DataBase ...'),
('How To Use MySQL Well','After you went through a ...'),
('Optimizing MySQL','In this tutorial we will show ...'),
('1001 MySQL Tricks','1. Never run mysqld as root. 2. ...'),
('MySQL vs. YourSQL','In the following database to database comparison ...'),
('MySQL Security','When configured properly, MySQL ...');

FLUSH STATUS;

SELECT title, MATCH(title, text) AGAINST ('database') AS score
FROM wp
WHERE MATCH(title, text) AGAINST ('database')
ORDER BY score DESC;

SHOW SESSION STATUS LIKE 'Sort%';
------------------------+

------------------------+

------------------------+
4 rows in set (0.00 sec)

There are more tests like that.

Upd: still fails in 10.0-monty (revno 3637) after merge with InnoDB of 5.6.10, although it works in MySQL 5.6.10, so if it's a bug, it might be MariaDB-only.

==============================================

CREATE TABLE user_stopword(value varchar(30)) ENGINE = InnoDB
DEFAULT CHARACTER SET latin1 COLLATE latin1_general_ci;
INSERT INTO user_stopword VALUES('lòve');
SET GLOBAL innodb_ft_server_stopword_table = 'test/user_stopword';

MySQL allows to set it, MariaDB says Variable 'innodb_ft_server_stopword_table' can't be set to the value of 'test/user_stopword'.
It should be possible, according to docs.

Upd: still fails in 10.0-monty (revno 3637) after merge with InnoDB of 5.6.10. It also fails in MySQL 5.6.10 (but works in 5.6.12).

==============================================

Minor issues: wrong error codes

Assorted assertion failures in concurrent tests:

130704 1:01:08 InnoDB: Assertion failure in thread 139896084535040 in file fts0fts.cc line 784
InnoDB: Failing assertion: ib_vector_size(table->fts->indexes) == ib_vector_size(table->fts->cache->indexes)

#5 0x00007f3c2fd6eb8b in __GI_abort () at abort.c:91
#6 0x0000000000d2d41c in fts_check_cached_index (table=0x7f3bf82bc368) at /10.0/storage/innobase/fts/fts0fts.cc:783
#7 0x0000000000bf2cd7 in ha_innobase::final_add_index (this=0x7f3bd80406c0, add_arg=0x7f3bf81d0570, commit=false) at /10.0/storage/innobase/handler/handler0alter.cc:1564
#8 0x00000000006d8143 in mysql_alter_table (thd=0x2cb6b18, new_db=0x7f3bf8030510 "test", new_name=0x7f3bf854a5e0 "table300_innodb_int_autoinc", create_info=0x7f3c186c7250, table_list=0x7f3bf80d3330, alter_info=0x7f3c186c7360, order_num=0, order=0x0, ignore=false, require_online=false) at /10.0/sql/sql_table.cc:7352
#9 0x0000000000989892 in Alter_table_statement::execute (this=0x7f3bf8101690, thd=0x2cb6b18) at /10.0/sql/sql_alter.cc:106
#10 0x000000000062a456 in mysql_execute_command (thd=0x2cb6b18) at /10.0/sql/sql_parse.cc:4873
#11 0x000000000062d680 in mysql_parse (thd=0x2cb6b18, rawbuf=0x7f3bf81219e0 "ALTER TABLE `table300_innodb_int_autoinc` ADD FULLTEXT INDEX idx_2 (`col_varchar_355_utf8_fulltext_key_not_null`)", length=113, parser_state=0x7f3c186c8500) at /10.0/sql/sql_parse.cc:6176
#12 0x0000000000620556 in dispatch_command (command=COM_QUERY, thd=0x2cb6b18, packet=0x2c729b9 "ALTER TABLE `table300_innodb_int_autoinc` ADD FULLTEXT INDEX idx_2 (`col_varchar_355_utf8_fulltext_key_not_null`)", packet_length=113) at /10.0/sql/sql_parse.cc:1274
#13 0x000000000061f9cc in do_command (thd=0x2cb6b18) at /10.0/sql/sql_parse.cc:983
#14 0x000000000073d820 in do_handle_one_connection (thd_arg=0x2cb6b18) at /10.0/sql/sql_connect.cc:1267
#15 0x000000000073d573 in handle_one_connection (arg=0x2cb6b18) at /10.0/sql/sql_connect.cc:1181
#16 0x00000000009d32b4 in pfs_spawn_thread (arg=0x2c79198) at /10.0/storage/perfschema/pfs.cc:1800
#17 0x00007f3c30d4ae9a in start_thread (arg=0x7f3c186c9700) at pthread_create.c:308

130704 0:34:40 InnoDB: Assertion failure in thread 139652942001920 in file handler0alter.cc line 1901
InnoDB: Failing assertion: fts_check_cached_index(prebuilt->table)

#5 0x00007f03ddf37b8b in __GI_abort () at abort.c:91
#6 0x0000000000bf38a1 in ha_innobase::final_drop_index (this=0x7f039409e840, iin_table=0x7f0394313978) at /10.0/storage/innobase/handler/handler0alter.cc:1901
#7 0x00000000006d7bf2 in mysql_alter_table (thd=0x480f4b8, new_db=0x7f0370066700 "test", new_name=0x7f0370077280 "table200_innodb_int_autoinc", create_info=0x7f037bffd2a0, table_list=0x7f037009c5d0, alter_info=0x7f037bffd3b0, order_num=0, order=0x0, ignore=false, require_online=false) at /10.0/sql/sql_table.cc:7227
#8 0x0000000000989892 in Alter_table_statement::execute (this=0x7f0370000ab0, thd=0x480f4b8) at /10.0/sql/sql_alter.cc:106
#9 0x000000000062a456 in mysql_execute_command (thd=0x480f4b8) at /10.0/sql/sql_parse.cc:4873
#10 0x000000000062d680 in mysql_parse (thd=0x480f4b8, rawbuf=0x7f0370059040 "ALTER TABLE `table200_innodb_int_autoinc` DROP INDEX idx_5", length=58, parser_state=0x7f037bffe550) at /10.0/sql/sql_parse.cc:6176
#11 0x0000000000620556 in dispatch_command (command=COM_QUERY, thd=0x480f4b8, packet=0x4815ab9 "ALTER TABLE `table200_innodb_int_autoinc` DROP INDEX idx_5", packet_length=58) at /10.0/sql/sql_parse.cc:1274
#12 0x000000000061f9cc in do_command (thd=0x480f4b8) at /10.0/sql/sql_parse.cc:983
#13 0x000000000073d820 in do_handle_one_connection (thd_arg=0x480f4b8) at /10.0/sql/sql_connect.cc:1267
#14 0x000000000073d573 in handle_one_connection (arg=0x480f4b8) at /10.0/sql/sql_connect.cc:1181
#15 0x00007f03def13e9a in start_thread (arg=0x7f037bfff700) at pthread_create.c:308
#16 0x00007f03ddff1cbd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112

WITH PARSER is not supported for InnoDB tables (neither in MariaDB nor in current MySQL), but MySQL produces the corresponding "not supported" error, while MariaDB pretends it uses the custom parser while in fact it doesn't.

Upd: It is still so in 10.0-monty (revno 3637) after merge with InnoDB of 5.6.10. And it works the same way with MySQL 5.6.10 (while MySQL 5.6.12 produces a proper error).

The following crashes.
The fix in MySQL 5.6 seems to be on InnoDB side only
revision-id: marko.makela@oracle.com-20121026125150-i7ag7ovgtdver04u

--source include/have_innodb.inc

create table t1 (a text) engine=InnoDB;
lock table t1 read;

--connect (con1,localhost,root,,)
set lock_wait_timeout=1;
--error ER_LOCK_WAIT_TIMEOUT
alter table t1 add fulltext (a);

--disconnect con1
--connection default
unlock tables;
drop table t1;

Upd: fixed in 10.0-monty (revno 3637) after merge with InnoDB of 5.6.10

Conclusion:

The behavior looks consistent with the state of InnoDB as of 5.6.5 which 10.0.3 includes. Fulltext indexes/search work to the same extent as they did in MySQL 5.6.5. There are numerous bugs and limitations that are reproducible on MySQL 5.6.5 (and not reproducible on MySQL 5.6.12). A superficial look at the code changes also confirms that related changes (initial WL and bugfixes) were on the InnoDB side.

I suggest to keep the functionality enabled in 10.0.4 if the new InnoDB is merged by then. If not, disable the functionality until it's merged (if it's easily disabled, otherwise keep it but document as pre-alpha quality).

After the InnoDB merge is done, I will re-run some basic tests before the release, and will later need to run concurrent tests targeted for ALTER ONLINE on tables with fulltext indexes (as the history shows, there was a special effort for ALTER .. LOCK=NONE and ALGORITHM=INPLACE);

Status after the merge with MySQL 5.6.10, as of 10.0-monty revno 3637
--------------------------------------------
the previous comments have been updated to indicate whether the problem is gone after the merge or still exists.
The behavior is generally consistent with MySQL 5.6.10, except for two issues that are now filed separately: MDEV-4801 (it newly appeared after the merge), MDEV-4802 (previously mentioned problem with filesort, still exists in MariaDB but not observed in MySQL 5.6.10).

I think that MDEV-4801 needs to be fixed before 10.0.4 release, and then InnoDB FT can stay in the release. Since it works pretty much as MySQL 5.6.10 which was GA, it should be good enough for MariaDB 10.0.4 which is still alpha.
MDEV-4802 is a minor issue which can be fixed later.

I will check once again after merge into the main 10.0 tree.

Not noted before, "Cannot find index FTS_DOC_ID_INDEX in InnoDB index translation table" errors are observed on the tree after the merge and on MySQL 5.6.10, but not on MySQL 5.6.12.

We still need to merge the innodb_fts suite into 10.0.

All trivial fixes made, so we have failing:
fulltext_misc
fulltext_order_by
innobase_drop_fts_index_table
innodb-fts-stopword
innodb_fts_misc
innodb_fts_opt

following tests should be fixed:
innobase_drop_fts_index_table
innodb-fts-stopword
innodb_fts_misc
innodb_fts_opt

innodb-fts-stopword (crash)
innodb_fts_opt

– source include/have_innodb.inc

select @@innodb_ft_user_stopword_table;
create table user_stopword(value varchar(30)) engine = innodb;
set session innodb_ft_user_stopword_table = "test/user_stopword";
select @@innodb_ft_user_stopword_table;
SET SESSION innodb_ft_user_stopword_table = default;
select @@innodb_ft_user_stopword_table;
drop table user_stopword;

innodb_fts_opt which tests optimization now the only failing test probably because the optimization do not work somewhere...

fixed fts optimization with limit and rolling back optimization with limit. still have some differences in the fts optimization applied...

pushed (without optimization and optimization test (should be more general solution)).