[MDEV-11233] CREATE FULLTEXT INDEX with a token longer than 127 bytes crashes server Created: 2016-11-04 Updated: 2017-01-27 Resolved: 2017-01-27 |
|
| Status: | Closed |
| Project: | MariaDB Server |
| Component/s: | Data Definition - Alter Table, Storage Engine - InnoDB |
| Affects Version/s: | 10.1.17, 10.1.18 |
| Fix Version/s: | 10.0.30, 10.1.20, 10.2.3 |
| Type: | Bug | Priority: | Critical |
| Reporter: | Viktor Zeman | Assignee: | Marko Mäkelä |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | index, innodb | ||
| Environment: |
Centos 6.5, Mariadb 10.1.18 and Mariadb 10.1.17 (and also on all older) |
||
| Issue Links: |
|
||||||||||||||||
| Sprint: | 10.1.20 | ||||||||||||||||
| Description |
|
Adding fulltext index on bigger table crash server
Problem was started by SQL: on table, which has about 200k rows
Server settings:
Exactly the same problem happen on another table when adding fulltext index (table size about 150MB) (reported in additional comments here: https://jira.mariadb.org/browse/MDEV-9129):
|
| Comments |
| Comment by Viktor Zeman [ 2016-11-04 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||
|
After next investigation we think, that size of DB table doesn't play a role here. e.g. we have about 800 databases on server with same structure, but only one DB crashed the server when changing qu_la_contacts table On other hand adding fulltext index on qu_g_mails crash server nearly on every db we have on server. This table holds email sources, which contains special characters nearly in every db. Our workaround was: | |||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Elena Stepanova [ 2016-11-12 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Is the problem reproducible if you reuse the tablespace instead of copying the data?
Does the server crash after that when you are using the new table? Thanks. | |||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Viktor Zeman [ 2016-11-18 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Elena, I don't understand your request with DISCARD TABLESPACE. | |||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Marko Mäkelä [ 2016-12-02 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Someone on irc repeated the same problem. I suggested the following:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Waqar Khan [ 2016-12-02 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||
|
After some testing with Marko, we have an update for this and how to replicate:
Upon CREATE FULLTEXT INDEX ft ON t(t); the server with crash with the following snippet from the log file: Dec 2 13:12:34 centos-mdb1 mysqld: 2016-12-02 13:12:34 7f4c2ebff700 InnoDB: Assertion failure in thread 139965178574592 in | |||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Marko Mäkelä [ 2016-12-02 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Also | |||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Marko Mäkelä [ 2016-12-05 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||
|
This bug is related to the following bug that was fixed in MySQL 5.6.29, 5.7.11 and 8.0.0:
If I revert the change of the latter fix to row_merge_fts_doc_tokenize(), the test case will not crash. Note: I was able to crash by using 3-byte UTF-8:
It seems that the correct fix is to keep the 84-character token length limit even when using utf8mb4. This would increase the storage size in bytes to 336. The MariaDB bug fix has the option to keep using 84*3 = 254 bytes as the maximum word size for those indexes that use encodings with less than 4 bytes per character. This would save 1 byte for fulltext index entries that where the length of an indexed word is between 128 and 254 bytes (43 to 84 three-byte characters, or 64 to 127 two-byte characters). | |||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Jan Lindström (Inactive) [ 2016-12-05 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||
|
ok to push. | |||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Elena Stepanova [ 2016-12-10 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Adding to make it searchable in JIRA | |||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Marko Mäkelä [ 2016-12-12 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||
|
elenst, that failure looks different, possibly a failure to create a thread:
There was no resolved stack trace with line numbers. The only place where ALTER TABLE is creating threads is for preprocessing data for fulltext indexes, the threads fts_parallel_tokenization (created by row_fts_start_psort()) and fts_parallel_merge (created by row_fts_start_parallel_merge()). The manual page for pthread_create() does not give hints why the pthread_create() might have crashed. Given that the platform is 32-bit (i386), one possibility could be exhaustion of the address space when trying to allocate stack for the thread. Maybe this could be repated by running the same tests on a single instance:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Manuel Arostegui [ 2017-01-26 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Quick note: this seems to affect also 10.0.28 and 10.0.29 but not 10.0.23. | |||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Marko Mäkelä [ 2017-01-27 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Reopening for backporting the fix to 10.0. | |||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Manuel Arostegui [ 2017-01-27 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Hey Marko - thanks. |