[MDEV-21040] InnoDB: fulltext search with ` in(...)` crashed on specific combination Created: 2019-11-12  Updated: 2020-04-01  Resolved: 2019-11-27

Status: Closed
Project: MariaDB Server
Component/s: Full-text Search, Storage Engine - InnoDB
Affects Version/s: 10.4.3, 10.4.10
Fix Version/s: 10.4.11

Type: Bug Priority: Major
Reporter: Moshe L Assignee: Igor Babaev
Resolution: Duplicate Votes: 0
Labels: crash, fulltext, regression
Environment:

Windows 10, x64

Issue Links:
Duplicate
duplicates MDEV-20056 Assertion `!prebuilt->index->is_prima... Closed
duplicates MDEV-20407 mysqld got signal 11; rowid filter Closed
is duplicated by MDEV-21099 MariaDB 10.4.10 crash Closed
Problem/Incident
is caused by MDEV-16188 Use in-memory PK filters built from r... Closed
Relates
relates to MDEV-20407 mysqld got signal 11; rowid filter Closed
relates to MDEV-21047 Crash on UTF-8 Full text search Closed
relates to MDEV-21125 Crash with FTS query Closed

 Description   

From the log :

191112 21:24:03 [ERROR] mysqld got exception 0xc0000005 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.

To report this bug, see https://mariadb.com/kb/en/reporting-bugs

We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.

Server version: 10.4.10-MariaDB
key_buffer_size=134217728
read_buffer_size=131072
max_used_connections=1
max_threads=65537
thread_count=7
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 136385 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x1c938d5b018
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
mysqld.exe!row_sel_field_store_in_mysql_format_func()[row0sel.cc:2864]
mysqld.exe!row_sel_store_mysql_field()[row0sel.cc:3011]
mysqld.exe!row_search_idx_cond_check()[row0sel.cc:3921]
mysqld.exe!row_search_mvcc()[row0sel.cc:5257]
mysqld.exe!ha_innobase::ft_read()[ha_innodb.cc:10092]
mysqld.exe!FT_SELECT::get_next()[opt_range.h:1719]
mysqld.exe!find_all_keys()[filesort.cc:764]
mysqld.exe!filesort()[filesort.cc:262]
mysqld.exe!create_sort_index()[sql_select.cc:23413]
mysqld.exe!st_join_table::sort_table()[sql_select.cc:21173]
mysqld.exe!join_init_read_record()[sql_select.cc:21108]
mysqld.exe!sub_select()[sql_select.cc:20188]
mysqld.exe!do_select()[sql_select.cc:19729]
mysqld.exe!JOIN::exec_inner()[sql_select.cc:4386]
mysqld.exe!JOIN::exec()[sql_select.cc:4169]
mysqld.exe!mysql_select()[sql_select.cc:4602]
mysqld.exe!handle_select()[sql_select.cc:408]
mysqld.exe!execute_sqlcom_select()[sql_parse.cc:6357]
mysqld.exe!mysql_execute_command()[sql_parse.cc:3899]
mysqld.exe!mysql_parse()[sql_parse.cc:7903]
mysqld.exe!dispatch_command()[sql_parse.cc:1844]
mysqld.exe!do_command()[sql_parse.cc:1359]
mysqld.exe!threadpool_process_request()[threadpool_common.cc:366]
mysqld.exe!tp_callback()[threadpool_common.cc:193]
ntdll.dll!RtlInitializeCriticalSection()
ntdll.dll!RtlReleaseSRWLockExclusive()
KERNEL32.DLL!BaseThreadInitThunk()
ntdll.dll!RtlUserThreadStart()

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x1c938d68310): SELECT search_sites.id, search_sites.start_date, company company, name,x,y, city_name ,
CAST( ST_Distance(@hereami, a_point) * 111195 as integer) distance
FROM search_sites2 search_sites WHERE
id in (select st_site from search_sites3 where st_site_visible=1 and st_type in (2546 ))
and city =414
and match(`name`, `description`, `company`, `require`, `address`, `city_name`, `company_orig_name`) against ('טקסט כלשהו בעברית')
ORDER BY distance LIMIT 0 ,10
Connection ID (thread ID): 8
Status: NOT_KILLED

Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on

The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
Writing a core file at E:\MySQL\
InnoDB: using atomic writes.

What I tried to do:
create empty table with the same columnts - works
remove the fulltext search - works
remove the where ... in ... - works
remove the city or change the city= to city in (...) - not work
remove some of the fields from the SELECT - not work
remove thr ORDER BY and the computed column for it - not work
tested with a 20 rows - works
30 rows - works
50 rows - crashed
start again - works
touch the table (INSERT, DELETE row, etc) - crashed again

Tested on 10.3 - works

Scheme used for testing:

CREATE TABLE `search_sites2` (
`id` int(11) NOT NULL,
`name` varchar(75) NOT NULL,
`company` varchar(50) NOT NULL DEFAULT ' ',
`description` text DEFAULT NULL,
`require` text DEFAULT NULL,
`has_phone` tinyint(1) NOT NULL,
`has_mail` tinyint(1) NOT NULL,
`mail` varchar(70) NOT NULL,
`home` tinyint(3) unsigned NOT NULL,
`visible` tinyint(1) DEFAULT NULL,
`link` varchar(100) DEFAULT NULL,
`manpower` tinyint(3) unsigned NOT NULL,
`start_date` datetime NOT NULL,
`update_date` datetime NOT NULL,
`require_cv` tinyint(3) unsigned NOT NULL,
`parent` int(11) NOT NULL,
`code` varchar(25) NOT NULL,
`submit_url` varchar(250) NOT NULL,
`x` double NOT NULL,
`y` double NOT NULL,
`address` varchar(90) DEFAULT NULL,
`address_id` int(11) DEFAULT NULL,
`sa_id` int(11) NOT NULL,
`a_point` point NOT NULL,
`city` int(11) DEFAULT NULL,
`city_name` varchar(50) NOT NULL,
`area` int(10) unsigned DEFAULT NULL,
`parent_area` int(10) unsigned DEFAULT NULL,
`company_id` int(11) NOT NULL,
`company_orig_name` varchar(50) DEFAULT NULL,
`logo` int(11) DEFAULT NULL,
`parent_id` int(11) DEFAULT NULL,
`company_personal` tinyint(3) unsigned DEFAULT NULL,
PRIMARY KEY (`sa_id`),
KEY `site_id` (`id`),
SPATIAL KEY `point` (`a_point`),
KEY `city` (`city`),
KEY `update_date` (`update_date`),
KEY `home` (`home`),
KEY `company` (`company_id`),
FULLTEXT KEY `fts` (`name`,`description`,`company`,`require`,`address`,`city_name`,`company_orig_name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

CREATE TABLE `search_sites3` (
`st_id` int(11) NOT NULL DEFAULT 0,
`st_site` int(11) NOT NULL,
`st_type` int(11) NOT NULL,
`st_speciality` int(11) NOT NULL,
`st_site_visible` tinyint(1) NOT NULL DEFAULT 1
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

I can send the table content, but not in public place.

 Comments   
Comment by Marko Mäkelä [ 2019-11-13 ]

Moshe L, you can upload the table data (a SQL dump would be preferred) to ftps://ftp.mariadb.com/ and note the file name here. I would suggest to start the file name with MDEV-21040.

Comment by Moshe L [ 2019-11-13 ]

Sent by email, as I could not upload to the FTPS - wrong user/password.

Thanks, Marko.

Comment by Marko Mäkelä [ 2019-11-13 ]

Thanks, Moshe L. Our FTP procedure was changed recently; maybe anonymous FTP (user name "anonymous" or "ftp") does not work any more.

I got the SQL dump, but I cannot repeat the crash with that locally. I tried a build of the latest 10.4 as well as 10.4.10.

Can you repeat the crash on a newly initialized database that has been loaded from the dump?

Comment by Moshe L [ 2019-11-13 ]

Yes.
I can repeat the crash on a new database and dump restore.
Windows 10 x64.

Comment by Moshe L [ 2019-11-14 ]

Tested with Aria, works.

Comment by Thirunarayanan Balathandayuthapani [ 2019-11-15 ]

I can able to repeat the issue with .sql provided via mail.

Comment by Thirunarayanan Balathandayuthapani [ 2019-11-15 ]

I can repeat the issue in 10.4.3, not in 10.3. The following stacktrace happens when the above select query:

#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
 
#1  0x00007f1d151658db in __GI_abort () at abort.c:100
 
#2  0x00007f1d1515539a in __assert_fail_base (fmt=0x7f1d152dc7d8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", 
    assertion=assertion@entry=0x55c73cbb8358 "n < rec_offs_n_fields(offsets)", 
    file=file@entry=0x55c73cbb8270 "/home/thiru/mariarepo/10.4/10.4-sample/storage/innobase/include/rem0rec.h", line=line@entry=665, 
    function=function@entry=0x55c73cbc0ee0 <rec_offs_nth_flag(unsigned long const*, unsigned long, unsigned long)::__PRETTY_FUNCTION__> "ulint rec_offs_nth_flag(const ulint*, ulint, ulint)") at assert.c:92
 
#3  0x00007f1d15155412 in __GI___assert_fail (assertion=0x55c73cbb8358 "n < rec_offs_n_fields(offsets)", 
    file=0x55c73cbb8270 "/home/thiru/mariarepo/10.4/10.4-sample/storage/innobase/include/rem0rec.h", line=665, 
    function=0x55c73cbc0ee0 <rec_offs_nth_flag(unsigned long const*, unsigned long, unsigned long)::__PRETTY_FUNCTION__> "ulint rec_offs_nth_flag(const ulint*, ulint, ulint)") at assert.c:101
 
#4  0x000055c73c2103e8 in rec_offs_nth_flag (offsets=0x7f1d1411cc80, n=3, flag=4611686018427387904)
 
    at /home/thiru/mariarepo/10.4/10.4-sample/storage/innobase/include/rem0rec.h:665
 
#5  0x000055c73c21058a in rec_offs_nth_extern (offsets=0x7f1d1411cc80, n=3)
 
    at /home/thiru/mariarepo/10.4/10.4-sample/storage/innobase/include/rem0rec.h:707
 
#6  0x000055c73c3a2cc8 in row_sel_store_mysql_field (mysql_rec=0x7f1c94c1a638 "\277\377", prebuilt=0x7f1c94bca298, rec=0x7f1cf556d0b1 "", 
    index=0x7f1c94ae2a58, offsets=0x7f1d1411cc80, field_no=3, templ=0x7f1c94073730)
 
    at /home/thiru/mariarepo/10.4/10.4-sample/storage/innobase/row/row0sel.cc:2908
 
#7  0x000055c73c3a57c4 in row_search_idx_cond_check (mysql_rec=0x7f1c94c1a638 "\277\377", prebuilt=0x7f1c94bca298, rec=0x7f1cf556d0b1 "", 
    offsets=0x7f1d1411cc80) at /home/thiru/mariarepo/10.4/10.4-sample/storage/innobase/row/row0sel.cc:3921
 
#8  0x000055c73c3a98c5 in row_search_mvcc (buf=0x7f1c94c1a638 "\277\377", mode=PAGE_CUR_GE, prebuilt=0x7f1c94bca298, match_mode=1, direction=0)
 
    at /home/thiru/mariarepo/10.4/10.4-sample/storage/innobase/row/row0sel.cc:5257
 
#9  0x000055c73c1ac39b in row_search_for_mysql (buf=0x7f1c94c1a638 "\277\377", mode=PAGE_CUR_GE, prebuilt=0x7f1c94bca298, match_mode=1, direction=0)
 
    at /home/thiru/mariarepo/10.4/10.4-sample/storage/innobase/include/row0sel.ic:137
 
#10 0x000055c73c1c1089 in ha_innobase::ft_read (this=0x7f1c9482db88, buf=0x7f1c94c1a638 "\277\377")
 
    at /home/thiru/mariarepo/10.4/10.4-sample/storage/innobase/handler/ha_innodb.cc:10090
 
#11 0x000055c73bcaa048 in handler::ha_ft_read (this=0x7f1c9482db88, buf=0x7f1c94c1a638 "\277\377")
 
    at /home/thiru/mariarepo/10.4/10.4-sample/sql/sql_class.h:6563
 
#12 0x000055c73c1102f7 in FT_SELECT::get_next (this=0x7f1c94d34560) at /home/thiru/mariarepo/10.4/10.4-sample/sql/opt_range.h:1719
 
#13 0x000055c73bf632f2 in find_all_keys (thd=0x7f1c94000d50, param=0x7f1d1411d970, select=0x7f1c9400ace8, fs_info=0x7f1c949911c0, 
    buffpek_pointers=0x7f1d1411db70, tempfile=0x7f1d1411da00, pq=0x0, found_rows=0x7f1c949913a0)
 
    at /home/thiru/mariarepo/10.4/10.4-sample/sql/filesort.cc:763
 
#14 0x000055c73bf6186b in filesort (thd=0x7f1c94000d50, table=0x7f1c94d6ada0, filesort=0x7f1c9400b4d8, tracker=0x7f1c9400bb98, join=0x7f1c94c5b128, 
    first_table_bit=1) at /home/thiru/mariarepo/10.4/10.4-sample/sql/filesort.cc:262
 
#15 0x000055c73bc96cf9 in create_sort_index (thd=0x7f1c94000d50, join=0x7f1c94c5b128, tab=0x7f1c940091a8, fsort=0x7f1c9400b4d8)
 
    at /home/thiru/mariarepo/10.4/10.4-sample/sql/sql_select.cc:23407
 
#16 0x000055c73bc90ff4 in st_join_table::sort_table (this=0x7f1c940091a8) at /home/thiru/mariarepo/10.4/10.4-sample/sql/sql_select.cc:21167
 
#17 0x000055c73bc90bc4 in join_init_read_record (tab=0x7f1c940091a8) at /home/thiru/mariarepo/10.4/10.4-sample/sql/sql_select.cc:21104
 
#18 0x000055c73bc8e98d in sub_select (join=0x7f1c94c5b128, join_tab=0x7f1c940091a8, end_of_records=false)
 
    at /home/thiru/mariarepo/10.4/10.4-sample/sql/sql_select.cc:20184
 
---Type <return> to continue, or q <return> to quit---
 
#19 0x000055c73bc8de7e in do_select (join=0x7f1c94c5b128, procedure=0x0) at /home/thiru/mariarepo/10.4/10.4-sample/sql/sql_select.cc:19725
 
#20 0x000055c73bc631d2 in JOIN::exec_inner (this=0x7f1c94c5b128) at /home/thiru/mariarepo/10.4/10.4-sample/sql/sql_select.cc:4386
 
#21 0x000055c73bc6230c in JOIN::exec (this=0x7f1c94c5b128) at /home/thiru/mariarepo/10.4/10.4-sample/sql/sql_select.cc:4168
 
#22 0x000055c73bc63a23 in mysql_select (thd=0x7f1c94000d50, tables=0x7f1c94014f98, wild_num=0, fields=..., conds=0x7f1c94017b80, og_num=1, 
    order=0x7f1c94019158, group=0x0, having=0x0, proc_param=0x0, select_options=2147748608, result=0x7f1c940193e0, unit=0x7f1c94004c80, 
    select_lex=0x7f1c94013b88) at /home/thiru/mariarepo/10.4/10.4-sample/sql/sql_select.cc:4600
 
#23 0x000055c73bc53993 in handle_select (thd=0x7f1c94000d50, lex=0x7f1c94004bb8, result=0x7f1c940193e0, setup_tables_done_option=0)
 
    at /home/thiru/mariarepo/10.4/10.4-sample/sql/sql_select.cc:408
 
#24 0x000055c73bc1a0a3 in execute_sqlcom_select (thd=0x7f1c94000d50, all_tables=0x7f1c94014f98)
 
    at /home/thiru/mariarepo/10.4/10.4-sample/sql/sql_parse.cc:6357
 
#25 0x000055c73bc0f780 in mysql_execute_command (thd=0x7f1c94000d50) at /home/thiru/mariarepo/10.4/10.4-sample/sql/sql_parse.cc:3899
 
#26 0x000055c73bc1e1a1 in mysql_parse (thd=0x7f1c94000d50, 
    rawbuf=0x7f1c94013778 "SELECT search_sites.id, search_sites.start_date, company company, name,x,y, city_name , CAST( ST_Distance(@hereami, a_point) * 111195 as integer) distance FROM search_sites2 search_sites WHERE id in ("..., length=470, parser_state=0x7f1d1411f1a0, is_com_multi=false, 
    is_next_command=false) at /home/thiru/mariarepo/10.4/10.4-sample/sql/sql_parse.cc:7898
 
#27 0x000055c73bc09387 in dispatch_command (command=COM_QUERY, thd=0x7f1c94000d50, packet=0x7f1c940af021 "", packet_length=470, is_com_multi=false, 
    is_next_command=false) at /home/thiru/mariarepo/10.4/10.4-sample/sql/sql_parse.cc:1841
 
#28 0x000055c73bc07a13 in do_command (thd=0x7f1c94000d50) at /home/thiru/mariarepo/10.4/10.4-sample/sql/sql_parse.cc:1359
 
#29 0x000055c73bd8f6e7 in do_handle_one_connection (connect=0x55c73f2fc8a0) at /home/thiru/mariarepo/10.4/10.4-sample/sql/sql_connect.cc:1412
 
#30 0x000055c73bd8f436 in handle_one_connection (arg=0x55c73f2fc8a0) at /home/thiru/mariarepo/10.4/10.4-sample/sql/sql_connect.cc:1316
 
#31 0x00007f1d160606db in start_thread (arg=0x7f1d14120700) at pthread_create.c:463
 
#32 0x00007f1d1524688f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

It looks like the regression of MDEV-16188. So assigning it to optimizer team.

Comment by Thirunarayanan Balathandayuthapani [ 2019-11-15 ]

Sent the repeating steps with .sql file via mail. Please take a look at it.

Comment by Thirunarayanan Balathandayuthapani [ 2019-11-27 ]

Fixed as a part of MDEV-20407

Comment by Marko Mäkelä [ 2019-11-27 ]

Actually, MDEV-20407 turned out to share a common fix with MDEV-20056.

Generated at Thu Feb 08 09:04:08 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.