[MDEV-9137] MariaDB Crash on Query Using Aria Engine Created: 2015-11-16  Updated: 2023-04-27

Status: Stalled
Project: MariaDB Server
Component/s: Storage Engine - Aria
Affects Version/s: 10.1.8, 5.5, 10.0, 10.1
Fix Version/s: 10.4

Type: Bug Priority: Major
Reporter: Benjamin Hodgetts Assignee: Michael Widenius
Resolution: Unresolved Votes: 0
Labels: crash
Environment:

Arch Linux 64bit


Issue Links:
Relates
relates to MDEV-18305 [Draft] Server crashed in delete_tree... Closed

 Description   

Certain imports appear to be crashing MariaDB but only when this table is set to Aria as the engine, it doesn't happen when the table is set to InnoDB or MyISAM. Here's one example. Each time MariaDB was restarted, it kept crashing on this same insert.

Thread pointer: 0x0x7fe6e0e54548
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7fea9ccb5d78 thread_stack 0x48000
/usr/bin/mysqld(my_print_stacktrace+0x2e)[0x55b637b50a0e]
/usr/bin/mysqld(handle_fatal_signal+0x34d)[0x55b63769fdcd]
/usr/lib/libpthread.so.0(+0x10d60)[0x7fea9ad4fd60]
/usr/bin/mysqld(+0x98299d)[0x55b637a9c99d]
/usr/bin/mysqld(+0x982c3b)[0x55b637a9cc3b]
/usr/bin/mysqld(+0x982a08)[0x55b637a9ca08]
/usr/bin/mysqld(+0x982a08)[0x55b637a9ca08]
/usr/bin/mysqld(+0x982e0b)[0x55b637a9ce0b]
/usr/bin/mysqld(+0x983230)[0x55b637a9d230]
/usr/bin/mysqld(+0x99cd43)[0x55b637ab6d43]
/usr/bin/mysqld(+0x97fd10)[0x55b637a99d10]
/usr/bin/mysqld(_ZN7handler12ha_write_rowEPh+0x42f)[0x55b6376a9cbf]
/usr/bin/mysqld(_Z12write_recordP3THDP5TABLEP12st_copy_info+0x6f)[0x55b6374fb2cf]
/usr/bin/mysqld(_Z12mysql_insertP3THDP10TABLE_LISTR4ListI4ItemERS3_IS5_ES6_S6_15enum_duplicatesb+0xf8e)[0x55b6374ff05e]
/usr/bin/mysqld(_Z21mysql_execute_commandP3THD+0x325a)[0x55b637518eca]
/usr/bin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x20e)[0x55b63751ecce]
/usr/bin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x24a7)[0x55b637522087]
/usr/bin/mysqld(_Z10do_commandP3THD+0x176)[0x55b6375227f6]
/usr/bin/mysqld(_Z26threadpool_process_requestP3THD+0x127)[0x55b637629667]
/usr/bin/mysqld(+0x55dc45)[0x55b637677c45]
/usr/lib/libpthread.so.0(+0x74a4)[0x7fea9ad464a4]
/usr/lib/libc.so.6(clone+0x6d)[0x7fea9a40413d]
 
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x7fe4d89648a0): INSERT INTO bookinfo          (title, author, asin, isbn, ean, url, salesrank, publisher, publishdate, pages,          overview, genre, cover, createddate, updateddate)         VALUES ('Standoff', 'Sandra Brown', '0743532627', '0743532627', '9780743532624', 'http://www.amazon.com/Standoff-Sandra-Brown/dp/0743532627%3FSubscriptionId%3DAKIAJ6YEOWNXMRGD5AFQ%26tag%3Dxnode0f-20%26linkCode%3Dxm2%26camp%3D2025%26creative%3D165953%26creativeASIN%3D0743532627', 1230300, 'Simon & Schuster Audio', '2003-05-01', 6, ' TV reporter Tiel McCoy is driving down Interstate 20 on her way to New Mexico for a well-earned vacation. But her itinerary is rewritten when she hears on the radio that the teenage daughter of a well-known Fort Worth multi-millionaire has been kidnapped, at least that\'s the official report. In truth, she is pregnant and has run away with her boyfriend. After calling her editor, Tiel abandons her holiday plans in favor of pursuing the story.  During an innocuous visit to a convenience store in Rojo Flats, Tiel gets up close and personal with the barrel of a gun, two desperate young lovers -- and the scoop of a lifetime.  In the electrifying standoff that follows, she learns why the two young runaways fear the wealthy father of one of them even more than the FBI...and why she must defend them against a world bent on their destruction.  Poignant and riveting, Standoff is suspense at its best, as only Sandra Brown can create it.', 'Suspense', 1, now(), now())
Connection ID (thread ID): 3259
Status: NOT_KILLED
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on
 
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
Mon Nov 16 13:18:23 2015 TokuFT recovery starting in env /var/lib/mysql/
Mon Nov 16 13:18:23 2015 TokuFT recovery scanning backward from 5805517708
Mon Nov 16 13:18:23 2015 TokuFT recovery bw_end_checkpoint at 5805517708 timestamp 1447679863009133 xid 5805505638 (bw_newer)
Mon Nov 16 13:18:23 2015 TokuFT recovery bw_begin_checkpoint at 5805505638 timestamp 1447679844844486 (bw_between)
Mon Nov 16 13:18:23 2015 TokuFT recovery turning around at begin checkpoint 5805505638 time 18164647
Mon Nov 16 13:18:23 2015 TokuFT recovery starts scanning forward to 5805517708 from 5805505638 left 12070 (fw_between)
Mon Nov 16 13:18:40 2015 TokuFT recovery scanning forward to 5805517708 at 5805507638 left 10070 (fw_between)
Mon Nov 16 13:18:55 2015 TokuFT recovery scanning forward to 5805517708 at 5805515638 left 2070 (fw_between)
Mon Nov 16 13:18:55 2015 TokuFT recovery closing 9765 dictionaries
Mon Nov 16 13:19:01 2015 TokuFT recovery making a checkpoint
Mon Nov 16 13:19:01 2015 TokuFT recovery done
2015-11-16 13:19:01 139810053760896 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2015-11-16 13:19:01 139810053760896 [Note] InnoDB: The InnoDB memory heap is disabled
2015-11-16 13:19:01 139810053760896 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2015-11-16 13:19:01 139810053760896 [Note] InnoDB: Memory barrier is not used
2015-11-16 13:19:01 139810053760896 [Note] InnoDB: Compressed tables use zlib 1.2.8
2015-11-16 13:19:01 139810053760896 [Note] InnoDB: Using Linux native AIO
2015-11-16 13:19:01 139810053760896 [Note] InnoDB: Using CPU crc32 instructions
2015-11-16 13:19:01 139810053760896 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2015-11-16 13:19:01 139810053760896 [Note] InnoDB: Completed initialization of buffer pool
2015-11-16 13:19:01 139810053760896 [Note] InnoDB: Highest supported file format is Barracuda.
2015-11-16 13:19:01 139810053760896 [Note] InnoDB: The log sequence numbers 706566915771 and 706566915771 in ibdata files do not match the log sequence number 706566915811 in the ib_logfiles!
2015-11-16 13:19:01 139810053760896 [Note] InnoDB: Database was not shutdown normally!
2015-11-16 13:19:01 139810053760896 [Note] InnoDB: Starting crash recovery.
2015-11-16 13:19:01 139810053760896 [Note] InnoDB: Reading tablespace information from the .ibd files...
2015-11-16 13:19:02 139810053760896 [Note] InnoDB: Restoring possible half-written data pages
2015-11-16 13:19:02 139810053760896 [Note] InnoDB: from the doublewrite buffer...
2015-11-16 13:19:02 139810053760896 [Note] InnoDB: 128 rollback segment(s) are active.
2015-11-16 13:19:02 139810053760896 [Note] InnoDB: Waiting for purge to start
2015-11-16 13:19:02 139810053760896 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.26-74.0 started; log sequence number 706566915811
2015-11-16 13:19:03 139808592598784 [Note] InnoDB: Dumping buffer pool(s) not yet started
2015-11-16 13:19:03 139810053760896 [Note] Plugin 'FEEDBACK' is disabled.
2015-11-16 13:19:03 139810053760896 [Note] Recovering after a crash using tc.log
2015-11-16 13:19:03 139810053760896 [Note] Starting crash recovery...
2015-11-16 13:19:03 139810053760896 [Note] Crash recovery finished.
2015-11-16 13:19:03 139810053760896 [Note] Event Scheduler: Loaded 0 events
2015-11-16 13:19:03 139810053760896 [Note] /usr/bin/mysqld: ready for connections.
Version: '10.1.8-MariaDB'  socket: '/var/run/mysqld/mysqld.sock'  port: 0  MariaDB Server

This is the table in question:

CREATE TABLE `bookinfo` (
 `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
 `title` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
 `author` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
 `asin` varchar(128) COLLATE utf8_unicode_ci DEFAULT NULL,
 `isbn` varchar(128) COLLATE utf8_unicode_ci DEFAULT NULL,
 `ean` varchar(128) COLLATE utf8_unicode_ci DEFAULT NULL,
 `url` varchar(1000) COLLATE utf8_unicode_ci DEFAULT NULL,
 `salesrank` int(10) unsigned DEFAULT NULL,
 `publisher` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
 `publishdate` datetime DEFAULT NULL,
 `pages` varchar(128) COLLATE utf8_unicode_ci DEFAULT NULL,
 `overview` varchar(10000) COLLATE utf8_unicode_ci DEFAULT NULL,
 `genre` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
 `cover` tinyint(1) unsigned NOT NULL DEFAULT '0',
 `createddate` datetime NOT NULL,
 `updateddate` datetime NOT NULL,
 PRIMARY KEY (`id`) KEY_BLOCK_SIZE=2048,
 UNIQUE KEY `ix_bookinfo_asin` (`asin`) KEY_BLOCK_SIZE=2048,
 FULLTEXT KEY `ix_bookinfo_author_title_ft` (`author`,`title`) KEY_BLOCK_SIZE=2048
) ENGINE=Aria AUTO_INCREMENT=212983 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci PACK_KEYS=1 PAGE_CHECKSUM=0 ROW_FORMAT=DYNAMIC TRANSACTIONAL=0 /* `PAGE_COMPRESSED`=1 */

(the table has been several different engines in the past which is why it appears to have retained many different and now invalid options).



 Comments   
Comment by Elena Stepanova [ 2015-11-18 ]

Enverex,

Is it possible that the table is corrupted? Did you run check table?

Does the table contain confidential information? Would it be possible for you to upload bookinfo.frm, bookinfo.MAD and bookinfo.MAI to our ftp.askmonty.org/private?

If you cannot do that, please at least paste the output of SHOW INDEX IN for the table.

Please also paste your cnf file(s).

Thanks.

Comment by Benjamin Hodgetts [ 2015-11-18 ]

There's nothing private in the table but it is very large. Here's the SHOW INDEX for now at least.

+----------+------------+-----------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table    | Non_unique | Key_name                    | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+----------+------------+-----------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| bookinfo |          0 | PRIMARY                     |            1 | id          | A         |      213322 |     NULL | NULL   |      | BTREE      |         |               |
| bookinfo |          0 | ix_bookinfo_asin            |            1 | asin        | A         |      213322 |     NULL | NULL   | YES  | BTREE      |         |               |
| bookinfo |          1 | ix_bookinfo_author_title_ft |            1 | author      | NULL      |      106661 |     NULL | NULL   |      | FULLTEXT   |         |               |
| bookinfo |          1 | ix_bookinfo_author_title_ft |            2 | title       | NULL      |      213322 |     NULL | NULL   |      | FULLTEXT   |         |               |
+----------+------------+-----------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+

Comment by Benjamin Hodgetts [ 2015-11-18 ]

I just tried to convert that table back from MyISAM to Aria to test for you and MariaDB crashed during the ALTER process so it really doesn't like the data. The crash this time was:

Thread pointer: 0x0x7f5094bf8c48
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7f514c847d78 thread_stack 0x48000
(my_addr_resolve failure: fork)
/usr/bin/mysqld(my_print_stacktrace+0x2e) [0x556bfffaba0e]
/usr/bin/mysqld(handle_fatal_signal+0x34d) [0x556bffafadcd]
/usr/lib/libpthread.so.0(+0x10d60) [0x7f5156b53d60]
/usr/bin/mysqld(+0x9ac6c9) [0x556bfff216c9]
/usr/bin/mysqld(+0x92a226) [0x556bffe9f226]
/usr/bin/mysqld(+0x44138f) [0x556bff9b638f]
/usr/bin/mysqld(sub_select(JOIN*, st_join_table*, bool)+0x164) [0x556bff9aaf64]
/usr/bin/mysqld(+0x44487d) [0x556bff9b987d]
/usr/bin/mysqld(JOIN::exec_inner()+0xc5e) [0x556bff9cc98e]
/usr/bin/mysqld(JOIN::exec()+0x54) [0x556bff9ce664]
/usr/bin/mysqld(mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*)+0x201) [0x556bff9cb071]
/usr/bin/mysqld(handle_select(THD*, LEX*, select_result*, unsigned long)+0x139) [0x556bff9cb929]
/usr/bin/mysqld(+0x3f5261) [0x556bff96a261]
/usr/bin/mysqld(mysql_execute_command(THD*)+0x6537) [0x556bff9771a7]
/usr/bin/mysqld(mysql_parse(THD*, char*, unsigned int, Parser_state*)+0x20e) [0x556bff979cce]
/usr/bin/mysqld(dispatch_command(enum_server_command, THD*, char*, unsigned int)+0x24a7) [0x556bff97d087]
/usr/bin/mysqld(do_command(THD*)+0x176) [0x556bff97d7f6]
/usr/bin/mysqld(threadpool_process_request(THD*)+0x127) [0x556bffa84667]
/usr/bin/mysqld(+0x55dc45) [0x556bffad2c45]
/usr/lib/libpthread.so.0(+0x74a4) [0x7f5156b4a4a4]
/usr/lib/libc.so.6(clone+0x6d) [0x7f515620813d]
 
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x7f300a0a34e0): SELECT * FROM bookinfo WHERE  MATCH(author, title) AGAINST('+Star +Trek +TOS +006 +The +Abode +of +Life +Lee +Correy' IN BOOLEAN MODE) LIMIT 1
Connection ID (thread ID): 134948
Status: NOT_KILLED
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on

I'll dump the table and upload it for you.

Running a repair on the table claims it's fine, but it still crashes when trying to convert to Aria. Initially it looked like trying to insert the data caused the crash. Now the data already exists in the DB, it crashes when trying to convert it to Aria.

Comment by Benjamin Hodgetts [ 2015-11-18 ]

Uploaded as bookinfo.tgz.

Comment by Benjamin Hodgetts [ 2015-11-18 ]

Just to clarify, it looks like MariaDB didn't crash during the conversion process to Aria in the post above, it crashed as soon as something tried to access some of the data in the table (as soon as it had finishing converting back to Aria from MyISAM).

Comment by Elena Stepanova [ 2015-11-20 ]

The table is actually corrupted:

MariaDB [test]> check table bookinfo;
+---------------+-------+----------+----------------------------------------------------+
| Table         | Op    | Msg_type | Msg_text                                           |
+---------------+-------+----------+----------------------------------------------------+
| test.bookinfo | check | error    | Page at 5029888 is not marked for index 1431655756 |
| test.bookinfo | check | error    | Page at 5013504 is not marked for index 1431655756 |
| test.bookinfo | check | error    | Page at 5021696 is not marked for index 1431655756 |
| test.bookinfo | check | error    | Key in wrong position at page 5070848              |
| test.bookinfo | check | error    | Corrupt                                            |
+---------------+-------+----------+----------------------------------------------------+
5 rows in set (0.11 sec)

Indeed, ordinary repair does not show it, and does not work:

MariaDB [test]> repair table bookinfo;
+---------------+--------+----------+----------+
| Table         | Op     | Msg_type | Msg_text |
+---------------+--------+----------+----------+
| test.bookinfo | repair | status   | OK       |
+---------------+--------+----------+----------+
1 row in set (8.27 sec)
 
MariaDB [test]> check table bookinfo;
+---------------+-------+----------+----------------------------------------------------+
| Table         | Op    | Msg_type | Msg_text                                           |
+---------------+-------+----------+----------------------------------------------------+
| test.bookinfo | check | error    | Page at 5029888 is not marked for index 1431655756 |
| test.bookinfo | check | error    | Page at 5013504 is not marked for index 1431655756 |
| test.bookinfo | check | error    | Page at 5021696 is not marked for index 1431655756 |
| test.bookinfo | check | error    | Key in wrong position at page 5070848              |
| test.bookinfo | check | error    | Corrupt                                            |
+---------------+-------+----------+----------------------------------------------------+
5 rows in set (0.10 sec)

But repair extended seems to do the job:

MariaDB [test]> repair table bookinfo extended;
+---------------+--------+----------+----------+
| Table         | Op     | Msg_type | Msg_text |
+---------------+--------+----------+----------+
| test.bookinfo | repair | status   | OK       |
+---------------+--------+----------+----------+
1 row in set (1 min 48.79 sec)
 
MariaDB [test]> check table bookinfo;
+---------------+-------+----------+----------+ 
| Table         | Op    | Msg_type | Msg_text |
+---------------+-------+----------+----------+
| test.bookinfo | check | status   | OK       |
+---------------+-------+----------+----------+
1 row in set (0.74 sec)

And INSERT works all right for me after that. Could you please also try to run repair table bookinfo extended and see if it helps?

Comment by Benjamin Hodgetts [ 2015-11-20 ]

I'll run that now.

But that begs the question; how have I converted this table to and from Aria several times without the corruption being detected? Surely to convert the table it would need to read and write all the information present?

Comment by Benjamin Hodgetts [ 2015-11-20 ]

Table seems to be happy now. I'm guessing MariaDB crashing out entirely isn't expected behaviour on a secretly corrupted table though?

Comment by Elena Stepanova [ 2015-11-21 ]

Regarding the question why some things work and some not, I guess it's hard to expect a reliable crash when a table is corrupted, it's rather the matter of luck (or a lack thereof).

Regarding the point that the whole server should not crash on a corrupt table, I totally agree.
Thanks for the report.


monty, the problem here is two-fold.
First, REPAIR TABLE not only fails to repair the table (which might be normal, otherwise why would we need REPAIR ... EXTENDED, right?), but it also does not report that the table is corrupted, it just returns OK while the table stays in the bad shape:

MariaDB [test]> repair table bookinfo;
+---------------+--------+----------+----------+ge done
| Table         | Op     | Msg_type | Msg_text |
+---------------+--------+----------+----------+
| test.bookinfo | repair | status   | OK       |
+---------------+--------+----------+----------+
1 row in set (21.69 sec)
 
MariaDB [test]> check table bookinfo;
+---------------+-------+----------+----------------------------------------------------+
| Table         | Op    | Msg_type | Msg_text                                           |
+---------------+-------+----------+----------------------------------------------------+
| test.bookinfo | check | error    | Page at 5029888 is not marked for index 4294967284 |
| test.bookinfo | check | error    | Page at 5013504 is not marked for index 4294967284 |
| test.bookinfo | check | error    | Page at 5021696 is not marked for index 4294967284 |
| test.bookinfo | check | error    | Key in wrong position at page 5070848              |
| test.bookinfo | check | error    | Corrupt                                            |
+---------------+-------+----------+----------------------------------------------------+
5 rows in set (0.31 sec)

Secondly, the crash upon DML. It would be fine to report that the operation failed because the table is corrupted, but certainly the server shouldn't crash.

The behavior is a bit different on 5.5 and 10.0 vs 10.1.

On 5.5 and 10.0 (debug builds) I'm getting the following assertion failure while running a DELETE:

Stack trace from 5.5 commit e669a5fd87b28158cbb4454b4bcce7fe4a4ef8b8

5.5/storage/maria/ma_page.c:220: _ma_write_keypage: Assertion `((uchar) buff[(share)->keypage_header - 1 - 1 - 2]) == page->keyinfo->key_nr' failed.
151121  0:47:59 [ERROR] mysqld got signal 6 ;
 
...
 
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x7f22fd445078): delete from bookinfo where asin = '0743532627'
Connection ID (thread ID): 1
Status: NOT_KILLED

#6  0x00007f2326c69311 in *__GI___assert_fail (assertion=0xed94a0 "((uchar) buff[(share)->keypage_header - 1 - 1 - 2]) == page->keyinfo->key_nr", file=<optimized out>, line=220, function=0xed9680 "_ma_write_keypage") at assert.c:81
#7  0x0000000000bdaf5b in _ma_write_keypage (page=0x7f2328e1bc80, lock=PAGECACHE_LOCK_LEFT_WRITELOCKED, level=3) at 5.5/storage/maria/ma_page.c:220
#8  0x0000000000c0beeb in underflow (info=0x7f22fd47b060, keyinfo=0x7f22fd44a428, anc_page=0x7f2328e1ec20, leaf_page=0x7f2328e1bc80, keypos=0x7f2328e1c25a "?r\024\224") at 5.5/storage/maria/ma_delete.c:1260
#9  0x0000000000c09591 in d_search (info=0x7f22fd47b060, key=0x7f2328e1ed20, comp_flag=4, anc_page=0x7f2328e1ec20) at 5.5/storage/maria/ma_delete.c:466
#10 0x0000000000c08a4c in _ma_ck_real_delete (info=0x7f22fd47b060, key=0x7f2328e1ed20, root=0x7f2328e1ed58) at 5.5/storage/maria/ma_delete.c:228
#11 0x0000000000c090af in d_search (info=0x7f22fd47b060, key=0x7f2328e28370, comp_flag=4, anc_page=0x7f2328e21da0) at 5.5/storage/maria/ma_delete.c:370
#12 0x0000000000c092f7 in d_search (info=0x7f22fd47b060, key=0x7f2328e28370, comp_flag=131137, anc_page=0x7f2328e24dc0) at 5.5/storage/maria/ma_delete.c:414
#13 0x0000000000c092f7 in d_search (info=0x7f22fd47b060, key=0x7f2328e28370, comp_flag=131137, anc_page=0x7f2328e27d60) at 5.5/storage/maria/ma_delete.c:414
#14 0x0000000000c08a4c in _ma_ck_real_delete (info=0x7f22fd47b060, key=0x7f2328e28370, root=0x7f2328e28308) at 5.5/storage/maria/ma_delete.c:228
#15 0x0000000000c0879c in _ma_ck_delete (info=0x7f22fd47b060, key=0x7f2328e28370) at 5.5/storage/maria/ma_delete.c:178
#16 0x0000000000c27348 in _ma_ft_erase (info=0x7f22fd47b060, keynr=2, keybuf=0x7f22fd47fa9e "\005Brown?wvi", wlist=0x7f22fd4fb078, filepos=272931324) at 5.5/storage/maria/ma_ft_update.c:159
#17 0x0000000000c279a3 in _ma_ft_del (info=0x7f22fd47b060, keynr=2, keybuf=0x7f22fd47fa9e "\005Brown?wvi", record=0x7f22fd4a2078 "", pos=272931324) at 5.5/storage/maria/ma_ft_update.c:278
#18 0x0000000000c081d4 in maria_delete (info=0x7f22fd47b060, record=0x7f22fd4a2078 "") at 5.5/storage/maria/ma_delete.c:82
#19 0x0000000000b8f378 in ha_maria::delete_row (this=0x7f22fd4b9878, buf=0x7f22fd4a2078 "") at 5.5/storage/maria/ha_maria.cc:2292
#20 0x00000000007f48bf in handler::ha_delete_row (this=0x7f22fd4b9878, buf=0x7f22fd4a2078 "") at 5.5/sql/handler.cc:5258
#21 0x0000000000934f49 in mysql_delete (thd=0x7f230dcac060, table_list=0x7f22fd445190, conds=0x7f22fd445930, order_list=0x7f230dcafca0, limit=18446744073709551615, options=0) at 5.5/sql/sql_delete.cc:346
#22 0x000000000063811c in mysql_execute_command (thd=0x7f230dcac060) at 5.5/sql/sql_parse.cc:3105
#23 0x000000000063f8a5 in mysql_parse (thd=0x7f230dcac060, rawbuf=0x7f22fd445078 "delete from bookinfo where asin = '0743532627'", length=46, parser_state=0x7f2328e29670) at 5.5/sql/sql_parse.cc:5914
#24 0x00000000006334a9 in dispatch_command (command=COM_QUERY, thd=0x7f230dcac060, packet=0x7f22ff416061 "delete from bookinfo where asin = '0743532627'", packet_length=46) at 5.5/sql/sql_parse.cc:1079
#25 0x0000000000632635 in do_command (thd=0x7f230dcac060) at 5.5/sql/sql_parse.cc:793
#26 0x0000000000735c19 in do_handle_one_connection (thd_arg=0x7f230dcac060) at 5.5/sql/sql_connect.cc:1269
#27 0x0000000000735993 in handle_one_connection (arg=0x7f230dcac060) at 5.5/sql/sql_connect.cc:1185
#28 0x00007f2328a63b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#29 0x00007f2326d1995d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112

On 10.1, also debug build, DELETE works all right, but the INSERT crashes:

Stack trace from 10.1 commit 836275bb203a47104eb7f28aa409924b91abc801

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x7fcb4f8ab020): INSERT INTO bookinfo          (title, author, asin, isbn, ean, url, salesrank, publisher, publishdate, pages,          overview, genre, cover, createddate, updateddate)         VALUES ('Standoff', 'Sandra Brown', '0743532627', '0743532627', '9780743532624', 'http://www.amazon.com/Standoff-Sandra-Brown/dp/0743532627%3FSubscriptionId%3DAKIAJ6YEOWNXMRGD5AFQ%26tag%3Dxnode0f-20%26linkCode%3Dxm2%26camp%3D2025%26creative%3D165953%26creativeASIN%3D0743532627', 1230300, 'Simon & Schuster Audio', '2003-05-01', 6, ' TV reporter Tiel McCoy is driving down Interstate 20 on her way to New Mexico for a well-earned vacation. But her itinerary is rewritten when she hears on the radio that the teenage daughter of a well-known Fort Worth multi-millionaire has been kidnapped, at least that\'s the official report. In truth, she is pregnant and has run away with her boyfriend. After calling her editor, Tiel abandons her holiday plans in favor of pursuing the story.  During an innocuous visit to a convenience store in Rojo Flats, Tiel gets up close and personal with the barrel of a gun, two desperate young lovers -- and the scoop of a lifetime.  In the electrifying standoff that follows, she learns why the two young runaways fear the wealthy father of one of them even more than the FBI...and why she must defend them against a world bent on their destruction.  Poignant and riveting, Standoff is suspense at its best, as only Sandra Brown can create it.', 'Suspense', 1, now(), now())
Connection ID (thread ID): 3
Status: NOT_KILLED

#2  <signal handler called>
#3  0x00007fcb7e56f1df in w_search (info=info@entry=0x7fcb4f8e6008, comp_flag=comp_flag@entry=4, key=key@entry=0x7fcb7dba20d0, page_pos=<optimized out>, father_page=father_page@entry=0x0, father_keypos=father_keypos@entry=0x0, insert_last=insert_last@entry=1 '\001') at 10.1/storage/maria/ma_write.c:639
#4  0x00007fcb7e56f57c in _ma_ck_real_write_btree (info=info@entry=0x7fcb4f8e6008, key=0x7fcb7dba20d0, root=root@entry=0x7fcb7db99e48, comp_flag=comp_flag@entry=4) at 10.1/storage/maria/ma_write.c:527
#5  0x00007fcb7e56f48e in w_search (info=info@entry=0x7fcb4f8e6008, comp_flag=4, comp_flag@entry=131137, key=<optimized out>, key@entry=0x7fcb7dba07f0, page_pos=<optimized out>, father_page=father_page@entry=0x7fcb7db9cdc0, father_keypos=0x7fcb7db9b6e3 "\200\002\003uce?t\275", <incomplete sequence \353>, insert_last=insert_last@entry=0 '\000') at 10.1/storage/maria/ma_write.c:677
#6  0x00007fcb7e56f249 in w_search (info=info@entry=0x7fcb4f8e6008, comp_flag=comp_flag@entry=131137, key=key@entry=0x7fcb7dba07f0, page_pos=<optimized out>, father_page=father_page@entry=0x7fcb7db9fd10, father_keypos=0x7fcb7db9d32a "", insert_last=<optimized out>, insert_last@entry=0 '\000') at 10.1/storage/maria/ma_write.c:726
#7  0x00007fcb7e56f249 in w_search (info=info@entry=0x7fcb4f8e6008, comp_flag=comp_flag@entry=131137, key=key@entry=0x7fcb7dba07f0, page_pos=<optimized out>, father_page=father_page@entry=0x0, father_keypos=father_keypos@entry=0x0, insert_last=<optimized out>, insert_last@entry=1 '\001') at 10.1/storage/maria/ma_write.c:726
#8  0x00007fcb7e56f57c in _ma_ck_real_write_btree (info=info@entry=0x7fcb4f8e6008, key=key@entry=0x7fcb7dba07f0, root=root@entry=0x7fcb7dba02b8, comp_flag=comp_flag@entry=131137) at 10.1/storage/maria/ma_write.c:527
#9  0x00007fcb7e56f61f in _ma_ck_write_btree_with_log (info=info@entry=0x7fcb4f8e6008, key=key@entry=0x7fcb7dba07f0, root=0x7fcb4f890270, comp_flag=131137) at 10.1/storage/maria/ma_write.c:494
#10 0x00007fcb7e56f79d in _ma_ck_write_btree (info=0x7fcb4f8e6008, key=0x7fcb7dba07f0) at 10.1/storage/maria/ma_write.c:452
#11 0x00007fcb7e56fa2d in _ma_ck_write (info=info@entry=0x7fcb4f8e6008, key=key@entry=0x7fcb7dba07f0) at 10.1/storage/maria/ma_write.c:437
#12 0x00007fcb7e586062 in _ma_ft_store (filepos=<optimized out>, wlist=0x7fcb4f9cf020, keybuf=<optimized out>, keynr=<optimized out>, info=<optimized out>) at 10.1/storage/maria/ma_ft_update.c:143
#13 _ma_ft_add (info=info@entry=0x7fcb4f8e6008, keynr=keynr@entry=2, keybuf=keybuf@entry=0x7fcb4f8ea9c6 "\005Brown?wvi", record=record@entry=0x7fcb4f97e020 "", pos=pos@entry=272931324) at 10.1/storage/maria/ma_ft_update.c:260
#14 0x00007fcb7e56cbb0 in maria_write (info=0x7fcb4f8e6008, record=0x7fcb4f97e020 "") at 10.1/storage/maria/ma_write.c:178
#15 0x00007fcb7e1e46ff in handler::ha_write_row (this=0x7fcb4f906620, buf=0x7fcb4f97e020 "") at 10.1/sql/handler.cc:5878
#16 0x00007fcb7e0670be in write_record (thd=thd@entry=0x7fcb7af11008, table=table@entry=0x7fcb4f869808, info=info@entry=0x7fcb7dba0b90) at 10.1/sql/sql_insert.cc:1877
#17 0x00007fcb7e06a1c8 in mysql_insert (thd=0x7fcb7af11008, table_list=0x7fcb4f8abc78, fields=..., values_list=..., update_fields=..., update_values=..., duplic=DUP_ERROR, ignore=false) at 10.1/sql/sql_insert.cc:987
#18 0x00007fcb7e07f70f in mysql_execute_command (thd=thd@entry=0x7fcb7af11008) at 10.1/sql/sql_parse.cc:3898
#19 0x00007fcb7e086c8e in mysql_parse (parser_state=0x7fcb7dba2cd0, thd=0x7fcb7af11008, rawbuf=<optimized out>, length=<optimized out>) at 10.1/sql/sql_parse.cc:7282
#20 mysql_parse (thd=0x7fcb7af11008, rawbuf=<optimized out>, length=<optimized out>, parser_state=0x7fcb7dba2cd0) at 10.1/sql/sql_parse.cc:7203
#21 0x00007fcb7e08966e in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x7fcb7af11008, packet=packet@entry=0x7fcb57aeb009 "INSERT INTO bookinfo          (title, author, asin, isbn, ean, url, salesrank, publisher, publishdate, pages,          overview, genre, cover, createddate, updateddate)         VALUES ('Standoff', 'Sa"..., packet_length=1334489120, packet_length@entry=1482) at 10.1/sql/sql_parse.cc:1488
#22 0x00007fcb7e089c10 in do_command (thd=0x7fcb7af11008) at 10.1/sql/sql_parse.cc:1110
#23 0x00007fcb7e138f3a in do_handle_one_connection (thd_arg=thd_arg@entry=0x7fcb7af11008) at 10.1/sql/sql_connect.cc:1350
#24 0x00007fcb7e1390a6 in handle_one_connection (arg=0x7fcb7af11008) at 10.1/sql/sql_connect.cc:1262
#25 0x00007fcb7d7f0b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#26 0x00007fcb7b67c95d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112

I've set the fix version to 5.5 and higher, but if the fix turns out too big for 5.5 or 10.0, please adjust the field accordingly.

Comment by Benjamin Hodgetts [ 2015-11-22 ]

At least I thought that fixed it, it keeps happening still. It looks like it keeps corrupting that specific table somehow which in turn causes the crashing to return (until an extended repair is done).

Thread pointer: 0x0x7fea4ed4d7c8
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7fea379b3d78 thread_stack 0x48000
/usr/bin/mysqld(my_print_stacktrace+0x2e)[0x55dd849bba0e]
/usr/bin/mysqld(handle_fatal_signal+0x34d)[0x55dd8450adcd]
/usr/lib/libpthread.so.0(+0x10d60)[0x7fea72daad60]
/usr/bin/mysqld(+0x9ac6c9)[0x55dd849316c9]
/usr/bin/mysqld(+0x92a226)[0x55dd848af226]
/usr/bin/mysqld(+0x44138f)[0x55dd843c638f]
/usr/bin/mysqld(_Z10sub_selectP4JOINP13st_join_tableb+0x164)[0x55dd843baf64]
/usr/bin/mysqld(+0x44487d)[0x55dd843c987d]
/usr/bin/mysqld(_ZN4JOIN10exec_innerEv+0xc5e)[0x55dd843dc98e]
/usr/bin/mysqld(_ZN4JOIN4execEv+0x54)[0x55dd843de664]
/usr/bin/mysqld(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x201)[0x55dd843db071]
/usr/bin/mysqld(_Z13handle_selectP3THDP3LEXP13select_resultm+0x139)[0x55dd843db929]
/usr/bin/mysqld(+0x3f5261)[0x55dd8437a261]
/usr/bin/mysqld(_Z21mysql_execute_commandP3THD+0x6537)[0x55dd843871a7]
/usr/bin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x20e)[0x55dd84389cce]
/usr/bin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x24a7)[0x55dd8438d087]
/usr/bin/mysqld(_Z10do_commandP3THD+0x176)[0x55dd8438d7f6]
/usr/bin/mysqld(_Z26threadpool_process_requestP3THD+0x127)[0x55dd84494667]
/usr/bin/mysqld(+0x55dc45)[0x55dd844e2c45]
/usr/lib/libpthread.so.0(+0x74a4)[0x7fea72da14a4]
/usr/lib/libc.so.6(clone+0x6d)[0x7fea7245f13d]
 
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x7fea422184e0): SELECT * FROM bookinfo WHERE  MATCH(author, title) AGAINST('+John +Gartner +Moriartyc +John +Gartner +Moriarty7z001' IN BOOLEAN MODE) LIMIT 1
Connection ID (thread ID): 31
Status: NOT_KILLED
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on

Comment by Elena Stepanova [ 2015-11-22 ]

Enverex,
This crash on SELECT looks like a separate problem.
It's usual that after a crash an Aria/MyISAM table which was open during the crash is considered to be corrupt; it's unusual however that it's corrupted that much, but we'll see what's going on after we can reproduce it. If you don't mind, to avoid confusion, please file a separate bug report about this second crash (and importantly, please attach your cnf file(s) to it), while this issue will stay about crashing on an already corrupted table, and about ordinary REPAIR failing to fix it.

Comment by Elena Stepanova [ 2018-06-15 ]

Here is another dataset for the reported problem, produced during concurrent tests, there is no sensitive data: ftp://ftp.askmonty.org/public/mdev9137.tar.gz
Effect is the same:

10.1 c69357d8d459

MariaDB [test]> check table table1_key_pk_parts_2_int_autoinc;
+----------------------------------------+-------+----------+--------------------------------------------------+
| Table                                  | Op    | Msg_type | Msg_text                                         |
+----------------------------------------+-------+----------+--------------------------------------------------+
| test.table1_key_pk_parts_2_int_autoinc | check | error    | Page at 24576 is not marked for index 2863311518 |
| test.table1_key_pk_parts_2_int_autoinc | check | error    | Page at 8192 is not marked for index 2863311518  |
| test.table1_key_pk_parts_2_int_autoinc | check | error    | Page at 16384 is not marked for index 2863311518 |
| test.table1_key_pk_parts_2_int_autoinc | check | error    | Key in wrong position at page 114688             |
| test.table1_key_pk_parts_2_int_autoinc | check | error    | Corrupt                                          |
+----------------------------------------+-------+----------+--------------------------------------------------+
5 rows in set (0.00 sec)
 
MariaDB [test]> INSERT INTO `table1_key_pk_parts_2_int_autoinc` SELECT * FROM `table1_key_pk_parts_2_int_autoinc`;
ERROR 2013 (HY000): Lost connection to MySQL server during query

#3  <signal handler called>
#4  0x00007f7b1e9a79c6 in w_search (info=0x7f7aeea3e070, comp_flag=4, key=0x7f7b1f142a00, page_pos=24576, father_page=0x0, father_keypos=0x0, insert_last=1 '\001') at /data/src/10.1/storage/maria/ma_write.c:630
#5  0x00007f7b1e9a7501 in _ma_ck_real_write_btree (info=0x7f7aeea3e070, key=0x7f7b1f142a00, root=0x7f7b1f141d98, comp_flag=4) at /data/src/10.1/storage/maria/ma_write.c:527
#6  0x00007f7b1e9a7d55 in w_search (info=0x7f7aeea3e070, comp_flag=4, key=0x7f7b1f142a00, page_pos=114688, father_page=0x0, father_keypos=0x0, insert_last=1 '\001') at /data/src/10.1/storage/maria/ma_write.c:677
#7  0x00007f7b1e9a7501 in _ma_ck_real_write_btree (info=0x7f7aeea3e070, key=0x7f7b1f1429e0, root=0x7f7b1f1423f0, comp_flag=131137) at /data/src/10.1/storage/maria/ma_write.c:527
#8  0x00007f7b1e9a737b in _ma_ck_write_btree_with_log (info=0x7f7aeea3e070, key=0x7f7b1f1429e0, root=0x7f7aeea082b0, comp_flag=131137) at /data/src/10.1/storage/maria/ma_write.c:494
#9  0x00007f7b1e9a71cc in _ma_ck_write_btree (info=0x7f7aeea3e070, key=0x7f7b1f1429e0) at /data/src/10.1/storage/maria/ma_write.c:452
#10 0x00007f7b1e9aafac in keys_free (key=0x7f7aeeafb118 "\t136052736?}", <incomplete sequence \350>, mode=free_free, param=0x7f7aee89b3c0) at /data/src/10.1/storage/maria/ma_write.c:1708
#11 0x00007f7b1eaa943e in delete_tree_element (tree=0x7f7aee89ae70, element=0x7f7aeeafb0f8) at /data/src/10.1/mysys/tree.c:186
#12 0x00007f7b1eaa93e3 in delete_tree_element (tree=0x7f7aee89ae70, element=0x7f7aeeafb210) at /data/src/10.1/mysys/tree.c:184
#13 0x00007f7b1eaa93e3 in delete_tree_element (tree=0x7f7aee89ae70, element=0x7f7aeeafb440) at /data/src/10.1/mysys/tree.c:184
#14 0x00007f7b1eaa93e3 in delete_tree_element (tree=0x7f7aee89ae70, element=0x7f7aeeafb8a0) at /data/src/10.1/mysys/tree.c:184
#15 0x00007f7b1eaa93e3 in delete_tree_element (tree=0x7f7aee89ae70, element=0x7f7aeeafc160) at /data/src/10.1/mysys/tree.c:184
#16 0x00007f7b1eaa93e3 in delete_tree_element (tree=0x7f7aee89ae70, element=0x7f7aeeafd2e0) at /data/src/10.1/mysys/tree.c:184
#17 0x00007f7b1eaa93e3 in delete_tree_element (tree=0x7f7aee89ae70, element=0x7f7aeeaff5e0) at /data/src/10.1/mysys/tree.c:184
#18 0x00007f7b1eaa93e3 in delete_tree_element (tree=0x7f7aee89ae70, element=0x7f7aeeb03be0) at /data/src/10.1/mysys/tree.c:184
#19 0x00007f7b1eaa93e3 in delete_tree_element (tree=0x7f7aee89ae70, element=0x7f7aeeb0c7e0) at /data/src/10.1/mysys/tree.c:184
#20 0x00007f7b1eaa93e3 in delete_tree_element (tree=0x7f7aee89ae70, element=0x7f7aeeafb168) at /data/src/10.1/mysys/tree.c:184
#21 0x00007f7b1eaa93e3 in delete_tree_element (tree=0x7f7aee89ae70, element=0x7f7aeeafb088) at /data/src/10.1/mysys/tree.c:184
#22 0x00007f7b1eaa92df in free_tree (tree=0x7f7aee89ae70, free_flags=2) at /data/src/10.1/mysys/tree.c:154
#23 0x00007f7b1eaa93a9 in reset_tree (tree=0x7f7aee89ae70) at /data/src/10.1/mysys/tree.c:176
#24 0x00007f7b1eaa95f4 in tree_insert (tree=0x7f7aee89ae70, key=0x7f7aeea42a06, key_size=20, custom_arg=0x7f7aee89b3c0) at /data/src/10.1/mysys/tree.c:236
#25 0x00007f7b1e9aad54 in _ma_ck_write_tree (info=0x7f7aeea3e070, key=0x7f7b1f1431f0) at /data/src/10.1/storage/maria/ma_write.c:1656
#26 0x00007f7b1e9a7111 in _ma_ck_write (info=0x7f7aeea3e070, key=0x7f7b1f1431f0) at /data/src/10.1/storage/maria/ma_write.c:435
#27 0x00007f7b1e9cd90f in _ma_ft_store (info=0x7f7aeea3e070, keynr=0, keybuf=0x7f7aeea42a06 "\t136052736?}", <incomplete sequence \350>, wlist=0x7f7aee9e9088, filepos=493540) at /data/src/10.1/storage/maria/ma_ft_update.c:143
#28 0x00007f7b1e9cdf74 in _ma_ft_add (info=0x7f7aeea3e070, keynr=0, keybuf=0x7f7aeea42a06 "\t136052736?}", <incomplete sequence \350>, record=0x7f7aee85a788 <incomplete sequence \375>, pos=493540) at /data/src/10.1/storage/maria/ma_ft_update.c:260
#29 0x00007f7b1e9a651a in maria_write (info=0x7f7aeea3e070, record=0x7f7aee85a788 <incomplete sequence \375>) at /data/src/10.1/storage/maria/ma_write.c:178
#30 0x00007f7b1e92ff3d in ha_maria::write_row (this=0x7f7aee8b5888, buf=0x7f7aee85a788 <incomplete sequence \375>) at /data/src/10.1/storage/maria/ha_maria.cc:1279
#31 0x00007f7b1e44c991 in handler::ha_write_row (this=0x7f7aee8b5888, buf=0x7f7aee85a788 <incomplete sequence \375>) at /data/src/10.1/sql/handler.cc:5941
#32 0x00007f7b1e1e1e09 in write_record (thd=0x7f7af82e7070, table=0x7f7aee8d5270, info=0x7f7aee826820) at /data/src/10.1/sql/sql_insert.cc:1915
#33 0x00007f7b1e1e6a69 in select_insert::send_data (this=0x7f7aee8267e0, values=...) at /data/src/10.1/sql/sql_insert.cc:3711
#34 0x00007f7b1e26e029 in end_send (join=0x7f7aee826880, join_tab=0x7f7aee8298a0, end_of_records=false) at /data/src/10.1/sql/sql_select.cc:19584
#35 0x00007f7b1e26bda2 in evaluate_join_record (join=0x7f7aee826880, join_tab=0x7f7aee829558, error=0) at /data/src/10.1/sql/sql_select.cc:18673
#36 0x00007f7b1e26b848 in sub_select (join=0x7f7aee826880, join_tab=0x7f7aee829558, end_of_records=false) at /data/src/10.1/sql/sql_select.cc:18489
#37 0x00007f7b1e26af17 in do_select (join=0x7f7aee826880, fields=0x7f7aee826c50, table=0x0, procedure=0x0) at /data/src/10.1/sql/sql_select.cc:18105
#38 0x00007f7b1e245896 in JOIN::exec_inner (this=0x7f7aee826880) at /data/src/10.1/sql/sql_select.cc:3261
#39 0x00007f7b1e242b27 in JOIN::exec (this=0x7f7aee826880) at /data/src/10.1/sql/sql_select.cc:2548
#40 0x00007f7b1e2460bd in mysql_select (thd=0x7f7af82e7070, rref_pointer_array=0x7f7af82eb550, tables=0x7f7aee8259b0, wild_num=1, fields=..., conds=0x0, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=3489925888, result=0x7f7aee8267e0, unit=0x7f7af82eaba8, select_lex=0x7f7af82eb2a8) at /data/src/10.1/sql/sql_select.cc:3485
#41 0x00007f7b1e23b93a in handle_select (thd=0x7f7af82e7070, lex=0x7f7af82eaae0, result=0x7f7aee8267e0, setup_tables_done_option=1073741824) at /data/src/10.1/sql/sql_select.cc:388
#42 0x00007f7b1e204849 in mysql_execute_command (thd=0x7f7af82e7070) at /data/src/10.1/sql/sql_parse.cc:4036
#43 0x00007f7b1e20ee4b in mysql_parse (thd=0x7f7af82e7070, rawbuf=0x7f7aee825088 "INSERT INTO `table1_key_pk_parts_2_int_autoinc` SELECT * FROM `table1_key_pk_parts_2_int_autoinc`", length=97, parser_state=0x7f7b1f144630) at /data/src/10.1/sql/sql_parse.cc:7449
#44 0x00007f7b1e1fd885 in dispatch_command (command=COM_QUERY, thd=0x7f7af82e7070, packet=0x7f7b1b7f9071 "INSERT INTO `table1_key_pk_parts_2_int_autoinc` SELECT * FROM `table1_key_pk_parts_2_int_autoinc`", packet_length=97) at /data/src/10.1/sql/sql_parse.cc:1492
#45 0x00007f7b1e1fc60a in do_command (thd=0x7f7af82e7070) at /data/src/10.1/sql/sql_parse.cc:1121
#46 0x00007f7b1e336557 in do_handle_one_connection (thd_arg=0x7f7af82e7070) at /data/src/10.1/sql/sql_connect.cc:1330
#47 0x00007f7b1e3362bb in handle_one_connection (arg=0x7f7af82e7070) at /data/src/10.1/sql/sql_connect.cc:1242
#48 0x00007f7b1d8f5064 in start_thread (arg=0x7f7b1f145b00) at pthread_create.c:309
#49 0x00007f7b1bd1b62d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Comment by Michael Widenius [ 2018-08-30 ]

Unfortunately for this I need a little more information to be able to repeat (especially as stack trace doesn't tell me on which line things crashed)

I assume the import is done to an empty table ?
Can you give me access to what you tried to import

Comment by Benjamin Hodgetts [ 2018-08-30 ]

Sorry Michael, I gave up on Toku long ago and the project this was part of has since been abandoned.

Comment by Elena Stepanova [ 2018-08-30 ]

monty,
The way to reproduce is actually in my previous comment, but they're not very well defined, so, to summarize:

The datadir contains a corrupt table table1_key_pk_parts_2_int_autoinc. The problem to look at in the scope of this report is two-fold:
1) REPAIR TABLE pretends everything is fine, but doesn't repair anything:

MariaDB [test]> check table table1_key_pk_parts_2_int_autoinc;
+----------------------------------------+-------+----------+--------------------------------------------------+
| Table                                  | Op    | Msg_type | Msg_text                                         |
+----------------------------------------+-------+----------+--------------------------------------------------+
| test.table1_key_pk_parts_2_int_autoinc | check | error    | Page at 24576 is not marked for index 2863311518 |
| test.table1_key_pk_parts_2_int_autoinc | check | error    | Page at 8192 is not marked for index 2863311518  |
| test.table1_key_pk_parts_2_int_autoinc | check | error    | Page at 16384 is not marked for index 2863311518 |
| test.table1_key_pk_parts_2_int_autoinc | check | error    | Key in wrong position at page 122880             |
| test.table1_key_pk_parts_2_int_autoinc | check | error    | Corrupt                                          |
+----------------------------------------+-------+----------+--------------------------------------------------+
5 rows in set (0.01 sec)
 
MariaDB [test]> repair table table1_key_pk_parts_2_int_autoinc;
+----------------------------------------+--------+----------+----------+
| Table                                  | Op     | Msg_type | Msg_text |
+----------------------------------------+--------+----------+----------+
| test.table1_key_pk_parts_2_int_autoinc | repair | status   | OK       |
+----------------------------------------+--------+----------+----------+
1 row in set (0.34 sec)
 
MariaDB [test]> check table table1_key_pk_parts_2_int_autoinc;
+----------------------------------------+-------+----------+--------------------------------------------------+
| Table                                  | Op    | Msg_type | Msg_text                                         |
+----------------------------------------+-------+----------+--------------------------------------------------+
| test.table1_key_pk_parts_2_int_autoinc | check | error    | Page at 24576 is not marked for index 2863311518 |
| test.table1_key_pk_parts_2_int_autoinc | check | error    | Page at 8192 is not marked for index 2863311518  |
| test.table1_key_pk_parts_2_int_autoinc | check | error    | Page at 16384 is not marked for index 2863311518 |
| test.table1_key_pk_parts_2_int_autoinc | check | error    | Key in wrong position at page 122880             |
| test.table1_key_pk_parts_2_int_autoinc | check | error    | Corrupt                                          |
+----------------------------------------+-------+----------+--------------------------------------------------+
5 rows in set (0.01 sec)

2) INSERT .. SELECT crashes:

MariaDB [test]> INSERT INTO `table1_key_pk_parts_2_int_autoinc` SELECT * FROM `table1_key_pk_parts_2_int_autoinc`;
ERROR 2013 (HY000): Lost connection to MySQL server during query

10.1 6c588c92a9b

#3  <signal handler called>
#4  0x0000561d48c53e1c in w_search (info=0x7f82d7a1d070, comp_flag=4, key=0x7f8306f47a00, page_pos=40960, father_page=0x0, father_keypos=0x0, insert_last=1 '\001') at /data/src/10.1/storage/maria/ma
_write.c:630
#5  0x0000561d48c53957 in _ma_ck_real_write_btree (info=0x7f82d7a1d070, key=0x7f8306f47a00, root=0x7f8306f46d98, comp_flag=4) at /data/src/10.1/storage/maria/ma_write.c:527
#6  0x0000561d48c541ab in w_search (info=0x7f82d7a1d070, comp_flag=4, key=0x7f8306f47a00, page_pos=196608, father_page=0x0, father_keypos=0x0, insert_last=1 '\001') at /data/src/10.1/storage/maria/m
a_write.c:677
#7  0x0000561d48c53957 in _ma_ck_real_write_btree (info=0x7f82d7a1d070, key=0x7f8306f479e0, root=0x7f8306f473f0, comp_flag=131137) at /data/src/10.1/storage/maria/ma_write.c:527
#8  0x0000561d48c537d1 in _ma_ck_write_btree_with_log (info=0x7f82d7a1d070, key=0x7f8306f479e0, root=0x7f82d7a08320, comp_flag=131137) at /data/src/10.1/storage/maria/ma_write.c:494
#9  0x0000561d48c53622 in _ma_ck_write_btree (info=0x7f82d7a1d070, key=0x7f8306f479e0) at /data/src/10.1/storage/maria/ma_write.c:452
#10 0x0000561d48c57402 in keys_free (key=0x7f82d7b9a118 "\t136052736?}", <incomplete sequence \350>, mode=free_free, param=0x7f82d789b3c0) at /data/src/10.1/storage/maria/ma_write.c:1708
#11 0x0000561d490802a5 in delete_tree_element (tree=0x7f82d789ae70, element=0x7f82d7b9a0f8) at /data/src/10.1/mysys/tree.c:186
#12 0x0000561d4908024a in delete_tree_element (tree=0x7f82d789ae70, element=0x7f82d7b9a210) at /data/src/10.1/mysys/tree.c:184
#13 0x0000561d4908024a in delete_tree_element (tree=0x7f82d789ae70, element=0x7f82d7b9a440) at /data/src/10.1/mysys/tree.c:184
#14 0x0000561d4908024a in delete_tree_element (tree=0x7f82d789ae70, element=0x7f82d7b9a8a0) at /data/src/10.1/mysys/tree.c:184
#15 0x0000561d4908024a in delete_tree_element (tree=0x7f82d789ae70, element=0x7f82d7b9b160) at /data/src/10.1/mysys/tree.c:184
#16 0x0000561d4908024a in delete_tree_element (tree=0x7f82d789ae70, element=0x7f82d7b9c2e0) at /data/src/10.1/mysys/tree.c:184
#17 0x0000561d4908024a in delete_tree_element (tree=0x7f82d789ae70, element=0x7f82d7b9e5e0) at /data/src/10.1/mysys/tree.c:184
#18 0x0000561d4908024a in delete_tree_element (tree=0x7f82d789ae70, element=0x7f82d7ba2be0) at /data/src/10.1/mysys/tree.c:184
#19 0x0000561d4908024a in delete_tree_element (tree=0x7f82d789ae70, element=0x7f82d7bab7e0) at /data/src/10.1/mysys/tree.c:184
#20 0x0000561d4908024a in delete_tree_element (tree=0x7f82d789ae70, element=0x7f82d7b9a168) at /data/src/10.1/mysys/tree.c:184
#21 0x0000561d4908024a in delete_tree_element (tree=0x7f82d789ae70, element=0x7f82d7b9a088) at /data/src/10.1/mysys/tree.c:184
#22 0x0000561d49080146 in free_tree (tree=0x7f82d789ae70, free_flags=2) at /data/src/10.1/mysys/tree.c:154
#23 0x0000561d49080210 in reset_tree (tree=0x7f82d789ae70) at /data/src/10.1/mysys/tree.c:176
#24 0x0000561d4908045b in tree_insert (tree=0x7f82d789ae70, key=0x7f82d7a21a06, key_size=20, custom_arg=0x7f82d789b3c0) at /data/src/10.1/mysys/tree.c:236
#25 0x0000561d48c571aa in _ma_ck_write_tree (info=0x7f82d7a1d070, key=0x7f8306f481f0) at /data/src/10.1/storage/maria/ma_write.c:1656
#26 0x0000561d48c53567 in _ma_ck_write (info=0x7f82d7a1d070, key=0x7f8306f481f0) at /data/src/10.1/storage/maria/ma_write.c:435
#27 0x0000561d48c79d65 in _ma_ft_store (info=0x7f82d7a1d070, keynr=0, keybuf=0x7f82d7a21a06 "\t136052736?}", <incomplete sequence \350>, wlist=0x7f82d79e9088, filepos=878560) at /data/src/10.1/storage/maria/ma_ft_update.c:143
#28 0x0000561d48c7a3ca in _ma_ft_add (info=0x7f82d7a1d070, keynr=0, keybuf=0x7f82d7a21a06 "\t136052736?}", <incomplete sequence \350>, record=0x7f82d785ac88 <incomplete sequence \375>, pos=878560) at /data/src/10.1/storage/maria/ma_ft_update.c:260
#29 0x0000561d48c52970 in maria_write (info=0x7f82d7a1d070, record=0x7f82d785ac88 <incomplete sequence \375>) at /data/src/10.1/storage/maria/ma_write.c:178
#30 0x0000561d48bdb1db in ha_maria::write_row (this=0x7f82d78b6088, buf=0x7f82d785ac88 <incomplete sequence \375>) at /data/src/10.1/storage/maria/ha_maria.cc:1279
#31 0x0000561d48a223cd in handler::ha_write_row (this=0x7f82d78b6088, buf=0x7f82d785ac88 <incomplete sequence \375>) at /data/src/10.1/sql/handler.cc:5963
#32 0x0000561d487b624b in write_record (thd=0x7f82e1351070, table=0x7f82d78d5270, info=0x7f82d7826080) at /data/src/10.1/sql/sql_insert.cc:1915
#33 0x0000561d487baeab in select_insert::send_data (this=0x7f82d7826040, values=...) at /data/src/10.1/sql/sql_insert.cc:3711
#34 0x0000561d48842757 in end_send (join=0x7f82d78260e0, join_tab=0x7f82d7829100, end_of_records=false) at /data/src/10.1/sql/sql_select.cc:19605
#35 0x0000561d488404d0 in evaluate_join_record (join=0x7f82d78260e0, join_tab=0x7f82d7828db8, error=0) at /data/src/10.1/sql/sql_select.cc:18694
#36 0x0000561d4883ff76 in sub_select (join=0x7f82d78260e0, join_tab=0x7f82d7828db8, end_of_records=false) at /data/src/10.1/sql/sql_select.cc:18510
#37 0x0000561d4883f645 in do_select (join=0x7f82d78260e0, fields=0x7f82d78264b0, table=0x0, procedure=0x0) at /data/src/10.1/sql/sql_select.cc:18126
#38 0x0000561d48819ea8 in JOIN::exec_inner (this=0x7f82d78260e0) at /data/src/10.1/sql/sql_select.cc:3275
#39 0x0000561d48817139 in JOIN::exec (this=0x7f82d78260e0) at /data/src/10.1/sql/sql_select.cc:2562
#40 0x0000561d4881a6cf in mysql_select (thd=0x7f82e1351070, rref_pointer_array=0x7f82e1355560, tables=0x7f82d78259b0, wild_num=1, fields=..., conds=0x0, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=3489925888, result=0x7f82d7826040, unit=0x7f82e1354bb8, select_lex=0x7f82e13552b8) at /data/src/10.1/sql/sql_select.cc:3499
#41 0x0000561d4880fd88 in handle_select (thd=0x7f82e1351070, lex=0x7f82e1354af0, result=0x7f82d7826040, setup_tables_done_option=1073741824) at /data/src/10.1/sql/sql_select.cc:388
#42 0x0000561d487d8cc1 in mysql_execute_command (thd=0x7f82e1351070) at /data/src/10.1/sql/sql_parse.cc:4039
#43 0x0000561d487e3368 in mysql_parse (thd=0x7f82e1351070, rawbuf=0x7f82d7825088 "INSERT INTO `table1_key_pk_parts_2_int_autoinc` SELECT * FROM `table1_key_pk_parts_2_int_autoinc`", length=97, parser_state=0x7f8306f49630) at /data/src/10.1/sql/sql_parse.cc:7463
#44 0x0000561d487d1cfd in dispatch_command (command=COM_QUERY, thd=0x7f82e1351070, packet=0x7f82e12b6071 "INSERT INTO `table1_key_pk_parts_2_int_autoinc` SELECT * FROM `table1_key_pk_parts_2_int_autoinc`", packet_length=97) at /data/src/10.1/sql/sql_parse.cc:1495
#45 0x0000561d487d0a82 in do_command (thd=0x7f82e1351070) at /data/src/10.1/sql/sql_parse.cc:1124
#46 0x0000561d4890aecd in do_handle_one_connection (thd_arg=0x7f82e1351070) at /data/src/10.1/sql/sql_connect.cc:1330

Generated at Thu Feb 08 07:32:24 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.