[MDEV-9233] Copying MySQL 5.5 data directory to 10.0 with partition tables crashes on insert Created: 2015-12-02  Updated: 2016-01-26  Resolved: 2015-12-14

Status: Closed
Project: MariaDB Server
Component/s: Partitioning, Storage Engine - InnoDB
Affects Version/s: 10.0.19, 5.5, 10.0, 10.1
Fix Version/s: 5.5.47, 10.0.23, 10.1.10

Type: Bug Priority: Major
Reporter: Kyle Joiner (Inactive) Assignee: Jan Lindström (Inactive)
Resolution: Fixed Votes: 0
Labels: partioning
Environment:

Linux

Attachments: File mdev9233_data.tar.gz    
Issue Links:
Relates
relates to MDEV-9471 Server crashes or returns an error wh... Closed
Sprint: 10.0.23

 Description   

There is a problem when performing a binary copy of the data directory from a 5.5.28 windows machine to Linux machine running 10.0.19.

MariaDB complains of partition names and the crashes on a specific insert.

They have generated a core file and the error log that are in the 20152012.zip file. A dump of the schema definitions is in schema_dump_20151125.7z and the crash can be seen in Linux_Slave.out.

Here is the stack trace when I reproduce it:

Thread pointer: 0x0x7f00d83a4008
 
Attempting backtrace. You can use the following information to find out
 
where mysqld died. If you see no messages after this, something went
 
terribly wrong...
 
stack_bottom = 0x7f00f2223e50 thread_stack 0x48000
 
mysys/stacktrace.c:247(my_print_stacktrace)[0xbca37e]
 
sql/signal_handler.cc:153(handle_fatal_signal)[0x71fe5c]
 
/lib/x86_64-linux-gnu/libpthread.so.0(+0x10340)[0x7f00f1e82340]
 
handler/ha_innodb.cc:7487(ha_innobase::write_row(unsigned char*))[0x8b2336]
 
sql/handler.cc:5963(handler::ha_write_row(unsigned char*))[0x728113]
 
sql/ha_partition.cc:4166(ha_partition::write_row(unsigned char*))[0xb59621]
 
sql/handler.cc:5963(handler::ha_write_row(unsigned char*))[0x728113]
 
sql/sql_insert.cc:1848(write_record(THD*, TABLE*, st_copy_info*))[0x57d391]
 
sql/sql_insert.cc:965(mysql_insert(THD*, TABLE_LIST*, List<Item>&, List<List<Item> >&, List<Item>&, List<Item>&, enum_duplicates, bool))[0x58591f]
 
sql/sql_parse.cc:3457(mysql_execute_command(THD*))[0x59e47e]
 
sql/sql_parse.cc:6529(mysql_parse(THD*, char*, unsigned int, Parser_state*))[0x59f866]
 
sql/sql_parse.cc:1310(dispatch_command(enum_server_command, THD*, char*, unsigned int))[0x5a1797]
 
sql/sql_parse.cc:999(do_command(THD*))[0x5a1ed9]
 
sql/sql_connect.cc:1375(do_handle_one_connection(THD*))[0x66bfd4]
 
sql/sql_connect.cc:1291(handle_one_connection)[0x66c112]
 
/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182)[0x7f00f1e7a182]
 
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f00f0d5647d]



 Comments   
Comment by Sergei Petrunia [ 2015-12-03 ]

Downloaded the binary (MariaDB-10.0.21-centos6-x86_64-server.rpm ) and loaded the core file. "thread apply where" gives this: https://gist.github.com/spetrunia/84990c3e9543ff02160a

Comment by Sergei Petrunia [ 2015-12-03 ]

Tried loading core file on "CentOS release 6.5" and I got this stack trace: https://gist.github.com/spetrunia/abf60446cecf9ec08dfb . Looks more realistic but stacks for some threads still look weird. I am not sure if I am using the right OS/ minor release.

Comment by Elena Stepanova [ 2015-12-04 ]

MySQL and upgrade are not important; reproducible upon moving a binary datadir of MariaDB 5.5 from Windows to Linux to the same or higher version of MariaDB.

Stack trace from 10.0 commit ba8e630d97af2b2ed3e527070f1cab05571911fd

 
#3  <signal handler called>
 
#4  0x0000000000b500cd in ha_innobase::write_row (this=0x7fcf85cab088, record=0x7fcf8db4f098 "\377%H[\321S\022") at 10.0/storage/xtradb/handler/ha_innodb.cc:7528
 
#5  0x000000000086bacb in handler::ha_write_row (this=0x7fcf85cab088, buf=0x7fcf8db4f098 "\377%H[\321S\022") at 10.0/sql/handler.cc:6004
 
#6  0x0000000000e36722 in ha_partition::write_row (this=0x7fcf85caa888, buf=0x7fcf8db4f098 "\377%H[\321S\022") at 10.0/sql/ha_partition.cc:4165
 
#7  0x000000000086bacb in handler::ha_write_row (this=0x7fcf85caa888, buf=0x7fcf8db4f098 "\377%H[\321S\022") at 10.0/sql/handler.cc:6004
 
#8  0x0000000000642b05 in write_record (thd=0x7fcf94b29070, table=0x7fcf85cd8c70, info=0x7fcfb20ba8f0) at 10.0/sql/sql_insert.cc:1847
 
#9  0x00000000006406fd in mysql_insert (thd=0x7fcf94b29070, table_list=0x7fcf85c22190, fields=..., values_list=..., update_fields=..., update_values=..., duplic=DUP_ERROR, ignore=false) at 10.0/sql/sql_insert.cc:964
 
#10 0x0000000000660aa5 in mysql_execute_command (thd=0x7fcf94b29070) at 10.0/sql/sql_parse.cc:3448
 
#11 0x00000000006693cd in mysql_parse (thd=0x7fcf94b29070, rawbuf=0x7fcf85c22088 "INSERT INTO t1 (creation_time) VALUES (NOW())", length=45, parser_state=0x7fcfb20bb5f0) at 10.0/sql/sql_parse.cc:6564
 
#12 0x000000000065b8f5 in dispatch_command (command=COM_QUERY, thd=0x7fcf94b29070, packet=0x7fcf8dbd8071 "INSERT INTO t1 (creation_time) VALUES (NOW())", packet_length=45) at 10.0/sql/sql_parse.cc:1307
 
#13 0x000000000065ab94 in do_command (thd=0x7fcf94b29070) at 10.0/sql/sql_parse.cc:998
 
#14 0x0000000000785580 in do_handle_one_connection (thd_arg=0x7fcf94b29070) at 10.0/sql/sql_connect.cc:1377
 
#15 0x00000000007852f2 in handle_one_connection (arg=0x7fcf94b29070) at 10.0/sql/sql_connect.cc:1292
 
#16 0x00007fcfb1253e9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
 
#17 0x00007fcfb0143cbd in clone () from /lib/x86_64-linux-gnu/libc.so.6
 
#18 0x0000000000000000 in ?? ()

Stack trace from 5.5 commit d87bc55b05046d6484659286fa831ff10dbd3af9

 
#3  <signal handler called>
 
#4  0x0000000000ab5aa1 in ha_innobase::write_row (this=0x7fa6a70b3078, record=0x7fa6a7074688 "\377\245Y[\321S\022") at 5.5/storage/xtradb/handler/ha_innodb.cc:6193
 
#5  0x00000000007dfc19 in handler::ha_write_row (this=0x7fa6a70b3078, buf=0x7fa6a7074688 "\377\245Y[\321S\022") at 5.5/sql/handler.cc:5203
 
#6  0x0000000000cbc498 in ha_partition::write_row (this=0x7fa6a70b6d78, buf=0x7fa6a7074688 "\377\245Y[\321S\022") at 5.5/sql/ha_partition.cc:3596
 
#7  0x00000000007dfc19 in handler::ha_write_row (this=0x7fa6a70b6d78, buf=0x7fa6a7074688 "\377\245Y[\321S\022") at 5.5/sql/handler.cc:5203
 
#8  0x00000000005fa1c6 in write_record (thd=0x7fa6ad67d060, table=0x7fa6a7023460, info=0x7fa6d16ddab0) at 5.5/sql/sql_insert.cc:1843
 
#9  0x00000000005f7e08 in mysql_insert (thd=0x7fa6ad67d060, table_list=0x7fa6a704c180, fields=..., values_list=..., update_fields=..., update_values=..., duplic=DUP_ERROR, ignore=false) at 5.5/sql/sql_insert.cc:986
 
#10 0x0000000000616a98 in mysql_execute_command (thd=0x7fa6ad67d060) at 5.5/sql/sql_parse.cc:2974
 
#11 0x000000000061ee04 in mysql_parse (thd=0x7fa6ad67d060, rawbuf=0x7fa6a704c078 "INSERT INTO t1 (creation_time) VALUES (NOW())", length=45, parser_state=0x7fa6d16de610) at 5.5/sql/sql_parse.cc:5914
 
#12 0x00000000006121bf in dispatch_command (command=COM_QUERY, thd=0x7fa6ad67d060, packet=0x7fa6ad791061 "INSERT INTO t1 (creation_time) VALUES (NOW())", packet_length=45) at 5.5/sql/sql_parse.cc:1079
 
#13 0x0000000000611310 in do_command (thd=0x7fa6ad67d060) at 5.5/sql/sql_parse.cc:793
 
#14 0x000000000071f011 in do_handle_one_connection (thd_arg=0x7fa6ad67d060) at 5.5/sql/sql_connect.cc:1269
 
#15 0x000000000071ed9e in handle_one_connection (arg=0x7fa6ad67d060) at 5.5/sql/sql_connect.cc:1185
 
#16 0x00007fa6d068fe9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
 
#17 0x00007fa6cfdbccbd in clone () from /lib/x86_64-linux-gnu/libc.so.6

To reproduce:

  • unpack the attached mdev9233_data.tar.gz and use it as the datadir;
  • start server with --lower-case-table-names=1, otherwise defaults;
  • run INSERT INTO t1 (creation_time) VALUES (NOW());
  • observe the crash

Additionally, 5.5 goes down with an assertion failure upon FLUSH TABLES performed on the same datadir. It does not happen with 10.0.

InnoDB: Assertion failure in thread 140107472586496 in file dict0dict.c line 363
 
InnoDB: Failing assertion: table->n_mysql_handles_opened > 0
 
 
 
#5  0x00007f8878b08b8b in abort () from /lib/x86_64-linux-gnu/libc.so.6
 
#6  0x0000000000bb84ce in dict_table_decrement_handle_count (table=0x7f88500c95f8, dict_locked=0) at 5.5/storage/xtradb/dict/dict0dict.c:363
 
#7  0x0000000000ae7c49 in row_prebuilt_free (prebuilt=0x7f88500eae78, dict_locked=0) at 5.5/storage/xtradb/row/row0mysql.c:848
 
#8  0x0000000000ab395d in ha_innobase::close (this=0x7f88500b5078) at 5.5/storage/xtradb/handler/ha_innodb.cc:5058
 
#9  0x00000000007da404 in handler::ha_close (this=0x7f88500b5078) at 5.5/sql/handler.cc:2327
 
#10 0x0000000000cbba9c in ha_partition::close (this=0x7f88500b8d78) at 5.5/sql/ha_partition.cc:3167
 
#11 0x00000000007da404 in handler::ha_close (this=0x7f88500b8d78) at 5.5/sql/handler.cc:2327
 
#12 0x00000000006e4a6c in closefrm (table=0x7f8850024860, free_share=true) at 5.5/sql/table.cc:2769
 
#13 0x00000000005b0f43 in intern_close_table (table=0x7f8850024860) at 5.5/sql/sql_base.cc:929
 
#14 0x00000000005b0fb1 in free_cache_entry (table=0x7f8850024860) at 5.5/sql/sql_base.cc:952
 
#15 0x00000000005b132c in close_cached_tables (thd=0x7f88566e9060, tables=0x0, wait_for_refresh=true, timeout=31536000) at 5.5/sql/sql_base.cc:1059
 
#16 0x000000000074b79f in reload_acl_and_cache (thd=0x7f88566e9060, options=4, tables=0x0, write_to_binlog=0x7f887a4e3eb0) at 5.5/sql/sql_reload.cc:272
 
#17 0x00000000006193a9 in mysql_execute_command (thd=0x7f88566e9060) at 5.5/sql/sql_parse.cc:3724
 
#18 0x000000000061ee04 in mysql_parse (thd=0x7f88566e9060, rawbuf=0x7f885004e078 "flush tables", length=12, parser_state=0x7f887a4e4610) at 5.5/sql/sql_parse.cc:5914
 
#19 0x00000000006121bf in dispatch_command (command=COM_QUERY, thd=0x7f88566e9060, packet=0x7f8856791061 "flush tables", packet_length=12) at 5.5/sql/sql_parse.cc:1079
 
#20 0x0000000000611310 in do_command (thd=0x7f88566e9060) at 5.5/sql/sql_parse.cc:793
 
#21 0x000000000071f011 in do_handle_one_connection (thd_arg=0x7f88566e9060) at 5.5/sql/sql_connect.cc:1269
 
#22 0x000000000071ed9e in handle_one_connection (arg=0x7f88566e9060) at 5.5/sql/sql_connect.cc:1185
 
#23 0x00007f8879495e9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
 
#24 0x00007f8878bc2cbd in clone () from /lib/x86_64-linux-gnu/libc.so.6

The datadir only contains one custom table:

DROP TABLE IF EXISTS t1;
 
CREATE TABLE t1 (
 
  `creation_time` datetime NOT NULL
 
) ENGINE=InnoDB 
PARTITION BY RANGE (TO_DAYS(creation_time))
 
(PARTITION LastPart VALUES LESS THAN MAXVALUE);

Comment by Jan Lindström (Inactive) [ 2015-12-04 ]

For 5.5:

commit 082b859d0d8afbe49be770c2f18cf39bbd939aa8
Author: Jan Lindström <jan.lindstrom@mariadb.com>
Date: Fri Dec 4 14:24:03 2015 +0200

MDEV-9233: Copying MySQL 5.5 data directory to 10.0 with partition tables crashes on insert

Analysis: There were two problems. (1) if partition table was
created using lower_case_tables = 1 on windows we did find the
correct table but we did not set share->ib_table correctly.
(2) we did open table on dictionary but did not increase
mysql_open_tables.

Fix: In xtradb allow access to tables with incorrect
lower case names (warning is printed to error log). If
table is opened increase mysql_open_tables count to avoid
crash on flush tables.

Comment by Jan Lindström (Inactive) [ 2015-12-04 ]

Need to retest 10.0 when 5.5 -> 10.0 merge is done.

Generated at Thu Feb 08 07:33:08 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.