There is a problem when performing a binary copy of the data directory from a 5.5.28 windows machine to Linux machine running 10.0.19.
MariaDB complains of partition names and the crashes on a specific insert.
They have generated a core file and the error log that are in the 20152012.zip file. A dump of the schema definitions is in schema_dump_20151125.7z and the crash can be seen in Linux_Slave.out.
Here is the stack trace when I reproduce it:
Thread pointer: 0x0x7f00d83a4008
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
Sergei Petrunia
added a comment - Downloaded the binary (MariaDB-10.0.21-centos6-x86_64-server.rpm ) and loaded the core file. "thread apply where" gives this: https://gist.github.com/spetrunia/84990c3e9543ff02160a
Tried loading core file on "CentOS release 6.5" and I got this stack trace: https://gist.github.com/spetrunia/abf60446cecf9ec08dfb . Looks more realistic but stacks for some threads still look weird. I am not sure if I am using the right OS/ minor release.
Sergei Petrunia
added a comment - Tried loading core file on "CentOS release 6.5" and I got this stack trace: https://gist.github.com/spetrunia/abf60446cecf9ec08dfb . Looks more realistic but stacks for some threads still look weird. I am not sure if I am using the right OS/ minor release.
MySQL and upgrade are not important; reproducible upon moving a binary datadir of MariaDB 5.5 from Windows to Linux to the same or higher version of MariaDB.
Stack trace from 10.0 commit ba8e630d97af2b2ed3e527070f1cab05571911fd
#3 <signal handler called>
#4 0x0000000000b500cd in ha_innobase::write_row (this=0x7fcf85cab088, record=0x7fcf8db4f098 "\377%H[\321S\022") at 10.0/storage/xtradb/handler/ha_innodb.cc:7528
#5 0x000000000086bacb in handler::ha_write_row (this=0x7fcf85cab088, buf=0x7fcf8db4f098 "\377%H[\321S\022") at 10.0/sql/handler.cc:6004
#6 0x0000000000e36722 in ha_partition::write_row (this=0x7fcf85caa888, buf=0x7fcf8db4f098 "\377%H[\321S\022") at 10.0/sql/ha_partition.cc:4165
#7 0x000000000086bacb in handler::ha_write_row (this=0x7fcf85caa888, buf=0x7fcf8db4f098 "\377%H[\321S\022") at 10.0/sql/handler.cc:6004
#8 0x0000000000642b05 in write_record (thd=0x7fcf94b29070, table=0x7fcf85cd8c70, info=0x7fcfb20ba8f0) at 10.0/sql/sql_insert.cc:1847
#9 0x00000000006406fd in mysql_insert (thd=0x7fcf94b29070, table_list=0x7fcf85c22190, fields=..., values_list=..., update_fields=..., update_values=..., duplic=DUP_ERROR, ignore=false) at 10.0/sql/sql_insert.cc:964
#10 0x0000000000660aa5 in mysql_execute_command (thd=0x7fcf94b29070) at 10.0/sql/sql_parse.cc:3448
#11 0x00000000006693cd in mysql_parse (thd=0x7fcf94b29070, rawbuf=0x7fcf85c22088 "INSERT INTO t1 (creation_time) VALUES (NOW())", length=45, parser_state=0x7fcfb20bb5f0) at 10.0/sql/sql_parse.cc:6564
#12 0x000000000065b8f5 in dispatch_command (command=COM_QUERY, thd=0x7fcf94b29070, packet=0x7fcf8dbd8071 "INSERT INTO t1 (creation_time) VALUES (NOW())", packet_length=45) at 10.0/sql/sql_parse.cc:1307
#13 0x000000000065ab94 in do_command (thd=0x7fcf94b29070) at 10.0/sql/sql_parse.cc:998
#14 0x0000000000785580 in do_handle_one_connection (thd_arg=0x7fcf94b29070) at 10.0/sql/sql_connect.cc:1377
#15 0x00000000007852f2 in handle_one_connection (arg=0x7fcf94b29070) at 10.0/sql/sql_connect.cc:1292
#16 0x00007fcfb1253e9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#17 0x00007fcfb0143cbd in clone () from /lib/x86_64-linux-gnu/libc.so.6
#18 0x0000000000000000 in ?? ()
Stack trace from 5.5 commit d87bc55b05046d6484659286fa831ff10dbd3af9
#3 <signal handler called>
#4 0x0000000000ab5aa1 in ha_innobase::write_row (this=0x7fa6a70b3078, record=0x7fa6a7074688 "\377\245Y[\321S\022") at 5.5/storage/xtradb/handler/ha_innodb.cc:6193
#5 0x00000000007dfc19 in handler::ha_write_row (this=0x7fa6a70b3078, buf=0x7fa6a7074688 "\377\245Y[\321S\022") at 5.5/sql/handler.cc:5203
#6 0x0000000000cbc498 in ha_partition::write_row (this=0x7fa6a70b6d78, buf=0x7fa6a7074688 "\377\245Y[\321S\022") at 5.5/sql/ha_partition.cc:3596
#7 0x00000000007dfc19 in handler::ha_write_row (this=0x7fa6a70b6d78, buf=0x7fa6a7074688 "\377\245Y[\321S\022") at 5.5/sql/handler.cc:5203
#8 0x00000000005fa1c6 in write_record (thd=0x7fa6ad67d060, table=0x7fa6a7023460, info=0x7fa6d16ddab0) at 5.5/sql/sql_insert.cc:1843
#9 0x00000000005f7e08 in mysql_insert (thd=0x7fa6ad67d060, table_list=0x7fa6a704c180, fields=..., values_list=..., update_fields=..., update_values=..., duplic=DUP_ERROR, ignore=false) at 5.5/sql/sql_insert.cc:986
#10 0x0000000000616a98 in mysql_execute_command (thd=0x7fa6ad67d060) at 5.5/sql/sql_parse.cc:2974
#11 0x000000000061ee04 in mysql_parse (thd=0x7fa6ad67d060, rawbuf=0x7fa6a704c078 "INSERT INTO t1 (creation_time) VALUES (NOW())", length=45, parser_state=0x7fa6d16de610) at 5.5/sql/sql_parse.cc:5914
#12 0x00000000006121bf in dispatch_command (command=COM_QUERY, thd=0x7fa6ad67d060, packet=0x7fa6ad791061 "INSERT INTO t1 (creation_time) VALUES (NOW())", packet_length=45) at 5.5/sql/sql_parse.cc:1079
#13 0x0000000000611310 in do_command (thd=0x7fa6ad67d060) at 5.5/sql/sql_parse.cc:793
#14 0x000000000071f011 in do_handle_one_connection (thd_arg=0x7fa6ad67d060) at 5.5/sql/sql_connect.cc:1269
#15 0x000000000071ed9e in handle_one_connection (arg=0x7fa6ad67d060) at 5.5/sql/sql_connect.cc:1185
#16 0x00007fa6d068fe9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#17 0x00007fa6cfdbccbd in clone () from /lib/x86_64-linux-gnu/libc.so.6
To reproduce:
unpack the attached mdev9233_data.tar.gz and use it as the datadir;
start server with --lower-case-table-names=1, otherwise defaults;
run INSERT INTO t1 (creation_time) VALUES (NOW());
observe the crash
Additionally, 5.5 goes down with an assertion failure upon FLUSH TABLES performed on the same datadir. It does not happen with 10.0.
InnoDB: Assertion failure in thread 140107472586496 in file dict0dict.c line 363
#5 0x00007f8878b08b8b in abort () from /lib/x86_64-linux-gnu/libc.so.6
#6 0x0000000000bb84ce in dict_table_decrement_handle_count (table=0x7f88500c95f8, dict_locked=0) at 5.5/storage/xtradb/dict/dict0dict.c:363
#7 0x0000000000ae7c49 in row_prebuilt_free (prebuilt=0x7f88500eae78, dict_locked=0) at 5.5/storage/xtradb/row/row0mysql.c:848
#8 0x0000000000ab395d in ha_innobase::close (this=0x7f88500b5078) at 5.5/storage/xtradb/handler/ha_innodb.cc:5058
#9 0x00000000007da404 in handler::ha_close (this=0x7f88500b5078) at 5.5/sql/handler.cc:2327
#10 0x0000000000cbba9c in ha_partition::close (this=0x7f88500b8d78) at 5.5/sql/ha_partition.cc:3167
#11 0x00000000007da404 in handler::ha_close (this=0x7f88500b8d78) at 5.5/sql/handler.cc:2327
#12 0x00000000006e4a6c in closefrm (table=0x7f8850024860, free_share=true) at 5.5/sql/table.cc:2769
#13 0x00000000005b0f43 in intern_close_table (table=0x7f8850024860) at 5.5/sql/sql_base.cc:929
#14 0x00000000005b0fb1 in free_cache_entry (table=0x7f8850024860) at 5.5/sql/sql_base.cc:952
#15 0x00000000005b132c in close_cached_tables (thd=0x7f88566e9060, tables=0x0, wait_for_refresh=true, timeout=31536000) at 5.5/sql/sql_base.cc:1059
#16 0x000000000074b79f in reload_acl_and_cache (thd=0x7f88566e9060, options=4, tables=0x0, write_to_binlog=0x7f887a4e3eb0) at 5.5/sql/sql_reload.cc:272
#17 0x00000000006193a9 in mysql_execute_command (thd=0x7f88566e9060) at 5.5/sql/sql_parse.cc:3724
#18 0x000000000061ee04 in mysql_parse (thd=0x7f88566e9060, rawbuf=0x7f885004e078 "flush tables", length=12, parser_state=0x7f887a4e4610) at 5.5/sql/sql_parse.cc:5914
#19 0x00000000006121bf in dispatch_command (command=COM_QUERY, thd=0x7f88566e9060, packet=0x7f8856791061 "flush tables", packet_length=12) at 5.5/sql/sql_parse.cc:1079
#20 0x0000000000611310 in do_command (thd=0x7f88566e9060) at 5.5/sql/sql_parse.cc:793
#21 0x000000000071f011 in do_handle_one_connection (thd_arg=0x7f88566e9060) at 5.5/sql/sql_connect.cc:1269
#22 0x000000000071ed9e in handle_one_connection (arg=0x7f88566e9060) at 5.5/sql/sql_connect.cc:1185
#23 0x00007f8879495e9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#24 0x00007f8878bc2cbd in clone () from /lib/x86_64-linux-gnu/libc.so.6
The datadir only contains one custom table:
DROP TABLE IF EXISTS t1;
CREATE TABLE t1 (
`creation_time` datetime NOT NULL
) ENGINE=InnoDB
PARTITION BY RANGE (TO_DAYS(creation_time))
(PARTITION LastPart VALUES LESS THAN MAXVALUE);
Elena Stepanova
added a comment - - edited MySQL and upgrade are not important; reproducible upon moving a binary datadir of MariaDB 5.5 from Windows to Linux to the same or higher version of MariaDB.
Stack trace from 10.0 commit ba8e630d97af2b2ed3e527070f1cab05571911fd
#3 <signal handler called>
#4 0x0000000000b500cd in ha_innobase::write_row (this=0x7fcf85cab088, record=0x7fcf8db4f098 "\377%H[\321S\022") at 10.0/storage/xtradb/handler/ha_innodb.cc:7528
#5 0x000000000086bacb in handler::ha_write_row (this=0x7fcf85cab088, buf=0x7fcf8db4f098 "\377%H[\321S\022") at 10.0/sql/handler.cc:6004
#6 0x0000000000e36722 in ha_partition::write_row (this=0x7fcf85caa888, buf=0x7fcf8db4f098 "\377%H[\321S\022") at 10.0/sql/ha_partition.cc:4165
#7 0x000000000086bacb in handler::ha_write_row (this=0x7fcf85caa888, buf=0x7fcf8db4f098 "\377%H[\321S\022") at 10.0/sql/handler.cc:6004
#8 0x0000000000642b05 in write_record (thd=0x7fcf94b29070, table=0x7fcf85cd8c70, info=0x7fcfb20ba8f0) at 10.0/sql/sql_insert.cc:1847
#9 0x00000000006406fd in mysql_insert (thd=0x7fcf94b29070, table_list=0x7fcf85c22190, fields=..., values_list=..., update_fields=..., update_values=..., duplic=DUP_ERROR, ignore=false) at 10.0/sql/sql_insert.cc:964
#10 0x0000000000660aa5 in mysql_execute_command (thd=0x7fcf94b29070) at 10.0/sql/sql_parse.cc:3448
#11 0x00000000006693cd in mysql_parse (thd=0x7fcf94b29070, rawbuf=0x7fcf85c22088 "INSERT INTO t1 (creation_time) VALUES (NOW())", length=45, parser_state=0x7fcfb20bb5f0) at 10.0/sql/sql_parse.cc:6564
#12 0x000000000065b8f5 in dispatch_command (command=COM_QUERY, thd=0x7fcf94b29070, packet=0x7fcf8dbd8071 "INSERT INTO t1 (creation_time) VALUES (NOW())", packet_length=45) at 10.0/sql/sql_parse.cc:1307
#13 0x000000000065ab94 in do_command (thd=0x7fcf94b29070) at 10.0/sql/sql_parse.cc:998
#14 0x0000000000785580 in do_handle_one_connection (thd_arg=0x7fcf94b29070) at 10.0/sql/sql_connect.cc:1377
#15 0x00000000007852f2 in handle_one_connection (arg=0x7fcf94b29070) at 10.0/sql/sql_connect.cc:1292
#16 0x00007fcfb1253e9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#17 0x00007fcfb0143cbd in clone () from /lib/x86_64-linux-gnu/libc.so.6
#18 0x0000000000000000 in ?? ()
Stack trace from 5.5 commit d87bc55b05046d6484659286fa831ff10dbd3af9
#3 <signal handler called>
#4 0x0000000000ab5aa1 in ha_innobase::write_row (this=0x7fa6a70b3078, record=0x7fa6a7074688 "\377\245Y[\321S\022") at 5.5/storage/xtradb/handler/ha_innodb.cc:6193
#5 0x00000000007dfc19 in handler::ha_write_row (this=0x7fa6a70b3078, buf=0x7fa6a7074688 "\377\245Y[\321S\022") at 5.5/sql/handler.cc:5203
#6 0x0000000000cbc498 in ha_partition::write_row (this=0x7fa6a70b6d78, buf=0x7fa6a7074688 "\377\245Y[\321S\022") at 5.5/sql/ha_partition.cc:3596
#7 0x00000000007dfc19 in handler::ha_write_row (this=0x7fa6a70b6d78, buf=0x7fa6a7074688 "\377\245Y[\321S\022") at 5.5/sql/handler.cc:5203
#8 0x00000000005fa1c6 in write_record (thd=0x7fa6ad67d060, table=0x7fa6a7023460, info=0x7fa6d16ddab0) at 5.5/sql/sql_insert.cc:1843
#9 0x00000000005f7e08 in mysql_insert (thd=0x7fa6ad67d060, table_list=0x7fa6a704c180, fields=..., values_list=..., update_fields=..., update_values=..., duplic=DUP_ERROR, ignore=false) at 5.5/sql/sql_insert.cc:986
#10 0x0000000000616a98 in mysql_execute_command (thd=0x7fa6ad67d060) at 5.5/sql/sql_parse.cc:2974
#11 0x000000000061ee04 in mysql_parse (thd=0x7fa6ad67d060, rawbuf=0x7fa6a704c078 "INSERT INTO t1 (creation_time) VALUES (NOW())", length=45, parser_state=0x7fa6d16de610) at 5.5/sql/sql_parse.cc:5914
#12 0x00000000006121bf in dispatch_command (command=COM_QUERY, thd=0x7fa6ad67d060, packet=0x7fa6ad791061 "INSERT INTO t1 (creation_time) VALUES (NOW())", packet_length=45) at 5.5/sql/sql_parse.cc:1079
#13 0x0000000000611310 in do_command (thd=0x7fa6ad67d060) at 5.5/sql/sql_parse.cc:793
#14 0x000000000071f011 in do_handle_one_connection (thd_arg=0x7fa6ad67d060) at 5.5/sql/sql_connect.cc:1269
#15 0x000000000071ed9e in handle_one_connection (arg=0x7fa6ad67d060) at 5.5/sql/sql_connect.cc:1185
#16 0x00007fa6d068fe9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#17 0x00007fa6cfdbccbd in clone () from /lib/x86_64-linux-gnu/libc.so.6
To reproduce:
unpack the attached mdev9233_data.tar.gz and use it as the datadir;
start server with --lower-case-table-names=1 , otherwise defaults;
run INSERT INTO t1 (creation_time) VALUES (NOW());
observe the crash
Additionally, 5.5 goes down with an assertion failure upon FLUSH TABLES performed on the same datadir. It does not happen with 10.0.
InnoDB: Assertion failure in thread 140107472586496 in file dict0dict.c line 363
InnoDB: Failing assertion: table->n_mysql_handles_opened > 0
#5 0x00007f8878b08b8b in abort () from /lib/x86_64-linux-gnu/libc.so.6
#6 0x0000000000bb84ce in dict_table_decrement_handle_count (table=0x7f88500c95f8, dict_locked=0) at 5.5/storage/xtradb/dict/dict0dict.c:363
#7 0x0000000000ae7c49 in row_prebuilt_free (prebuilt=0x7f88500eae78, dict_locked=0) at 5.5/storage/xtradb/row/row0mysql.c:848
#8 0x0000000000ab395d in ha_innobase::close (this=0x7f88500b5078) at 5.5/storage/xtradb/handler/ha_innodb.cc:5058
#9 0x00000000007da404 in handler::ha_close (this=0x7f88500b5078) at 5.5/sql/handler.cc:2327
#10 0x0000000000cbba9c in ha_partition::close (this=0x7f88500b8d78) at 5.5/sql/ha_partition.cc:3167
#11 0x00000000007da404 in handler::ha_close (this=0x7f88500b8d78) at 5.5/sql/handler.cc:2327
#12 0x00000000006e4a6c in closefrm (table=0x7f8850024860, free_share=true) at 5.5/sql/table.cc:2769
#13 0x00000000005b0f43 in intern_close_table (table=0x7f8850024860) at 5.5/sql/sql_base.cc:929
#14 0x00000000005b0fb1 in free_cache_entry (table=0x7f8850024860) at 5.5/sql/sql_base.cc:952
#15 0x00000000005b132c in close_cached_tables (thd=0x7f88566e9060, tables=0x0, wait_for_refresh=true, timeout=31536000) at 5.5/sql/sql_base.cc:1059
#16 0x000000000074b79f in reload_acl_and_cache (thd=0x7f88566e9060, options=4, tables=0x0, write_to_binlog=0x7f887a4e3eb0) at 5.5/sql/sql_reload.cc:272
#17 0x00000000006193a9 in mysql_execute_command (thd=0x7f88566e9060) at 5.5/sql/sql_parse.cc:3724
#18 0x000000000061ee04 in mysql_parse (thd=0x7f88566e9060, rawbuf=0x7f885004e078 "flush tables", length=12, parser_state=0x7f887a4e4610) at 5.5/sql/sql_parse.cc:5914
#19 0x00000000006121bf in dispatch_command (command=COM_QUERY, thd=0x7f88566e9060, packet=0x7f8856791061 "flush tables", packet_length=12) at 5.5/sql/sql_parse.cc:1079
#20 0x0000000000611310 in do_command (thd=0x7f88566e9060) at 5.5/sql/sql_parse.cc:793
#21 0x000000000071f011 in do_handle_one_connection (thd_arg=0x7f88566e9060) at 5.5/sql/sql_connect.cc:1269
#22 0x000000000071ed9e in handle_one_connection (arg=0x7f88566e9060) at 5.5/sql/sql_connect.cc:1185
#23 0x00007f8879495e9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#24 0x00007f8878bc2cbd in clone () from /lib/x86_64-linux-gnu/libc.so.6
The datadir only contains one custom table:
DROP TABLE IF EXISTS t1;
CREATE TABLE t1 (
`creation_time` datetime NOT NULL
) ENGINE=InnoDB
PARTITION BY RANGE (TO_DAYS(creation_time))
(PARTITION LastPart VALUES LESS THAN MAXVALUE);
commit 082b859d0d8afbe49be770c2f18cf39bbd939aa8
Author: Jan Lindström <jan.lindstrom@mariadb.com>
Date: Fri Dec 4 14:24:03 2015 +0200
MDEV-9233: Copying MySQL 5.5 data directory to 10.0 with partition tables crashes on insert
Analysis: There were two problems. (1) if partition table was
created using lower_case_tables = 1 on windows we did find the
correct table but we did not set share->ib_table correctly.
(2) we did open table on dictionary but did not increase
mysql_open_tables.
Fix: In xtradb allow access to tables with incorrect
lower case names (warning is printed to error log). If
table is opened increase mysql_open_tables count to avoid
crash on flush tables.
Jan Lindström (Inactive)
added a comment - For 5.5:
commit 082b859d0d8afbe49be770c2f18cf39bbd939aa8
Author: Jan Lindström <jan.lindstrom@mariadb.com>
Date: Fri Dec 4 14:24:03 2015 +0200
MDEV-9233 : Copying MySQL 5.5 data directory to 10.0 with partition tables crashes on insert
Analysis: There were two problems. (1) if partition table was
created using lower_case_tables = 1 on windows we did find the
correct table but we did not set share->ib_table correctly.
(2) we did open table on dictionary but did not increase
mysql_open_tables.
Fix: In xtradb allow access to tables with incorrect
lower case names (warning is printed to error log). If
table is opened increase mysql_open_tables count to avoid
crash on flush tables.
Downloaded the binary (MariaDB-10.0.21-centos6-x86_64-server.rpm ) and loaded the core file. "thread apply where" gives this: https://gist.github.com/spetrunia/84990c3e9543ff02160a