[MDEV-18295] IMPORT TABLESPACE fails with instant-altered tables Created: 2019-01-18  Updated: 2024-02-07  Resolved: 2019-02-11

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 10.3, 10.4
Fix Version/s: 10.4.3, 10.3.13

Type: Bug Priority: Major
Reporter: Aleksey Midenkov Assignee: Marko Mäkelä
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Problem/Incident
causes MDEV-21549 IMPORT TABLESPACE fails to adjust all... Closed
causes MDEV-33400 Adaptive hash index corruption after ... Stalled
is caused by MDEV-11369 Instant add column for InnoDB Closed
Relates
relates to MDEV-18543 IMPORT TABLESPACE fails after instant... Closed

 Description   

Reproduce 1 (instant add)

--source include/have_innodb.inc
 
set default_storage_engine=innodb;
 
 
 
create or replace table t2 (x int, z int);
 
alter table t2 discard tablespace;
 
 
 
create or replace table t1 (x int);
 
insert into t1 values (1);
 
alter table t1 add z int, algorithm instant;
 
select * from t1;
 
flush tables t1 for export;
 
--let $MYSQLD_DATADIR= `select @@datadir`
 
--move_file $MYSQLD_DATADIR/test/t1.cfg $MYSQLD_DATADIR/test/t2.cfg
 
--copy_file $MYSQLD_DATADIR/test/t1.ibd $MYSQLD_DATADIR/test/t2.ibd
 
unlock tables;
 
 
 
alter table t2 import tablespace;
 
 
 
select * from t2;
 
check table t2 extended;
 
 
 
create or replace database test;

Result 1

mysqld: /home/midenok/src/mariadb/trunk2/src/storage/innobase/include/dict0mem.h:673: const byte *dict_col_t::instant_value(ulint *) const: Assertion `is_added()' failed.
 
 
 
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
 
#1  0x00007ffff6d04535 in __GI_abort () at abort.c:79
 
#2  0x00007ffff6d0440f in __assert_fail_base (fmt=0x7ffff6e90858 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x169a46e "is_added()", file=0x1688af8 "/home/midenok/src/mariadb/trunk2/src/storage/innobase/include/dict0mem.h", line=673, function=<optimized out>) at assert.c:92
 
#3  0x00007ffff6d14142 in __GI___assert_fail (assertion=0x169a46e "is_added()", file=0x1688af8 "/home/midenok/src/mariadb/trunk2/src/storage/innobase/include/dict0mem.h", line=673, function=0x16958c5 "const byte *dict_col_t::instant_value(ulint *) const") at assert.c:101
 
#4  0x0000000000e1e728 in dict_col_t::instant_value (this=0x7fff9c0269c8, len=0x7ffff006ebe8) at /home/midenok/src/mariadb/trunk2/src/storage/innobase/include/dict0mem.h:673
 
#5  0x0000000000e1e220 in dict_index_t::instant_field_value (this=0x7fff9c029d38, n=4, len=0x7ffff006ebe8) at /home/midenok/src/mariadb/trunk2/src/storage/innobase/include/dict0mem.h:1172
 
#6  0x0000000000fc55b3 in row_sel_store_mysql_field_func (mysql_rec=0x7fff9c025908 "\375\001", prebuilt=0x7fff9c02d048, rec=0x7ffff0638087 "", index=0x7fff9c029d38, offsets=0x7ffff006fb40, field_no=4, templ=0x7fff9c08abd0) at /home/midenok/src/mariadb/trunk2/src/storage/innobase/row/row0sel.cc:2982
 
#7  0x0000000000fbc227 in row_sel_store_mysql_rec (mysql_rec=0x7fff9c025908 "\375\001", prebuilt=0x7fff9c02d048, rec=0x7ffff0638087 "", vrow=0x0, rec_clust=false, index=0x7fff9c029d38, offsets=0x7ffff006fb40) at /home/midenok/src/mariadb/trunk2/src/storage/innobase/row/row0sel.cc:3160
 
#8  0x0000000000fba13a in row_search_mvcc (buf=0x7fff9c025908 "\375\001", mode=PAGE_CUR_G, prebuilt=0x7fff9c02d048, match_mode=0, direction=0) at /home/midenok/src/mariadb/trunk2/src/storage/innobase/row/row0sel.cc:5440
 
#9  0x0000000000dba82e in ha_innobase::index_read (this=0x7fff9c02a730, buf=0x7fff9c025908 "\375\001", key_ptr=0x0, key_len=0, find_flag=HA_READ_AFTER_KEY) at /home/midenok/src/mariadb/trunk2/src/storage/innobase/handler/ha_innodb.cc:9257
 
#10 0x0000000000dbb3cb in ha_innobase::index_first (this=0x7fff9c02a730, buf=0x7fff9c025908 "\375\001") at /home/midenok/src/mariadb/trunk2/src/storage/innobase/handler/ha_innodb.cc:9634
 
#11 0x0000000000dbb664 in ha_innobase::rnd_next (this=0x7fff9c02a730, buf=0x7fff9c025908 "\375\001") at /home/midenok/src/mariadb/trunk2/src/storage/innobase/handler/ha_innodb.cc:9727
 
#12 0x0000000000b39447 in handler::ha_rnd_next (this=0x7fff9c02a730, buf=0x7fff9c025908 "\375\001") at /home/midenok/src/mariadb/trunk2/src/sql/handler.cc:2800
 
#13 0x0000000000d10344 in rr_sequential (info=0x7fff9c018b20) at /home/midenok/src/mariadb/trunk2/src/sql/records.cc:481
 
#14 0x0000000000710beb in READ_RECORD::read_record (this=0x7fff9c018b20) at /home/midenok/src/mariadb/trunk2/src/sql/records.h:73
 
#15 0x000000000084046c in join_init_read_record (tab=0x7fff9c018a58) at /home/midenok/src/mariadb/trunk2/src/sql/sql_select.cc:20326
 
#16 0x000000000085ba9a in sub_select (join=0x7fff9c017630, join_tab=0x7fff9c018a58, end_of_records=false) at /home/midenok/src/mariadb/trunk2/src/sql/sql_select.cc:19387
 
#17 0x00000000008442a9 in do_select (join=0x7fff9c017630, procedure=0x0) at /home/midenok/src/mariadb/trunk2/src/sql/sql_select.cc:18930
 
#18 0x000000000084303e in JOIN::exec_inner (this=0x7fff9c017630) at /home/midenok/src/mariadb/trunk2/src/sql/sql_select.cc:4094
 
#19 0x000000000084210e in JOIN::exec (this=0x7fff9c017630) at /home/midenok/src/mariadb/trunk2/src/sql/sql_select.cc:3888
 
#20 0x0000000000819671 in mysql_select (thd=0x7fff9c000cf8, tables=0x7fff9c0166f0, wild_num=1, fields=..., conds=0x0, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147748608, result=0x7fff9c017608, unit=0x7fff9c004bd0, select_lex=0x7fff9c016168) at /home/midenok/src/mariadb/trunk2/src/sql/sql_select.cc:4293
 
#21 0x0000000000818eb1 in handle_select (thd=0x7fff9c000cf8, lex=0x7fff9c004b08, result=0x7fff9c017608, setup_tables_done_option=0) at /home/midenok/src/mariadb/trunk2/src/sql/sql_select.cc:373
 
#22 0x00000000007dadba in execute_sqlcom_select (thd=0x7fff9c000cf8, all_tables=0x7fff9c0166f0) at /home/midenok/src/mariadb/trunk2/src/sql/sql_parse.cc:6579
 
#23 0x00000000007cfe75 in mysql_execute_command (thd=0x7fff9c000cf8) at /home/midenok/src/mariadb/trunk2/src/sql/sql_parse.cc:3777
 
#24 0x00000000007ca940 in mysql_parse (thd=0x7fff9c000cf8, rawbuf=0x7fff9c0160e0 "select * from t2", length=16, parser_state=0x7ffff00745e0, is_com_multi=false, is_next_command=false) at /home/midenok/src/mariadb/trunk2/src/sql/sql_parse.cc:8116

Reproduce 2 (instant drop)

--source include/have_innodb.inc
 
set default_storage_engine=innodb;
 
 
 
create or replace table t2 (x int);
 
alter table t2 discard tablespace;
 
 
 
create or replace table t1 (x int, y int);
 
insert into t1 values (1);
 
alter table t1 drop y int, algorithm instant;
 
select * from t1;
 
flush tables t1 for export;
 
--let $MYSQLD_DATADIR= `select @@datadir`
 
--move_file $MYSQLD_DATADIR/test/t1.cfg $MYSQLD_DATADIR/test/t2.cfg
 
--copy_file $MYSQLD_DATADIR/test/t1.ibd $MYSQLD_DATADIR/test/t2.ibd
 
unlock tables;
 
 
 
alter table t2 import tablespace;
 
 
 
select * from t2;
 
check table t2 extended;
 
 
 
create or replace database test;

Result 2

mysqltest: At line 8: query 'insert into t1 values (1)' failed: 1136: Column count doesn't match value count at row 1



 Comments   
Comment by Marko Mäkelä [ 2019-01-18 ]

I can repeat a crash with 10.3 as well. In MDEV-11369 I added code that was supposed to handle this (search for btr_cur_instant_root_init in storage/innobase/row/row0import.cc), but maybe the logic got broken at some point, because I forgot to add a regression test for it.

10.3 4e75bfcb21469c9efbbc55d2ff3978e11475d40d

 
2019-01-18 14:13:33 9 [Note] InnoDB: Phase I - Update all pages
 
mysqld: /mariadb/10.3/storage/innobase/include/dict0mem.h:637: const byte* dict_col_t::instant_value(ulint*) const: Assertion `is_instant()' failed.
 
…
 
#8  0x000055b747823727 in dict_col_t::instant_value (this=0x7f366409d7b8, len=0x7f36b441c3c0) at /mariadb/10.3/storage/innobase/include/dict0mem.h:637
 
#9  0x000055b747823a91 in dict_index_t::instant_field_value (this=0x7f366409e3c8, n=4, len=0x7f36b441c3c0) at /mariadb/10.3/storage/innobase/include/dict0mem.h:1093
 
#10 0x000055b7478e5037 in rec_init_offsets_comp_ordinary (rec=0x7f366419007e "", index=0x7f366409e3c8, offsets=0x7f36b441d048, n_core=4, def_val=0x0, format=REC_LEAF_ORDINARY) at /mariadb/10.3/storage/innobase/rem/rem0rec.cc:368
 
#11 0x000055b7478e5e92 in rec_init_offsets (rec=0x7f366419007e "", index=0x7f366409e3c8, leaf=true, offsets=0x7f36b441d048) at /mariadb/10.3/storage/innobase/rem/rem0rec.cc:610
 
#12 0x000055b7478e6a34 in rec_get_offsets_func (rec=0x7f366419007e "", index=0x7f366409e3c8, offsets=0x7f36b441d048, leaf=true, n_fields=18446744073709551615, file=0x55b748048ba0 "/mariadb/10.3/storage/innobase/row/row0import.cc", line=1742, heap=0x7f36b441d370) at /mariadb/10.3/storage/innobase/rem/rem0rec.cc:868
 
#13 0x000055b7479013e5 in PageConverter::update_records (this=0x7f36b441cfb0, block=0x7f36641282b0) at /mariadb/10.3/storage/innobase/row/row0import.cc:1740
 
#14 0x000055b74790186a in PageConverter::update_index_page (this=0x7f36b441cfb0, block=0x7f36641282b0) at /mariadb/10.3/storage/innobase/row/row0import.cc:1859
 
#15 0x000055b747901d03 in PageConverter::update_page (this=0x7f36b441cfb0, block=0x7f36641282b0, page_type=@0x7f36b441c958: 18) at /mariadb/10.3/storage/innobase/row/row0import.cc:1941
 
#16 0x000055b7478f96f5 in PageConverter::operator() (this=0x7f36b441cfb0, block=0x7f36641282b0) at /mariadb/10.3/storage/innobase/row/row0import.cc:1989
 
#17 0x000055b7478fd251 in fil_iterate (iter=..., block=0x7f36641282b0, callback=...) at /mariadb/10.3/storage/innobase/row/row0import.cc:3473
 
#18 0x000055b7478fe0e0 in fil_tablespace_iterate (table=0x7f366409d488, n_io_buffers=64, callback=...) at /mariadb/10.3/storage/innobase/row/row0import.cc:3722
 
#19 0x000055b7478fea79 in row_import_for_mysql (table=0x7f366409d488, prebuilt=0x7f366402f0b8) at /mariadb/10.3/storage/innobase/row/row0import.cc:3907

Comment by Marko Mäkelä [ 2019-02-11 ]

In 10.3, the initial part of the intended logic works, because btr_cur_instant_root_init() is being called. What is missing is the call to btr_cur_instant_init_low(), because we do not use the buffer pool during the initial phase of the import. We should simply fill in some dummy values to keep dict_col_t::instant_value() happy. Finally, when the IMPORT TABLEPSACE has finished, we should reload the table definition.

It seems to me that there could be further problems if IMPORT TABLESPACE is executed on a table on which instant ADD COLUMN or DROP COLUMN had been invoked in the past. I will write a test case that will cover this as well.

Comment by Marko Mäkelä [ 2019-02-11 ]

In 10.4, we will actually have to load the metadata from the being-imported table, or otherwise we will be unable to adjust the tablespace IDs in BLOB pointers that belong to instantly dropped columns. That is a non-trivial change, because we will have to access the leftmost leaf page while bypassing the buffer pool. We may have to block the import of tables where instant DROP COLUMN or permutation of columns (MDEV-15562) has been executed.

Comment by Marko Mäkelä [ 2019-02-11 ]

I fixed IMPORT TABLESPACE after instant ADD COLUMN.

MDEV-18543 tracks the inability of MariaDB 10.4 to IMPORT TABLESPACE after instant DROP COLUMN or changing the order of columns.

Generated at Thu Feb 08 08:43:00 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.