[MDEV-13015] After restart, InnoDB wrongly thinks that a SEQUENCE is a TABLE Created: 2017-06-06  Updated: 2018-01-15  Resolved: 2017-06-15

Status: Closed
Project: MariaDB Server
Component/s: Sequences, Storage Engine - InnoDB
Affects Version/s: 10.3
Fix Version/s: 10.3.1

Type: Bug Priority: Major
Reporter: Elena Stepanova Assignee: Marko Mäkelä
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Duplicate
is duplicated by MDEV-13018 Assertion `!(((trx))->auto_commit && ... Closed
Relates
relates to MDEV-13016 Testing SEQUENCE object phase 2 Closed
relates to MDEV-12873 InnoDB SYS_TABLES.TYPE incompatibilit... Closed

 Description    

--source include/have_innodb.inc
 
 
 
create sequence s1 engine=InnoDB;
 
--source include/restart_mysqld.inc
 
select * from s1;
 
 
 
# cleanup
 
drop sequence s1;

10.3 86b9417035295

 
mysqld: /data/src/10.3/storage/innobase/include/read0types.h:166: bool ReadView::changes_visible(trx_id_t, const table_name_t&) const: Assertion `id > 0' failed.
 
170607  0:53:09 [ERROR] mysqld got signal 6 ;
 
 
 
#7  0x00007faf19f1fee2 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6
 
#8  0x000055cb14612cc2 in ReadView::changes_visible (this=0x55cb1866cc00, id=0, name=...) at /data/src/10.3/storage/innobase/include/read0types.h:166
 
#9  0x000055cb145fdbd2 in lock_clust_rec_cons_read_sees (rec=0x7faf0f8d007d "", index=0x7faec013ce88, offsets=0x7faf0c3422c0, view=0x55cb1866cc00) at /data/src/10.3/storage/innobase/lock/lock0lock.cc:428
 
#10 0x000055cb14705041 in row_search_mvcc (buf=0x7faec0136c38 '\245' <repeats 128 times>, mode=PAGE_CUR_G, prebuilt=0x7faec013d8e8, match_mode=0, direction=0) at /data/src/10.3/storage/innobase/row/row0sel.cc:5098
 
#11 0x000055cb14590f27 in ha_innobase::index_read (this=0x7faec0142118, buf=0x7faec0136c38 '\245' <repeats 128 times>, key_ptr=0x0, key_len=0, find_flag=HA_READ_AFTER_KEY) at /data/src/10.3/storage/innobase/handler/ha_innodb.cc:9837
 
#12 0x000055cb145922ca in ha_innobase::index_first (this=0x7faec0142118, buf=0x7faec0136c38 '\245' <repeats 128 times>) at /data/src/10.3/storage/innobase/handler/ha_innodb.cc:10275
 
#13 0x000055cb1459254a in ha_innobase::rnd_next (this=0x7faec0142118, buf=0x7faec0136c38 '\245' <repeats 128 times>) at /data/src/10.3/storage/innobase/handler/ha_innodb.cc:10371
 
#14 0x000055cb149b080b in ha_sequence::rnd_next (this=0x7faec0136758, buf=0x7faec0136c38 '\245' <repeats 128 times>) at /data/src/10.3/sql/ha_sequence.h:111
 
#15 0x000055cb1427dd6f in handler::ha_rnd_next (this=0x7faec0136758, buf=0x7faec0136c38 '\245' <repeats 128 times>) at /data/src/10.3/sql/handler.cc:2594
 
#16 0x000055cb1427f497 in handler::read_first_row (this=0x7faec0136758, buf=0x7faec0136c38 '\245' <repeats 128 times>, primary_key=64) at /data/src/10.3/sql/handler.cc:2831
 
#17 0x000055cb14050967 in handler::ha_read_first_row (this=0x7faec0136758, buf=0x7faec0136c38 '\245' <repeats 128 times>, primary_key=64) at /data/src/10.3/sql/sql_class.h:5849
 
#18 0x000055cb141a4838 in SEQUENCE::read_stored_values (this=0x7faec013a068) at /data/src/10.3/sql/sql_sequence.cc:473
 
#19 0x000055cb141a4679 in SEQUENCE::read_initial_values (this=0x7faec013a068, table_arg=0x7faec0135b50) at /data/src/10.3/sql/sql_sequence.cc:440
 
#20 0x000055cb149af9e9 in ha_sequence::open (this=0x7faec0136758, name=0x7faec0139fa8 "./test/s1", mode=2, flags=18) at /data/src/10.3/sql/ha_sequence.cc:110
 
#21 0x000055cb1427d754 in handler::ha_open (this=0x7faec0136758, table_arg=0x7faec0135b50, name=0x7faec0139fa8 "./test/s1", mode=2, test_if_locked=18) at /data/src/10.3/sql/handler.cc:2516
 
#22 0x000055cb140cc5d5 in open_table_from_share (thd=0x7faec0000b00, share=0x7faec0139a98, alias=0x7faec0014b48 "s1", db_stat=33, prgflag=8, ha_open_flags=18, outparam=0x7faec0135b50, is_create_table=false) at /data/src/10.3/sql/table.cc:3310
 
#23 0x000055cb13f4e2a7 in open_table (thd=0x7faec0000b00, table_list=0x7faec0014b50, ot_ctx=0x7faf0c343740) at /data/src/10.3/sql/sql_base.cc:1877
 
#24 0x000055cb13f50d8a in open_and_process_table (thd=0x7faec0000b00, lex=0x7faec00045d0, tables=0x7faec0014b50, counter=0x7faf0c3437d4, flags=0, prelocking_strategy=0x7faf0c343850, has_prelocking_list=false, ot_ctx=0x7faf0c343740) at /data/src/10.3/sql/sql_base.cc:3414
 
#25 0x000055cb13f51ee0 in open_tables (thd=0x7faec0000b00, options=..., start=0x7faf0c3437b8, counter=0x7faf0c3437d4, flags=0, prelocking_strategy=0x7faf0c343850) at /data/src/10.3/sql/sql_base.cc:3934
 
#26 0x000055cb13f536d4 in open_and_lock_tables (thd=0x7faec0000b00, options=..., tables=0x7faec0014b50, derived=true, flags=0, prelocking_strategy=0x7faf0c343850) at /data/src/10.3/sql/sql_base.cc:4689
 
#27 0x000055cb13f46b23 in open_and_lock_tables (thd=0x7faec0000b00, tables=0x7faec0014b50, derived=true, flags=0) at /data/src/10.3/sql/sql_base.h:495
 
#28 0x000055cb13fd396c in execute_sqlcom_select (thd=0x7faec0000b00, all_tables=0x7faec0014b50) at /data/src/10.3/sql/sql_parse.cc:6386
 
#29 0x000055cb13fca19c in mysql_execute_command (thd=0x7faec0000b00) at /data/src/10.3/sql/sql_parse.cc:3581
 
#30 0x000055cb13fd7860 in mysql_parse (thd=0x7faec0000b00, rawbuf=0x7faec0014968 "select * from s1", length=16, parser_state=0x7faf0c345200, is_com_multi=false, is_next_command=false) at /data/src/10.3/sql/sql_parse.cc:7927
 
#31 0x000055cb13fc5518 in dispatch_command (command=COM_QUERY, thd=0x7faec0000b00, packet=0x7faec000add1 "select * from s1", packet_length=16, is_com_multi=false, is_next_command=false) at /data/src/10.3/sql/sql_parse.cc:1817
 
#32 0x000055cb13fc3ec1 in do_command (thd=0x7faec0000b00) at /data/src/10.3/sql/sql_parse.cc:1380
 
#33 0x000055cb141113f4 in do_handle_one_connection (connect=0x55cb184ddaf0) at /data/src/10.3/sql/sql_connect.cc:1354
 
#34 0x000055cb14111181 in handle_one_connection (arg=0x55cb184ddaf0) at /data/src/10.3/sql/sql_connect.cc:1260
 
#35 0x000055cb1456da8d in pfs_spawn_thread (arg=0x55cb184e6390) at /data/src/10.3/storage/perfschema/pfs.cc:1862
 
#36 0x00007faf1bc59494 in start_thread (arg=0x7faf0c346700) at pthread_create.c:333
 
#37 0x00007faf19fdc93f in clone () from /lib/x86_64-linux-gnu/libc.so.6

Also reproducible on bb-10.2-ext 3d428e017dbc.

 Comments   
Comment by Marko Mäkelä [ 2017-06-07 ]

The problem is that InnoDB is not storing the NO_ROLLBACK flag to SYS_TABLES.TYPE. This is because dict_sys_tables_type_to_tf() and dict_tf_to_sys_tables_type() are clearing all flags except the following:

	flags |= type & (DICT_TF_MASK_ZIP_SSIZE
 
			 | DICT_TF_MASK_ATOMIC_BLOBS
 
			 | DICT_TF_MASK_DATA_DIR
 
			 | DICT_TF_MASK_PAGE_COMPRESSION
 
			 | DICT_TF_MASK_PAGE_COMPRESSION_LEVEL);

In addition to the NO_ROLLBACK flag that was introduced in MDEV-10139, the following flags are not being persisted: ATOMIC_WRITES, PAGE_ENCRYPTION, PAGE_ENCRYPTION_KEY. The ATOMIC_WRITES flag was introduced in MariaDB 10.1, and it is being preserved there. But there is apparently no use of it.

The PAGE_ENCRYPTION flags are not being written to SYS_TABLES.TYPE in MariaDB 10.1 either. It looks like these flags should be removed or moved to DICT_TF2 or directly to a bitfield of dict_table_t.

So, to fix this bug properly, we need to refactor the flags a bit, to ensure that backward and forward compatibility with regard to SYS_TABLES.TYPE is guaranteed. With this quick&dirty fix the test case passes:

diff --git a/storage/innobase/include/dict0dict.ic b/storage/innobase/include/dict0dict.ic
 
index e9507302e66..830a271459e 100644
 
--- a/storage/innobase/include/dict0dict.ic
 
+++ b/storage/innobase/include/dict0dict.ic
 
@@ -1039,6 +1039,7 @@ dict_sys_tables_type_to_tf(
 
 	flags |= type & (DICT_TF_MASK_ZIP_SSIZE
 
 			 | DICT_TF_MASK_ATOMIC_BLOBS
 
 			 | DICT_TF_MASK_DATA_DIR
 
+			 | (1U << DICT_TF_POS_NO_ROLLBACK)
 
 			 | DICT_TF_MASK_PAGE_COMPRESSION
 
 			 | DICT_TF_MASK_PAGE_COMPRESSION_LEVEL);
 
 
@@ -1074,6 +1075,7 @@ dict_tf_to_sys_tables_type(
 
 	type |= flags & (DICT_TF_MASK_ZIP_SSIZE
 
 			 | DICT_TF_MASK_ATOMIC_BLOBS
 
 			 | DICT_TF_MASK_DATA_DIR
 
+			 | (1U << DICT_TF_POS_NO_ROLLBACK)
 
 			 | DICT_TF_MASK_PAGE_COMPRESSION
 
 			 | DICT_TF_MASK_PAGE_COMPRESSION_LEVEL);

Comment by Marko Mäkelä [ 2017-06-07 ]

Pushed to bb-10.3-marko. I would backport the removal of the flags also to 10.2 in order to clean up the code there already. Also, 10.2 should prevent access to SEQUENCEs.

Comment by Jan Lindström (Inactive) [ 2017-06-08 ]

ok to push.

Comment by Marko Mäkelä [ 2017-06-08 ]

Thanks! There was an inconsistency in the patch. I revised it with the introduction of DICT_TF_MASK_NO_ROLLBACK. I will push the applicable parts of it already to 10.2 along with a test that edits some flags in SYS_TABLES.TYPE while the server is offline.

Comment by Marko Mäkelä [ 2017-06-12 ]

There is a further compatibility problem: In MariaDB 10.2.2 (with the merge of MySQL 5.7.9), the SYS_TABLES.TYPE was made incompatible with MariaDB 10.1.

MySQL 5.7 introduced the SHARED_SPACE flag for indicating that a table uses an explicit TABLESPACE attribute. MariaDB 10.2 does not support CREATE TABLESPACE for InnoDB tables. In the above-mentioned commit, the existing MariaDB fields PAGE_COMPRESSION, PAGE_COMPRESSION_LEVEL, ATOMIC_WRITES were shifted by 1, making the flags incompatible with 10.1 and earlier versions.

The proper fix seems to be to remove the SHARED_SPACE flag from 10.2 and 10.3, and to move PAGE_COMPRESSION, PAGE_COMPRESSION_LEVEL, ATOMIC_WRITES to their original position. Unfortunately this may mean that if any tables were created with the page_compression flag in the 10.2.6 GA release, those tables will be unreadable by other versions of MariaDB. Import/export is not affected, because these flags are only persisted in SYS_TABLES.TYPE, which resides in the InnoDB system tablespace.

Comment by Marko Mäkelä [ 2017-06-15 ]

Finally, I fixed this together with the merge of MDEV-12873 in bb-10.2-ext.

Generated at Thu Feb 08 08:02:15 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.