[MDEV-13015] After restart, InnoDB wrongly thinks that a SEQUENCE is a TABLE - Jira

Details

Type: Bug
Status: Closed (View Workflow)
Priority: Major
Resolution: Fixed
Affects Version/s: 10.3(EOL)
Fix Version/s: 10.3.1
Component/s: Sequences, Storage Engine - InnoDB
Labels:
None

Description

--source include/have_innodb.inc

create sequence s1 engine=InnoDB;

--source include/restart_mysqld.inc

select * from s1;

# cleanup

drop sequence s1;

10.3 86b9417035295
mysqld: /data/src/10.3/storage/innobase/include/read0types.h:166: bool ReadView::changes_visible(trx_id_t, const table_name_t&) const: Assertion `id > 0' failed.
170607 0:53:09 [ERROR] mysqld got signal 6 ;

#7 0x00007faf19f1fee2 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6
#8 0x000055cb14612cc2 in ReadView::changes_visible (this=0x55cb1866cc00, id=0, name=...) at /data/src/10.3/storage/innobase/include/read0types.h:166
#9 0x000055cb145fdbd2 in lock_clust_rec_cons_read_sees (rec=0x7faf0f8d007d "", index=0x7faec013ce88, offsets=0x7faf0c3422c0, view=0x55cb1866cc00) at /data/src/10.3/storage/innobase/lock/lock0lock.cc:428
#10 0x000055cb14705041 in row_search_mvcc (buf=0x7faec0136c38 '\245' <repeats 128 times>, mode=PAGE_CUR_G, prebuilt=0x7faec013d8e8, match_mode=0, direction=0) at /data/src/10.3/storage/innobase/row/row0sel.cc:5098
#11 0x000055cb14590f27 in ha_innobase::index_read (this=0x7faec0142118, buf=0x7faec0136c38 '\245' <repeats 128 times>, key_ptr=0x0, key_len=0, find_flag=HA_READ_AFTER_KEY) at /data/src/10.3/storage/innobase/handler/ha_innodb.cc:9837
#12 0x000055cb145922ca in ha_innobase::index_first (this=0x7faec0142118, buf=0x7faec0136c38 '\245' <repeats 128 times>) at /data/src/10.3/storage/innobase/handler/ha_innodb.cc:10275
#13 0x000055cb1459254a in ha_innobase::rnd_next (this=0x7faec0142118, buf=0x7faec0136c38 '\245' <repeats 128 times>) at /data/src/10.3/storage/innobase/handler/ha_innodb.cc:10371
#14 0x000055cb149b080b in ha_sequence::rnd_next (this=0x7faec0136758, buf=0x7faec0136c38 '\245' <repeats 128 times>) at /data/src/10.3/sql/ha_sequence.h:111
#15 0x000055cb1427dd6f in handler::ha_rnd_next (this=0x7faec0136758, buf=0x7faec0136c38 '\245' <repeats 128 times>) at /data/src/10.3/sql/handler.cc:2594
#16 0x000055cb1427f497 in handler::read_first_row (this=0x7faec0136758, buf=0x7faec0136c38 '\245' <repeats 128 times>, primary_key=64) at /data/src/10.3/sql/handler.cc:2831
#17 0x000055cb14050967 in handler::ha_read_first_row (this=0x7faec0136758, buf=0x7faec0136c38 '\245' <repeats 128 times>, primary_key=64) at /data/src/10.3/sql/sql_class.h:5849
#18 0x000055cb141a4838 in SEQUENCE::read_stored_values (this=0x7faec013a068) at /data/src/10.3/sql/sql_sequence.cc:473
#19 0x000055cb141a4679 in SEQUENCE::read_initial_values (this=0x7faec013a068, table_arg=0x7faec0135b50) at /data/src/10.3/sql/sql_sequence.cc:440
#20 0x000055cb149af9e9 in ha_sequence::open (this=0x7faec0136758, name=0x7faec0139fa8 "./test/s1", mode=2, flags=18) at /data/src/10.3/sql/ha_sequence.cc:110
#21 0x000055cb1427d754 in handler::ha_open (this=0x7faec0136758, table_arg=0x7faec0135b50, name=0x7faec0139fa8 "./test/s1", mode=2, test_if_locked=18) at /data/src/10.3/sql/handler.cc:2516
#22 0x000055cb140cc5d5 in open_table_from_share (thd=0x7faec0000b00, share=0x7faec0139a98, alias=0x7faec0014b48 "s1", db_stat=33, prgflag=8, ha_open_flags=18, outparam=0x7faec0135b50, is_create_table=false) at /data/src/10.3/sql/table.cc:3310
#23 0x000055cb13f4e2a7 in open_table (thd=0x7faec0000b00, table_list=0x7faec0014b50, ot_ctx=0x7faf0c343740) at /data/src/10.3/sql/sql_base.cc:1877
#24 0x000055cb13f50d8a in open_and_process_table (thd=0x7faec0000b00, lex=0x7faec00045d0, tables=0x7faec0014b50, counter=0x7faf0c3437d4, flags=0, prelocking_strategy=0x7faf0c343850, has_prelocking_list=false, ot_ctx=0x7faf0c343740) at /data/src/10.3/sql/sql_base.cc:3414
#25 0x000055cb13f51ee0 in open_tables (thd=0x7faec0000b00, options=..., start=0x7faf0c3437b8, counter=0x7faf0c3437d4, flags=0, prelocking_strategy=0x7faf0c343850) at /data/src/10.3/sql/sql_base.cc:3934
#26 0x000055cb13f536d4 in open_and_lock_tables (thd=0x7faec0000b00, options=..., tables=0x7faec0014b50, derived=true, flags=0, prelocking_strategy=0x7faf0c343850) at /data/src/10.3/sql/sql_base.cc:4689
#27 0x000055cb13f46b23 in open_and_lock_tables (thd=0x7faec0000b00, tables=0x7faec0014b50, derived=true, flags=0) at /data/src/10.3/sql/sql_base.h:495
#28 0x000055cb13fd396c in execute_sqlcom_select (thd=0x7faec0000b00, all_tables=0x7faec0014b50) at /data/src/10.3/sql/sql_parse.cc:6386
#29 0x000055cb13fca19c in mysql_execute_command (thd=0x7faec0000b00) at /data/src/10.3/sql/sql_parse.cc:3581
#30 0x000055cb13fd7860 in mysql_parse (thd=0x7faec0000b00, rawbuf=0x7faec0014968 "select * from s1", length=16, parser_state=0x7faf0c345200, is_com_multi=false, is_next_command=false) at /data/src/10.3/sql/sql_parse.cc:7927
#31 0x000055cb13fc5518 in dispatch_command (command=COM_QUERY, thd=0x7faec0000b00, packet=0x7faec000add1 "select * from s1", packet_length=16, is_com_multi=false, is_next_command=false) at /data/src/10.3/sql/sql_parse.cc:1817
#32 0x000055cb13fc3ec1 in do_command (thd=0x7faec0000b00) at /data/src/10.3/sql/sql_parse.cc:1380
#33 0x000055cb141113f4 in do_handle_one_connection (connect=0x55cb184ddaf0) at /data/src/10.3/sql/sql_connect.cc:1354
#34 0x000055cb14111181 in handle_one_connection (arg=0x55cb184ddaf0) at /data/src/10.3/sql/sql_connect.cc:1260
#35 0x000055cb1456da8d in pfs_spawn_thread (arg=0x55cb184e6390) at /data/src/10.3/storage/perfschema/pfs.cc:1862
#36 0x00007faf1bc59494 in start_thread (arg=0x7faf0c346700) at pthread_create.c:333
#37 0x00007faf19fdc93f in clone () from /lib/x86_64-linux-gnu/libc.so.6

Also reproducible on bb-10.2-ext 3d428e017dbc.

Attachments

Issue Links

is duplicated by

MDEV-13018 Assertion `!(((trx))->auto_commit && ((trx))->will_lock == 0)' failed in lock_table_create upon reading from locked InnoDB sequence

Closed

relates to

MDEV-13016 Testing SEQUENCE object phase 2

Closed

MDEV-12873 InnoDB SYS_TABLES.TYPE incompatibility for PAGE_COMPRESSED=YES in MariaDB 10.2.2 to 10.2.6

Closed

Activity

Ascending order - Click to sort in descending order

Marko Mäkelä added a comment - 2017-06-07 12:07

The problem is that InnoDB is not storing the NO_ROLLBACK flag to SYS_TABLES.TYPE. This is because dict_sys_tables_type_to_tf() and dict_tf_to_sys_tables_type() are clearing all flags except the following:

	flags |= type & (DICT_TF_MASK_ZIP_SSIZE

			 | DICT_TF_MASK_ATOMIC_BLOBS

			 | DICT_TF_MASK_DATA_DIR

			 | DICT_TF_MASK_PAGE_COMPRESSION

			 | DICT_TF_MASK_PAGE_COMPRESSION_LEVEL);

In addition to the NO_ROLLBACK flag that was introduced in ~~MDEV-10139~~, the following flags are not being persisted: ATOMIC_WRITES, PAGE_ENCRYPTION, PAGE_ENCRYPTION_KEY. The ATOMIC_WRITES flag was introduced in MariaDB 10.1, and it is being preserved there. But there is apparently no use of it.

The PAGE_ENCRYPTION flags are not being written to SYS_TABLES.TYPE in MariaDB 10.1 either. It looks like these flags should be removed or moved to DICT_TF2 or directly to a bitfield of dict_table_t.

So, to fix this bug properly, we need to refactor the flags a bit, to ensure that backward and forward compatibility with regard to SYS_TABLES.TYPE is guaranteed. With this quick&dirty fix the test case passes:

diff --git a/storage/innobase/include/dict0dict.ic b/storage/innobase/include/dict0dict.ic

index e9507302e66..830a271459e 100644

--- a/storage/innobase/include/dict0dict.ic

+++ b/storage/innobase/include/dict0dict.ic

@@ -1039,6 +1039,7 @@ dict_sys_tables_type_to_tf(

 	flags |= type & (DICT_TF_MASK_ZIP_SSIZE

 			 | DICT_TF_MASK_ATOMIC_BLOBS

 			 | DICT_TF_MASK_DATA_DIR

+			 | (1U << DICT_TF_POS_NO_ROLLBACK)

 			 | DICT_TF_MASK_PAGE_COMPRESSION

 			 | DICT_TF_MASK_PAGE_COMPRESSION_LEVEL);

@@ -1074,6 +1075,7 @@ dict_tf_to_sys_tables_type(

 	type |= flags & (DICT_TF_MASK_ZIP_SSIZE

 			 | DICT_TF_MASK_ATOMIC_BLOBS

 			 | DICT_TF_MASK_DATA_DIR

+			 | (1U << DICT_TF_POS_NO_ROLLBACK)

 			 | DICT_TF_MASK_PAGE_COMPRESSION

 			 | DICT_TF_MASK_PAGE_COMPRESSION_LEVEL);

Marko Mäkelä added a comment - 2017-06-07 12:07 The problem is that InnoDB is not storing the NO_ROLLBACK flag to SYS_TABLES.TYPE. This is because dict_sys_tables_type_to_tf() and dict_tf_to_sys_tables_type() are clearing all flags except the following: flags |= type & (DICT_TF_MASK_ZIP_SSIZE | DICT_TF_MASK_ATOMIC_BLOBS | DICT_TF_MASK_DATA_DIR | DICT_TF_MASK_PAGE_COMPRESSION | DICT_TF_MASK_PAGE_COMPRESSION_LEVEL); In addition to the NO_ROLLBACK flag that was introduced in MDEV-10139 , the following flags are not being persisted: ATOMIC_WRITES, PAGE_ENCRYPTION, PAGE_ENCRYPTION_KEY. The ATOMIC_WRITES flag was introduced in MariaDB 10.1, and it is being preserved there. But there is apparently no use of it. The PAGE_ENCRYPTION flags are not being written to SYS_TABLES.TYPE in MariaDB 10.1 either. It looks like these flags should be removed or moved to DICT_TF2 or directly to a bitfield of dict_table_t. So, to fix this bug properly, we need to refactor the flags a bit, to ensure that backward and forward compatibility with regard to SYS_TABLES.TYPE is guaranteed. With this quick&dirty fix the test case passes: diff --git a/storage/innobase/include/dict0dict.ic b/storage/innobase/include/dict0dict.ic index e9507302e66..830a271459e 100644 --- a/storage/innobase/include/dict0dict.ic +++ b/storage/innobase/include/dict0dict.ic @@ -1039,6 +1039,7 @@ dict_sys_tables_type_to_tf( flags |= type & (DICT_TF_MASK_ZIP_SSIZE | DICT_TF_MASK_ATOMIC_BLOBS | DICT_TF_MASK_DATA_DIR + | (1U << DICT_TF_POS_NO_ROLLBACK) | DICT_TF_MASK_PAGE_COMPRESSION | DICT_TF_MASK_PAGE_COMPRESSION_LEVEL); @@ -1074,6 +1075,7 @@ dict_tf_to_sys_tables_type( type |= flags & (DICT_TF_MASK_ZIP_SSIZE | DICT_TF_MASK_ATOMIC_BLOBS | DICT_TF_MASK_DATA_DIR + | (1U << DICT_TF_POS_NO_ROLLBACK) | DICT_TF_MASK_PAGE_COMPRESSION | DICT_TF_MASK_PAGE_COMPRESSION_LEVEL);

Marko Mäkelä added a comment - 2017-06-07 13:12 - edited

Pushed to bb-10.3-marko. I would backport the removal of the flags also to 10.2 in order to clean up the code there already. Also, 10.2 should prevent access to SEQUENCEs.

Marko Mäkelä added a comment - 2017-06-07 13:12 - edited Pushed to bb-10.3-marko . I would backport the removal of the flags also to 10.2 in order to clean up the code there already. Also, 10.2 should prevent access to SEQUENCEs.

Jan Lindström (Inactive) added a comment - 2017-06-08 05:07

ok to push.

Jan Lindström (Inactive) added a comment - 2017-06-08 05:07 ok to push.

Marko Mäkelä added a comment - 2017-06-08 05:29

Thanks! There was an inconsistency in the patch. I revised it with the introduction of DICT_TF_MASK_NO_ROLLBACK. I will push the applicable parts of it already to 10.2 along with a test that edits some flags in SYS_TABLES.TYPE while the server is offline.

Marko Mäkelä added a comment - 2017-06-08 05:29 Thanks! There was an inconsistency in the patch. I revised it with the introduction of DICT_TF_MASK_NO_ROLLBACK. I will push the applicable parts of it already to 10.2 along with a test that edits some flags in SYS_TABLES.TYPE while the server is offline.

Marko Mäkelä added a comment - 2017-06-12 15:28

There is a further compatibility problem: In MariaDB 10.2.2 (with the merge of MySQL 5.7.9), the SYS_TABLES.TYPE was made incompatible with MariaDB 10.1.

MySQL 5.7 introduced the SHARED_SPACE flag for indicating that a table uses an explicit TABLESPACE attribute. MariaDB 10.2 does not support CREATE TABLESPACE for InnoDB tables. In the above-mentioned commit, the existing MariaDB fields PAGE_COMPRESSION, PAGE_COMPRESSION_LEVEL, ATOMIC_WRITES were shifted by 1, making the flags incompatible with 10.1 and earlier versions.

The proper fix seems to be to remove the SHARED_SPACE flag from 10.2 and 10.3, and to move PAGE_COMPRESSION, PAGE_COMPRESSION_LEVEL, ATOMIC_WRITES to their original position. Unfortunately this may mean that if any tables were created with the page_compression flag in the 10.2.6 GA release, those tables will be unreadable by other versions of MariaDB. Import/export is not affected, because these flags are only persisted in SYS_TABLES.TYPE, which resides in the InnoDB system tablespace.

Marko Mäkelä added a comment - 2017-06-12 15:28 There is a further compatibility problem: In MariaDB 10.2.2 (with the merge of MySQL 5.7.9 ), the SYS_TABLES.TYPE was made incompatible with MariaDB 10.1. MySQL 5.7 introduced the SHARED_SPACE flag for indicating that a table uses an explicit TABLESPACE attribute. MariaDB 10.2 does not support CREATE TABLESPACE for InnoDB tables. In the above-mentioned commit, the existing MariaDB fields PAGE_COMPRESSION, PAGE_COMPRESSION_LEVEL, ATOMIC_WRITES were shifted by 1, making the flags incompatible with 10.1 and earlier versions. The proper fix seems to be to remove the SHARED_SPACE flag from 10.2 and 10.3, and to move PAGE_COMPRESSION, PAGE_COMPRESSION_LEVEL, ATOMIC_WRITES to their original position. Unfortunately this may mean that if any tables were created with the page_compression flag in the 10.2.6 GA release, those tables will be unreadable by other versions of MariaDB. Import/export is not affected, because these flags are only persisted in SYS_TABLES.TYPE, which resides in the InnoDB system tablespace.

Marko Mäkelä added a comment - 2017-06-15 15:10

Finally, I fixed this together with the merge of ~~MDEV-12873~~ in bb-10.2-ext.

Marko Mäkelä added a comment - 2017-06-15 15:10 Finally, I fixed this together with the merge of MDEV-12873 in bb-10.2-ext.

People

Assignee:: Marko Mäkelä

Reporter:: Elena Stepanova

Votes:: 0 Vote for this issue

Watchers:: 4 Start watching this issue

Dates

Created:: 2017-06-06 22:03

Updated:: 2018-01-15 18:07

Resolved:: 2017-06-15 15:10

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.

MariaDB Server