Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Fixed
-
10.3.0, 10.2.12, 10.2.13, 10.2.14
-
Ubuntu 16.04, x86-64bits
Description
MariaDb crashes if you run the attached sql file as soon as you start the service.
I'm sending you the error log with the error. If you can see de file, the startup and the crash inmediatly next. This error generate a database crash with signal 6, but affter the crash we can't startup the database again.
Could you tell me if you have this error in your roadmap?
Thanks!
Attachments
Issue Links
- relates to
-
-
- Closed
-
-
-
- Closed
-
-
-
- Open
-
-
-
- Closed
-
Activity
On a closer investigation, this particular bug cannot affect MariaDB versions before 10.2.2, because CREATE TEMPORARY TABLE would internally be handled not much differently from CREATE TABLE, and creating a persistent table would cause InnoDB to commit the current transaction.
Here is a reduced test case that reproduces the problem:
--source include/have_innodb.inc
|
SET @saved_frequency = @@GLOBAL.innodb_purge_rseg_truncate_frequency; |
SET GLOBAL innodb_purge_rseg_truncate_frequency = 1; |
CREATE TABLE t1(i TEXT NOT NULL) ENGINE=InnoDB; |
BEGIN; |
INSERT t1 SET i=REPEAT('1234567890',840); |
UPDATE t1 SET i=''; |
CREATE TEMPORARY TABLE t2(i SERIAL) ENGINE=InnoDB; |
INSERT INTO t2 () VALUES (); |
ROLLBACK; |
--source include/wait_all_purged.inc
|
DROP TABLE t1; |
SET GLOBAL innodb_purge_rseg_truncate_frequency = @saved_frequency; |
The assertion fails on the first (only) BLOB page. The page was originally freed as part of the ROLLBACK of the INSERT:
#0 buf_page_set_file_page_was_freed (page_id=...)
|
at /mariadb/10.2/storage/innobase/buf/buf0buf.cc:3645
|
#1 0x000055555766b057 in fseg_free_page_func (seg_header=0x7fffecbb804a "", space_id=4, page=4,
|
ahi=true, mtr=0x7fffde4992b0) at /mariadb/10.2/storage/innobase/fsp/fsp0fsp.cc:3180
|
#2 0x0000555557420c8f in btr_page_free_low (index=0x618000040508, block=0x7fffec6bf8f0, level=0,
|
blob=true, mtr=0x7fffde4992b0) at /mariadb/10.2/storage/innobase/btr/btr0btr.cc:849
|
#3 0x00005555574824c8 in btr_free_externally_stored_field (index=0x618000040508,
|
field_ref=0x7fffecbb8092 "", rec=0x7fffecbb807f "", offsets=0x61a000082908, page_zip=0x0, i=3,
|
rollback=true, local_mtr=0x7fffde499d50) at /mariadb/10.2/storage/innobase/btr/btr0cur.cc:7345
|
#4 0x0000555557482868 in btr_rec_free_externally_stored_fields (index=0x618000040508,
|
rec=0x7fffecbb807f "", offsets=0x61a000082908, page_zip=0x0, rollback=true, mtr=0x7fffde499d50)
|
at /mariadb/10.2/storage/innobase/btr/btr0cur.cc:7397
|
#5 0x0000555557478f78 in btr_cur_pessimistic_delete (err=0x7fffde499c90, has_reserved_extents=0,
|
cursor=0x61b000074778, flags=0, rollback=true, mtr=0x7fffde499d50)
|
at /mariadb/10.2/storage/innobase/btr/btr0cur.cc:5143
|
#6 0x00005555577599d5 in row_undo_ins_remove_clust_rec (node=0x61b000074708)
|
at /mariadb/10.2/storage/innobase/row/row0uins.cc:160
|
#7 0x000055555775b928 in row_undo_ins (node=0x61b000074708, thr=0x6170000437e0)
|
at /mariadb/10.2/storage/innobase/row/row0uins.cc:508
|
#8 0x00005555572d08cc in row_undo (node=0x61b000074708, thr=0x6170000437e0)
|
at /mariadb/10.2/storage/innobase/row/row0undo.cc:299
|
In the rollback of a newly inserted record it is the perfectly valid action to free the BLOBs and to delete the record.
Side note: CREATE TEMPORARY TABLE is allocating a new transaction without cleaning up the previous transaction:
#0 0x00005555573cfd52 in trx_sys_get_new_trx_id ()
|
at /mariadb/10.2/storage/innobase/include/trx0sys.ic:401
|
#1 0x00005555573d8232 in trx_start_low (trx=0x7fffecfd1fd8, read_write=true)
|
at /mariadb/10.2/storage/innobase/trx/trx0trx.cc:1216
|
#2 0x00005555573e2840 in trx_start_if_not_started_xa_low (trx=0x7fffecfd1fd8, read_write=true)
|
at /mariadb/10.2/storage/innobase/trx/trx0trx.cc:2813
|
#3 0x000055555721bff9 in row_table_add_foreign_constraints (trx=0x7fffecfd1fd8,
|
sql_string=0x62b000000288 "CREATE TEMPORARY TABLE t2(i SERIAL) ENGINE=InnoDB", sql_length=49,
|
name=0x7fffde4975e0 "mysqld.1/#sql75b4_8_0", reject_fks=1)
|
at /mariadb/10.2/storage/innobase/row/row0mysql.cc:2551
|
#4 0x0000555556f41067 in create_table_info_t::create_table (this=0x7fffde497560)
|
at /mariadb/10.2/storage/innobase/handler/ha_innodb.cc:12808
|
#5 0x0000555556f4258d in ha_innobase::create (this=0x61d000169f08,
|
name=0x7fffde49b0e0 "/mariadb/10.2/bld/mysql-test/var/tmp/mysqld.1/#sql75b4_8_0",
|
form=0x7fffde497b60, create_info=0x7fffde49c1d0)
|
at /mariadb/10.2/storage/innobase/handler/ha_innodb.cc:13010
|
#6 0x00005555569d56db in handler::ha_create (this=0x61d000169f08,
|
name=0x7fffde49b0e0 "/mariadb/10.2/bld/mysql-test/var/tmp/mysqld.1/#sql75b4_8_0",
|
form=0x7fffde497b60, info_arg=0x7fffde49c1d0) at /mariadb/10.2/sql/handler.cc:4371
|
It is of course completely bogus to add any FOREIGN KEY constraints to TEMPORARY TABLE. But removing that transaction start does not fix the crash:
diff --git a/storage/innobase/handler/ha_innodb.cc b/storage/innobase/handler/ha_innodb.cc
|
index 3f707e5e631..77915c095e5 100644
|
--- a/storage/innobase/handler/ha_innodb.cc
|
+++ b/storage/innobase/handler/ha_innodb.cc
|
@@ -12674,8 +12674,6 @@ create_table_info_t::create_table()
|
int primary_key_no;
|
uint i;
|
dict_table_t* innobase_table = NULL;
|
- const char* stmt;
|
- size_t stmt_len;
|
|
DBUG_ENTER("create_table");
|
|
@@ -12800,9 +12798,9 @@ create_table_info_t::create_table()
|
dict_table_get_all_fts_indexes(innobase_table, fts->indexes);
|
}
|
|
- stmt = innobase_get_stmt_unsafe(m_thd, &stmt_len);
|
-
|
- if (stmt) {
|
+ size_t stmt_len;
|
+ if (const char* stmt = (m_flags2 & DICT_TF2_TEMPORARY)
|
+ ? NULL : innobase_get_stmt_unsafe(m_thd, &stmt_len)) {
|
dberr_t err = row_table_add_foreign_constraints(
|
m_trx, stmt, stmt_len, m_table_name,
|
m_create_info->options & HA_LEX_CREATE_TMP_TABLE); |
The rollback is correctly initiated on the older transaction. With the above patch, no transaction ID is allocated for the CREATE TEMPORARY TABLE.
The problem is that after the record was deleted by rollback, purge is trying to deallocate the same BLOB another time. This looks like a flaw in the logic for BLOB ownership.
In InnoDB, BLOBs are normally copy-on-write and the BLOB values must be freed by purge, once old records become inaccessible. However, BLOB values are also inherited when the PRIMARY KEY of a record is updated. That is, the BLOB ownership will be transferred from the old delete-marked PRIMARY KEY record to the new record. On the purge of the old record, the BLOB must not be freed, because the record no longer 'owns' it.
In the case of ROLLBACK, the BLOB was already freed, but the update_undo log of the UPDATE was transferred to purge, and for some reason purge attempts to free the BLOB. Purge would do this by directly following the BLOB pointer in the undo log record, without trying to look up the record.
The fix might involve an update of the undo log record in ROLLBACK, so that purge will not attempt to free the BLOB. This must be done very carefully, so that in other cases (such as updating a record that was originally inserted by a different transaction) purge will not fail to free BLOBs that become inaccessible to active read views.
thiru, can you please think of a solution while I will be taking some days off from work? MDEV-13697 may provide some inspiration. Some purge-like actions are already performed in some cases of ROLLBACK.
I tried to debug this further on the long flight, but did not make a breakthrough yet.
I believe that that the reason why this test cannot be repeated without the TEMPORARY TABLE (or on MariaDB versions earlier than 10.2) is that purge fails to run (MDEV-11802).
The following is the most minimal test I was able to create so far:
--source include/have_innodb.inc
|
SET @saved_frequency = @@GLOBAL.innodb_purge_rseg_truncate_frequency; |
SET GLOBAL innodb_purge_rseg_truncate_frequency = 1; |
CREATE TEMPORARY TABLE t2(i SERIAL) ENGINE=InnoDB; |
CREATE TABLE t1(i TEXT NOT NULL) ENGINE=InnoDB; |
BEGIN; |
INSERT t1 SET i=REPEAT('1234567890',840); |
UPDATE t1 SET i=''; |
INSERT INTO t2 () VALUES (); |
ROLLBACK; |
--source include/wait_all_purged.inc
|
DROP TABLE t1; |
SET GLOBAL innodb_purge_rseg_truncate_frequency = @saved_frequency; |
I tried setting a watchpoint on trx_sys->rseg_history_len and breakpoints on row_purge_record_func and trx_undo_set_state_at_finish.
If I remove the INSERT INTO t2, purge will not run for UPDATE t1, even though trx_sys->rseg_history_len will be incremented and decremented for it.
During rollback we should rollback from last undo log record. But if the last undo log record belongs to
temporary table then we are not following that order.Rollback first process the redo based rollback segments
and then it processes the no-redo rollback segments.
In our case,
trx_roll_try_truncate()
|
executed after rollback operation of all undo logs. InnoDB executed
the no-redo rollback segments at last and its undo_no is also greater than redo rollback segment.
By result of this, it will never remove undo logs belongs to redo log segment.
I have good reason to believe that this bug is likely to affect all InnoDB versions. I remember this class of failures already in MySQL 5.6 or maybe 5.5, but back then it would not reproduce easily.