Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Fixed
-
10.5, 10.6, 10.7(EOL), 10.8(EOL), 10.9(EOL)
-
64-bit
Description
The setting innodb_flush_neighbors is broken on 64-bit platforms ever since the code was refactored in MDEV-23399 to avoid recomputing page_id_t::fold() in buf_flush_check_neighbors(). The following fixes the problem that was repeatable when attempting to flush pages in the temporary tablespace around page number 0x200002:
diff --git a/storage/innobase/include/buf0types.h b/storage/innobase/include/buf0types.h
|
--- a/storage/innobase/include/buf0types.h
|
+++ b/storage/innobase/include/buf0types.h
|
@@ -162,7 +162,10 @@ class page_id_t
|
|
/** Retrieve the fold value.
|
@return fold value */
|
- ulint fold() const { return (space() << 20) + space() + page_no(); }
|
+ ulint fold() const
|
+ {
|
+ return sizeof(ulint) == 8 ? m_id : (space() << 20) + space() + page_no();
|
+ }
|
|
/** Reset the page number only.
|
@param[in] page_no page number */ |
The problem is that the return type of page_id_t::space() is uint32_t, and therefore on 64-bit systems the fold value would not be the nonzero value (((ulint)0xfffffffe)<<20)+0xfffffffe+0x200002 (as it could be assumed on a quick read of the code), but 0, because that is what (((uint32_t)0xfffffffe)<<20)+0xfffffffe+0x200002 evaluates to.
Furthermore, there is some indication that despite MDEV-17380, neighbour flushing was attempted on a NVMe device (major number: 259, minor number 0).
The performance of the above patch will have to be tested.
Attachments
Issue Links
- is caused by
-
-
- Closed
-
Activity
I tested the performance on 10.9 with the microbenchmark that I had used in MDEV-26603. The proposed fix would actually cause a significant regression (5% at low-ish concurrency). The following patch is what was originally intended. I will test this fix with the RQG invocation.
diff --git a/storage/innobase/include/buf0types.h b/storage/innobase/include/buf0types.h
|
index 6bc46dbbc04..84e1f46cd9b 100644
|
--- a/storage/innobase/include/buf0types.h
|
+++ b/storage/innobase/include/buf0types.h
|
@@ -131,7 +131,7 @@ class page_id_t
|
|
/** Retrieve the fold value.
|
@return fold value */
|
- ulint fold() const { return (space() << 20) + space() + page_no(); }
|
+ ulint fold() const { return (ulint{space()} << 20) + space() + page_no(); }
|
|
/** Reset the page number only.
|
@param[in] page_no page number */ |
Furthermore, I found out that fil_node_t::find_metadata() realized that MDEV-25121 probably broke the SSD detection for the InnoDB system and temporary tablespace.
The test encryption.innochecksum revealed that we did not attempt to detect SSD on the temporary tablespace. The following fixes that:
diff --git a/storage/innobase/fil/fil0fil.cc b/storage/innobase/fil/fil0fil.cc
|
index 95c90ed2c1b..b461396031e 100644
|
--- a/storage/innobase/fil/fil0fil.cc
|
+++ b/storage/innobase/fil/fil0fil.cc
|
@@ -1139,7 +1139,10 @@ bool fil_space_t::open(bool create_new_db)
|
}
|
|
if (create_new_db)
|
+ {
|
+ node->find_metadata(node->handle);
|
continue;
|
+ }
|
if (skip_read)
|
{
|
size+= node->size; |
RQG test in case it needs to be done again at some point:
git clone https://github.com/MariaDB/randgen --branch mdev28371 rqg-mdev28371cd rqg-mdev28371/. ./mdev28371.cmd <basedir>Increase the number of threads in mdev28371.cmd for some concurrency, or vardir in mdev28371.cmd to change from shm to a disk, or the number of tables in mdev28371.zz for whatever.
The test didn't fail for me on 10.5, but failed reliably on 10.6+, at least in shm and on SSD.
And the stack trace to make JIRA search happy:
10.6 d1edb011
mysqld: /data/src/10.6/storage/innobase/buf/buf0flu.cc:927: bool buf_flush_check_neighbor(page_id_t, ulint, bool): Assertion `fold == id.fold()' failed.220420 19:24:07 [ERROR] mysqld got signal 6 ;#7 0x00007fdcb4807662 in __GI___assert_fail (assertion=0x55f44e3a9186 "fold == id.fold()", file=0x55f44e3a8400 "/data/src/10.6/storage/innobase/buf/buf0flu.cc", line=927, function=0x55f44e3a9150 "bool buf_flush_check_neighbor(page_id_t, ulint, bool)") at assert.c:101#8 0x000055f44dd04112 in buf_flush_check_neighbor (id=..., fold=18446744073709551615, lru=true) at /data/src/10.6/storage/innobase/buf/buf0flu.cc:927#9 0x000055f44dd0449e in buf_flush_check_neighbors (space=..., id=..., contiguous=true, lru=true) at /data/src/10.6/storage/innobase/buf/buf0flu.cc:985#10 0x000055f44dd04be2 in buf_flush_try_neighbors (space=0x55f450d9cdb8, page_id=..., contiguous=true, lru=true, n_flushed=0, n_to_flush=32) at /data/src/10.6/storage/innobase/buf/buf0flu.cc:1087#11 0x000055f44dd05dcc in buf_flush_LRU_list_batch (max=32, n=0x7fdc9ce603c0) at /data/src/10.6/storage/innobase/buf/buf0flu.cc:1308#12 0x000055f44dd06056 in buf_do_LRU_batch (max=32) at /data/src/10.6/storage/innobase/buf/buf0flu.cc:1350#13 0x000055f44dd073c8 in buf_flush_LRU (max_n=32) at /data/src/10.6/storage/innobase/buf/buf0flu.cc:1691#14 0x000055f44dd1020c in buf_LRU_get_free_block (have_mutex=false) at /data/src/10.6/storage/innobase/buf/buf0lru.cc:488#15 0x000055f44dd8fd21 in fsp_page_create (space=0x55f450d9cdb8, offset=2128790, mtr=0x7fdc9ce60b30) at /data/src/10.6/storage/innobase/fsp/fsp0fsp.cc:1078#16 0x000055f44dd94985 in fseg_alloc_free_page_low (space=0x55f450d9cdb8, seg_inode=0x7fdc9790cdb2 "", iblock=0x7fdc905d52c0, hint=2128790, direction=113 'q', has_done_reservation=true, mtr=0x7fdc9ce60b30, init_mtr=0x7fdc9ce60b30) at /data/src/10.6/storage/innobase/fsp/fsp0fsp.cc:2252#17 0x000055f44dd94b06 in fseg_alloc_free_page_general (seg_header=0x7fdc92d3004a "\377\377\377\376", hint=2128790, direction=113 'q', has_done_reservation=true, mtr=0x7fdc9ce60b30, init_mtr=0x7fdc9ce60b30) at /data/src/10.6/storage/innobase/fsp/fsp0fsp.cc:2301#18 0x000055f44dc84cbb in btr_page_alloc_low (index=0x7fdc4c5535c8, hint_page_no=2128790, file_direction=113 'q', level=0, mtr=0x7fdc9ce60b30, init_mtr=0x7fdc9ce60b30) at /data/src/10.6/storage/innobase/btr/btr0btr.cc:499#19 0x000055f44dc84d2f in btr_page_alloc (index=0x7fdc4c5535c8, hint_page_no=2128790, file_direction=113 'q', level=0, mtr=0x7fdc9ce60b30, init_mtr=0x7fdc9ce60b30) at /data/src/10.6/storage/innobase/btr/btr0btr.cc:530#20 0x000055f44dcc5a34 in btr_store_big_rec_extern_fields (pcur=0x7fdc9ce61050, offsets=0x7fdc9ce617e0, big_rec_vec=0x7fdc4c5bcab8, btr_mtr=0x7fdc9ce61140, op=BTR_STORE_INSERT) at /data/src/10.6/storage/innobase/btr/btr0cur.cc:7029#21 0x000055f44db9d258 in row_ins_index_entry_big_rec (entry=0x7fdc4c0e90e8, big_rec=0x7fdc4c5bcab8, offsets=0x7fdc9ce617e0, heap=0x7fdc9ce61670, index=0x7fdc4c5535c8, thd=0x7fdc4c000db8) at /data/src/10.6/storage/innobase/row/row0ins.cc:2483#22 0x000055f44db9e82f in row_ins_clust_index_entry_low (flags=2, mode=2, index=0x7fdc4c5535c8, n_uniq=1, entry=0x7fdc4c0e90e8, n_ext=0, thr=0x7fdc4c57bf08) at /data/src/10.6/storage/innobase/row/row0ins.cc:2798#23 0x000055f44db9fe97 in row_ins_clust_index_entry (index=0x7fdc4c5535c8, entry=0x7fdc4c0e90e8, thr=0x7fdc4c57bf08, n_ext=0) at /data/src/10.6/storage/innobase/row/row0ins.cc:3231#24 0x000055f44dba03d7 in row_ins_index_entry (index=0x7fdc4c5535c8, entry=0x7fdc4c0e90e8, thr=0x7fdc4c57bf08) at /data/src/10.6/storage/innobase/row/row0ins.cc:3357#25 0x000055f44dba0ca3 in row_ins_index_entry_step (node=0x7fdc4c57bb78, thr=0x7fdc4c57bf08) at /data/src/10.6/storage/innobase/row/row0ins.cc:3525#26 0x000055f44dba11bc in row_ins (node=0x7fdc4c57bb78, thr=0x7fdc4c57bf08) at /data/src/10.6/storage/innobase/row/row0ins.cc:3671#27 0x000055f44dba1abc in row_ins_step (thr=0x7fdc4c57bf08) at /data/src/10.6/storage/innobase/row/row0ins.cc:3817#28 0x000055f44dbc2f49 in row_insert_for_mysql (mysql_rec=0x7fdc4c577538 "\001", prebuilt=0x7fdc4c57b438, ins_mode=ROW_INS_NORMAL) at /data/src/10.6/storage/innobase/row/row0mysql.cc:1318#29 0x000055f44da03341 in ha_innobase::write_row (this=0x7fdc4c486030, record=0x7fdc4c577538 "\001") at /data/src/10.6/storage/innobase/handler/ha_innodb.cc:7922#30 0x000055f44d60174d in handler::ha_write_row (this=0x7fdc4c486030, buf=0x7fdc4c577538 "\001") at /data/src/10.6/sql/handler.cc:7545#31 0x000055f44d20c609 in write_record (thd=0x7fdc4c000db8, table=0x7fdc4c5593a8, info=0x7fdc4c015ec0, sink=0x0) at /data/src/10.6/sql/sql_insert.cc:2146#32 0x000055f44d212149 in select_insert::send_data (this=0x7fdc4c015e70, values=...) at /data/src/10.6/sql/sql_insert.cc:4110#33 0x000055f44d2fddb5 in select_result_sink::send_data_with_check (this=0x7fdc4c015e70, items=..., u=0x7fdc4c005120, sent=28) at /data/src/10.6/sql/sql_class.h:5643#34 0x000055f44d2e375e in end_send (join=0x7fdc4c015f30, join_tab=0x7fdc4c0187f8, end_of_records=false) at /data/src/10.6/sql/sql_select.cc:22337#35 0x000055f44d2e0c67 in evaluate_join_record (join=0x7fdc4c015f30, join_tab=0x7fdc4c018448, error=0) at /data/src/10.6/sql/sql_select.cc:21331#36 0x000055f44d2e0660 in sub_select (join=0x7fdc4c015f30, join_tab=0x7fdc4c018448, end_of_records=false) at /data/src/10.6/sql/sql_select.cc:21140#37 0x000055f44d2df952 in do_select (join=0x7fdc4c015f30, procedure=0x0) at /data/src/10.6/sql/sql_select.cc:20646#38 0x000055f44d2b2fd6 in JOIN::exec_inner (this=0x7fdc4c015f30) at /data/src/10.6/sql/sql_select.cc:4753#39 0x000055f44d2b204b in JOIN::exec (this=0x7fdc4c015f30) at /data/src/10.6/sql/sql_select.cc:4531#40 0x000055f44d2b3963 in mysql_select (thd=0x7fdc4c000db8, tables=0x7fdc4c014ef0, fields=..., conds=0x0, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2202244745984, result=0x7fdc4c015e70, unit=0x7fdc4c005120, select_lex=0x7fdc4c014918) at /data/src/10.6/sql/sql_select.cc:5010#41 0x000055f44d2a2c7f in handle_select (thd=0x7fdc4c000db8, lex=0x7fdc4c005058, result=0x7fdc4c015e70, setup_tables_done_option=1073741824) at /data/src/10.6/sql/sql_select.cc:545#42 0x000055f44d25e072 in mysql_execute_command (thd=0x7fdc4c000db8, is_called_from_prepared_stmt=false) at /data/src/10.6/sql/sql_parse.cc:4726#43 0x000055f44d268fea in mysql_parse (thd=0x7fdc4c000db8, rawbuf=0x7fdc4c014100 "INSERT /* QNO 3420 CON_ID 10 */ INTO `tmp` SELECT * FROM t1", length=59, parser_state=0x7fdc9ce63500) at /data/src/10.6/sql/sql_parse.cc:8045#44 0x000055f44d255625 in dispatch_command (command=COM_QUERY, thd=0x7fdc4c000db8, packet=0x7fdc4c00b769 "INSERT /* QNO 3420 CON_ID 10 */ INTO `tmp` SELECT * FROM t1 ", packet_length=60, blocking=true) at /data/src/10.6/sql/sql_parse.cc:1912#45 0x000055f44d253f52 in do_command (thd=0x7fdc4c000db8, blocking=true) at /data/src/10.6/sql/sql_parse.cc:1409#46 0x000055f44d40d97e in do_handle_one_connection (connect=0x55f450e22cf8, put_in_cache=true) at /data/src/10.6/sql/sql_connect.cc:1418#47 0x000055f44d40d61d in handle_one_connection (arg=0x55f450e22cf8) at /data/src/10.6/sql/sql_connect.cc:1312#48 0x000055f44d927fc0 in pfs_spawn_thread (arg=0x55f450e0a898) at /data/src/10.6/storage/perfschema/pfs.cc:2201#49 0x00007fdcb4cd3ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477#50 0x00007fdcb48d0def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95