[MDEV-15528] Avoid writing freed InnoDB pages - Jira

Details

Type: Bug
Status: Closed (View Workflow)
Priority: Major
Resolution: Fixed
Affects Version/s: 10.1.3, 10.2.0, 10.3.0
Fix Version/s: 10.5.2
Component/s: Storage Engine - InnoDB
Labels:
- performance

Description

When a InnoDB data file page is freed, its contents becomes garbage, and any storage allocated in the data file is wasted.

MariaDB 10.4 introduced an InnoDB redo log record MLOG_INIT_FREE_PAGE for marking pages as freed. In ~~MDEV-12353~~ (MariaDB 10.5.2), that record was replaced with FREE_PAGE. This record could be treated as no-op, or we can punch a hole for page_compressed=1 tables.

If innodb_immediate_scrub_data_uncompressed is set, we should initialize the page with zeros. This will replace some of the non-working scrubbing logic (~~MDEV-8139~~). The scrubbing will be fixed further in ~~MDEV-8139~~.

The following parameters will be deprecated and ignored and the problematic ‘background scrubbing’ code removed:

innodb-background-scrub-data-uncompressed
innodb-background-scrub-data-compressed
innodb-background-scrub-data-interval
innodb-background-scrub-data-check-interval

For page_compressed tables the freed page will be hole-punched

Attachments

Issue Links

blocks

MDEV-12227 Defer writes to the InnoDB temporary tablespace

Closed

MDEV-17596 Assertion `block->page.flush_observer == __null || block->page.flush_observer == observer' failed in buf_flush_note_modification

Closed

MDEV-18724 Replace buf_block_t::mutex with more std::atomic

Closed

causes

MDEV-22096 Mariabackup copied too old page or too new checkpoint

Closed

MDEV-22097 Not applying DELETE_ROW_FORMAT_REDUNDANT due to corruption

Closed

MDEV-22103 INNODB_ENCRYPTION_NUM_KEY_REQUESTS is missing from INFORMATION_SCHEMA.GLOBAL_STATUS

Closed

MDEV-22139 fseg_free_page_low() fails to write FREE_PAGE record, breaking recovery

Closed

MDEV-22169 Recovery fails after failing to insert into mlog_init

Closed

MDEV-22495 Assertion mode == 16 || mode == 12 || fix_block->page.status != buf_page_t::FREED failed in ibuf_read_merge_pages()

Closed

MDEV-22710 Assertion `mode == 16 || mode == 12 || fix_block->page.status != buf_page_t::FREED' failed in buf_page_get_low

Closed

MDEV-23252 Assertion failure 'req_type.is_dblwr_recover() || err == DB_SUCCESS' for page_compressed tables

Closed

MDEV-24695 Encryption is modifying a freed page

Closed

MDEV-27500 buf_page_free() fails to drop the adaptive hash index

Closed

MDEV-27985 buf_flush_freed_pages() causes InnoDB to hang

Closed

MDEV-30404 Inconsistent updates of PAGE_MAX_TRX_ID on ROW_FORMAT=COMPRESSED pages

Closed

MDEV-30438 innodb.undo_truncate,4k fails when --innodb-immediate-scrub-data-uncompressed is enabled

Closed

MDEV-32552 Write-ahead logging is broken for freed pages

Closed

relates to

MDEV-11696 Page Compression Has No Effect on Table Data File Size

Closed

MDEV-16526 Overhaul the InnoDB page flushing

Closed

MDEV-17380 innodb_flush_neighbors=ON should be ignored on SSD

Closed

MDEV-21952 ibdata1 file size growing in MariaDB

Closed

MDEV-23973 Change buffer corruption when reallocating an recently freed page

Closed

MDEV-24569 Assertion `mach_read_from_4(frame + 4U) == block.page.id().page_no()' failed in log_phys_t::apply

Closed

MDEV-25026 Various code paths are accessing freed pages

Closed

MDEV-28699 Shrink temporary tablespaces without restart

Closed

MDEV-31816 buf_LRU_free_page() does not preserve ROW_FORMAT=COMPRESSED block state

Closed

MDEV-33112 innodb_undo_log_truncate=ON is blocking page writes

Closed

MDEV-8139 Fix scrubbing

Closed

MDEV-11068 Review which innodb_compression_algorithm to support in binary packages

Closed

MDEV-12226 Avoid writes of freed (garbage) pages to InnoDB data files

Closed

MDEV-12699 Improve crash recovery of corrupted data pages

Closed

MDEV-15949 InnoDB: Failing assertion: space->n_pending_ops == 0 in fil_delete_tablespace upon DROP TABLE

Closed

MDEV-16796 TRUNCATE TABLE slowdown with innodb_file_per_table=ON

Closed

MDEV-17380 innodb_flush_neighbors=ON should be ignored on SSD

Closed

MDEV-18698 Show InnoDB's internal background threads in SHOW ENGINE INNODB STATUS

Open

MDEV-20813 Assertion `!srv_safe_truncate || !newest_lsn || fil_page_get_type(page)' or `!newest_lsn || fil_page_get_type(page)' failed in buf_flush_init_for_writing

Closed

MDEV-22190 After IMPORT: InnoDB: Record 126 is above rec heap top 120

Closed

MDEV-22839 ROW_FORMAT=COMPRESSED vs PAGE_COMPRESSION=1 - size comparison

Open

MDEV-24780 [FATAL] InnoDB: Trying to read page number <...> with encryption and innodb-background-scrub-data-uncompressed

Closed

(12 causes, 22 relates to)

Activity

Ascending order - Click to sort in descending order

Marko Mäkelä added a comment - 2019-02-08 17:18

MariaDB 10.4.3 will introduce a MLOG_INIT_FREE_PAGE record for this purpose. This allows this bug to be fixed in MariaDB 10.4 later without breaking crash-downgrade to earlier MariaDB 10.4 versions.

In ~~MDEV-12699~~, MLOG_INIT_FREE_PAGE should be handled in a similar way as the existing record MLOG_INIT_FILE_PAGE2.

Marko Mäkelä added a comment - 2019-02-08 17:18 MariaDB 10.4.3 will introduce a MLOG_INIT_FREE_PAGE record for this purpose. This allows this bug to be fixed in MariaDB 10.4 later without breaking crash-downgrade to earlier MariaDB 10.4 versions. In MDEV-12699 , MLOG_INIT_FREE_PAGE should be handled in a similar way as the existing record MLOG_INIT_FILE_PAGE2 .

Marko Mäkelä added a comment - 2019-04-08 14:56

I think that this is realistic to do in the 10.4 time frame. Assigning to myself, because this is closely related to ~~MDEV-12699~~, which I am working on.

Marko Mäkelä added a comment - 2019-04-08 14:56 I think that this is realistic to do in the 10.4 time frame. Assigning to myself, because this is closely related to MDEV-12699 , which I am working on.

Marko Mäkelä added a comment - 2019-04-18 14:37

When scrubbing is not enabled and the page is not on SSD and the file is not page_compressed, we can discard the page from the buf_pool->flush_list once the MLOG_INIT_FREE_PAGE record has been written. For scrubbing, it is kind of mandatory to initialize the page.

For non-compressed tables on SSD, we will have to evaluate whether punching holes is going to improve or hurt performance on the average.

Marko Mäkelä added a comment - 2019-04-18 14:37 When scrubbing is not enabled and the page is not on SSD and the file is not page_compressed , we can discard the page from the buf_pool->flush_list once the MLOG_INIT_FREE_PAGE record has been written. For scrubbing, it is kind of mandatory to initialize the page. For non-compressed tables on SSD, we will have to evaluate whether punching holes is going to improve or hurt performance on the average.

Marko Mäkelä added a comment - 2020-01-27 15:56

As part of this, we should simplify the freeing of B-tree root pages:

diff --git a/storage/innobase/btr/btr0btr.cc b/storage/innobase/btr/btr0btr.cc

index 753ed9b077c..21609edb7f3 100644

--- a/storage/innobase/btr/btr0btr.cc

+++ b/storage/innobase/btr/btr0btr.cc

@@ -961,12 +961,9 @@ constexpr index_id_t	BTR_FREED_INDEX_ID = 0;

 /** Free a B-tree root page. btr_free_but_not_root() must already

 have been called.

-In a persistent tablespace, the caller must invoke fsp_init_file_page()

-before mtr.commit().

 @param[in,out]	block		index root page

-@param[in,out]	mtr		mini-transaction

-@param[in]	invalidate	whether to invalidate PAGE_INDEX_ID */

-static void btr_free_root(buf_block_t *block, mtr_t *mtr, bool invalidate)

+@param[in,out]	mtr		mini-transaction */

+static void btr_free_root(buf_block_t *block, mtr_t *mtr)

   ut_ad(mtr_memo_contains_flagged(mtr, block,

 				  MTR_MEMO_PAGE_X_FIX | MTR_MEMO_PAGE_SX_FIX));

(Obviously, also the code inside if (invalidate) should be removed and the callers be adjusted.)
We should ensure that MLOG_INIT_FREE_PAGE records will be emitted by fseg_free_step(), and thus no log needs to be written in this function.

Marko Mäkelä added a comment - 2020-01-27 15:56 As part of this, we should simplify the freeing of B-tree root pages: diff --git a/storage/innobase/btr/btr0btr.cc b/storage/innobase/btr/btr0btr.cc index 753ed9b077c..21609edb7f3 100644 --- a/storage/innobase/btr/btr0btr.cc +++ b/storage/innobase/btr/btr0btr.cc @@ -961,12 +961,9 @@ constexpr index_id_t BTR_FREED_INDEX_ID = 0; /** Free a B-tree root page. btr_free_but_not_root() must already have been called. -In a persistent tablespace, the caller must invoke fsp_init_file_page() -before mtr.commit(). @param[in,out] block index root page -@param[in,out] mtr mini-transaction -@param[in] invalidate whether to invalidate PAGE_INDEX_ID */ -static void btr_free_root(buf_block_t *block, mtr_t *mtr, bool invalidate) +@param[in,out] mtr mini-transaction */ +static void btr_free_root(buf_block_t *block, mtr_t *mtr) { ut_ad(mtr_memo_contains_flagged(mtr, block, MTR_MEMO_PAGE_X_FIX | MTR_MEMO_PAGE_SX_FIX)); (Obviously, also the code inside if (invalidate) should be removed and the callers be adjusted.) We should ensure that MLOG_INIT_FREE_PAGE records will be emitted by fseg_free_step() , and thus no log needs to be written in this function.

Marko Mäkelä added a comment - 2020-03-09 14:45

This looks OK to me, after addressing my feedback for commit 1 of 2. We will address the following later, ideally before the 10.5 GA release:

clean up btr_free_root() as I suggested in my previous comment
address the FIXME comments in buf_page_free() by introducing a data structure that covers freed page ranges also for pages that are not in the buffer pool, and enable the scrubbing tests (~~MDEV-8139~~)

Marko Mäkelä added a comment - 2020-03-09 14:45 This looks OK to me, after addressing my feedback for commit 1 of 2 . We will address the following later, ideally before the 10.5 GA release: clean up btr_free_root() as I suggested in my previous comment address the FIXME comments in buf_page_free() by introducing a data structure that covers freed page ranges also for pages that are not in the buffer pool, and enable the scrubbing tests ( MDEV-8139 )

Thirunarayanan Balathandayuthapani added a comment - 2020-03-10 06:20

commit a5584b13d1e04f38b843602413669591aa65c359 (HEAD -> 10.5, origin/10.5, 10.5-work)

Author: Thirunarayanan Balathandayuthapani <thiru@mariadb.com>

Date:   Tue Mar 10 10:47:24 2020 +0530

commit a35b4ae89871d8184f04abb112c385481d557dbb

Author: Thirunarayanan Balathandayuthapani <thiru@mariadb.com>

Date:   Mon Mar 9 13:25:33 2020 +0530

    MDEV-15528 Punch holes when pages are freed

Patch has been pushed to 10.5. Thanks to marko and @matthias.leich for testing abd reviewing it

Thirunarayanan Balathandayuthapani added a comment - 2020-03-10 06:20 commit a5584b13d1e04f38b843602413669591aa65c359 (HEAD -> 10.5, origin/10.5, 10.5-work) Author: Thirunarayanan Balathandayuthapani <thiru@mariadb.com> Date: Tue Mar 10 10:47:24 2020 +0530 commit a35b4ae89871d8184f04abb112c385481d557dbb Author: Thirunarayanan Balathandayuthapani <thiru@mariadb.com> Date: Mon Mar 9 13:25:33 2020 +0530 MDEV-15528 Punch holes when pages are freed Patch has been pushed to 10.5. Thanks to marko and @matthias.leich for testing abd reviewing it

People

Assignee:: Thirunarayanan Balathandayuthapani

Reporter:: Marko Mäkelä

Votes:: 0 Vote for this issue

Watchers:: 5 Start watching this issue

Dates

Created:: 2018-03-09 13:00

Updated:: 2024-08-08 06:33

Resolved:: 2020-03-10 06:20

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.

MariaDB Server