[MDEV-14085] Merge new release of InnoDB MySQL 5.7.20 to 10.2 Created: 2017-10-18  Updated: 2018-01-16  Resolved: 2017-10-18

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 10.2, 10.3
Fix Version/s: 10.2.10, 10.3.3

Type: Bug Priority: Major
Reporter: Marko Mäkelä Assignee: Marko Mäkelä
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Blocks
blocks MDEV-14958 Merge new release of InnoDB MySQL 5.7... Closed
is blocked by MDEV-11751 Merge new release of InnoDB MySQL 5.7... Closed
is blocked by MDEV-13481 Merge new release of InnoDB MySQL 5.7... Closed
Relates
relates to MDEV-14086 Setting innodb_buffer_pool_load_now o... Closed
relates to MDEV-12547 InnoDB FULLTEXT index has too strict ... Closed

 Description   

Fixes to InnoDB native partitioning (not present in MariaDB)

These affect ha_innopart (InnoDB native partitioning) only. MariaDB 10.2 uses the partitioning engine for all storage engines, and ha_innopart has been removed.

commit 3f4b658db7ad6165b7d6413312e2da8e5d1ff432
Merge: e361855f6ff 97c4cd56f26
Author: Nisha Gopalakrishnan <nisha.gopalakrishnan@oracle.com>
Date:   Wed Aug 23 13:04:52 2017 +0530
 
    Merge branch 'mysql-5.6' into mysql-5.7

commit 234a56e4fb9407b48b6d74f5076ff6cace7c646e
Author: Aditya A <aditya.a@oracle.com>
Date:   Tue Jun 13 18:41:54 2017 +0530

This fixes a memory leak in ha_innopart::records_in_range(). The class does not exist in MariaDB; partitioned InnoDB tables would use ha_innobase::records_in_range().

commit 06ccfade4cf2b1ccd8b934c2a3b08dfebf1e5a5a
Author: Aditya A <aditya.a@oracle.com>
Date:   Mon Jun 12 23:08:20 2017 +0530

Another fix to ha_innobase::records_in_range(), related to the use of DISCARD TABLESPACE.

Fixes to partitioning outside InnoDB

commit 9414c7035020a1915db56101aa7929d8b4069d83
Author: Aditya A <aditya.a@oracle.com>
Date:   Fri Aug 4 18:18:47 2017 +0530
 
    Bug#25687813    REPLICATION REGRESSION WITH RBR AND PARTITIONED TABLES
    
    PROBLEM
    -------
    
    While applying update the slave is trying to initialize all partitions
    before reading from rnd_pos() call. This is done for each row update
    because of which the performance is getting effected.
    
    FIX
    ---
    
    Initialize only the partition on which rnd_pos() is called.

(No test case is included.)
This is merging a change from MySQL 5.5. Maybe we should merge this performance fix to MariaDB 5.5 if it is not already present?

commit be901b60ae59c93848c829d1b0b2cb523ab8692e
Author: Nisha Gopalakrishnan <nisha.gopalakrishnan@oracle.com>
Date:   Wed Aug 16 13:58:25 2017 +0530
 
    Bug#26390632: CREATE TABLE CAN CAUSE MYSQL TO EXIT.
    
    Analysis
    ========
    CREATE TABLE of InnoDB table with a partition name
    which exceeds the path limit can cause the server
    to exit.
    
    During the preparation of the partition name,
    there was no check to identify whether the complete
    path name for partition exceeds the max supported
    path length, causing the server to exit during
    subsequent processing.
    
    Fix
    ===
    During the preparation of partition name, check and report
    an error if the partition path name exceeds the maximum path
    name limit.
    
    This is a 5.5 patch.

(No test case is included.)
A similar fix is already present in MariaDB 5.5.57, 10.0.32, 10.1.26, 10.2.7, 10.3.1.

Fixes applicable to Oracle encryption only (not present in MariaDB)

commit 84c8b66cb809d5a57fd82865c7c1d28ecb6191da
Author: Allen Lai <zheng.lai@oracle.com>
Date:   Mon Aug 14 19:17:33 2017 +0800
 
    Fixed bug#26595476 INCORRECT ERROR HANDLING IN ROW_IMPORT_FOR_MYSQL
    
    We need to remove all the invalid code in these error handling code.
    
    Approved by Sunny Bains <sunny.bains@oracle.com>
    RB: 17022
 
commit 2626dc593f0707e0d7e5e82ad84bab1b5b9a463c
Author: Allen Lai <zheng.lai@oracle.com>
Date:   Mon Jun 12 14:39:14 2017 +0800
 
    Bug#26243264 ALTER TABLE ON ENCRYPTED TABLE CAUSES FRM OUT OF SYNC AND DECRYPTS DATA
    
    This bug is cased by that we didn't set the encryption attribuite properly in altering table.
    
    Approved by Jimmy Yang <Jimmy.Yang@oracle.com>

MariaDB will keep using the encryption code that was introduced in 10.1, much earlier than MySQL 5.7.9.

Follow-up fixes to Bug #23481444 OPTIMISER CALL ROW_SEARCH_MVCC() …

As noted in MDEV-11751, MariaDB rejected these changes that were originally introduced in MySQL 5.6.36, 5.7.18, and MySQL 8.0.1, because the benefit seemed questionable and the the changes looked risky for a GA release. There have been numerous follow-up fixes, including these ones:

commit f5a400f43ee8ceb0c7bab1c6f34e10fafc2a162f
Merge: 350b58f1764 14e5c10f873
Author: Thirunarayanan Balathandayuthapani <thirunarayanan.balathandayuth@oracle.com>
Date:   Wed Aug 2 17:15:26 2017 +0530
 
    Merge branch 'mysql-5.6' into mysql-5.7
 
commit 14e5c10f873992d7fe581c10d68dd9b379af02f6
Author: Thirunarayanan Balathandayuthapani <thirunarayanan.balathandayuth@oracle.com>
Date:   Wed Aug 2 17:14:34 2017 +0530
 
commit 350b58f1764ad1f98a72f2ad0b03b41ceb1a285f
Merge: 5bbf4d4f37a 9a1c47f36f9
Author: Thirunarayanan Balathandayuthapani <thirunarayanan.balathandayuth@oracle.com>
Date:   Fri Jul 28 15:43:52 2017 +0530
 
commit 9a1c47f36f9a2dea3ce49f35ea158f2bbd7591ab
Author: Thirunarayanan Balathandayuthapani <thirunarayanan.balathandayuth@oracle.com>
Date:   Fri Jul 28 15:41:13 2017 +0530

There are not only no test cases, but also no commit messages. Could this be a new Oracle policy for security bugs?

Other fixes that do not really apply

commit 48e07ec3935ab229c8ec5aede71ae77ede65a789
Author: Aakanksha Verma <aakanksha.verma@oracle.com>
Date:   Wed Aug 2 18:12:02 2017 +0530

There is no commit message, but there is a test case, so this probably cannot be a security bug, unless the test case was published by accident. This is a one-line addition to fts_query_filter_doc_ids(), to reset the field query->n_docs=0:

@@ -3321,6 +3321,7 @@ func_exit:
	if (query->total_size > fts_result_cache_limit) {
		return(DB_FTS_EXCEED_RESULT_CACHE_LIMIT);
	} else {
+		query->n_docs = 0;
		return(DB_SUCCESS);
	}
 }

The test case uses the "ngram" fulltext parser plugin and a series of queries like this, with the LIMIT increasing from 1 to 6.
I added a similar test to 10.0.
Apparently the change is attempting to fix a bug in the LIMIT pushdown that was introduced in MySQL 5.7.14 in
Bug#22709692 FTS QUERY EXCEEDS RESULT CACHE LIMIT.
That code was never enabled in MariaDB. I removed the dead code from 10.2.

commit a7388dddaa853db1224eccf0d1df8d83af9e796f
Merge: 852fcd21c3e 068f8261d4c
Author: Aakanksha Verma <aakanksha.verma@oracle.com>
Date:   Thu Jul 13 11:25:30 2017 +0530
 
    Merge branch 'mysql-5.6' into mysql-5.7
 
commit 068f8261d4c1e134965383ff974ddf30c0758f51
Author: Aakanksha Verma <aakanksha.verma@oracle.com>
Date:   Thu Jul 13 11:21:24 2017 +0530
 
    Bug #24938374   MYSQL CRASHED AFTER LONG WAIT ON DICT OPERATION LOCK
    WHILE SYNCING FTS INDEX
    
    PROBLEM
    
    As part of rb#12340, dict_operation_lock was acquired to avoid dropping
    of FTS index/ table while the sync is in progress in the background. Due
    to this change server gets killed on long wait for dict_operation_lock
    while a long sync is in progress.
    
    SOLUTION
    
    We remove the dict operation lock that is held by sync (fts) in
    background. We will manage the concurrency of drop index and sync index
    in 2 ways:
    1) If sync is already in progress, Drop index would wait for sync to
    complete.
    2) If alter is already happening and sync is invoked then sync would do
    a check for index/table to be dropped flags are set or not, if it is it
    would skip syncing that index.
    
    Reviewed by: Jimmy Yang <Jimmy.Yang@oracle.com>
    RB: 15344

We will get this by a merge from MySQL 5.6.38 to MariaDB 10.0. There is no need to pick this from 5.7.20.

commit 259ee176fe54a99a838203c8bf9b2bb5e8434b0a
Author: Thirunarayanan Balathandayuthapani <thirunarayanan.balathandayuth@oracle.com>
Date:   Thu Jul 6 18:28:26 2017 +0530

There is a test case, but no commit message.
It is obvious that srv_buf_dump_event is only created if innodb_read_only=0. But, the proper target is 10.0, not 10.2. Why did not Oracle fix this potentially crashing bug in 5.6?
I pushed a more complete fix and a more efficient test in
MDEV-14086 Setting innodb_buffer_pool_load_now or innodb_buffer_load_abort will crash if innodb_read_only

commit c39ffbd0947b48b05916d0163d2c64e976d49ba0
Merge: 1e2fc773309 d56147ea0c4
Author: Nisha Gopalakrishnan <nisha.gopalakrishnan@oracle.com>
Date:   Tue Jun 20 16:16:02 2017 +0530
 
    Merge branch 'mysql-5.6' into mysql-5.7
 
commit d56147ea0c4844102f35b3c033fd4fb1724c1c67
Author: Nisha Gopalakrishnan <nisha.gopalakrishnan@oracle.com>
Date:   Tue Jun 20 16:12:54 2017 +0530

This one introduces a test case and quite a few changes to fulltext code outside InnoDB. The test case even discloses the bug number and title:

Bug#21140111: Explain ... match against: Assertion failed: ret ...

Apparently no fix is strictly needed in MariaDB, because the test case does not crash 10.0, 10.1 or 10.2.
I pushed an adapted and extended version of the test case to 10.0.
In InnoDB, the only change is plugging a potential memory leak and non-debug assertion failure on query->intersection in fts_query_free(). I was unable to repeat any problem, but I pushed a fix to 10.0 just in case.

Changes that we can pick from MySQL 5.7.20 to MariaDB 10.2

commit b7e496f0b23530566211a410a903aedb97c4292c
Author: Darshan M N <darshan.m.n@oracle.com>
Date:   Thu Jun 1 10:14:05 2017 +0530
 
    BUG#25479538 ASSERT:SIZE == SPACE->SIZE DURING BUF_READ_AHEAD_RANDOM
    
    Issue
    =====
    The original issue was that the size of a fil_per_table tablespace was calculated
    incorrectly during truncate in the presence of an fts index. This incorrect calculation
    was fixed as part of BUG#25053705 along with a testcase to reproduce the bug. The
    assert that was added as part of it to reproduce the bug was wrong and resulted in
    this bug.
    
    Fix
    ===
    Although the assert was removed earlier in a seperate commit as it was blocking the
    ntest, this patch replaces the other parts of the code that were added to reproduce
    the bug and replaces it with code that tries to reproduce the bug in a different way.
    
    The new code basically tries to tweak conditions so as to simulate the random read
    where a page that doesn't exist is tried to be read.
    
    RB: 15890
    Reviewed-by: Jimmy Yang <Jimmy.Yang@oracle.com>
    Reviewed-by: Satya Bodapati <satya.bodapati@oracle.com>

This patch is only adding debug instrumentation and a test. Should be safe to apply. This is applicable to 10.2 only, because this bug was introduced by the WL#6501 TRUNCATE changes.


Generated at Thu Feb 08 08:10:48 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.