Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-14085

Merge new release of InnoDB MySQL 5.7.20 to 10.2

    XMLWordPrintable

    Details

      Description

      Fixes to InnoDB native partitioning (not present in MariaDB)

      These affect ha_innopart (InnoDB native partitioning) only. MariaDB 10.2 uses the partitioning engine for all storage engines, and ha_innopart has been removed.

      commit 3f4b658db7ad6165b7d6413312e2da8e5d1ff432
      Merge: e361855f6ff 97c4cd56f26
      Author: Nisha Gopalakrishnan <nisha.gopalakrishnan@oracle.com>
      Date:   Wed Aug 23 13:04:52 2017 +0530
       
          Merge branch 'mysql-5.6' into mysql-5.7
      

      commit 234a56e4fb9407b48b6d74f5076ff6cace7c646e
      Author: Aditya A <aditya.a@oracle.com>
      Date:   Tue Jun 13 18:41:54 2017 +0530
      

      This fixes a memory leak in ha_innopart::records_in_range(). The class does not exist in MariaDB; partitioned InnoDB tables would use ha_innobase::records_in_range().

      commit 06ccfade4cf2b1ccd8b934c2a3b08dfebf1e5a5a
      Author: Aditya A <aditya.a@oracle.com>
      Date:   Mon Jun 12 23:08:20 2017 +0530
      

      Another fix to ha_innobase::records_in_range(), related to the use of DISCARD TABLESPACE.

      Fixes to partitioning outside InnoDB

      commit 9414c7035020a1915db56101aa7929d8b4069d83
      Author: Aditya A <aditya.a@oracle.com>
      Date:   Fri Aug 4 18:18:47 2017 +0530
       
          Bug#25687813    REPLICATION REGRESSION WITH RBR AND PARTITIONED TABLES
          
          PROBLEM
          -------
          
          While applying update the slave is trying to initialize all partitions
          before reading from rnd_pos() call. This is done for each row update
          because of which the performance is getting effected.
          
          FIX
          ---
          
          Initialize only the partition on which rnd_pos() is called.
      

      (No test case is included.)
      This is merging a change from MySQL 5.5. Maybe we should merge this performance fix to MariaDB 5.5 if it is not already present?

      commit be901b60ae59c93848c829d1b0b2cb523ab8692e
      Author: Nisha Gopalakrishnan <nisha.gopalakrishnan@oracle.com>
      Date:   Wed Aug 16 13:58:25 2017 +0530
       
          Bug#26390632: CREATE TABLE CAN CAUSE MYSQL TO EXIT.
          
          Analysis
          ========
          CREATE TABLE of InnoDB table with a partition name
          which exceeds the path limit can cause the server
          to exit.
          
          During the preparation of the partition name,
          there was no check to identify whether the complete
          path name for partition exceeds the max supported
          path length, causing the server to exit during
          subsequent processing.
          
          Fix
          ===
          During the preparation of partition name, check and report
          an error if the partition path name exceeds the maximum path
          name limit.
          
          This is a 5.5 patch.
      

      (No test case is included.)
      A similar fix is already present in MariaDB 5.5.57, 10.0.32, 10.1.26, 10.2.7, 10.3.1.

      Fixes applicable to Oracle encryption only (not present in MariaDB)

      commit 84c8b66cb809d5a57fd82865c7c1d28ecb6191da
      Author: Allen Lai <zheng.lai@oracle.com>
      Date:   Mon Aug 14 19:17:33 2017 +0800
       
          Fixed bug#26595476 INCORRECT ERROR HANDLING IN ROW_IMPORT_FOR_MYSQL
          
          We need to remove all the invalid code in these error handling code.
          
          Approved by Sunny Bains <sunny.bains@oracle.com>
          RB: 17022
       
      commit 2626dc593f0707e0d7e5e82ad84bab1b5b9a463c
      Author: Allen Lai <zheng.lai@oracle.com>
      Date:   Mon Jun 12 14:39:14 2017 +0800
       
          Bug#26243264 ALTER TABLE ON ENCRYPTED TABLE CAUSES FRM OUT OF SYNC AND DECRYPTS DATA
          
          This bug is cased by that we didn't set the encryption attribuite properly in altering table.
          
          Approved by Jimmy Yang <Jimmy.Yang@oracle.com>
      

      MariaDB will keep using the encryption code that was introduced in 10.1, much earlier than MySQL 5.7.9.

      Follow-up fixes to Bug #23481444 OPTIMISER CALL ROW_SEARCH_MVCC() …

      As noted in MDEV-11751, MariaDB rejected these changes that were originally introduced in MySQL 5.6.36, 5.7.18, and MySQL 8.0.1, because the benefit seemed questionable and the the changes looked risky for a GA release. There have been numerous follow-up fixes, including these ones:

      commit f5a400f43ee8ceb0c7bab1c6f34e10fafc2a162f
      Merge: 350b58f1764 14e5c10f873
      Author: Thirunarayanan Balathandayuthapani <thirunarayanan.balathandayuth@oracle.com>
      Date:   Wed Aug 2 17:15:26 2017 +0530
       
          Merge branch 'mysql-5.6' into mysql-5.7
       
      commit 14e5c10f873992d7fe581c10d68dd9b379af02f6
      Author: Thirunarayanan Balathandayuthapani <thirunarayanan.balathandayuth@oracle.com>
      Date:   Wed Aug 2 17:14:34 2017 +0530
       
      commit 350b58f1764ad1f98a72f2ad0b03b41ceb1a285f
      Merge: 5bbf4d4f37a 9a1c47f36f9
      Author: Thirunarayanan Balathandayuthapani <thirunarayanan.balathandayuth@oracle.com>
      Date:   Fri Jul 28 15:43:52 2017 +0530
       
      commit 9a1c47f36f9a2dea3ce49f35ea158f2bbd7591ab
      Author: Thirunarayanan Balathandayuthapani <thirunarayanan.balathandayuth@oracle.com>
      Date:   Fri Jul 28 15:41:13 2017 +0530
      

      There are not only no test cases, but also no commit messages. Could this be a new Oracle policy for security bugs?

      Other fixes that do not really apply

      commit 48e07ec3935ab229c8ec5aede71ae77ede65a789
      Author: Aakanksha Verma <aakanksha.verma@oracle.com>
      Date:   Wed Aug 2 18:12:02 2017 +0530
      

      There is no commit message, but there is a test case, so this probably cannot be a security bug, unless the test case was published by accident. This is a one-line addition to fts_query_filter_doc_ids(), to reset the field query->n_docs=0:

      @@ -3321,6 +3321,7 @@ func_exit:
      	if (query->total_size > fts_result_cache_limit) {
      		return(DB_FTS_EXCEED_RESULT_CACHE_LIMIT);
      	} else {
      +		query->n_docs = 0;
      		return(DB_SUCCESS);
      	}
       }
      

      The test case uses the "ngram" fulltext parser plugin and a series of queries like this, with the LIMIT increasing from 1 to 6.
      I added a similar test to 10.0.
      Apparently the change is attempting to fix a bug in the LIMIT pushdown that was introduced in MySQL 5.7.14 in
      Bug#22709692 FTS QUERY EXCEEDS RESULT CACHE LIMIT.
      That code was never enabled in MariaDB. I removed the dead code from 10.2.

      commit a7388dddaa853db1224eccf0d1df8d83af9e796f
      Merge: 852fcd21c3e 068f8261d4c
      Author: Aakanksha Verma <aakanksha.verma@oracle.com>
      Date:   Thu Jul 13 11:25:30 2017 +0530
       
          Merge branch 'mysql-5.6' into mysql-5.7
       
      commit 068f8261d4c1e134965383ff974ddf30c0758f51
      Author: Aakanksha Verma <aakanksha.verma@oracle.com>
      Date:   Thu Jul 13 11:21:24 2017 +0530
       
          Bug #24938374   MYSQL CRASHED AFTER LONG WAIT ON DICT OPERATION LOCK
          WHILE SYNCING FTS INDEX
          
          PROBLEM
          
          As part of rb#12340, dict_operation_lock was acquired to avoid dropping
          of FTS index/ table while the sync is in progress in the background. Due
          to this change server gets killed on long wait for dict_operation_lock
          while a long sync is in progress.
          
          SOLUTION
          
          We remove the dict operation lock that is held by sync (fts) in
          background. We will manage the concurrency of drop index and sync index
          in 2 ways:
          1) If sync is already in progress, Drop index would wait for sync to
          complete.
          2) If alter is already happening and sync is invoked then sync would do
          a check for index/table to be dropped flags are set or not, if it is it
          would skip syncing that index.
          
          Reviewed by: Jimmy Yang <Jimmy.Yang@oracle.com>
          RB: 15344
      

      We will get this by a merge from MySQL 5.6.38 to MariaDB 10.0. There is no need to pick this from 5.7.20.

      commit 259ee176fe54a99a838203c8bf9b2bb5e8434b0a
      Author: Thirunarayanan Balathandayuthapani <thirunarayanan.balathandayuth@oracle.com>
      Date:   Thu Jul 6 18:28:26 2017 +0530
      

      There is a test case, but no commit message.
      It is obvious that srv_buf_dump_event is only created if innodb_read_only=0. But, the proper target is 10.0, not 10.2. Why did not Oracle fix this potentially crashing bug in 5.6?
      I pushed a more complete fix and a more efficient test in
      MDEV-14086 Setting innodb_buffer_pool_load_now or innodb_buffer_load_abort will crash if innodb_read_only

      commit c39ffbd0947b48b05916d0163d2c64e976d49ba0
      Merge: 1e2fc773309 d56147ea0c4
      Author: Nisha Gopalakrishnan <nisha.gopalakrishnan@oracle.com>
      Date:   Tue Jun 20 16:16:02 2017 +0530
       
          Merge branch 'mysql-5.6' into mysql-5.7
       
      commit d56147ea0c4844102f35b3c033fd4fb1724c1c67
      Author: Nisha Gopalakrishnan <nisha.gopalakrishnan@oracle.com>
      Date:   Tue Jun 20 16:12:54 2017 +0530
      

      This one introduces a test case and quite a few changes to fulltext code outside InnoDB. The test case even discloses the bug number and title:

      Bug#21140111: Explain ... match against: Assertion failed: ret ...
      

      Apparently no fix is strictly needed in MariaDB, because the test case does not crash 10.0, 10.1 or 10.2.
      I pushed an adapted and extended version of the test case to 10.0.
      In InnoDB, the only change is plugging a potential memory leak and non-debug assertion failure on query->intersection in fts_query_free(). I was unable to repeat any problem, but I pushed a fix to 10.0 just in case.

      Changes that we can pick from MySQL 5.7.20 to MariaDB 10.2

      commit b7e496f0b23530566211a410a903aedb97c4292c
      Author: Darshan M N <darshan.m.n@oracle.com>
      Date:   Thu Jun 1 10:14:05 2017 +0530
       
          BUG#25479538 ASSERT:SIZE == SPACE->SIZE DURING BUF_READ_AHEAD_RANDOM
          
          Issue
          =====
          The original issue was that the size of a fil_per_table tablespace was calculated
          incorrectly during truncate in the presence of an fts index. This incorrect calculation
          was fixed as part of BUG#25053705 along with a testcase to reproduce the bug. The
          assert that was added as part of it to reproduce the bug was wrong and resulted in
          this bug.
          
          Fix
          ===
          Although the assert was removed earlier in a seperate commit as it was blocking the
          ntest, this patch replaces the other parts of the code that were added to reproduce
          the bug and replaces it with code that tries to reproduce the bug in a different way.
          
          The new code basically tries to tweak conditions so as to simulate the random read
          where a page that doesn't exist is tried to be read.
          
          RB: 15890
          Reviewed-by: Jimmy Yang <Jimmy.Yang@oracle.com>
          Reviewed-by: Satya Bodapati <satya.bodapati@oracle.com>
      

      This patch is only adding debug instrumentation and a test. Should be safe to apply. This is applicable to 10.2 only, because this bug was introduced by the WL#6501 TRUNCATE changes.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              marko Marko Mäkelä
              Reporter:
              marko Marko Mäkelä
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: