Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-25909

Unnecessary calls to fil_ibd_load() slow down recovery

    XMLWordPrintable

    Details

      Description

      As I noted in MDEV-25852, recovery should probably maintain a mapping from file names to tablespace IDs, so that it can avoid repeated reads of the same page. We are not only unnecessarily re-reading files during recovery, but also displaying messages about it:

      10.6 6fbf978eec4506eb46737ac4da00ea04403ae855

      2021-06-10 20:59:34 0 [Note] InnoDB: Ignoring data file './test/#sql-alter-3116aa-53.ibd' with space ID 1594. Another data file called ./cool_down/t3.ibd exists with the same space ID.
      2021-06-10 20:59:34 0 [Note] InnoDB: Ignoring data file './test/#sql-alter-3116aa-53.ibd' with space ID 1594. Another data file called ./cool_down/t3.ibd exists with the same space ID.
      2021-06-10 20:59:34 0 [Note] InnoDB: Ignoring data file './test/#sql-alter-3116aa-53.ibd' with space ID 1594. Another data file called ./cool_down/t3.ibd exists with the same space ID.
      2021-06-10 20:59:34 0 [Note] InnoDB: Ignoring data file './test/#sql-alter-3116aa-53.ibd' with space ID 1606. Another data file called ./cool_down/t1.ibd exists with the same space ID.
      2021-06-10 20:59:34 0 [Note] InnoDB: Ignoring data file './test/#sql-alter-3116aa-53.ibd' with space ID 1606. Another data file called ./cool_down/t1.ibd exists with the same space ID.
      2021-06-10 20:59:34 0 [Note] InnoDB: Ignoring data file './test/#sql-alter-3116aa-53.ibd' with space ID 1606. Another data file called ./cool_down/t1.ibd exists with the same space ID.
      2021-06-10 20:59:34 0 [Note] InnoDB: Ignoring data file './test/#sql-alter-3116aa-57.ibd' with space ID 1624. Another data file called ./cool_down/t4.ibd exists with the same space ID.
      2021-06-10 20:59:34 0 [Note] InnoDB: Ignoring data file './test/#sql-alter-3116aa-57.ibd' with space ID 1624. Another data file called ./cool_down/t4.ibd exists with the same space ID.
      2021-06-10 20:59:34 0 [Note] InnoDB: Ignoring data file './test/#sql-alter-3116aa-57.ibd' with space ID 1624. Another data file called ./cool_down/t4.ibd exists with the same space ID.
      2021-06-10 20:59:35 0 [Note] InnoDB: Ignoring data file './test/#sql-alter-3116aa-57.ibd' with space ID 1811. Another data file called ./test/t4.ibd exists with the same space ID.
      2021-06-10 20:59:35 0 [Note] InnoDB: Ignoring data file './test/#sql-alter-3116aa-57.ibd' with space ID 1811. Another data file called ./test/t4.ibd exists with the same space ID.
      2021-06-10 20:59:35 0 [Note] InnoDB: Ignoring data file './test/#sql-alter-3116aa-57.ibd' with space ID 1811. Another data file called ./test/t4.ibd exists with the same space ID.
      

      In deferred_spaces.add() the existence of a name-to-ID mapping would avoid a full traversal of the data structure:

          char *fil_path= fil_make_filepath(nullptr, {filename, strlen(filename)},
                                            IBD, false);
          const item defer{lsn, fil_path, false};
          ut_free(fil_path);
       
          /* The file name must be unique. Keep the one with the latest LSN. */
          auto d= defers.begin();
       
          while (d != defers.end())
          {
            if (d->second.file_name != defer.file_name)
              ++d;
      

      Here, also the call to fil_make_filepath() could be unnecessary and possibly should be removed. (It has to be tested extensively with DATA DIRECTORY.)
      Last, somewhat related to this (and MDEV-24626), I think that Datafile::read_first_page() needs to be simplified. We should only read the first innodb_page_size bytes from the file. If the file is shorter than that, or all bytes are NUL, we will use deferred_spaces. If the FSP_SPACE_FLAGS matches our innodb_page_size and the checksum are valid, we will return the tablespace ID. Else, we will use deferred_spaces.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              thiru Thirunarayanan Balathandayuthapani
              Reporter:
              marko Marko Mäkelä
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:

                  Git Integration