Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-7577

LOAD DATA INFILE is fast on a master but generates much more I/O when replicated

    XMLWordPrintable

Details

    Description

      LOAD DATA INFILE is intended for fast, bulk imports into MySQL. This works pretty well.

      However, when replicating the content of the file is saved to bin logs on the master, but on the slave, it is:
      1. downloaded into the relay logs [normal processing]
      2. extracted from the relay logs into a temporary file
      3. re-imported from the temporary file into the slave
      4. the temporary file is removed.

      This extra step 2 generates unneeded I/O and can slow the process on the downstream slaves.
      Optimizing it might be good for replication performance when the loaded data sizes are large.

      One thought is to convert the LOAD DATA INFILE content when writing the bin logs into a "normal RBR binlog stream”,
      so that on the slave it would be processed normally. Given this behavior is different to the default behavior it
      would probably need to be optionally enabled with a specific config option on the master.

      This feature request is created to document a previous discussion on that topic with the MariaDB engineering team.

      Attachments

        Activity

          People

            Unassigned Unassigned
            ivan.stoykov@skysql.com Stoykov (Inactive)
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: