Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-30266

Preparing a mariabackup on a mounted CIFS / SMB file system fails

Details

    • Bug
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • 10.6.11, 10.7.7, 10.8.6, 10.9.4, 10.10.2, 10.11.1
    • 10.6, 10.11
    • Backup
    • None

    Description

      Running the --backup stage of mariabackup with the target-dir being on a mounted CIFS share works fine, but the --prepare stage then fails early with an IO error:

      root@ubuntu-jammy:~# strace -f -o log -s10000  mariabackup --prepare --target-dir=/mnt/X
      mariabackup based on MariaDB server 10.9.4-MariaDB debian-linux-gnu (x86_64)
      [00] 2022-12-19 07:59:12 cd to /mnt/X/
      [00] 2022-12-19 07:59:12 open files limit requested 0, set to 1024
      [00] 2022-12-19 07:59:12 This target seems to be not prepared yet.
      [00] 2022-12-19 07:59:12 mariabackup: using the following InnoDB configuration for recovery:
      [00] 2022-12-19 07:59:12 innodb_data_home_dir = .
      [00] 2022-12-19 07:59:12 innodb_data_file_path = ibdata1:12M:autoextend
      [00] 2022-12-19 07:59:12 innodb_log_group_home_dir = .
      [00] 2022-12-19 07:59:12 Starting InnoDB instance for recovery.
      [00] 2022-12-19 07:59:12 mariabackup: Using 104857600 bytes for buffer pool (set by --use-memory parameter)
      2022-12-19  7:59:12 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
      2022-12-19  7:59:12 0 [Note] InnoDB: Number of transaction pools: 1
      2022-12-19  7:59:12 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
      2022-12-19  7:59:12 0 [Note] InnoDB: Initializing buffer pool, total size = 100.000MiB, chunk size = 100.000MiB
      2022-12-19  7:59:12 0 [Note] InnoDB: Completed initialization of buffer pool
      [00] 2022-12-19 07:59:12 mariadb-backup: srv_start() returned 100 (I/O error).
      

      strace shows that the ibdata1 file can be opened just fine, but reading from it fails with "invalid argument"

      2434  openat(AT_FDCWD, "./ibdata1", O_RDWR|O_CLOEXEC <unfinished ...>
      2434  <... openat resumed>)             = 6
      2434  fcntl(6, F_SETFL, O_RDONLY|O_DIRECT <unfinished ...>
      2434  <... fcntl resumed>)              = 0
      2434  fcntl(6, F_SETLK, {l_type=F_WRLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0
      2434  newfstatat(6, "", {st_mode=S_IFREG|0755, st_size=12582912, ...}, AT_EMPTY_PATH) = 0
      2434  pread64(6, 0x55af89b64000, 65536, 0) = -1 EINVAL (Invalid argument)
      2434  newfstatat(AT_FDCWD, "/etc/localtime", {st_mode=S_IFREG|0644, st_size=114, ...}, 0) = 0
      2434  write(2, "[00] 2022-12-19 07:59:12 mariadb-backup: srv_start() returned 100 (I/O error).\n", 79) = 79
      

      For testing I used a Samba 4.16 server on Linux as the CIFS server.

      Attachments

        Issue Links

          Activity

            Turns out this only started with 10.6, earlier versions can prepare the backup on the CIFS share just fine

            hholzgra Hartmut Holzgraefe added a comment - Turns out this only started with 10.6, earlier versions can prepare the backup on the CIFS share just fine

            Why "Fix for" for 10.3 to 10.5? This only started with 10.6, earlier versions work fine. So it should only be "Fix for" for 10.6 and above?

            hholzgra Hartmut Holzgraefe added a comment - Why "Fix for" for 10.3 to 10.5? This only started with 10.6, earlier versions work fine. So it should only be "Fix for" for 10.6 and above?
            danblack Daniel Black added a comment -

            I think this is in the general class of bugs that O_DIRECT and CIFS is rather specific about the mount option and the kernel versions.

            From MDEV-26970, cache=none on the mount options might help or innodb_flush_method=fsync.

            danblack Daniel Black added a comment - I think this is in the general class of bugs that O_DIRECT and CIFS is rather specific about the mount option and the kernel versions. From MDEV-26970 , cache=none on the mount options might help or innodb_flush_method=fsync .

            cache=none indeed seems to do the trick

            not checked innodb_flush_method yet, but as that changed its default value in 10.6 that would explain why I was only hitting it in 10.6+

            hholzgra Hartmut Holzgraefe added a comment - cache=none indeed seems to do the trick not checked innodb_flush_method yet, but as that changed its default value in 10.6 that would explain why I was only hitting it in 10.6+

            --innnodb-flush-method=fsync on the command line also works fine.

            I couldn't get it to read such setting from my.cnf though.

            When trying

            [mariabackup]
            innnodb-flush-method=fsync
            

            I'm just getting the same error, and the variables entry in the --help output still shows O_DIRECT

            When trying

            [server]
            innnodb-flush-method=fsync
            

            I see the variable value in --help output change to "fsync", but when running --prepare it still runs into the same error.

            That's probably material for a separate jira ticket though ...

            hholzgra Hartmut Holzgraefe added a comment - --innnodb-flush-method=fsync on the command line also works fine. I couldn't get it to read such setting from my.cnf though. When trying [mariabackup] innnodb-flush-method=fsync I'm just getting the same error, and the variables entry in the --help output still shows O_DIRECT When trying [server] innnodb-flush-method=fsync I see the variable value in --help output change to "fsync", but when running --prepare it still runs into the same error. That's probably material for a separate jira ticket though ...

            People

              bar Alexander Barkov
              hholzgra Hartmut Holzgraefe
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.