Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-21719

Set rsync to use MariaDB error.log

    XMLWordPrintable

Details

    Description

      Right now, when using rsync SST, the rsync messages go to syslog. This is confusing as a system administrator as MariaDB has its own log, and even contains SOME messages regarding rsync. The issue is that these errors in the MariaDB log regarding rsync are highly cryptic or misleading, since the real information was actually sent to syslog. Since the rsync SST is part of the MariaDB Galera process, all logs regarding this particular process should remain together.

      When rsync fails for any reason, MariaDB's error log will show something like the following. The error message implies that MariaDB possibly crashed (terminated unexpectedly) when the actual issue was rsync erroring out.

      2020-02-11 15:11:23 0 [Note] WSREP: PC protocol downgrade 1 -> 0
      2020-02-11 15:11:23 0 [Note] WSREP: view((empty))
      2020-02-11 15:11:23 0 [Note] WSREP: gcomm: closed
      2020-02-11 15:11:23 0 [Note] WSREP: mysqld: Terminated.
      WSREP_SST: [ERROR] Parent mysqld process (PID:26635) terminated unexpectedly. (20200211 15:11:24.N)
      /usr/local/bin/wsrep_sst_rsync: line 506: kill: (-26635) - No such process
      WSREP_SST: [INFO] Joiner cleanup. rsync PID: 26693 (20200211 15:11:24.N)
      WSREP_SST: [INFO] Joiner cleanup done. (20200211 15:11:24.N)

      This particular instance came up due to a ZFS configuration that I had on a particular server. However, other rsync errors happen from time to time too, such as networking errors. When diagnosing the issue with MDEV-21709, this is the error message presented in the syslog which would have been significantly more helpful (much less time consuming as a sysadmin) to just have it in the MariaDB error.log, so less hunting/debugging/coalescing logs is needed in the field.

      Feb 11 00:09:49 sea-cospix-maria-02 rsyncd[95201]: rsync: readdir(".zfs" (in rsync_sst)): Invalid argument (22)
      Feb 11 00:09:49 sea-cospix-maria-02 rsyncd[95201]: rsync: delete_file: rmdir(.zfs/snapshot) failed: Operation not supported (45)

      Attachments

        Issue Links

          Activity

            People

              sysprg Julius Goryavsky
              darkain Vincent Milum Jr
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.