Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-31018

Replica of 10.3, 10.4, <10.5.19 and <10.6.12 to 10.11 will not work when using non-default charset

Details

    Description

      10.9.2 introduced a new regression MDEV-30824. When a binlog generated from any engine versions is replayed on 10.9.2+ engines, depends on the character set settings, the replay could be immediately broken. This will break any functionalities relying on binlog such as replication.

      Invalid line in binlog:

      /*!\C utf8mb4 *//*!*/;
       
      SET @@session.character_set_client=224,@@session.collation_connection=224,@@session.collation_server=33/*!*/;
      

      Error:

      ERROR 1115 (42000) at line 38: Unknown character set: '224'
      

      This is later fixed with https://github.com/MariaDB/server/pull/2557 merged into 10.5, and is available in the upcoming minors 10.5.20, 10.6.12 etc. The fix prevents the previous releases from generating the invalid binlog that is not recognized by higher releases.

      Merging this to latest minors does not address the issue when replicating from older minors to 10.9.2+. That means, any replication from 10.3, 10.4, <10.5.19 and <10.6.12 to 10.9.2, 10.10 and 10.11 will still be broken if non-default character set is used.

      IMHO, we should also come up with another solution, that instead of fixing the binlog generation on previous majors, but to fix 10.9.2+, for backward compatibility.

      Attachments

        Issue Links

          Activity

            bar Alexander Barkov added a comment - Hello Elkin , Can you please review a patch: https://github.com/MariaDB/server/commit/ef911e08afae4f12ce5c1ea080208f0a7d15c3c7 ? Thanks!
            Tingynia Tingyao Nian added a comment - - edited

            Thanks Alex for the fast response! I left the same comment in the CR:

            Good idea of converting non-default collation to default collation, which IMHO is the root cause of the very original issue MDEV-28769(https://jira.mariadb.org/browse/MDEV-28769), and it does not change system behavior, because eventually the charset are still the same.

            I have 1 question/concern, could we apply this logic to all situation? Not just slave thread? If the concern is MDEV-28769(https://jira.mariadb.org/browse/MDEV-28769) may re-appear, I believe converting collation before setting should resolve it. On the other hand, only applying this change to slave thread means we are making assumptions that binlog is only used on slave threads, which may not always be true as sometimes it's also used during recovery?

            Tingynia Tingyao Nian added a comment - - edited Thanks Alex for the fast response! I left the same comment in the CR: Good idea of converting non-default collation to default collation, which IMHO is the root cause of the very original issue MDEV-28769 ( https://jira.mariadb.org/browse/MDEV-28769 ), and it does not change system behavior, because eventually the charset are still the same. I have 1 question/concern, could we apply this logic to all situation? Not just slave thread? If the concern is MDEV-28769 ( https://jira.mariadb.org/browse/MDEV-28769 ) may re-appear, I believe converting collation before setting should resolve it. On the other hand, only applying this change to slave thread means we are making assumptions that binlog is only used on slave threads, which may not always be true as sometimes it's also used during recovery?

            People

              bar Alexander Barkov
              Tingynia Tingyao Nian
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.