Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-10191

non convertible chars convert() resulted in Null instead "?" on Windows

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • 10.1.4, 10.1(EOL), 10.2(EOL)
    • 10.1.27, 10.2.9, 10.3.2
    • Character Sets
    • None
    • Windows, reproduced on Windows Server 2012 R2 Standard Edition 64bit

    Description

      All Linux Versions and Windows before MariaDB 10.1.4
      select convert('ABCMogę jeśćDEF' using latin1)
      ->'ABCMog? je??DEF'

      On Windows and all Versions starting from 10.1.4
      select convert('ABCMogę jeśćDEF' using latin1)
      --> NULL

      Attachments

        Activity

          Richard Richard Stracke added a comment - - edited

          It returns null.

          Version is 9.1.0.4867.

          Please consider, that a customer have the same results.

          Valerii have checked on console as well.

          I quote his note:

          Let me share my test results. Windows first:

          [openxs@fc23 maria10.1]$ bin/mysql -A -uroot -proot --host=192.168.0.60 --port=3316 test
          Welcome to the MariaDB monitor. Commands end with ; or \g.
          Your MariaDB connection id is 2
          Server version: 10.1.13-MariaDB mariadb.org binary distribution
           
          Copyright (c) 2000, 2016, Oracle, MariaDB Corporation Ab and others.
           
          Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
           
          MariaDB [test]> show variables like 'char%';
          +--------------------------+-----------------------------------------------------+
          | Variable_name | Value |
          +--------------------------+-----------------------------------------------------+
          | character_set_client | utf8 |
          | character_set_connection | utf8 |
          | character_set_database | latin1 |
          | character_set_filesystem | binary |
          | character_set_results | utf8 |
          | character_set_server | latin1 |
          | character_set_system | utf8 |
          | character_sets_dir | C:\Program Files (x86)\MariaDB 10.1\share\charsets\ |
          +--------------------------+-----------------------------------------------------+
          8 rows in set (0.01 sec)
           
          MariaDB [test]> select convert('ABCMogę jeśćDEF' using latin1);
          +--------------------------------------------+
          | convert('ABCMogę jeśćDEF' using latin1) |
          +--------------------------------------------+
          | NULL |
          +--------------------------------------------+
          1 row in set, 1 warning (0.00 sec)
          

          Richard

          Richard Richard Stracke added a comment - - edited It returns null. Version is 9.1.0.4867. Please consider, that a customer have the same results. Valerii have checked on console as well. I quote his note: Let me share my test results. Windows first: [openxs@fc23 maria10.1]$ bin/mysql -A -uroot -proot --host=192.168.0.60 --port=3316 test Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 2 Server version: 10.1.13-MariaDB mariadb.org binary distribution   Copyright (c) 2000, 2016, Oracle, MariaDB Corporation Ab and others.   Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.   MariaDB [test]> show variables like 'char%'; +--------------------------+-----------------------------------------------------+ | Variable_name | Value | +--------------------------+-----------------------------------------------------+ | character_set_client | utf8 | | character_set_connection | utf8 | | character_set_database | latin1 | | character_set_filesystem | binary | | character_set_results | utf8 | | character_set_server | latin1 | | character_set_system | utf8 | | character_sets_dir | C:\Program Files (x86)\MariaDB 10.1\share\charsets\ | +--------------------------+-----------------------------------------------------+ 8 rows in set (0.01 sec)   MariaDB [test]> select convert('ABCMogę jeśćDEF' using latin1); +--------------------------------------------+ | convert('ABCMogę jeśćDEF' using latin1) | +--------------------------------------------+ | NULL | +--------------------------------------------+ 1 row in set, 1 warning (0.00 sec) Richard

          Hi,

          i forgot the warning.
          Code 1976
          Cannot convert 'utf8mb4' character 0xC499 to 'latin1'

          Richard Richard Stracke added a comment - Hi, i forgot the warning. Code 1976 Cannot convert 'utf8mb4' character 0xC499 to 'latin1'

          Valerii helped to track the problem down.
          The result depends only on sql_mode.

          SET sql_mode='';
          SELECT CONVERT(_utf8 0xC499 USING latin1);
          

          returns

          +------------------------------------+
          | CONVERT(_utf8 0xC499 USING latin1) |
          +------------------------------------+
          | ?                                  |
          +------------------------------------+
          1 row in set, 1 warning (0.00 sec)
          

          while this:

          SET sql_mode='STRICT_TRANS_TABLES';
          SELECT CONVERT(_utf8 0xC499 USING latin1);
          

          returns

          +------------------------------------+
          | CONVERT(_utf8 0xC499 USING latin1) |
          +------------------------------------+
          | NULL                               |
          +------------------------------------+
          1 row in set, 1 warning (0.00 sec)
          

          bar Alexander Barkov added a comment - Valerii helped to track the problem down. The result depends only on sql_mode. SET sql_mode= '' ; SELECT CONVERT (_utf8 0xC499 USING latin1); returns +------------------------------------+ | CONVERT(_utf8 0xC499 USING latin1) | +------------------------------------+ | ? | +------------------------------------+ 1 row in set, 1 warning (0.00 sec) while this: SET sql_mode= 'STRICT_TRANS_TABLES' ; SELECT CONVERT (_utf8 0xC499 USING latin1); returns +------------------------------------+ | CONVERT(_utf8 0xC499 USING latin1) | +------------------------------------+ | NULL | +------------------------------------+ 1 row in set, 1 warning (0.00 sec)

          Windows, MSI installatiion is setting sql_mode to contain STRICT_TRANS_TABLES. This has its roots back to how MySQL handled it (since 4.1 or so), and also was discussed with QA engineer when MSI installer was implemented.

          Also MariaDB 10.2 is using this bit STRICT_TRANS_TABLES by default cross-plattform.

          Everyone can set sql_mode to another value, if defaults do not work well with the app.

          wlad Vladislav Vaintroub added a comment - Windows, MSI installatiion is setting sql_mode to contain STRICT_TRANS_TABLES. This has its roots back to how MySQL handled it (since 4.1 or so), and also was discussed with QA engineer when MSI installer was implemented. Also MariaDB 10.2 is using this bit STRICT_TRANS_TABLES by default cross-plattform. Everyone can set sql_mode to another value, if defaults do not work well with the app.

          reassigning to bar to check.

          wlad Vladislav Vaintroub added a comment - reassigning to bar to check.

          People

            bar Alexander Barkov
            Richard Richard Stracke
            Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.