Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-8402

Bug #77473 Truncated data with subquery & UTF8

    Details

    • Sprint:
      5.5.50

      Description

      REPEAT() and RPAD() results in a subquery get truncated if the length parameter is between 32768 and 65536 for 2-byte UTF-8 sequences, or between 21845 and 65535 . Below and above that point the result is correct.

      set names utf8;
      select length(data) as len from ( select repeat('ä', ...) as data ) as Sub;
       
      parameter -> result
       
      36766 -> 65532
      32767 -> 65534
      32678 -> 65534
      ...
      65535 -> 65534
      65536 -> 131072
      65537 -> 131074

      This is not a problem of the length function in the outer query, it is already the string returned from the inner query that gets truncated as can be seen with

      mysql -Ne "set names utf8; select data from (select repeat('é', 40000) as data ) as Sub;" | wc
            1       1   65535

      Looks as if the intermediate result length is calculated by character count and not by byte length, and so the wrong data type seems to be used to store the intermediate result. E.g. when using a unicode character that has a 3 byte UTF-8 representation:

      select length(data) as len from ( select repeat('☃', ...) as data ) as Sub;
       
      parameter -> result
       
      21844 -> 65532
      21845 -> 65535
      21846 -> 65535 
      ...
      65535 -> 65535
      65536 -> 196608

        Attachments

          Activity

            People

            • Assignee:
              bar Alexander Barkov
              Reporter:
              hholzgra Hartmut Holzgraefe
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: