[MDEV-18052] Unexpected calculated column lengths Created: 2018-12-21 Updated: 2019-01-16 Resolved: 2019-01-10 |
|
| Status: | Closed |
| Project: | MariaDB Server |
| Component/s: | Data types, Views |
| Affects Version/s: | 5.5, 10.0, 10.1, 10.2.17, 10.2, 10.3, 10.4 |
| Fix Version/s: | N/A |
| Type: | Bug | Priority: | Minor |
| Reporter: | Jonathan Monahan | Assignee: | Alexander Barkov |
| Resolution: | Not a Bug | Votes: | 0 |
| Labels: | upstream | ||
| Description |
|
The length of a calculated column in a VIEW is sometimes incorrect, for example:
The base values clearly have fixed lengths, and the calculation will clearly always produce a fixed length of 8 characters, but the VIEW description insists that the column is 18 characters wide. The character set is 'utf8' on both client and server, but the data contains just single-byte characters, so I don't think it is a confusion between characters and bytes. |
| Comments |
| Comment by Elena Stepanova [ 2018-12-28 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
It is so in all active versions of MariaDB and MySQL. I expect that bar will be able to explain why it is so. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Alexander Barkov [ 2019-01-10 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
A shorter script demonstrating the same problem:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Alexander Barkov [ 2019-01-10 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
This is an intentional legacy behavior. The column "v2" has a data type of int(1). https://dev.mysql.com/doc/refman/8.0/en/numeric-type-attributes.html So a column of an int(1) data type can still store all values in the range -2147483648..2147483647, which requires 11 characters. You can use something like this as a workaround:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Jonathan Monahan [ 2019-01-10 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Ah, that makes sense. Unfortunately I was using the length reported by SHOW FIELDS to calculate an appropriate index prefix for a temporary table to load a subset of the VIEW into, e.g.:
The first 10 characters for VARCHAR and TEXT columns is sufficient to make a reasonable index, and for all other data types I can just index the whole column. Of course I must not exceed the 1000 characters limit on all indexes on the temporary table, and I must not specify an index prefix that is longer than the width of a VARCHAR. TEXT columns are always longer than 10 characters. Hence, I have logic like this (in Ruby):
In the example above, the column_definition.limit (from SHOW FIELDS) is 18, and so my code applies an index prefix of 10 characters, but that fails:
So, if I cannot rely on the width of a column from SHOW FIELDS, how should I discover the maximum width of a column in a VIEW that is triggering the error message? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Alexander Barkov [ 2019-01-10 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
I cannot reproduce this. This script works fine for me:
Can you please paste the full SQL script that generates this error message? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Jonathan Monahan [ 2019-01-11 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
This is odd. I have expanded your script in various ways in trying to reproduce the problem again, but it works on MariaDb 5.5 and MariaDb 10.2. I have created a source table that the VIEWs pull data from, added multiple values including NULL values in case that is confusing things, and added a final VIEW that simply copies through the calculated values which is similar to our production scenario:
Note that:
In my (much more complex) scenario, the CREATE TEMPORARY TABLE statement fails with "Incorrect prefix key; the used key part isn't a string, the used length is longer than the key part, or the storage engine doesn't support unique prefix keys", and I thought that was because the actual values were all less than 10 characters. Clearly your script, and my expanded script show that the index prefix does not have to be less than the length of the values, but that contradicts my experience https://dev.mysql.com/doc/refman/5.5/en/create-index.html#create-index-column-prefixes states Prefix limits are measured in bytes. However, prefix lengths for index specifications in CREATE TABLE, ALTER TABLE, and CREATE INDEX statements are interpreted as number of characters for nonbinary string types (CHAR, VARCHAR, TEXT) and number of bytes for binary string types (BINARY, VARBINARY, BLOB). Take this into account when specifying a prefix length for a nonbinary string column that uses a multibyte character set. but I don't think that is relevant. Perhaps I am misunderstanding the error message: the used key part is a string (generated by CONCAT), and the storage engine is InnoDb which does support unique prefix keys, so I am assuming that the error is referring to "the used length is longer than the key part". Can you provide an example in which that part of the error is triggered? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Alexander Barkov [ 2019-01-15 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
The "the used length is longer than the key part" is triggered in cases like this:
Notice, the column size is 10, and the prefix size if 20. This is not allowed. The prefix size must be less than (or equal to) the column size. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Jonathan Monahan [ 2019-01-15 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yes, I understand that simple scenario. We have a much more complex scenario that caused the error, with calculated values, multiple levels of VIEWs and then creating a temporary table from the final VIEW. I tried to cut the problem down to raise this issue, but clearly over-simplified it. In my previous comment you can see that the t2 temporary table has a 10 character index prefix on the e3 column, which seems to be allowed because the schema for the e3 column in the v3 VIEW is VARCHAR(18), but the values in the VIEW and the temporary table are all less than 10 characters. In our much more complex scenario we got the error, and the only explanation for it is the "the used length is longer than the key part", i.e. the index prefix of 10 characters is longer than the 8 character actual values. I will try to reproduce our complex scenario again, and come up with a better reproducible script. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Alexander Barkov [ 2019-01-16 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
This error is raised at the table creation time (not at the table populating time). |