[MDEV-28152] Features for sequence Created: 2022-03-22  Updated: 2024-01-16

Status: Stalled
Project: MariaDB Server
Component/s: Sequences
Fix Version/s: 11.5

Type: New Feature Priority: Critical
Reporter: Rucha Deodhar Assignee: Yuchen Pei
Resolution: Unresolved Votes: 1
Labels: None

Issue Links:
Blocks
is blocked by MDEV-33012 Alter sequence with second execution ... Open
Problem/Incident
causes MDEV-33012 Alter sequence with second execution ... Open
Relates
relates to MDEV-13005 Fixing bugs in SEQUENCE, part 3 Closed
relates to MDEV-25729 Fetch MariaDB Sequences metadata Closed

 Description   

This is a list of trivial features for Sequence (originally mentioned as part of MDEV-13005)

  • Support syntax "CREATE SEQUENCE x AS <some_int_type>';
    <some_int_type> can mean all the integers that we support
    Currently (as per documentation) maximum value that sequence can support is 9223372036854775806.
    But unsigned bigint can have maximum (unsigned) value: 18446744073709551615
    So adding support for this maximum (unsigned) value is needed.
    Also, set maximum minimum based on the datatype
  • Allow in parser very big numbers (decimal) for MAXVALUE but change these to LONGLONG_MAX-1, with a note for the user. This is needes as it's common practice to use
    MAXVALUE 9999999999999999999999999999 in Oracle examples.
  • Create information_schema.sequences that lists all sequences and their information. Should work similar to views:
    select * from information_schema.sequences where table_schema="test";
    The column names are:
    SEQUENCE_CATALOG, SEQUENCE_SCHEMA, SEQUENCE_NAME, DATA_TYPE, NUMERIC_PRECISION, NUMERIC_PRECISION_RADIX, NUMERIC_SCALE, START_VALUE, MINIMUM_VALUE, MAXIMUM_VALUE, INCREMENT, CYCLE_OPTION, DECLARED_DATA_TYPE, DECLARED_NUMERIC_PRECISION, DECLARED_NUMERIC_SCALE

Some suggested values:
DATA_TYPE BIGINT
DECLARED_DATA_TYPE BIGINT
NUMERIC_PRECISION 19
NUMERIC_PRECISION_RADIX 10
DECLARED_DATA_TYPE NULL
DECLARED_NUMERIC_PRECISION NULL
DECLARED_NUMERIC_SCALE NULL



 Comments   
Comment by Yuchen Pei [ 2023-02-09 ]

Initial version of the second item done: https://github.com/MariaDB/server/commit/84675ed91f4, where for consistency we

1. truncate not only decimals, but also ulonglong, -ulonglong, LONGLONG_MAX and LONGLONG_MIN
2. truncate not only maxvalue but also minvalue (obviously)

Now working on the first item.

Comment by Yuchen Pei [ 2023-02-10 ]

Initial version of the first item, part 1: https://github.com/MariaDB/server/commit/8a96330b07a. It supports signed datatypes for sequence values (unsigned datatypes will be in a separate patch). A pending task for this patch is to choose a proper representation of the the value datatypes in the sequence table. Currently I am lazily using the number representation of the C++ enum, but need to find out whether there is a builtin sql enum already in the server code, or we have to use something else, like string (Type_handler::name()).

Comment by Yuchen Pei [ 2023-02-13 ]

I could not find any built-in sql enums corresponding to the c++ enum_field_types, so I updated my patch to use string instead.

However, the change from tinyint to string fails some tests. AFAIK, there are two types of failure:

  • "ER_SEQUENCE_INVALID_TABLE_STRUCTURE (4086): Sequence 'test.t1' table structure is invalid (value_type)", which is caused by the varchar field defaulting to my_charset_bin, which adds a binary flag to the field.
  • aria table removing the row in the sequence when cycling back (sequence.next testcase):

    CREATE or REPLACE SEQUENCE t1 minvalue 1 maxvalue 2 increment by -1 cache 2 cycle engine=aria;
    select next value for t1;
    select * from t1;
    CREATE or REPLACE SEQUENCE t1 minvalue 1 maxvalue 2 increment by -1 cache 2 cycle engine=aria;
    select next value for t1;
    select * from t1;
      

    The first select * from t1 (with myisam) returns the correct result:

      next_not_cached_value	minimum_value	maximum_value	start_value
        increment	cache_size	cycle_option	cycle_count	value_type
    0	1	2	2	-1	2	1	0	bigint
      

    But the second one (with aria) returns empty:

        next_not_cached_value	minimum_value	maximum_value	start_value
        increment	cache_size	cycle_option	cycle_count	value_type
      

I have therefore kept this change in a separate commit (https://github.com/MariaDB/server/commit/f9cb7cbb7dc).

I will continue debugging these problems. Meanwhile, sanja could you take a look at the three patches and let me know what you think (e.g. whether you agree with the general approach)? After that I will continue with adding support for unsigned value types and the information schema tables.

One concern with adding a column for value types is migration: how do we handle schema changes caused by server upgrade?

Comment by Yuchen Pei [ 2023-02-13 ]

After discussions with sanja, I have revised the commits:

Comment by Yuchen Pei [ 2023-03-03 ]

Following up on discussions, the parsing is fixed - new non-terminals have been added and there's no longer newly introduced conflicts. sanja igor.

Current progress: Truncation of maxvalue/minvalue (item 2) done. Introducing types (item 1) still wip, mainly pending implementing ALTER SEQUENCE ... AS <new_type> (commit fd5a6414f43b2a3 below), but also need to fix a couple tests (see commit msg of 79d4283490cee) and maybe add more tests. information_schema.sequences (item 3) not started.

Commits in order (will squash when no longer wip):

Comment by Yuchen Pei [ 2023-03-07 ]

As an update, Item 3 is implemented, and all default and sql_sequence tests pass except compat/oracle.sequence which probably requires backward compatibility with oracle that does not support the new AS <TYPE> syntax[1]. The remaining non-trivial issue and blocker is the implementation of ALTER SEQUENCE ... AS <NEW_TYPE>.

Commits in order (the "alter as new_type" wip commit is in a different branch [2] and has not changed much since the last update):

[1] https://docs.oracle.com/en/database/oracle/oracle-database/21/sqlrf/CREATE-SEQUENCE.html
[2] https://github.com/MariaDB/server/compare/bb-11.0-mdev-28152-alter-value-type

Comment by Yuchen Pei [ 2023-03-09 ]

As discussed with serg and sanja, we implement an incomplete alter feature so that it is more likely to be done during this development cycle. The missing feature is: in ALTER SEQUENCE, AS <new_type> cannot be combined with any other alter options. That is, we can do alter sequence s as int, or alter sequence s maxvalue 100 minvalue 20, but not alter sequence s as int maxvalue 100 minvalue 20.

Patches in order:

Passing all tests / CI is green tick. sanja can you take a look or assign to someone else if you don't have time?

Comment by Yuchen Pei [ 2023-03-17 ]

I realised I need to update the implementation of setval() too for unsigned types. Working on it.

Meanwhile I have addressed all the comments in the first round review at https://lists.launchpad.net/maria-developers/msg13297.html

New patch:
https://github.com/MariaDB/server/commit/eef42da6b59

Diff with the previous (squashed) patch:
https://github.com/MariaDB/server/commit/8c2738b21da

Comment by Yuchen Pei [ 2023-03-20 ]

sanja In case you haven't had a chance to look at the patches in the previous comment yet, here is the updated patch with setval() and auto_increment() updated for unsigned types:

https://github.com/MariaDB/server/commit/30425c431c5

The diff with the previous patch that you have reviewed:

https://github.com/MariaDB/server/commit/bd4a2fac119

Comment by Oleksandr Byelkin [ 2023-11-15 ]

Thank you looks OK now

Comment by Yuchen Pei [ 2023-11-28 ]

rebased to 11.4:

9548c168fb26083852118402bf6d809a77ae57b8 upstream/bb-11.4-mdev-28152 MDEV-28152 Features for sequences

Locally tests pass on the sql_sequence suite.

waiting for CI...

Comment by Yuchen Pei [ 2023-11-29 ]

Hi lstartseva, ptal at the following commit, thanks

445ce402b01 upstream/bb-11.4-mdev-28152 MDEV-28152 Features for sequences

Comment by Lena Startseva [ 2023-12-14 ]

Testing is completed but MDEV-33012 needs to be fixed before pushing

Comment by Yuchen Pei [ 2024-01-09 ]

Hi lstartseva, could you test the patch on top of the MDEV-33169 patch that fixes MDEV-33012?

4d2224702fc upstream/bb-11.4-mdev-28152 MDEV-28152 Features for sequences
09a710036b6 MDEV-33169 Reset sequence used fields after check in alter sequence

Comment by Lena Startseva [ 2024-01-15 ]

Really, it looks like patch for MDEV-33169 fixes problems MDEV-33012. Tests passed.
Feature is ok to push with this changes.

Comment by Yuchen Pei [ 2024-01-15 ]

Thanks lstartseva. The fixversion was changed to 11.5 and the feature was not included in 11.4-preview, so we have to wait for an 11.5 branch to be created first.

Generated at Thu Feb 08 09:58:28 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.