[MDEV-27293] Allow converting a versioned table from implicit to explicit row_start/row_end columns Created: 2021-12-17  Updated: 2023-11-30

Status: Open
Project: MariaDB Server
Component/s: Versioned Tables
Fix Version/s: None

Type: New Feature Priority: Major
Reporter: Valerii Kravchuk Assignee: Oleksandr Byelkin
Resolution: Unresolved Votes: 0
Labels: None


 Description   

It would make sense to allow converting a table from implicit to explicit row start/end columns. Like this:

create table t1 (...) with system versioning;
alter table t1 add column rs timestamp(6) as row start, add column re timestamp(6) as row end, add period for system_time (rs,re);

This would allow to add indexes on these columns post factum, when the table history grows, and still preserve the history.



 Comments   
Comment by Sergei Golubchik [ 2022-07-25 ]

agree, this feature makes perfect sense

Comment by Nikita Malyavin [ 2023-06-19 ]

Should be rather trivial, however a lot of defence is written around.
Note that initially CHANGE of versioning columns was forbidden by tempesta's patch in scope of a bug fix: https://github.com/tempesta-tech/mariadb/issues/213

Also we should make that a syntax is supported properly:

 ADD rs TIMESTAMP(6) AS ROW START,
 ADD rs TIMESTAMP(6) AS ROW END,
 ADD PERIOD FOR sYSTEM_TIME(rs, re)

should actually change existing system columns and a period.

We could favor instead the simpler version both for us and for the user:

ALTER TABLE t1
 CHANGE row_start rs TIMESTAMP(6) AS ROW START,
 CHANGE row_end re TIMESTAMP(6) AS ROW END;

This requires less code and testing, but we'll refer an invisible system column, which wasn't possible before.

Given all this, and some decision-making, plus review fixes, I estimate the critical path as 7 days.

Comment by Nikita Malyavin [ 2023-06-27 ]

ralf.gebhardt we have to decide which semantics are we going to implement. The two above are mutually exclusive imo:
the first one takes into account that there is no user-exposed column row_start/row_end. And also the user didn't specify PERIOR FOR SYSTEM_TIME explicitly. It was silently created.
So it makes sense to write {{ ADD COLUMN rs..., ADD COLUMN re ..., ADD PERIOD ... }} since we never actually added it. CREATE TABLE WAS like:

CREATE TABLE t1(x int, y text) WITH SYSTEM VERSIONING;

But technically it would mean a rename of the system column.

Or we may acknowledge that row_start and row_end are the names of invisible system columns and simply allow to rename them, even though we never stated their creation.

The related question can be, can we rename an invisible system column in any other case? I know one such case – it's long unique index:

CREATE TABLE textie(t text UNIQUE);

There, some invisible system virtual column is created AS HASH(t), which actually has a key over it.
I don't know whether we can rename it or not, but I suspect that the behavior is undefined anyway.

So if we can rename an invisible column row_start, why can't we rename this one as well?

These questions must be answered and the behavior should be clarified. And for example one must make sure that they didn't apparently allow modifying all other system fields, when it shouldn't be done.

Comment by Ralf Gebhardt [ 2023-07-11 ]

nikitamalyavin,

SHOW CREATE TABLE does not show the the row_start/row_end columns. Would desc <table> show them as INVISIBLE?
If yes, couldn't an ALTER TABLE <table> MODIFY be used to make them visible to archive the same as if an explicit CREATE TABLE was used?

Comment by Ralf Gebhardt [ 2023-07-22 ]

Ok, DESC does not show it either.
nikitamalyavin Do we have any way to find these invisible columns. information_schema. columns does not show them either

Comment by Nikita Malyavin [ 2023-07-26 ]

I think we don't. midenok maybe you know?

Comment by Aleksey Midenkov [ 2023-08-15 ]

I don't understand question about "find", but it was designed to "drop" as making them invisible. Invisible means pretend they not exist and inaccessible for user. ADD syntax in description is correct.

Comment by Ralf Gebhardt [ 2023-09-25 ]

serg, here seems to be missing a decision regarding implementation details, mainly to implement ADD for a field which exists, but is invisible. Or to MODIFY a field we do not know that it exists. From my point of view, given that they seems to exist no way for even a SUPER user get an information from the server, that the field exists, ADD is the better option. What do you think?

Comment by Sergei Golubchik [ 2023-09-26 ]

Let's get the terminology right: INVISIBLE columns are columns that are shown in SHOW CREATE TABLE and INFORMATION_SCHEMA and carry an INVISIBLE attribute. System versioning columns that weren't explicitly created are not "invisible", they aren't columns and aren't part of the table definition. Internally, yes, the server reserves some space in the row image and uses it for storing internal technical system versioning information. And the user can use pseudo-columns ROW_START and ROW_END to access that system versioning information.

So midenok is absolutely right, there is no column to modify. You want a column — you have to add it.

Generated at Thu Feb 08 09:51:49 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.