[MDEV-32188] make TIMESTAMP use whole 32-bit unsigned range Created: 2023-09-16  Updated: 2024-02-07

Status: In Testing
Project: MariaDB Server
Component/s: Temporal Types
Fix Version/s: 11.4

Type: New Feature Priority: Critical
Reporter: Sergei Golubchik Assignee: Elena Stepanova
Resolution: Unresolved Votes: 2
Labels: Preview_11.4

Issue Links:
Duplicate
is duplicated by MDEV-9444 MariaDB needs to be Y2038-compliant Closed
is duplicated by MDEV-20192 Change timestamp to 64-bit integer fo... Closed
is duplicated by MDEV-26736 UNIX_TIMESTAMP doesn't work beyond 20... Closed
PartOf
Problem/Incident
causes MDEV-32930 mysqlbinlog --verbose shows incorrect... Open
causes MDEV-33013 versioning.insert2 fails on 32-bit wi... Closed
causes MDEV-33017 Assertion `temp > 1969' fails in Fiel... Closed
causes MDEV-33032 Live and dump upgrade from previous v... Closed
causes MDEV-33239 mysqlbinlog cannot parse events writt... Open
causes MDEV-33248 Connect engine does not support 32-bi... Closed
causes MDEV-33302 Columnstore does not support 32-bit t... Open
causes MDEV-33310 mariadb-upgrade does not fix InnoDB v... Closed
causes MDEV-33358 ROW END needs to be adjusted to 32-bi... Open
Relates
relates to MDEV-15750 preserve MYSQL_TYPE_TIMESTAMP in temp... Stalled
relates to MDEV-32582 Events fuctionality after 2038 Open
relates to MDEV-33381 armhf: MTR failures in Spider and ver... Open
relates to MDEV-341 64-bit support in FROM_UNIXTIME() Open
relates to MDEV-10018 Timestamp with time zone Open
relates to MDEV-32496 TIMESTAMP variant to support YEAR ran... Open

 Description   

Currently timestamp is 32-bit signed, although negative values are rejected, so it is de facto 31-bit, allowing values from 1970 to 2038.

We can make it 32-bit unsigned extending timestamp range to 2106, and it won't require many changes to the storage.

Storage of timestamp is not affected but the change.
However storage of 'row_end' for system versioned tables must change from storing a 2038 based timestamp to a 2106 based one.



 Comments   
Comment by Michael Widenius [ 2023-09-19 ]

Bar, please review bb-11.3-monty

Comment by Alexander Barkov [ 2023-10-23 ]

monty, there is a summary of my review comments (previously discussed on slack):

  • This code is going to change under terms of MDEV-32496 and MDEV-10018. Therefore, it's better to take into account planned changes, to stabilize the underlying code one time instead of two times.
  • I don't like that you use a differently defined my_time_t on different platforms. If you want to preserve my_time_t for plugin compatibility, then the server code should use a different data type. Let's call it for example my_seconds_t, and it should be defines as longlong (signed).
  • This new data type for seconds should be a part of ABI.
  • The type my_time_t should not be touched, and it should stay in ABI. I don't like that you removed it. I think this is meaningless to remove my_time_t from ABI but insist that we need it for backward compatibility.
  • The data type in most server files should be changed from my_time_t to my_seconds_t. Functions/methods which depend on my_time_t that you want still to be available for plugings should be duplicated: for my_time_t and for my_seconds_t.
  • my_timeval should be changed as follows:

    struct my_timeval
    {
      longlong tv_sec; -- change to my_seconds_t
      long tv_usec; -- change to uint32
    };
    

  • To convert between my_seconds_t and MYSQL_TIME on more 32bit systems, we should add a CMake check for functions localtime64_r (Unix) and locatime64_s (Windows) and use these functions when available.
Comment by Alexander Barkov [ 2023-10-24 ]

The patches in bb-11.3-monty are Ok to push.
I'd still prefer though if we used a system independent replacement for my_time_t.

Comment by Elena Stepanova [ 2024-01-07 ]

The branch is now bb-11.4-timestamp.

Comment by Marko Mäkelä [ 2024-01-30 ]

I encountered the changeset Updated ALTER TABLE to convert old row_end timestamps to new timestamp range that is related to this task. I think that a change of a data type (from unsigned 31-bit timestamps to unsigned 32-bit) needs to be signaled to the storage engines by setting the ALTER_STORED_COLUMN_TYPE flag. In that way, InnoDB will refuse to perform any native ALTER TABLE operation just fine, and any fixup attempts like this one should be unnecessary. The idea of handler::check_if_supported_inplace_alter() is to let the storage engine choose the most efficient algorithm, unless the user has instructed otherwise, by specifying ALGORITHM=COPY or setting the parameters old_alter_table=1 or (ever since MDEV-13134) alter_algorithm=copy.

Comment by Marko Mäkelä [ 2024-01-30 ]

The ALTER TABLE attribute FORCE was originally introduced in MySQL 5.6.17 by the following bug fix, to allow the MySQL 5.6.8 WL#6255 online table rebuild to work on OPTIMIZE TABLE:
BUG# 13975227: ONLINE OPTIMIZE TABLE FOR INNODB TABLES
The idea is that OPTIMIZE TABLE will map to ALTER TABLE…FORCE in the case that handler::optimize() is not implemented or supported in the storage engine.

I do not know which was the first GA release of MariaDB Server 10.0, but I think that this change was part of that release series from rather early on.

Generated at Thu Feb 08 10:29:30 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.