[MDEV-33032] Live and dump upgrade from previous versions to the extended timestamp fails for system-versioned tables Created: 2023-12-14  Updated: 2023-12-19  Resolved: 2023-12-19

Status: Closed
Project: MariaDB Server
Component/s: Temporal Types, Versioned Tables
Affects Version/s: N/A
Fix Version/s: N/A

Type: Bug Priority: Critical
Reporter: Elena Stepanova Assignee: Michael Widenius
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Problem/Incident
is caused by MDEV-32188 make TIMESTAMP use whole 32-bit unsig... In Testing

 Description   

If a system-versioned (timestamp-based) table is created on pre-11.4 server, its current records have the row_end within 2038-01.
It doesn't change upon live upgrade to the server with the extended timestamp, even after running mysql_upgrade, and thus these records become historical.

Also, if mysqldump --dump-history is run on the old version, it again contains 2038-01 for current records, so restoring this dump doesn't help either. If mysqldump is run without --dump-history, it works (for invisible columns at least), but then all versioning history will be lost.



 Comments   
Comment by Michael Widenius [ 2023-12-18 ]

This is explained in the commit message:

However for system versioned tables we currently store the 2038 year max
timestamp as the 'max timestamp' which is used to detect current values.
This patch stores the new 2106 max value as the max timestamp.
For now, this handled by treating old rows with the old 2038 timestamp
as it would be the new max 2106 timestamp.
At some point, before 2016, we have to fix mysql_upgrade to update all
system versioning timestamps to use the new 2106 max timestamp.

Comment by Elena Stepanova [ 2023-12-18 ]

But it's not treated as max timestamp, that's the point. Sorry I should have added an example from the beginning, but here it goes now.

On the old server (I was using the current 11.3):

MariaDB [test]> create table t (a int) with system versioning;
Query OK, 0 rows affected (0.041 sec)
 
MariaDB [test]> insert into t values (1);
Query OK, 1 row affected (0.014 sec)
 
MariaDB [test]> select a, row_start, row_end from t;
+------+----------------------------+----------------------------+
| a    | row_start                  | row_end                    |
+------+----------------------------+----------------------------+
|    1 | 2023-12-18 15:44:17.724987 | 2038-01-19 05:14:07.999999 |
+------+----------------------------+----------------------------+
1 row in set (0.002 sec)

So, the record is current and it is seen by normal select.

Now, starting bb-11.3-timestamp on this datadir:

MariaDB [test]> select a, row_start, row_end from t;
Empty set (0.003 sec)

The record is still there, but it became historical and is only seen by select for system_time all:

MariaDB [test]> select a, row_start, row_end from t for system_time all;
+------+----------------------------+----------------------------+
| a    | row_start                  | row_end                    |
+------+----------------------------+----------------------------+
|    1 | 2023-12-18 15:44:17.724987 | 2038-01-19 05:14:07.999999 |
+------+----------------------------+----------------------------+
1 row in set (0.001 sec)

My timezone is EET (taken from the system), in case it's important.

Comment by Michael Widenius [ 2023-12-18 ]

It looks like the approach I envisioned will not work for upgrades.
I am now adding support for fixing row_end as part of mariadb-upgrade.
For mysqldump we will need an option to update row_end from 2038-01-19 03:14:07.999999 to
2106-02-07 06:28:15.999999 for system versioned tables. We can backport this option to earlier version, like 10.5 which should fix this issue.

Comment by Michael Widenius [ 2023-12-19 ]

Fixed in bb-10.4-timestamp

Generated at Thu Feb 08 10:35:53 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.