[MDEV-16355] Add option for mysqldump to read data as of specific timestamp from system-versioned tables Created: 2018-05-31  Updated: 2023-11-27  Due: 2021-09-14  Resolved: 2021-10-09

Status: Closed
Project: MariaDB Server
Component/s: Scripts & Clients, Versioned Tables
Fix Version/s: 10.7.0

Type: Task Priority: Critical
Reporter: Valerii Kravchuk Assignee: Sergei Golubchik
Resolution: Fixed Votes: 3
Labels: mysqldump

Issue Links:
Problem/Incident
causes MDEV-26946 Various small mysqldump manual improv... Open
causes MDEV-27749 Binary changes for --as-of and --tz-u... Closed
causes MDEV-28254 Wrong position for row_start, row_end... Closed
Relates
relates to MDEV-16029 mysqldump: dump and restore historica... Closed
relates to MDEV-16798 need mysqldump option to include SYST... Closed

 Description   

It would be great to have an option for mysqldump to read data as of specific timestamp from system-versioned tables. One should just use:

SELECT * FROM <t> FOR SYSTEM_TIME AS OF TIMESTAMP '...';

Instead of simple SELECT for such tables if the option is provided.

I think it makes sense to implement this separately from MDEV-16029



 Comments   
Comment by Aleksey Midenkov [ 2018-06-01 ]

There should be options for:

1. dump with history or current snapshot only (MDEV-16029);
2. dump at specific point in time.

1. and 2. should be compatible, providing the ability to dump at specific point in time with all prior history.

Comment by Aleksey Midenkov [ 2018-07-17 ]

Task statement

New mysqldump option: --asof-timestamp=T.

If both --dump-history and --asof-timestamp=T are specified, then history is dumped from the beginning up to T. The query is executed roughly like this:

select *, row_start, row_end from t1 for system_time between timestamp '0-0-0 0:0' and timestamp T;

If only --asof-timestamp=T, then data is dumped without prior history. The query is executed roughly like this:

select * from t1 for system_time as of timestamp T;

Note

This option specifies only timestamp literal.

It does not specify TRX_ID for precise-versioned tables (though specifying a timestamp is of course valid). This is the subject for further development (e.g. --asof-transaction).

It does not specify any functional expressions.

Security check

Fail on single quote (') symbol to avoid forged queries.

Comment by Aleksey Midenkov [ 2018-07-27 ]

MDEV-16798 suggests an option to dump "all system versioning history back to a specified date".

Comment by Robert Dyas [ 2018-09-12 ]

Although we don't need this feature, we do need MDEV-16029 which has been unchanged for quite a while.

Comment by Sergei Golubchik [ 2021-08-15 ]

for the reference:

$ git fetch https://github.com/tempesta-tech/mariadb.git a19495728dea

Comment by Aleksey Midenkov [ 2021-08-19 ]

I'd expect this pushed together with MDEV-16546 and MDEV-16029.
Please review bb-10.7-midenok

Comment by Sergei Golubchik [ 2021-08-19 ]

This is a very clear, independent, and rather small feature. Let's not delay it unnecessary by combining it with two larger and way more arguable features. It can get pushed much faster it it won't wait for two other MDEVs

Comment by Roel Van de Paar [ 2021-10-09 ]

Not sure if I missing something here: TODO-3118 Lists this feature as present in 10.7 trunk, and this ticket is closed with a fix version of 10.7. However, neither 10.7 trunk, nor 10.5.11 (the github.com/tempesta-tech repo listed above) nor 10.5 trunk seems to have the feature implemented:

10.7.1 5cc9cf9a05582307abefd8f3b57548945da86b1a and others as described above

$ ./bin/mysqldump --help --verbose | grep -i asof
$ ./bin/mysqldump --asof-timestamp
./bin/mysqldump: unknown option '--asof-timestamp'
$ ./bin/mysqldump --dump-history
./bin/mysqldump: unknown option '--dump-history'

Comment by Sergei Golubchik [ 2021-10-09 ]

10.7 any time in Sept/Oct

$ ./bin/mysqldump --help --verbose | grep -i as-of
  --as-of=name        Dump system versioned table as of specified timestamp.
as-of                             (No default value)

Comment by Roel Van de Paar [ 2021-10-12 ]

Lol. The comments above mentioned --asof... not --as-of.... Thanks!

Comment by Roel Van de Paar [ 2021-10-29 ]

To clarify, --dump-history was (eventually) not added here, only --as-of.

The --as-of option was documented.

Comment by Roel Van de Paar [ 2021-10-29 ]

Logged MDEV-26943 mysqldump --as-of option ignores hour/minute precision

Comment by Roel Van de Paar [ 2021-10-31 ]

For any customers/users reviewing the implementation of -as-of please note there is up-to microsecond precision:

$ ./bin/mysqldump -uroot -S./socket.sock --compact test t1 --as-of="2021-11-01 06:23:43.5" --skip-tz-utc
...
INSERT INTO `t1` VALUES
(1),
(2),
(3);
$ ./bin/mysqldump -uroot -S./socket.sock --compact test t1 --as-of="2021-11-01 06:23:43.7" --skip-tz-utc
...
INSERT INTO `t1` VALUES
(2),
(3);

Comment by Roel Van de Paar [ 2021-10-31 ]

Logged MDEV-26946 Various small mysqldump manual improvements for the --as-of and --tz-utc options

Generated at Thu Feb 08 08:28:17 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.