[MDEV-28808] Test MDEV-16329 (ALTER ONLINE TABLE) - Core server part - Jira

Elena Stepanova made changes - 2024-02-14 12:36

Fix Version/s		N/A [ 14700 ]
Fix Version/s	11.2 [ 28603 ]
Resolution		Done [ 10200 ]
Status	Stalled [ 10000 ]	Closed [ 6 ]

Elena Stepanova made changes - 2023-08-12 22:46

Status

In Progress [ 3 ]

Stalled [ 10000 ]

Elena Stepanova made changes - 2023-08-12 18:12

Link

This issue causes MDEV-31906 [ MDEV-31906 ]

Elena Stepanova made changes - 2023-08-12 16:50

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [c29ff60b|https://github.com/MariaDB/server/commit/c29ff60b2c847b21d1cb914ac38f49d238234b35]
Baseline: [e81fa345 (11.2)|https://github.com/MariaDB/server/commit/e81fa345020ec6a067583db6a7019d6404b26f93]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- Exceptions (limitations) need to be enumerated and documented, see ~~MDEV-16329~~
- New flag in old_mode: {{LOCK_ALTER_TABLE_COPY}}

Engines:
InnoDB, MyISAM, Aria, Heap, CSV, RocksDB, Archive, Mroonga -- applies
MERGE, Spider, Blackhole, Sphinx -- skips online alter (doesn't hit debug sync point), probably because there is no real data
Columnstore -- skips online alter (doesn't increase Binlog_bytes_written, cannot test debug (!), it fails there, to be checked whether it's specific to the branch)
S3, OQGRAPH, FEDERATED, FEDERATEDX -- ALTER is prohibited in general
PERFORMANCE_SCHEMA -- cannot be ALTERed
Conversion to S3 -- prohibited. Conversion from S3 -- applies (?).
Sequences -- prohibited.
Connect -- prohibited.

*Testing*

(/) +Buildbot+
- (/) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (/) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (/) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=bb-11.2-oalter&revision=c29ff60b2c847b21d1cb914ac38f49d238234b35&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(off) +Code review notes+

(?) +Extra MTR+
- (/) GCOV for the patch
  default set + galera set, nm-big run: Lines with zero coverage (67/2603) [^missings-mtr-c29ff60b.txt]
- (/) SAN tests
** (/) ASAN default big / nm + ps + view
** (/) MSAN default big / nm + ps + view
** (/) UBSAN default big / nm + ps + view
** (/) ASAN plugins big/nm
** (/) ASAN galera big/nm

(/) +Manual Testing+
- (/) By spec/exploratory
- (/) integration map traverse

(/) +Crash recovery+
- No new problems with stability observed
- testing of recovery correctness in comparison to binlog is limited due to a number of legacy issues in binlog replay

(/) +Replication+
- No new problems with server (primary or replica) stability observed
- testing of replicaton stability and correctness is limited due to a number of legacy issues in replication

(/) +Larger integration+
- (off) Galera
  outside the scope of this item, done in ~~MDEV-27986~~
- (/) Columnstore
  does not support online alter
- (/) MariaBackup
  no specific issues observed
- (off) Protocol
- (off) Connectors

(?) +Upgrade+
- (off) Live upgrade
- (off) Dump upgrade
- (off) Package upgrade
- (/) MariaBackup upgrade
- (!) OM => NS replication upgrade
Some old bugs fixed along with online alter development (while not directly related to the feature), the fixes may cause OM => NS replication failures. E.g. if ALGORITHM=INSTANT was silently ignored in a previous version and ALTER with it succeeded, even though not executed as INSTANT, now it is rejected, thus upon replicating such events the replication will abort.
- (off) Galera rolling upgrade
  out of the scope of this item

(/) +Random+
- (!) RQG GCOV for the patch
  not satisfactory, to be further worked on after the feature release
- (/) RQG-100 ASAN
  50% tests fail due to unrelated issues
- (?) RQG-100 Release

(?) +Compatibility+
- (off) Windows
- (off) ARM
- (off) FreeBSD
- (i) i686
  ~~MDEV-31646~~ downgraded to non-RC-blocking

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+
Big DML events in concurrency with ALTER on tables without PK is a concern. old_mode is to be set for instances which can have such scenarios.

(/) +Updates/additions to non-MTR regression tests+
- various small ALTER-related changes
- syncpoint grammar extensions
- binlog replay improvements
- new old_mode in parameter presets and dynamic variables

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
c29ff60b2c8 fix cleenup??
00accd5e465 read_log_event: extract a shortcut with dedicated max_allowed_packet arg
1fa0e86f3cc ~~MDEV-31646~~ fixes
f5191233418 ~~MDEV-31812~~ Add switch to old_mode to disable non-locking ALTER
da81aa232c8 fix slave_exec_mode initialization
dd4e5d543d5 ~~MDEV-31804~~ Assertion `thd->m_transaction_psi == __null' fails
f4c5b3bc1ba Cleanup: make slave_exec_mode of its enum type and pack Log_event better
b6b0a830774 ~~MDEV-31838~~ Assertion fails upon replication online alter with MINIMAL row
a88019c0e42 ~~MDEV-31781~~ ALTER TABLE ENGINE=s3 fails
7e9ea6317e1 ~~MDEV-31777~~ ER_GET_ERRNO upon online alter on CONNECT table
0a32e10cb68 ~~MDEV-31631~~ Adding auto-increment to table with history online misbehaves
69c7c84583d ~~MDEV-31776~~ Online ALTER reports the number of affected rows incorrectly
0400e0014c7 Rows_log_event: reorganize the class layout for a lower memory footprint
ed44d2f69d0 ~~MDEV-31775~~ Server crash upon online alter on sequence
b749e1287f9 add to binlog_bytes_written for an initial description event
41f7852d9f1 make a proper cleanup if online_alter_binlog is failed to create
8750a2e9ee8 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
ebc53664028 fix key detection on replica with extra columns
6e5e24eac2a ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
df2b8694fef ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
ca32d2342df ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
d80cb40f132 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
7ed19179c69 fix -Werror build
c164dd95bd4 follow-up ~~MDEV-30430~~: fix versioning.rpl
73bbbc07719 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
8d5a9341781 ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
3b8525ac5ee Add const to get_foreign_key_list/get_parent_foreign_key_list
c3ea029832a Add const to alloc-related thd methods
d07bd4316b5 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
388d2039c5d ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
7f0b27a33ce refactor unpack_row
ed8a15d44b5 unpack_row: unpack a correct number of fields
03f806acb95 ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
d61761f2cb4 unpack_row: set the correct fields in has_value_set for online alter
b599911c92a ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
1a5b9fcc541 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
af106e27e27 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
5258c19894c ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
ab3b7b8e2dc clean up Rows_log_event virtual methods
057bb76d6bb ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
4819c13a496 add partition test
b609b9c6d10 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
c66d605c80e ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
ff0290047d9 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
ef1caa6263b ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
ad27cab7806 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
6b7a16a2f97 fix main.alter_table_{online,lock}
70e385bb6c9 ~~MDEV-29069~~ follow-up: improve DEFAULT rules
3694afb4e6e ~~MDEV-29069~~ follow-up: support partially usable keys
069a2063c44 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
23f7233fb4c ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
8292140d1d5 cleanup: cache the result of Rows_log_event::find_key()
39820b1ef87 set table->pos_in_table_list in online alter
0d8a4a8dd19 cleanup: remove rpl_group_info::get_table_data()
0495785dd4f cleanup: ifdefs
8be6788bd88 few rgi assertions. this can proof that rgi is always present
6b90a751c57 rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
f55f6916d2c ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
35401ae1e53 log_event.h: remove junk EOL spaces
a7e3bb4521d ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
ce7c344f861 Fix write_set too
ef10d88a43a ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
26c94583e42 rpl: check should go after defaults and vcols update
dd1bde0a035 ~~MDEV-29067~~ Online alter ignores check constraint violation
7b16df60410 don't do ALTER IGNORE TABLE online
c4d8107063b ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
295156d1a54 Do not ignore sql_mode when replicating
37fecbc237f Simplify rgi->get_table_data call
3ad859d37ae reorder RPL_TABLE_LIST fields for better packing
6df8322726c ~~MDEV-29021~~ fixup
4afa0b2e844 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
36764a6f278 cleanup, remove dead code
f950f4453fd ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
50ea1b9f3de ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
c61df2b4a10 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
c6d89f0a236 remove redundant warnings in RBR and online alter
e27314cf40b cleanup: whitespace, etc
ed186479fea ~~MDEV-28959~~ Online alter ignores strict table mode
43a364143ef ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
2a15af9ae9a control Cache_flip_event_log lifetime with reference count
7da5157220b ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
462bc39a120 test rename alter_table_online -> alter_table_online_debug
d560f19f813 separate online_alter_cache_data from binlog_cache_data
e968c475e7a put binlog_cache_data on a memroot
e6d69e3f952 always commit for non-trans engines
a7b11479979 savepoints
e9891c063ff don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
62fb6a20eb9 don't use start_consistent_snapshot
997aa8885b7 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
28cc0fcf5a4 don't do DROP SYSTEM VERSIONING online
d96ffbde842 set read_set early, before row reads
adbba1c0920 no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
6fb0d9dc0a3 online alter always uses ALGORITHM=COPY, LOCK=NONE
9da6f1f4687 remove handler::open_read_view()
63d43ff120b cleanup
338fe3c6962 support 'alter online table t1 page_checksum=0'
64c21c742b2 tests: move around, add new
1af3b635610 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
c4bb6280b2d ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
bf15bfb49c7 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
d61f467ef4e ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
2fdc269ee25 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
--- 8c8027970ba rpl: repack table_def
--- 7c25ddfd5ff Copy_field: add const to arguments
--- 40bed167bb2 rename tests
--- 2d837d42759 binlog_combinations.inc -> binlog_format_combinations.inc
--- 979f808883b cleanup: remove vcol_info->stored_in_db
--- 10561a10ed1 Fix recalculation of vcols in binlog_row_image=minimal
--- e7f8898393b cleanup: clarify ha_innobase::column_bitmaps_signal()
--- 74b94197eb2 allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [c29ff60b|https://github.com/MariaDB/server/commit/c29ff60b2c847b21d1cb914ac38f49d238234b35]
Baseline: [e81fa345 (11.2)|https://github.com/MariaDB/server/commit/e81fa345020ec6a067583db6a7019d6404b26f93]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- New flag in old_mode: {{LOCK_ALTER_TABLE_COPY}}. Big DML events in concurrency with ALTER on tables without PK is a concern. old_mode is to be set for instances which can have such scenarios.

- Exceptions (limitations) when the new online alter is not applicable are enumerated in ~~MDEV-16329~~

Per-engines exploratory observations:
- InnoDB, MyISAM, Aria, Heap, CSV, RocksDB, Archive, Mroonga -- applies
- MERGE, Spider, Blackhole, Sphinx -- skips online alter (doesn't hit debug sync point), probably because there is no real data
- Columnstore -- skips online alter (doesn't increase Binlog_bytes_written, cannot test debug (!), it fails there, to be checked whether it's specific to the branch)
S3, OQGRAPH, FEDERATED, FEDERATEDX -- ALTER is prohibited in general
- PERFORMANCE_SCHEMA -- cannot be ALTERed
- Conversion to S3 -- prohibited. Conversion from S3 -- applies (?).
- Sequences -- prohibited.
- Connect -- prohibited.

*Testing*

(/) +Buildbot+
- (/) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (/) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (/) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=bb-11.2-oalter&revision=c29ff60b2c847b21d1cb914ac38f49d238234b35&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(off) +Code review notes+

(/) +Extra MTR+
- (/) GCOV for the patch
  default set + galera set, nm-big run: Lines with zero coverage (67/2603) [^missings-mtr-c29ff60b.txt]
- (/) SAN tests
** (/) ASAN default big / nm + ps + view
** (/) MSAN default big / nm + ps + view
** (/) UBSAN default big / nm + ps + view
** (/) ASAN plugins big/nm
** (/) ASAN galera big/nm

(/) +Manual Testing+
- (/) By spec/exploratory
- (/) integration map traverse

(/) +Crash recovery+
- No new problems with stability observed
- testing of recovery correctness in comparison to binlog is limited due to a number of legacy issues in binlog replay

(/) +Replication+
- No new problems with server (primary or replica) stability observed
- testing of replicaton stability and correctness is limited due to a number of legacy issues in replication

(/) +Larger integration+
- (off) Galera
  outside the scope of this item, done in ~~MDEV-27986~~
- (/) Columnstore
  does not support online alter
- (/) MariaBackup
  no specific issues observed
- (off) Protocol
- (off) Connectors

(/) +Upgrade+
- (off) Live upgrade
- (off) Dump upgrade
- (off) Package upgrade
- (/) MariaBackup upgrade
- (!) OM => NS replication upgrade
Some old bugs fixed along with online alter development (while not directly related to the feature), the fixes may cause OM => NS replication failures. E.g. if ALGORITHM=INSTANT was silently ignored in a previous version and ALTER with it succeeded, even though not executed as INSTANT, now it is rejected, thus upon replicating such events the replication will abort.
- (off) Galera rolling upgrade
  out of the scope of this item

(?) +Random+
- (!) RQG GCOV for the patch
  not satisfactory, to be further worked on after the feature release
- (/) RQG-100 ASAN
  50% tests fail due to unrelated issues
- (?) RQG-100 Release

(off) +Compatibility+
- (off) Windows
- (off) ARM
- (off) FreeBSD
- (i) i686
  ~~MDEV-31646~~ downgraded to non-RC-blocking

(off) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(/) +Performance considerations+
Big DML events in concurrency with ALTER on tables without PK is a concern. old_mode is to be set for instances which can have such scenarios.

(/) +Updates/additions to non-MTR regression tests+
- various small ALTER-related changes
- syncpoint grammar extensions
- binlog replay improvements
- new old_mode in parameter presets and dynamic variables

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
c29ff60b2c8 fix cleenup??
00accd5e465 read_log_event: extract a shortcut with dedicated max_allowed_packet arg
1fa0e86f3cc ~~MDEV-31646~~ fixes
f5191233418 ~~MDEV-31812~~ Add switch to old_mode to disable non-locking ALTER
da81aa232c8 fix slave_exec_mode initialization
dd4e5d543d5 ~~MDEV-31804~~ Assertion `thd->m_transaction_psi == __null' fails
f4c5b3bc1ba Cleanup: make slave_exec_mode of its enum type and pack Log_event better
b6b0a830774 ~~MDEV-31838~~ Assertion fails upon replication online alter with MINIMAL row
a88019c0e42 ~~MDEV-31781~~ ALTER TABLE ENGINE=s3 fails
7e9ea6317e1 ~~MDEV-31777~~ ER_GET_ERRNO upon online alter on CONNECT table
0a32e10cb68 ~~MDEV-31631~~ Adding auto-increment to table with history online misbehaves
69c7c84583d ~~MDEV-31776~~ Online ALTER reports the number of affected rows incorrectly
0400e0014c7 Rows_log_event: reorganize the class layout for a lower memory footprint
ed44d2f69d0 ~~MDEV-31775~~ Server crash upon online alter on sequence
b749e1287f9 add to binlog_bytes_written for an initial description event
41f7852d9f1 make a proper cleanup if online_alter_binlog is failed to create
8750a2e9ee8 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
ebc53664028 fix key detection on replica with extra columns
6e5e24eac2a ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
df2b8694fef ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
ca32d2342df ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
d80cb40f132 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
7ed19179c69 fix -Werror build
c164dd95bd4 follow-up ~~MDEV-30430~~: fix versioning.rpl
73bbbc07719 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
8d5a9341781 ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
3b8525ac5ee Add const to get_foreign_key_list/get_parent_foreign_key_list
c3ea029832a Add const to alloc-related thd methods
d07bd4316b5 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
388d2039c5d ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
7f0b27a33ce refactor unpack_row
ed8a15d44b5 unpack_row: unpack a correct number of fields
03f806acb95 ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
d61761f2cb4 unpack_row: set the correct fields in has_value_set for online alter
b599911c92a ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
1a5b9fcc541 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
af106e27e27 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
5258c19894c ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
ab3b7b8e2dc clean up Rows_log_event virtual methods
057bb76d6bb ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
4819c13a496 add partition test
b609b9c6d10 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
c66d605c80e ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
ff0290047d9 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
ef1caa6263b ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
ad27cab7806 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
6b7a16a2f97 fix main.alter_table_{online,lock}
70e385bb6c9 ~~MDEV-29069~~ follow-up: improve DEFAULT rules
3694afb4e6e ~~MDEV-29069~~ follow-up: support partially usable keys
069a2063c44 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
23f7233fb4c ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
8292140d1d5 cleanup: cache the result of Rows_log_event::find_key()
39820b1ef87 set table->pos_in_table_list in online alter
0d8a4a8dd19 cleanup: remove rpl_group_info::get_table_data()
0495785dd4f cleanup: ifdefs
8be6788bd88 few rgi assertions. this can proof that rgi is always present
6b90a751c57 rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
f55f6916d2c ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
35401ae1e53 log_event.h: remove junk EOL spaces
a7e3bb4521d ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
ce7c344f861 Fix write_set too
ef10d88a43a ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
26c94583e42 rpl: check should go after defaults and vcols update
dd1bde0a035 ~~MDEV-29067~~ Online alter ignores check constraint violation
7b16df60410 don't do ALTER IGNORE TABLE online
c4d8107063b ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
295156d1a54 Do not ignore sql_mode when replicating
37fecbc237f Simplify rgi->get_table_data call
3ad859d37ae reorder RPL_TABLE_LIST fields for better packing
6df8322726c ~~MDEV-29021~~ fixup
4afa0b2e844 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
36764a6f278 cleanup, remove dead code
f950f4453fd ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
50ea1b9f3de ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
c61df2b4a10 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
c6d89f0a236 remove redundant warnings in RBR and online alter
e27314cf40b cleanup: whitespace, etc
ed186479fea ~~MDEV-28959~~ Online alter ignores strict table mode
43a364143ef ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
2a15af9ae9a control Cache_flip_event_log lifetime with reference count
7da5157220b ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
462bc39a120 test rename alter_table_online -> alter_table_online_debug
d560f19f813 separate online_alter_cache_data from binlog_cache_data
e968c475e7a put binlog_cache_data on a memroot
e6d69e3f952 always commit for non-trans engines
a7b11479979 savepoints
e9891c063ff don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
62fb6a20eb9 don't use start_consistent_snapshot
997aa8885b7 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
28cc0fcf5a4 don't do DROP SYSTEM VERSIONING online
d96ffbde842 set read_set early, before row reads
adbba1c0920 no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
6fb0d9dc0a3 online alter always uses ALGORITHM=COPY, LOCK=NONE
9da6f1f4687 remove handler::open_read_view()
63d43ff120b cleanup
338fe3c6962 support 'alter online table t1 page_checksum=0'
64c21c742b2 tests: move around, add new
1af3b635610 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
c4bb6280b2d ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
bf15bfb49c7 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
d61f467ef4e ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
2fdc269ee25 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
--- 8c8027970ba rpl: repack table_def
--- 7c25ddfd5ff Copy_field: add const to arguments
--- 40bed167bb2 rename tests
--- 2d837d42759 binlog_combinations.inc -> binlog_format_combinations.inc
--- 979f808883b cleanup: remove vcol_info->stored_in_db
--- 10561a10ed1 Fix recalculation of vcols in binlog_row_image=minimal
--- e7f8898393b cleanup: clarify ha_innobase::column_bitmaps_signal()
--- 74b94197eb2 allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

Elena Stepanova made changes - 2023-08-12 16:45

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [c29ff60b|https://github.com/MariaDB/server/commit/c29ff60b2c847b21d1cb914ac38f49d238234b35]
Baseline: [e81fa345 (11.2)|https://github.com/MariaDB/server/commit/e81fa345020ec6a067583db6a7019d6404b26f93]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- Exceptions (limitations) need to be enumerated and documented, see ~~MDEV-16329~~
- New flag in old_mode: {{LOCK_ALTER_TABLE_COPY}}

Engines:
InnoDB, MyISAM, Aria, Heap, CSV, RocksDB, Archive, Mroonga -- applies
MERGE, Spider, Blackhole, Sphinx -- skips online alter (doesn't hit debug sync point), probably because there is no real data
Columnstore -- skips online alter (doesn't increase Binlog_bytes_written, cannot test debug (!), it fails there, to be checked whether it's specific to the branch)
S3, OQGRAPH, FEDERATED, FEDERATEDX -- ALTER is prohibited in general
PERFORMANCE_SCHEMA -- cannot be ALTERed
Conversion to S3 -- prohibited. Conversion from S3 -- applies (?).
Sequences -- prohibited.
Connect -- prohibited.

*Testing*

(/) +Buildbot+
- (/) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (/) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (/) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=bb-11.2-oalter&revision=c29ff60b2c847b21d1cb914ac38f49d238234b35&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(off) +Code review notes+

(?) +Extra MTR+
- (?) GCOV for the patch
  default set + galera set, nm-big run: Lines with zero coverage (67/2603) [^missings-mtr-c29ff60b.txt]
- (?) SAN tests
** (/) ASAN default big / nm + ps + view
** (/) MSAN default big / nm + ps + view
** (/) UBSAN default big / nm + ps + view
** (/) ASAN plugins big/nm
** (?) ASAN galera big/nm
{noformat}
galera.galera_gcache_recover_manytrx
The TLS connection was non-properly terminated.
{noformat}

(?) +Manual Testing+
- (/) By spec/exploratory
- (/) integration map traverse

(/) +Crash recovery+
- No new problems with stability observed
- testing of recovery correctness in comparison to binlog is limited due to a number of legacy issues in binlog replay

(/) +Replication+
- No new problems with server (primary or replica) stability observed
- testing of replicaton stability and correctness is limited due to a number of legacy issues in replication

(?) +Larger integration+
- (off) Galera
  outside the scope of this item, done in ~~MDEV-27986~~
- (/) Columnstore
  does not support online alter
- (/) MariaBackup
  no specific issues observed
- (off) Protocol
- (off) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (off) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (off) Galera rolling upgrade
  out of the scope of this item

(?) +Random+
- (?) RQG GCOV for the patch
- (/) RQG-100 ASAN
  50% tests fail due to unrelated issues
- (?) RQG-100 Release

(?) +Compatibility+
- (off) Windows
- (off) ARM
- (off) FreeBSD
- (?) i686

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+
Big DML events in concurrency with ALTER on tables without PK is a concern. old_mode is to be set for instances which can have such scenarios.

(/) +Updates/additions to non-MTR regression tests+
- various small ALTER-related changes
- syncpoint grammar extensions
- binlog replay improvements
- new old_mode in parameter presets and dynamic variables

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
c29ff60b2c8 fix cleenup??
00accd5e465 read_log_event: extract a shortcut with dedicated max_allowed_packet arg
1fa0e86f3cc ~~MDEV-31646~~ fixes
f5191233418 ~~MDEV-31812~~ Add switch to old_mode to disable non-locking ALTER
da81aa232c8 fix slave_exec_mode initialization
dd4e5d543d5 ~~MDEV-31804~~ Assertion `thd->m_transaction_psi == __null' fails
f4c5b3bc1ba Cleanup: make slave_exec_mode of its enum type and pack Log_event better
b6b0a830774 ~~MDEV-31838~~ Assertion fails upon replication online alter with MINIMAL row
a88019c0e42 ~~MDEV-31781~~ ALTER TABLE ENGINE=s3 fails
7e9ea6317e1 ~~MDEV-31777~~ ER_GET_ERRNO upon online alter on CONNECT table
0a32e10cb68 ~~MDEV-31631~~ Adding auto-increment to table with history online misbehaves
69c7c84583d ~~MDEV-31776~~ Online ALTER reports the number of affected rows incorrectly
0400e0014c7 Rows_log_event: reorganize the class layout for a lower memory footprint
ed44d2f69d0 ~~MDEV-31775~~ Server crash upon online alter on sequence
b749e1287f9 add to binlog_bytes_written for an initial description event
41f7852d9f1 make a proper cleanup if online_alter_binlog is failed to create
8750a2e9ee8 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
ebc53664028 fix key detection on replica with extra columns
6e5e24eac2a ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
df2b8694fef ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
ca32d2342df ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
d80cb40f132 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
7ed19179c69 fix -Werror build
c164dd95bd4 follow-up ~~MDEV-30430~~: fix versioning.rpl
73bbbc07719 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
8d5a9341781 ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
3b8525ac5ee Add const to get_foreign_key_list/get_parent_foreign_key_list
c3ea029832a Add const to alloc-related thd methods
d07bd4316b5 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
388d2039c5d ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
7f0b27a33ce refactor unpack_row
ed8a15d44b5 unpack_row: unpack a correct number of fields
03f806acb95 ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
d61761f2cb4 unpack_row: set the correct fields in has_value_set for online alter
b599911c92a ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
1a5b9fcc541 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
af106e27e27 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
5258c19894c ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
ab3b7b8e2dc clean up Rows_log_event virtual methods
057bb76d6bb ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
4819c13a496 add partition test
b609b9c6d10 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
c66d605c80e ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
ff0290047d9 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
ef1caa6263b ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
ad27cab7806 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
6b7a16a2f97 fix main.alter_table_{online,lock}
70e385bb6c9 ~~MDEV-29069~~ follow-up: improve DEFAULT rules
3694afb4e6e ~~MDEV-29069~~ follow-up: support partially usable keys
069a2063c44 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
23f7233fb4c ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
8292140d1d5 cleanup: cache the result of Rows_log_event::find_key()
39820b1ef87 set table->pos_in_table_list in online alter
0d8a4a8dd19 cleanup: remove rpl_group_info::get_table_data()
0495785dd4f cleanup: ifdefs
8be6788bd88 few rgi assertions. this can proof that rgi is always present
6b90a751c57 rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
f55f6916d2c ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
35401ae1e53 log_event.h: remove junk EOL spaces
a7e3bb4521d ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
ce7c344f861 Fix write_set too
ef10d88a43a ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
26c94583e42 rpl: check should go after defaults and vcols update
dd1bde0a035 ~~MDEV-29067~~ Online alter ignores check constraint violation
7b16df60410 don't do ALTER IGNORE TABLE online
c4d8107063b ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
295156d1a54 Do not ignore sql_mode when replicating
37fecbc237f Simplify rgi->get_table_data call
3ad859d37ae reorder RPL_TABLE_LIST fields for better packing
6df8322726c ~~MDEV-29021~~ fixup
4afa0b2e844 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
36764a6f278 cleanup, remove dead code
f950f4453fd ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
50ea1b9f3de ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
c61df2b4a10 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
c6d89f0a236 remove redundant warnings in RBR and online alter
e27314cf40b cleanup: whitespace, etc
ed186479fea ~~MDEV-28959~~ Online alter ignores strict table mode
43a364143ef ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
2a15af9ae9a control Cache_flip_event_log lifetime with reference count
7da5157220b ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
462bc39a120 test rename alter_table_online -> alter_table_online_debug
d560f19f813 separate online_alter_cache_data from binlog_cache_data
e968c475e7a put binlog_cache_data on a memroot
e6d69e3f952 always commit for non-trans engines
a7b11479979 savepoints
e9891c063ff don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
62fb6a20eb9 don't use start_consistent_snapshot
997aa8885b7 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
28cc0fcf5a4 don't do DROP SYSTEM VERSIONING online
d96ffbde842 set read_set early, before row reads
adbba1c0920 no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
6fb0d9dc0a3 online alter always uses ALGORITHM=COPY, LOCK=NONE
9da6f1f4687 remove handler::open_read_view()
63d43ff120b cleanup
338fe3c6962 support 'alter online table t1 page_checksum=0'
64c21c742b2 tests: move around, add new
1af3b635610 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
c4bb6280b2d ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
bf15bfb49c7 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
d61f467ef4e ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
2fdc269ee25 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
--- 8c8027970ba rpl: repack table_def
--- 7c25ddfd5ff Copy_field: add const to arguments
--- 40bed167bb2 rename tests
--- 2d837d42759 binlog_combinations.inc -> binlog_format_combinations.inc
--- 979f808883b cleanup: remove vcol_info->stored_in_db
--- 10561a10ed1 Fix recalculation of vcols in binlog_row_image=minimal
--- e7f8898393b cleanup: clarify ha_innobase::column_bitmaps_signal()
--- 74b94197eb2 allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [c29ff60b|https://github.com/MariaDB/server/commit/c29ff60b2c847b21d1cb914ac38f49d238234b35]
Baseline: [e81fa345 (11.2)|https://github.com/MariaDB/server/commit/e81fa345020ec6a067583db6a7019d6404b26f93]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- Exceptions (limitations) need to be enumerated and documented, see ~~MDEV-16329~~
- New flag in old_mode: {{LOCK_ALTER_TABLE_COPY}}

Engines:
InnoDB, MyISAM, Aria, Heap, CSV, RocksDB, Archive, Mroonga -- applies
MERGE, Spider, Blackhole, Sphinx -- skips online alter (doesn't hit debug sync point), probably because there is no real data
Columnstore -- skips online alter (doesn't increase Binlog_bytes_written, cannot test debug (!), it fails there, to be checked whether it's specific to the branch)
S3, OQGRAPH, FEDERATED, FEDERATEDX -- ALTER is prohibited in general
PERFORMANCE_SCHEMA -- cannot be ALTERed
Conversion to S3 -- prohibited. Conversion from S3 -- applies (?).
Sequences -- prohibited.
Connect -- prohibited.

*Testing*

(/) +Buildbot+
- (/) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (/) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (/) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=bb-11.2-oalter&revision=c29ff60b2c847b21d1cb914ac38f49d238234b35&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(off) +Code review notes+

(?) +Extra MTR+
- (/) GCOV for the patch
  default set + galera set, nm-big run: Lines with zero coverage (67/2603) [^missings-mtr-c29ff60b.txt]
- (/) SAN tests
** (/) ASAN default big / nm + ps + view
** (/) MSAN default big / nm + ps + view
** (/) UBSAN default big / nm + ps + view
** (/) ASAN plugins big/nm
** (/) ASAN galera big/nm

(/) +Manual Testing+
- (/) By spec/exploratory
- (/) integration map traverse

(/) +Crash recovery+
- No new problems with stability observed
- testing of recovery correctness in comparison to binlog is limited due to a number of legacy issues in binlog replay

(/) +Replication+
- No new problems with server (primary or replica) stability observed
- testing of replicaton stability and correctness is limited due to a number of legacy issues in replication

(/) +Larger integration+
- (off) Galera
  outside the scope of this item, done in ~~MDEV-27986~~
- (/) Columnstore
  does not support online alter
- (/) MariaBackup
  no specific issues observed
- (off) Protocol
- (off) Connectors

(?) +Upgrade+
- (off) Live upgrade
- (off) Dump upgrade
- (off) Package upgrade
- (/) MariaBackup upgrade
- (!) OM => NS replication upgrade
Some old bugs fixed along with online alter development (while not directly related to the feature), the fixes may cause OM => NS replication failures. E.g. if ALGORITHM=INSTANT was silently ignored in a previous version and ALTER with it succeeded, even though not executed as INSTANT, now it is rejected, thus upon replicating such events the replication will abort.
- (off) Galera rolling upgrade
  out of the scope of this item

(/) +Random+
- (!) RQG GCOV for the patch
  not satisfactory, to be further worked on after the feature release
- (/) RQG-100 ASAN
  50% tests fail due to unrelated issues
- (?) RQG-100 Release

(?) +Compatibility+
- (off) Windows
- (off) ARM
- (off) FreeBSD
- (i) i686
  ~~MDEV-31646~~ downgraded to non-RC-blocking

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+
Big DML events in concurrency with ALTER on tables without PK is a concern. old_mode is to be set for instances which can have such scenarios.

(/) +Updates/additions to non-MTR regression tests+
- various small ALTER-related changes
- syncpoint grammar extensions
- binlog replay improvements
- new old_mode in parameter presets and dynamic variables

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
c29ff60b2c8 fix cleenup??
00accd5e465 read_log_event: extract a shortcut with dedicated max_allowed_packet arg
1fa0e86f3cc ~~MDEV-31646~~ fixes
f5191233418 ~~MDEV-31812~~ Add switch to old_mode to disable non-locking ALTER
da81aa232c8 fix slave_exec_mode initialization
dd4e5d543d5 ~~MDEV-31804~~ Assertion `thd->m_transaction_psi == __null' fails
f4c5b3bc1ba Cleanup: make slave_exec_mode of its enum type and pack Log_event better
b6b0a830774 ~~MDEV-31838~~ Assertion fails upon replication online alter with MINIMAL row
a88019c0e42 ~~MDEV-31781~~ ALTER TABLE ENGINE=s3 fails
7e9ea6317e1 ~~MDEV-31777~~ ER_GET_ERRNO upon online alter on CONNECT table
0a32e10cb68 ~~MDEV-31631~~ Adding auto-increment to table with history online misbehaves
69c7c84583d ~~MDEV-31776~~ Online ALTER reports the number of affected rows incorrectly
0400e0014c7 Rows_log_event: reorganize the class layout for a lower memory footprint
ed44d2f69d0 ~~MDEV-31775~~ Server crash upon online alter on sequence
b749e1287f9 add to binlog_bytes_written for an initial description event
41f7852d9f1 make a proper cleanup if online_alter_binlog is failed to create
8750a2e9ee8 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
ebc53664028 fix key detection on replica with extra columns
6e5e24eac2a ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
df2b8694fef ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
ca32d2342df ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
d80cb40f132 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
7ed19179c69 fix -Werror build
c164dd95bd4 follow-up ~~MDEV-30430~~: fix versioning.rpl
73bbbc07719 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
8d5a9341781 ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
3b8525ac5ee Add const to get_foreign_key_list/get_parent_foreign_key_list
c3ea029832a Add const to alloc-related thd methods
d07bd4316b5 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
388d2039c5d ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
7f0b27a33ce refactor unpack_row
ed8a15d44b5 unpack_row: unpack a correct number of fields
03f806acb95 ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
d61761f2cb4 unpack_row: set the correct fields in has_value_set for online alter
b599911c92a ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
1a5b9fcc541 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
af106e27e27 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
5258c19894c ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
ab3b7b8e2dc clean up Rows_log_event virtual methods
057bb76d6bb ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
4819c13a496 add partition test
b609b9c6d10 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
c66d605c80e ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
ff0290047d9 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
ef1caa6263b ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
ad27cab7806 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
6b7a16a2f97 fix main.alter_table_{online,lock}
70e385bb6c9 ~~MDEV-29069~~ follow-up: improve DEFAULT rules
3694afb4e6e ~~MDEV-29069~~ follow-up: support partially usable keys
069a2063c44 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
23f7233fb4c ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
8292140d1d5 cleanup: cache the result of Rows_log_event::find_key()
39820b1ef87 set table->pos_in_table_list in online alter
0d8a4a8dd19 cleanup: remove rpl_group_info::get_table_data()
0495785dd4f cleanup: ifdefs
8be6788bd88 few rgi assertions. this can proof that rgi is always present
6b90a751c57 rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
f55f6916d2c ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
35401ae1e53 log_event.h: remove junk EOL spaces
a7e3bb4521d ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
ce7c344f861 Fix write_set too
ef10d88a43a ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
26c94583e42 rpl: check should go after defaults and vcols update
dd1bde0a035 ~~MDEV-29067~~ Online alter ignores check constraint violation
7b16df60410 don't do ALTER IGNORE TABLE online
c4d8107063b ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
295156d1a54 Do not ignore sql_mode when replicating
37fecbc237f Simplify rgi->get_table_data call
3ad859d37ae reorder RPL_TABLE_LIST fields for better packing
6df8322726c ~~MDEV-29021~~ fixup
4afa0b2e844 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
36764a6f278 cleanup, remove dead code
f950f4453fd ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
50ea1b9f3de ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
c61df2b4a10 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
c6d89f0a236 remove redundant warnings in RBR and online alter
e27314cf40b cleanup: whitespace, etc
ed186479fea ~~MDEV-28959~~ Online alter ignores strict table mode
43a364143ef ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
2a15af9ae9a control Cache_flip_event_log lifetime with reference count
7da5157220b ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
462bc39a120 test rename alter_table_online -> alter_table_online_debug
d560f19f813 separate online_alter_cache_data from binlog_cache_data
e968c475e7a put binlog_cache_data on a memroot
e6d69e3f952 always commit for non-trans engines
a7b11479979 savepoints
e9891c063ff don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
62fb6a20eb9 don't use start_consistent_snapshot
997aa8885b7 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
28cc0fcf5a4 don't do DROP SYSTEM VERSIONING online
d96ffbde842 set read_set early, before row reads
adbba1c0920 no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
6fb0d9dc0a3 online alter always uses ALGORITHM=COPY, LOCK=NONE
9da6f1f4687 remove handler::open_read_view()
63d43ff120b cleanup
338fe3c6962 support 'alter online table t1 page_checksum=0'
64c21c742b2 tests: move around, add new
1af3b635610 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
c4bb6280b2d ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
bf15bfb49c7 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
d61f467ef4e ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
2fdc269ee25 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
--- 8c8027970ba rpl: repack table_def
--- 7c25ddfd5ff Copy_field: add const to arguments
--- 40bed167bb2 rename tests
--- 2d837d42759 binlog_combinations.inc -> binlog_format_combinations.inc
--- 979f808883b cleanup: remove vcol_info->stored_in_db
--- 10561a10ed1 Fix recalculation of vcols in binlog_row_image=minimal
--- e7f8898393b cleanup: clarify ha_innobase::column_bitmaps_signal()
--- 74b94197eb2 allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

Elena Stepanova made changes - 2023-08-12 16:38

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [c29ff60b|https://github.com/MariaDB/server/commit/c29ff60b2c847b21d1cb914ac38f49d238234b35]
Baseline: [e81fa345 (11.2)|https://github.com/MariaDB/server/commit/e81fa345020ec6a067583db6a7019d6404b26f93]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- Exceptions (limitations) need to be enumerated and documented, see ~~MDEV-16329~~
- New flag in old_mode: {{LOCK_ALTER_TABLE_COPY}}

Engines:
InnoDB, MyISAM, Aria, Heap, CSV, RocksDB, Archive, Mroonga -- applies
MERGE, Spider, Blackhole, Sphinx -- skips online alter (doesn't hit debug sync point), probably because there is no real data
Columnstore -- skips online alter (doesn't increase Binlog_bytes_written, cannot test debug (!), it fails there, to be checked whether it's specific to the branch)
S3, OQGRAPH, FEDERATED, FEDERATEDX -- ALTER is prohibited in general
PERFORMANCE_SCHEMA -- cannot be ALTERed
Conversion to S3 -- prohibited. Conversion from S3 -- applies (?).
Sequences -- prohibited.
Connect -- prohibited.

*Testing*

(/) +Buildbot+
- (/) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (/) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (/) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=bb-11.2-oalter&revision=c29ff60b2c847b21d1cb914ac38f49d238234b35&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR+
- (?) GCOV for the patch
  default set + galera set, nm-big run: Lines with zero coverage (67/2603) [^missings-mtr-c29ff60b.txt]
- (?) SAN tests
** (/) ASAN default big / nm + ps + view
** (/) MSAN default big / nm + ps + view
** (/) UBSAN default big / nm + ps + view
** (/) ASAN plugins big/nm
** (?) ASAN galera big/nm
{noformat}
galera.galera_gcache_recover_manytrx
The TLS connection was non-properly terminated.
{noformat}

(?) +Manual Testing+
- (/) By spec/exploratory
- (/) integration map traverse

(/) +Crash recovery+
- No new problems with stability observed
- testing of recovery correctness in comparison to binlog is limited due to a number of legacy issues in binlog replay

(/) +Replication+
- No new problems with server (primary or replica) stability observed
- testing of replicaton stability and correctness is limited due to a number of legacy issues in replication

(?) +Larger integration+
- (off) Galera
  outside the scope of this item, done in ~~MDEV-27986~~
- (/) Columnstore
  does not support online alter
- (/) MariaBackup
  no specific issues observed
- (off) Protocol
- (off) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (off) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (off) Galera rolling upgrade
  out of the scope of this item

(?) +Random+
- (?) RQG GCOV for the patch
- (/) RQG-100 ASAN
  50% tests fail due to unrelated issues
- (?) RQG-100 Release

(?) +Compatibility+
- (off) Windows
- (off) ARM
- (off) FreeBSD
- (?) i686

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+
Big DML events in concurrency with ALTER on tables without PK is a concern. old_mode is to be set for instances which can have such scenarios.

(/) +Updates/additions to non-MTR regression tests+
- various small ALTER-related changes
- syncpoint grammar extensions
- binlog replay improvements
- new old_mode in parameter presets and dynamic variables

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
c29ff60b2c8 fix cleenup??
00accd5e465 read_log_event: extract a shortcut with dedicated max_allowed_packet arg
1fa0e86f3cc ~~MDEV-31646~~ fixes
f5191233418 ~~MDEV-31812~~ Add switch to old_mode to disable non-locking ALTER
da81aa232c8 fix slave_exec_mode initialization
dd4e5d543d5 ~~MDEV-31804~~ Assertion `thd->m_transaction_psi == __null' fails
f4c5b3bc1ba Cleanup: make slave_exec_mode of its enum type and pack Log_event better
b6b0a830774 ~~MDEV-31838~~ Assertion fails upon replication online alter with MINIMAL row
a88019c0e42 ~~MDEV-31781~~ ALTER TABLE ENGINE=s3 fails
7e9ea6317e1 ~~MDEV-31777~~ ER_GET_ERRNO upon online alter on CONNECT table
0a32e10cb68 ~~MDEV-31631~~ Adding auto-increment to table with history online misbehaves
69c7c84583d ~~MDEV-31776~~ Online ALTER reports the number of affected rows incorrectly
0400e0014c7 Rows_log_event: reorganize the class layout for a lower memory footprint
ed44d2f69d0 ~~MDEV-31775~~ Server crash upon online alter on sequence
b749e1287f9 add to binlog_bytes_written for an initial description event
41f7852d9f1 make a proper cleanup if online_alter_binlog is failed to create
8750a2e9ee8 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
ebc53664028 fix key detection on replica with extra columns
6e5e24eac2a ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
df2b8694fef ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
ca32d2342df ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
d80cb40f132 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
7ed19179c69 fix -Werror build
c164dd95bd4 follow-up ~~MDEV-30430~~: fix versioning.rpl
73bbbc07719 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
8d5a9341781 ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
3b8525ac5ee Add const to get_foreign_key_list/get_parent_foreign_key_list
c3ea029832a Add const to alloc-related thd methods
d07bd4316b5 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
388d2039c5d ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
7f0b27a33ce refactor unpack_row
ed8a15d44b5 unpack_row: unpack a correct number of fields
03f806acb95 ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
d61761f2cb4 unpack_row: set the correct fields in has_value_set for online alter
b599911c92a ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
1a5b9fcc541 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
af106e27e27 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
5258c19894c ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
ab3b7b8e2dc clean up Rows_log_event virtual methods
057bb76d6bb ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
4819c13a496 add partition test
b609b9c6d10 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
c66d605c80e ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
ff0290047d9 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
ef1caa6263b ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
ad27cab7806 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
6b7a16a2f97 fix main.alter_table_{online,lock}
70e385bb6c9 ~~MDEV-29069~~ follow-up: improve DEFAULT rules
3694afb4e6e ~~MDEV-29069~~ follow-up: support partially usable keys
069a2063c44 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
23f7233fb4c ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
8292140d1d5 cleanup: cache the result of Rows_log_event::find_key()
39820b1ef87 set table->pos_in_table_list in online alter
0d8a4a8dd19 cleanup: remove rpl_group_info::get_table_data()
0495785dd4f cleanup: ifdefs
8be6788bd88 few rgi assertions. this can proof that rgi is always present
6b90a751c57 rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
f55f6916d2c ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
35401ae1e53 log_event.h: remove junk EOL spaces
a7e3bb4521d ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
ce7c344f861 Fix write_set too
ef10d88a43a ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
26c94583e42 rpl: check should go after defaults and vcols update
dd1bde0a035 ~~MDEV-29067~~ Online alter ignores check constraint violation
7b16df60410 don't do ALTER IGNORE TABLE online
c4d8107063b ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
295156d1a54 Do not ignore sql_mode when replicating
37fecbc237f Simplify rgi->get_table_data call
3ad859d37ae reorder RPL_TABLE_LIST fields for better packing
6df8322726c ~~MDEV-29021~~ fixup
4afa0b2e844 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
36764a6f278 cleanup, remove dead code
f950f4453fd ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
50ea1b9f3de ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
c61df2b4a10 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
c6d89f0a236 remove redundant warnings in RBR and online alter
e27314cf40b cleanup: whitespace, etc
ed186479fea ~~MDEV-28959~~ Online alter ignores strict table mode
43a364143ef ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
2a15af9ae9a control Cache_flip_event_log lifetime with reference count
7da5157220b ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
462bc39a120 test rename alter_table_online -> alter_table_online_debug
d560f19f813 separate online_alter_cache_data from binlog_cache_data
e968c475e7a put binlog_cache_data on a memroot
e6d69e3f952 always commit for non-trans engines
a7b11479979 savepoints
e9891c063ff don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
62fb6a20eb9 don't use start_consistent_snapshot
997aa8885b7 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
28cc0fcf5a4 don't do DROP SYSTEM VERSIONING online
d96ffbde842 set read_set early, before row reads
adbba1c0920 no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
6fb0d9dc0a3 online alter always uses ALGORITHM=COPY, LOCK=NONE
9da6f1f4687 remove handler::open_read_view()
63d43ff120b cleanup
338fe3c6962 support 'alter online table t1 page_checksum=0'
64c21c742b2 tests: move around, add new
1af3b635610 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
c4bb6280b2d ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
bf15bfb49c7 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
d61f467ef4e ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
2fdc269ee25 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
--- 8c8027970ba rpl: repack table_def
--- 7c25ddfd5ff Copy_field: add const to arguments
--- 40bed167bb2 rename tests
--- 2d837d42759 binlog_combinations.inc -> binlog_format_combinations.inc
--- 979f808883b cleanup: remove vcol_info->stored_in_db
--- 10561a10ed1 Fix recalculation of vcols in binlog_row_image=minimal
--- e7f8898393b cleanup: clarify ha_innobase::column_bitmaps_signal()
--- 74b94197eb2 allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [c29ff60b|https://github.com/MariaDB/server/commit/c29ff60b2c847b21d1cb914ac38f49d238234b35]
Baseline: [e81fa345 (11.2)|https://github.com/MariaDB/server/commit/e81fa345020ec6a067583db6a7019d6404b26f93]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- Exceptions (limitations) need to be enumerated and documented, see ~~MDEV-16329~~
- New flag in old_mode: {{LOCK_ALTER_TABLE_COPY}}

Engines:
InnoDB, MyISAM, Aria, Heap, CSV, RocksDB, Archive, Mroonga -- applies
MERGE, Spider, Blackhole, Sphinx -- skips online alter (doesn't hit debug sync point), probably because there is no real data
Columnstore -- skips online alter (doesn't increase Binlog_bytes_written, cannot test debug (!), it fails there, to be checked whether it's specific to the branch)
S3, OQGRAPH, FEDERATED, FEDERATEDX -- ALTER is prohibited in general
PERFORMANCE_SCHEMA -- cannot be ALTERed
Conversion to S3 -- prohibited. Conversion from S3 -- applies (?).
Sequences -- prohibited.
Connect -- prohibited.

*Testing*

(/) +Buildbot+
- (/) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (/) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (/) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=bb-11.2-oalter&revision=c29ff60b2c847b21d1cb914ac38f49d238234b35&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(off) +Code review notes+

(?) +Extra MTR+
- (?) GCOV for the patch
  default set + galera set, nm-big run: Lines with zero coverage (67/2603) [^missings-mtr-c29ff60b.txt]
- (?) SAN tests
** (/) ASAN default big / nm + ps + view
** (/) MSAN default big / nm + ps + view
** (/) UBSAN default big / nm + ps + view
** (/) ASAN plugins big/nm
** (?) ASAN galera big/nm
{noformat}
galera.galera_gcache_recover_manytrx
The TLS connection was non-properly terminated.
{noformat}

(?) +Manual Testing+
- (/) By spec/exploratory
- (/) integration map traverse

(/) +Crash recovery+
- No new problems with stability observed
- testing of recovery correctness in comparison to binlog is limited due to a number of legacy issues in binlog replay

(/) +Replication+
- No new problems with server (primary or replica) stability observed
- testing of replicaton stability and correctness is limited due to a number of legacy issues in replication

(?) +Larger integration+
- (off) Galera
  outside the scope of this item, done in ~~MDEV-27986~~
- (/) Columnstore
  does not support online alter
- (/) MariaBackup
  no specific issues observed
- (off) Protocol
- (off) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (off) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (off) Galera rolling upgrade
  out of the scope of this item

(?) +Random+
- (?) RQG GCOV for the patch
- (/) RQG-100 ASAN
  50% tests fail due to unrelated issues
- (?) RQG-100 Release

(?) +Compatibility+
- (off) Windows
- (off) ARM
- (off) FreeBSD
- (?) i686

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+
Big DML events in concurrency with ALTER on tables without PK is a concern. old_mode is to be set for instances which can have such scenarios.

(/) +Updates/additions to non-MTR regression tests+
- various small ALTER-related changes
- syncpoint grammar extensions
- binlog replay improvements
- new old_mode in parameter presets and dynamic variables

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
c29ff60b2c8 fix cleenup??
00accd5e465 read_log_event: extract a shortcut with dedicated max_allowed_packet arg
1fa0e86f3cc ~~MDEV-31646~~ fixes
f5191233418 ~~MDEV-31812~~ Add switch to old_mode to disable non-locking ALTER
da81aa232c8 fix slave_exec_mode initialization
dd4e5d543d5 ~~MDEV-31804~~ Assertion `thd->m_transaction_psi == __null' fails
f4c5b3bc1ba Cleanup: make slave_exec_mode of its enum type and pack Log_event better
b6b0a830774 ~~MDEV-31838~~ Assertion fails upon replication online alter with MINIMAL row
a88019c0e42 ~~MDEV-31781~~ ALTER TABLE ENGINE=s3 fails
7e9ea6317e1 ~~MDEV-31777~~ ER_GET_ERRNO upon online alter on CONNECT table
0a32e10cb68 ~~MDEV-31631~~ Adding auto-increment to table with history online misbehaves
69c7c84583d ~~MDEV-31776~~ Online ALTER reports the number of affected rows incorrectly
0400e0014c7 Rows_log_event: reorganize the class layout for a lower memory footprint
ed44d2f69d0 ~~MDEV-31775~~ Server crash upon online alter on sequence
b749e1287f9 add to binlog_bytes_written for an initial description event
41f7852d9f1 make a proper cleanup if online_alter_binlog is failed to create
8750a2e9ee8 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
ebc53664028 fix key detection on replica with extra columns
6e5e24eac2a ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
df2b8694fef ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
ca32d2342df ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
d80cb40f132 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
7ed19179c69 fix -Werror build
c164dd95bd4 follow-up ~~MDEV-30430~~: fix versioning.rpl
73bbbc07719 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
8d5a9341781 ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
3b8525ac5ee Add const to get_foreign_key_list/get_parent_foreign_key_list
c3ea029832a Add const to alloc-related thd methods
d07bd4316b5 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
388d2039c5d ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
7f0b27a33ce refactor unpack_row
ed8a15d44b5 unpack_row: unpack a correct number of fields
03f806acb95 ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
d61761f2cb4 unpack_row: set the correct fields in has_value_set for online alter
b599911c92a ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
1a5b9fcc541 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
af106e27e27 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
5258c19894c ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
ab3b7b8e2dc clean up Rows_log_event virtual methods
057bb76d6bb ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
4819c13a496 add partition test
b609b9c6d10 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
c66d605c80e ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
ff0290047d9 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
ef1caa6263b ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
ad27cab7806 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
6b7a16a2f97 fix main.alter_table_{online,lock}
70e385bb6c9 ~~MDEV-29069~~ follow-up: improve DEFAULT rules
3694afb4e6e ~~MDEV-29069~~ follow-up: support partially usable keys
069a2063c44 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
23f7233fb4c ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
8292140d1d5 cleanup: cache the result of Rows_log_event::find_key()
39820b1ef87 set table->pos_in_table_list in online alter
0d8a4a8dd19 cleanup: remove rpl_group_info::get_table_data()
0495785dd4f cleanup: ifdefs
8be6788bd88 few rgi assertions. this can proof that rgi is always present
6b90a751c57 rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
f55f6916d2c ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
35401ae1e53 log_event.h: remove junk EOL spaces
a7e3bb4521d ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
ce7c344f861 Fix write_set too
ef10d88a43a ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
26c94583e42 rpl: check should go after defaults and vcols update
dd1bde0a035 ~~MDEV-29067~~ Online alter ignores check constraint violation
7b16df60410 don't do ALTER IGNORE TABLE online
c4d8107063b ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
295156d1a54 Do not ignore sql_mode when replicating
37fecbc237f Simplify rgi->get_table_data call
3ad859d37ae reorder RPL_TABLE_LIST fields for better packing
6df8322726c ~~MDEV-29021~~ fixup
4afa0b2e844 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
36764a6f278 cleanup, remove dead code
f950f4453fd ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
50ea1b9f3de ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
c61df2b4a10 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
c6d89f0a236 remove redundant warnings in RBR and online alter
e27314cf40b cleanup: whitespace, etc
ed186479fea ~~MDEV-28959~~ Online alter ignores strict table mode
43a364143ef ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
2a15af9ae9a control Cache_flip_event_log lifetime with reference count
7da5157220b ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
462bc39a120 test rename alter_table_online -> alter_table_online_debug
d560f19f813 separate online_alter_cache_data from binlog_cache_data
e968c475e7a put binlog_cache_data on a memroot
e6d69e3f952 always commit for non-trans engines
a7b11479979 savepoints
e9891c063ff don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
62fb6a20eb9 don't use start_consistent_snapshot
997aa8885b7 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
28cc0fcf5a4 don't do DROP SYSTEM VERSIONING online
d96ffbde842 set read_set early, before row reads
adbba1c0920 no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
6fb0d9dc0a3 online alter always uses ALGORITHM=COPY, LOCK=NONE
9da6f1f4687 remove handler::open_read_view()
63d43ff120b cleanup
338fe3c6962 support 'alter online table t1 page_checksum=0'
64c21c742b2 tests: move around, add new
1af3b635610 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
c4bb6280b2d ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
bf15bfb49c7 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
d61f467ef4e ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
2fdc269ee25 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
--- 8c8027970ba rpl: repack table_def
--- 7c25ddfd5ff Copy_field: add const to arguments
--- 40bed167bb2 rename tests
--- 2d837d42759 binlog_combinations.inc -> binlog_format_combinations.inc
--- 979f808883b cleanup: remove vcol_info->stored_in_db
--- 10561a10ed1 Fix recalculation of vcols in binlog_row_image=minimal
--- e7f8898393b cleanup: clarify ha_innobase::column_bitmaps_signal()
--- 74b94197eb2 allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

Elena Stepanova made changes - 2023-08-12 16:37

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [c29ff60b|https://github.com/MariaDB/server/commit/c29ff60b2c847b21d1cb914ac38f49d238234b35]
Baseline: [e81fa345 (11.2)|https://github.com/MariaDB/server/commit/e81fa345020ec6a067583db6a7019d6404b26f93]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- Exceptions (limitations) need to be enumerated and documented, see ~~MDEV-16329~~
- New flag in old_mode: {{LOCK_ALTER_TABLE_COPY}}

*Testing*

(/) +Buildbot+
- (/) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (/) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (/) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=bb-11.2-oalter&revision=c29ff60b2c847b21d1cb914ac38f49d238234b35&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR+
- (?) GCOV for the patch
  default set + galera set, nm-big run: Lines with zero coverage (67/2603) [^missings-mtr-c29ff60b.txt]
- (?) SAN tests
** (/) ASAN default big / nm + ps + view
** (/) MSAN default big / nm + ps + view
** (/) UBSAN default big / nm + ps + view
** (/) ASAN plugins big/nm
** (?) ASAN galera big/nm
{noformat}
galera.galera_gcache_recover_manytrx
The TLS connection was non-properly terminated.
{noformat}

(?) +Manual Testing+
- (/) By spec/exploratory
- (/) integration map traverse

(/) +Crash recovery+
- No new problems with stability observed
- testing of recovery correctness in comparison to binlog is limited due to a number of legacy issues in binlog replay

(/) +Replication+
- No new problems with server (primary or replica) stability observed
- testing of replicaton stability and correctness is limited due to a number of legacy issues in replication

(?) +Larger integration+
- (off) Galera
  outside the scope of this item, done in ~~MDEV-27986~~
- (/) Columnstore
  does not support online alter
- (/) MariaBackup
  no specific issues observed
- (off) Protocol
- (off) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (off) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (off) Galera rolling upgrade
  out of the scope of this item

(?) +Random+
- (?) RQG GCOV for the patch
- (/) RQG-100 ASAN
  50% tests fail due to unrelated issues
- (?) RQG-100 Release

(?) +Compatibility+
- (off) Windows
- (off) ARM
- (off) FreeBSD
- (?) i686

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+
Big DML events in concurrency with ALTER on tables without PK is a concern. old_mode is to be set for instances which can have such scenarios.

(/) +Updates/additions to non-MTR regression tests+
- various small ALTER-related changes
- syncpoint grammar extensions
- binlog replay improvements
- new old_mode in parameter presets and dynamic variables

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
c29ff60b2c8 fix cleenup??
00accd5e465 read_log_event: extract a shortcut with dedicated max_allowed_packet arg
1fa0e86f3cc ~~MDEV-31646~~ fixes
f5191233418 ~~MDEV-31812~~ Add switch to old_mode to disable non-locking ALTER
da81aa232c8 fix slave_exec_mode initialization
dd4e5d543d5 ~~MDEV-31804~~ Assertion `thd->m_transaction_psi == __null' fails
f4c5b3bc1ba Cleanup: make slave_exec_mode of its enum type and pack Log_event better
b6b0a830774 ~~MDEV-31838~~ Assertion fails upon replication online alter with MINIMAL row
a88019c0e42 ~~MDEV-31781~~ ALTER TABLE ENGINE=s3 fails
7e9ea6317e1 ~~MDEV-31777~~ ER_GET_ERRNO upon online alter on CONNECT table
0a32e10cb68 ~~MDEV-31631~~ Adding auto-increment to table with history online misbehaves
69c7c84583d ~~MDEV-31776~~ Online ALTER reports the number of affected rows incorrectly
0400e0014c7 Rows_log_event: reorganize the class layout for a lower memory footprint
ed44d2f69d0 ~~MDEV-31775~~ Server crash upon online alter on sequence
b749e1287f9 add to binlog_bytes_written for an initial description event
41f7852d9f1 make a proper cleanup if online_alter_binlog is failed to create
8750a2e9ee8 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
ebc53664028 fix key detection on replica with extra columns
6e5e24eac2a ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
df2b8694fef ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
ca32d2342df ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
d80cb40f132 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
7ed19179c69 fix -Werror build
c164dd95bd4 follow-up ~~MDEV-30430~~: fix versioning.rpl
73bbbc07719 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
8d5a9341781 ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
3b8525ac5ee Add const to get_foreign_key_list/get_parent_foreign_key_list
c3ea029832a Add const to alloc-related thd methods
d07bd4316b5 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
388d2039c5d ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
7f0b27a33ce refactor unpack_row
ed8a15d44b5 unpack_row: unpack a correct number of fields
03f806acb95 ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
d61761f2cb4 unpack_row: set the correct fields in has_value_set for online alter
b599911c92a ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
1a5b9fcc541 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
af106e27e27 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
5258c19894c ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
ab3b7b8e2dc clean up Rows_log_event virtual methods
057bb76d6bb ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
4819c13a496 add partition test
b609b9c6d10 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
c66d605c80e ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
ff0290047d9 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
ef1caa6263b ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
ad27cab7806 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
6b7a16a2f97 fix main.alter_table_{online,lock}
70e385bb6c9 ~~MDEV-29069~~ follow-up: improve DEFAULT rules
3694afb4e6e ~~MDEV-29069~~ follow-up: support partially usable keys
069a2063c44 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
23f7233fb4c ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
8292140d1d5 cleanup: cache the result of Rows_log_event::find_key()
39820b1ef87 set table->pos_in_table_list in online alter
0d8a4a8dd19 cleanup: remove rpl_group_info::get_table_data()
0495785dd4f cleanup: ifdefs
8be6788bd88 few rgi assertions. this can proof that rgi is always present
6b90a751c57 rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
f55f6916d2c ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
35401ae1e53 log_event.h: remove junk EOL spaces
a7e3bb4521d ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
ce7c344f861 Fix write_set too
ef10d88a43a ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
26c94583e42 rpl: check should go after defaults and vcols update
dd1bde0a035 ~~MDEV-29067~~ Online alter ignores check constraint violation
7b16df60410 don't do ALTER IGNORE TABLE online
c4d8107063b ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
295156d1a54 Do not ignore sql_mode when replicating
37fecbc237f Simplify rgi->get_table_data call
3ad859d37ae reorder RPL_TABLE_LIST fields for better packing
6df8322726c ~~MDEV-29021~~ fixup
4afa0b2e844 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
36764a6f278 cleanup, remove dead code
f950f4453fd ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
50ea1b9f3de ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
c61df2b4a10 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
c6d89f0a236 remove redundant warnings in RBR and online alter
e27314cf40b cleanup: whitespace, etc
ed186479fea ~~MDEV-28959~~ Online alter ignores strict table mode
43a364143ef ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
2a15af9ae9a control Cache_flip_event_log lifetime with reference count
7da5157220b ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
462bc39a120 test rename alter_table_online -> alter_table_online_debug
d560f19f813 separate online_alter_cache_data from binlog_cache_data
e968c475e7a put binlog_cache_data on a memroot
e6d69e3f952 always commit for non-trans engines
a7b11479979 savepoints
e9891c063ff don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
62fb6a20eb9 don't use start_consistent_snapshot
997aa8885b7 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
28cc0fcf5a4 don't do DROP SYSTEM VERSIONING online
d96ffbde842 set read_set early, before row reads
adbba1c0920 no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
6fb0d9dc0a3 online alter always uses ALGORITHM=COPY, LOCK=NONE
9da6f1f4687 remove handler::open_read_view()
63d43ff120b cleanup
338fe3c6962 support 'alter online table t1 page_checksum=0'
64c21c742b2 tests: move around, add new
1af3b635610 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
c4bb6280b2d ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
bf15bfb49c7 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
d61f467ef4e ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
2fdc269ee25 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
--- 8c8027970ba rpl: repack table_def
--- 7c25ddfd5ff Copy_field: add const to arguments
--- 40bed167bb2 rename tests
--- 2d837d42759 binlog_combinations.inc -> binlog_format_combinations.inc
--- 979f808883b cleanup: remove vcol_info->stored_in_db
--- 10561a10ed1 Fix recalculation of vcols in binlog_row_image=minimal
--- e7f8898393b cleanup: clarify ha_innobase::column_bitmaps_signal()
--- 74b94197eb2 allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

----

h2. Other notes

Engines:
InnoDB, MyISAM, Aria, Heap, CSV, RocksDB, Archive, Mroonga -- applies
MERGE, Spider, Blackhole, Sphinx -- skips online alter (doesn't hit debug sync point), probably because there is no real data
Columnstore -- skips online alter (doesn't increase Binlog_bytes_written, cannot test debug (!), it fails there, to be checked whether it's specific to the branch)
S3, OQGRAPH, FEDERATED, FEDERATEDX -- ALTER is prohibited in general
PERFORMANCE_SCHEMA -- cannot be ALTERed
(?) Conversion to S3 -- prohibited
(?) Sequences -- prohibited
(?) Connect -- prohibited

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [c29ff60b|https://github.com/MariaDB/server/commit/c29ff60b2c847b21d1cb914ac38f49d238234b35]
Baseline: [e81fa345 (11.2)|https://github.com/MariaDB/server/commit/e81fa345020ec6a067583db6a7019d6404b26f93]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- Exceptions (limitations) need to be enumerated and documented, see ~~MDEV-16329~~
- New flag in old_mode: {{LOCK_ALTER_TABLE_COPY}}

Engines:
InnoDB, MyISAM, Aria, Heap, CSV, RocksDB, Archive, Mroonga -- applies
MERGE, Spider, Blackhole, Sphinx -- skips online alter (doesn't hit debug sync point), probably because there is no real data
Columnstore -- skips online alter (doesn't increase Binlog_bytes_written, cannot test debug (!), it fails there, to be checked whether it's specific to the branch)
S3, OQGRAPH, FEDERATED, FEDERATEDX -- ALTER is prohibited in general
PERFORMANCE_SCHEMA -- cannot be ALTERed
Conversion to S3 -- prohibited. Conversion from S3 -- applies (?).
Sequences -- prohibited.
Connect -- prohibited.

*Testing*

(/) +Buildbot+
- (/) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (/) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (/) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=bb-11.2-oalter&revision=c29ff60b2c847b21d1cb914ac38f49d238234b35&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR+
- (?) GCOV for the patch
  default set + galera set, nm-big run: Lines with zero coverage (67/2603) [^missings-mtr-c29ff60b.txt]
- (?) SAN tests
** (/) ASAN default big / nm + ps + view
** (/) MSAN default big / nm + ps + view
** (/) UBSAN default big / nm + ps + view
** (/) ASAN plugins big/nm
** (?) ASAN galera big/nm
{noformat}
galera.galera_gcache_recover_manytrx
The TLS connection was non-properly terminated.
{noformat}

(?) +Manual Testing+
- (/) By spec/exploratory
- (/) integration map traverse

(/) +Crash recovery+
- No new problems with stability observed
- testing of recovery correctness in comparison to binlog is limited due to a number of legacy issues in binlog replay

(/) +Replication+
- No new problems with server (primary or replica) stability observed
- testing of replicaton stability and correctness is limited due to a number of legacy issues in replication

(?) +Larger integration+
- (off) Galera
  outside the scope of this item, done in ~~MDEV-27986~~
- (/) Columnstore
  does not support online alter
- (/) MariaBackup
  no specific issues observed
- (off) Protocol
- (off) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (off) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (off) Galera rolling upgrade
  out of the scope of this item

(?) +Random+
- (?) RQG GCOV for the patch
- (/) RQG-100 ASAN
  50% tests fail due to unrelated issues
- (?) RQG-100 Release

(?) +Compatibility+
- (off) Windows
- (off) ARM
- (off) FreeBSD
- (?) i686

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+
Big DML events in concurrency with ALTER on tables without PK is a concern. old_mode is to be set for instances which can have such scenarios.

(/) +Updates/additions to non-MTR regression tests+
- various small ALTER-related changes
- syncpoint grammar extensions
- binlog replay improvements
- new old_mode in parameter presets and dynamic variables

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
c29ff60b2c8 fix cleenup??
00accd5e465 read_log_event: extract a shortcut with dedicated max_allowed_packet arg
1fa0e86f3cc ~~MDEV-31646~~ fixes
f5191233418 ~~MDEV-31812~~ Add switch to old_mode to disable non-locking ALTER
da81aa232c8 fix slave_exec_mode initialization
dd4e5d543d5 ~~MDEV-31804~~ Assertion `thd->m_transaction_psi == __null' fails
f4c5b3bc1ba Cleanup: make slave_exec_mode of its enum type and pack Log_event better
b6b0a830774 ~~MDEV-31838~~ Assertion fails upon replication online alter with MINIMAL row
a88019c0e42 ~~MDEV-31781~~ ALTER TABLE ENGINE=s3 fails
7e9ea6317e1 ~~MDEV-31777~~ ER_GET_ERRNO upon online alter on CONNECT table
0a32e10cb68 ~~MDEV-31631~~ Adding auto-increment to table with history online misbehaves
69c7c84583d ~~MDEV-31776~~ Online ALTER reports the number of affected rows incorrectly
0400e0014c7 Rows_log_event: reorganize the class layout for a lower memory footprint
ed44d2f69d0 ~~MDEV-31775~~ Server crash upon online alter on sequence
b749e1287f9 add to binlog_bytes_written for an initial description event
41f7852d9f1 make a proper cleanup if online_alter_binlog is failed to create
8750a2e9ee8 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
ebc53664028 fix key detection on replica with extra columns
6e5e24eac2a ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
df2b8694fef ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
ca32d2342df ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
d80cb40f132 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
7ed19179c69 fix -Werror build
c164dd95bd4 follow-up ~~MDEV-30430~~: fix versioning.rpl
73bbbc07719 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
8d5a9341781 ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
3b8525ac5ee Add const to get_foreign_key_list/get_parent_foreign_key_list
c3ea029832a Add const to alloc-related thd methods
d07bd4316b5 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
388d2039c5d ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
7f0b27a33ce refactor unpack_row
ed8a15d44b5 unpack_row: unpack a correct number of fields
03f806acb95 ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
d61761f2cb4 unpack_row: set the correct fields in has_value_set for online alter
b599911c92a ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
1a5b9fcc541 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
af106e27e27 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
5258c19894c ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
ab3b7b8e2dc clean up Rows_log_event virtual methods
057bb76d6bb ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
4819c13a496 add partition test
b609b9c6d10 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
c66d605c80e ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
ff0290047d9 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
ef1caa6263b ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
ad27cab7806 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
6b7a16a2f97 fix main.alter_table_{online,lock}
70e385bb6c9 ~~MDEV-29069~~ follow-up: improve DEFAULT rules
3694afb4e6e ~~MDEV-29069~~ follow-up: support partially usable keys
069a2063c44 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
23f7233fb4c ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
8292140d1d5 cleanup: cache the result of Rows_log_event::find_key()
39820b1ef87 set table->pos_in_table_list in online alter
0d8a4a8dd19 cleanup: remove rpl_group_info::get_table_data()
0495785dd4f cleanup: ifdefs
8be6788bd88 few rgi assertions. this can proof that rgi is always present
6b90a751c57 rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
f55f6916d2c ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
35401ae1e53 log_event.h: remove junk EOL spaces
a7e3bb4521d ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
ce7c344f861 Fix write_set too
ef10d88a43a ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
26c94583e42 rpl: check should go after defaults and vcols update
dd1bde0a035 ~~MDEV-29067~~ Online alter ignores check constraint violation
7b16df60410 don't do ALTER IGNORE TABLE online
c4d8107063b ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
295156d1a54 Do not ignore sql_mode when replicating
37fecbc237f Simplify rgi->get_table_data call
3ad859d37ae reorder RPL_TABLE_LIST fields for better packing
6df8322726c ~~MDEV-29021~~ fixup
4afa0b2e844 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
36764a6f278 cleanup, remove dead code
f950f4453fd ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
50ea1b9f3de ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
c61df2b4a10 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
c6d89f0a236 remove redundant warnings in RBR and online alter
e27314cf40b cleanup: whitespace, etc
ed186479fea ~~MDEV-28959~~ Online alter ignores strict table mode
43a364143ef ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
2a15af9ae9a control Cache_flip_event_log lifetime with reference count
7da5157220b ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
462bc39a120 test rename alter_table_online -> alter_table_online_debug
d560f19f813 separate online_alter_cache_data from binlog_cache_data
e968c475e7a put binlog_cache_data on a memroot
e6d69e3f952 always commit for non-trans engines
a7b11479979 savepoints
e9891c063ff don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
62fb6a20eb9 don't use start_consistent_snapshot
997aa8885b7 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
28cc0fcf5a4 don't do DROP SYSTEM VERSIONING online
d96ffbde842 set read_set early, before row reads
adbba1c0920 no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
6fb0d9dc0a3 online alter always uses ALGORITHM=COPY, LOCK=NONE
9da6f1f4687 remove handler::open_read_view()
63d43ff120b cleanup
338fe3c6962 support 'alter online table t1 page_checksum=0'
64c21c742b2 tests: move around, add new
1af3b635610 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
c4bb6280b2d ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
bf15bfb49c7 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
d61f467ef4e ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
2fdc269ee25 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
--- 8c8027970ba rpl: repack table_def
--- 7c25ddfd5ff Copy_field: add const to arguments
--- 40bed167bb2 rename tests
--- 2d837d42759 binlog_combinations.inc -> binlog_format_combinations.inc
--- 979f808883b cleanup: remove vcol_info->stored_in_db
--- 10561a10ed1 Fix recalculation of vcols in binlog_row_image=minimal
--- e7f8898393b cleanup: clarify ha_innobase::column_bitmaps_signal()
--- 74b94197eb2 allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

Elena Stepanova made changes - 2023-08-11 16:05

Attachment		oalter-c29ff60b.diff [ 71538 ]
Attachment		coverage-mtr-c29ff60b.txt [ 71539 ]
Attachment		missings-mtr-c29ff60b.txt [ 71540 ]

Elena Stepanova made changes - 2023-08-11 16:04

Attachment

oalter-e599b8b4.diff [ 69372 ]

Elena Stepanova made changes - 2023-08-11 16:04

Attachment

missings-mtr-e599b8b4.txt [ 69370 ]

Elena Stepanova made changes - 2023-08-11 16:04

Attachment

coverage-mtr-e599b8b4.txt [ 69371 ]

Elena Stepanova made changes - 2023-08-11 16:04

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [c29ff60b|https://github.com/MariaDB/server/commit/c29ff60b2c847b21d1cb914ac38f49d238234b35]
Baseline: [e81fa345 (11.2)|https://github.com/MariaDB/server/commit/e81fa345020ec6a067583db6a7019d6404b26f93]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- Exceptions (limitations) need to be enumerated and documented, see ~~MDEV-16329~~
- New flag in old_mode: {{LOCK_ALTER_TABLE_COPY}}

*Testing*

(/) +Buildbot+
- (/) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (/) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (/) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=bb-11.2-oalter&revision=c29ff60b2c847b21d1cb914ac38f49d238234b35&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR+
- (?) GCOV for the patch
  default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (?) SAN tests
** (/) ASAN default big / nm + ps + view
** (/) MSAN default big / nm + ps + view
** (/) UBSAN default big / nm + ps + view
** (/) ASAN plugins big/nm
** (?) ASAN galera big/nm
{noformat}
galera.galera_gcache_recover_manytrx
The TLS connection was non-properly terminated.
{noformat}

(?) +Manual Testing+
- (/) By spec/exploratory
- (/) integration map traverse

(/) +Crash recovery+
- No new problems with stability observed
- testing of recovery correctness in comparison to binlog is limited due to a number of legacy issues in binlog replay

(/) +Replication+
- No new problems with server (primary or replica) stability observed
- testing of replicaton stability and correctness is limited due to a number of legacy issues in replication

(?) +Larger integration+
- (off) Galera
  outside the scope of this item, done in ~~MDEV-27986~~
- (/) Columnstore
  does not support online alter
- (/) MariaBackup
  no specific issues observed
- (off) Protocol
- (off) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (off) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (off) Galera rolling upgrade
  out of the scope of this item

(?) +Random+
- (?) RQG GCOV for the patch
- (/) RQG-100 ASAN
  50% tests fail due to unrelated issues
- (?) RQG-100 Release

(?) +Compatibility+
- (off) Windows
- (off) ARM
- (off) FreeBSD
- (?) i686

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+
Big DML events in concurrency with ALTER on tables without PK is a concern. old_mode is to be set for instances which can have such scenarios.

(/) +Updates/additions to non-MTR regression tests+
- various small ALTER-related changes
- syncpoint grammar extensions
- binlog replay improvements
- new old_mode in parameter presets and dynamic variables

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
c29ff60b2c8 fix cleenup??
00accd5e465 read_log_event: extract a shortcut with dedicated max_allowed_packet arg
1fa0e86f3cc ~~MDEV-31646~~ fixes
f5191233418 ~~MDEV-31812~~ Add switch to old_mode to disable non-locking ALTER
da81aa232c8 fix slave_exec_mode initialization
dd4e5d543d5 ~~MDEV-31804~~ Assertion `thd->m_transaction_psi == __null' fails
f4c5b3bc1ba Cleanup: make slave_exec_mode of its enum type and pack Log_event better
b6b0a830774 ~~MDEV-31838~~ Assertion fails upon replication online alter with MINIMAL row
a88019c0e42 ~~MDEV-31781~~ ALTER TABLE ENGINE=s3 fails
7e9ea6317e1 ~~MDEV-31777~~ ER_GET_ERRNO upon online alter on CONNECT table
0a32e10cb68 ~~MDEV-31631~~ Adding auto-increment to table with history online misbehaves
69c7c84583d ~~MDEV-31776~~ Online ALTER reports the number of affected rows incorrectly
0400e0014c7 Rows_log_event: reorganize the class layout for a lower memory footprint
ed44d2f69d0 ~~MDEV-31775~~ Server crash upon online alter on sequence
b749e1287f9 add to binlog_bytes_written for an initial description event
41f7852d9f1 make a proper cleanup if online_alter_binlog is failed to create
8750a2e9ee8 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
ebc53664028 fix key detection on replica with extra columns
6e5e24eac2a ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
df2b8694fef ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
ca32d2342df ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
d80cb40f132 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
7ed19179c69 fix -Werror build
c164dd95bd4 follow-up ~~MDEV-30430~~: fix versioning.rpl
73bbbc07719 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
8d5a9341781 ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
3b8525ac5ee Add const to get_foreign_key_list/get_parent_foreign_key_list
c3ea029832a Add const to alloc-related thd methods
d07bd4316b5 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
388d2039c5d ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
7f0b27a33ce refactor unpack_row
ed8a15d44b5 unpack_row: unpack a correct number of fields
03f806acb95 ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
d61761f2cb4 unpack_row: set the correct fields in has_value_set for online alter
b599911c92a ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
1a5b9fcc541 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
af106e27e27 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
5258c19894c ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
ab3b7b8e2dc clean up Rows_log_event virtual methods
057bb76d6bb ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
4819c13a496 add partition test
b609b9c6d10 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
c66d605c80e ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
ff0290047d9 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
ef1caa6263b ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
ad27cab7806 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
6b7a16a2f97 fix main.alter_table_{online,lock}
70e385bb6c9 ~~MDEV-29069~~ follow-up: improve DEFAULT rules
3694afb4e6e ~~MDEV-29069~~ follow-up: support partially usable keys
069a2063c44 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
23f7233fb4c ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
8292140d1d5 cleanup: cache the result of Rows_log_event::find_key()
39820b1ef87 set table->pos_in_table_list in online alter
0d8a4a8dd19 cleanup: remove rpl_group_info::get_table_data()
0495785dd4f cleanup: ifdefs
8be6788bd88 few rgi assertions. this can proof that rgi is always present
6b90a751c57 rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
f55f6916d2c ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
35401ae1e53 log_event.h: remove junk EOL spaces
a7e3bb4521d ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
ce7c344f861 Fix write_set too
ef10d88a43a ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
26c94583e42 rpl: check should go after defaults and vcols update
dd1bde0a035 ~~MDEV-29067~~ Online alter ignores check constraint violation
7b16df60410 don't do ALTER IGNORE TABLE online
c4d8107063b ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
295156d1a54 Do not ignore sql_mode when replicating
37fecbc237f Simplify rgi->get_table_data call
3ad859d37ae reorder RPL_TABLE_LIST fields for better packing
6df8322726c ~~MDEV-29021~~ fixup
4afa0b2e844 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
36764a6f278 cleanup, remove dead code
f950f4453fd ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
50ea1b9f3de ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
c61df2b4a10 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
c6d89f0a236 remove redundant warnings in RBR and online alter
e27314cf40b cleanup: whitespace, etc
ed186479fea ~~MDEV-28959~~ Online alter ignores strict table mode
43a364143ef ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
2a15af9ae9a control Cache_flip_event_log lifetime with reference count
7da5157220b ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
462bc39a120 test rename alter_table_online -> alter_table_online_debug
d560f19f813 separate online_alter_cache_data from binlog_cache_data
e968c475e7a put binlog_cache_data on a memroot
e6d69e3f952 always commit for non-trans engines
a7b11479979 savepoints
e9891c063ff don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
62fb6a20eb9 don't use start_consistent_snapshot
997aa8885b7 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
28cc0fcf5a4 don't do DROP SYSTEM VERSIONING online
d96ffbde842 set read_set early, before row reads
adbba1c0920 no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
6fb0d9dc0a3 online alter always uses ALGORITHM=COPY, LOCK=NONE
9da6f1f4687 remove handler::open_read_view()
63d43ff120b cleanup
338fe3c6962 support 'alter online table t1 page_checksum=0'
64c21c742b2 tests: move around, add new
1af3b635610 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
c4bb6280b2d ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
bf15bfb49c7 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
d61f467ef4e ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
2fdc269ee25 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
--- 8c8027970ba rpl: repack table_def
--- 7c25ddfd5ff Copy_field: add const to arguments
--- 40bed167bb2 rename tests
--- 2d837d42759 binlog_combinations.inc -> binlog_format_combinations.inc
--- 979f808883b cleanup: remove vcol_info->stored_in_db
--- 10561a10ed1 Fix recalculation of vcols in binlog_row_image=minimal
--- e7f8898393b cleanup: clarify ha_innobase::column_bitmaps_signal()
--- 74b94197eb2 allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

----

h2. Other notes

Engines:
InnoDB, MyISAM, Aria, Heap, CSV, RocksDB, Archive, Mroonga -- applies
MERGE, Spider, Blackhole, Sphinx -- skips online alter (doesn't hit debug sync point), probably because there is no real data
Columnstore -- skips online alter (doesn't increase Binlog_bytes_written, cannot test debug (!), it fails there, to be checked whether it's specific to the branch)
S3, OQGRAPH, FEDERATED, FEDERATEDX -- ALTER is prohibited in general
PERFORMANCE_SCHEMA -- cannot be ALTERed
(?) Conversion to S3 -- prohibited
(?) Sequences -- prohibited
(?) Connect -- prohibited

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [c29ff60b|https://github.com/MariaDB/server/commit/c29ff60b2c847b21d1cb914ac38f49d238234b35]
Baseline: [e81fa345 (11.2)|https://github.com/MariaDB/server/commit/e81fa345020ec6a067583db6a7019d6404b26f93]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- Exceptions (limitations) need to be enumerated and documented, see ~~MDEV-16329~~
- New flag in old_mode: {{LOCK_ALTER_TABLE_COPY}}

*Testing*

(/) +Buildbot+
- (/) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (/) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (/) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=bb-11.2-oalter&revision=c29ff60b2c847b21d1cb914ac38f49d238234b35&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR+
- (?) GCOV for the patch
  default set + galera set, nm-big run: Lines with zero coverage (67/2603) [^missings-mtr-c29ff60b.txt]
- (?) SAN tests
** (/) ASAN default big / nm + ps + view
** (/) MSAN default big / nm + ps + view
** (/) UBSAN default big / nm + ps + view
** (/) ASAN plugins big/nm
** (?) ASAN galera big/nm
{noformat}
galera.galera_gcache_recover_manytrx
The TLS connection was non-properly terminated.
{noformat}

(?) +Manual Testing+
- (/) By spec/exploratory
- (/) integration map traverse

(/) +Crash recovery+
- No new problems with stability observed
- testing of recovery correctness in comparison to binlog is limited due to a number of legacy issues in binlog replay

(/) +Replication+
- No new problems with server (primary or replica) stability observed
- testing of replicaton stability and correctness is limited due to a number of legacy issues in replication

(?) +Larger integration+
- (off) Galera
  outside the scope of this item, done in ~~MDEV-27986~~
- (/) Columnstore
  does not support online alter
- (/) MariaBackup
  no specific issues observed
- (off) Protocol
- (off) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (off) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (off) Galera rolling upgrade
  out of the scope of this item

(?) +Random+
- (?) RQG GCOV for the patch
- (/) RQG-100 ASAN
  50% tests fail due to unrelated issues
- (?) RQG-100 Release

(?) +Compatibility+
- (off) Windows
- (off) ARM
- (off) FreeBSD
- (?) i686

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+
Big DML events in concurrency with ALTER on tables without PK is a concern. old_mode is to be set for instances which can have such scenarios.

(/) +Updates/additions to non-MTR regression tests+
- various small ALTER-related changes
- syncpoint grammar extensions
- binlog replay improvements
- new old_mode in parameter presets and dynamic variables

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
c29ff60b2c8 fix cleenup??
00accd5e465 read_log_event: extract a shortcut with dedicated max_allowed_packet arg
1fa0e86f3cc ~~MDEV-31646~~ fixes
f5191233418 ~~MDEV-31812~~ Add switch to old_mode to disable non-locking ALTER
da81aa232c8 fix slave_exec_mode initialization
dd4e5d543d5 ~~MDEV-31804~~ Assertion `thd->m_transaction_psi == __null' fails
f4c5b3bc1ba Cleanup: make slave_exec_mode of its enum type and pack Log_event better
b6b0a830774 ~~MDEV-31838~~ Assertion fails upon replication online alter with MINIMAL row
a88019c0e42 ~~MDEV-31781~~ ALTER TABLE ENGINE=s3 fails
7e9ea6317e1 ~~MDEV-31777~~ ER_GET_ERRNO upon online alter on CONNECT table
0a32e10cb68 ~~MDEV-31631~~ Adding auto-increment to table with history online misbehaves
69c7c84583d ~~MDEV-31776~~ Online ALTER reports the number of affected rows incorrectly
0400e0014c7 Rows_log_event: reorganize the class layout for a lower memory footprint
ed44d2f69d0 ~~MDEV-31775~~ Server crash upon online alter on sequence
b749e1287f9 add to binlog_bytes_written for an initial description event
41f7852d9f1 make a proper cleanup if online_alter_binlog is failed to create
8750a2e9ee8 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
ebc53664028 fix key detection on replica with extra columns
6e5e24eac2a ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
df2b8694fef ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
ca32d2342df ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
d80cb40f132 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
7ed19179c69 fix -Werror build
c164dd95bd4 follow-up ~~MDEV-30430~~: fix versioning.rpl
73bbbc07719 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
8d5a9341781 ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
3b8525ac5ee Add const to get_foreign_key_list/get_parent_foreign_key_list
c3ea029832a Add const to alloc-related thd methods
d07bd4316b5 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
388d2039c5d ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
7f0b27a33ce refactor unpack_row
ed8a15d44b5 unpack_row: unpack a correct number of fields
03f806acb95 ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
d61761f2cb4 unpack_row: set the correct fields in has_value_set for online alter
b599911c92a ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
1a5b9fcc541 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
af106e27e27 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
5258c19894c ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
ab3b7b8e2dc clean up Rows_log_event virtual methods
057bb76d6bb ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
4819c13a496 add partition test
b609b9c6d10 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
c66d605c80e ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
ff0290047d9 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
ef1caa6263b ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
ad27cab7806 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
6b7a16a2f97 fix main.alter_table_{online,lock}
70e385bb6c9 ~~MDEV-29069~~ follow-up: improve DEFAULT rules
3694afb4e6e ~~MDEV-29069~~ follow-up: support partially usable keys
069a2063c44 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
23f7233fb4c ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
8292140d1d5 cleanup: cache the result of Rows_log_event::find_key()
39820b1ef87 set table->pos_in_table_list in online alter
0d8a4a8dd19 cleanup: remove rpl_group_info::get_table_data()
0495785dd4f cleanup: ifdefs
8be6788bd88 few rgi assertions. this can proof that rgi is always present
6b90a751c57 rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
f55f6916d2c ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
35401ae1e53 log_event.h: remove junk EOL spaces
a7e3bb4521d ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
ce7c344f861 Fix write_set too
ef10d88a43a ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
26c94583e42 rpl: check should go after defaults and vcols update
dd1bde0a035 ~~MDEV-29067~~ Online alter ignores check constraint violation
7b16df60410 don't do ALTER IGNORE TABLE online
c4d8107063b ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
295156d1a54 Do not ignore sql_mode when replicating
37fecbc237f Simplify rgi->get_table_data call
3ad859d37ae reorder RPL_TABLE_LIST fields for better packing
6df8322726c ~~MDEV-29021~~ fixup
4afa0b2e844 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
36764a6f278 cleanup, remove dead code
f950f4453fd ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
50ea1b9f3de ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
c61df2b4a10 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
c6d89f0a236 remove redundant warnings in RBR and online alter
e27314cf40b cleanup: whitespace, etc
ed186479fea ~~MDEV-28959~~ Online alter ignores strict table mode
43a364143ef ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
2a15af9ae9a control Cache_flip_event_log lifetime with reference count
7da5157220b ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
462bc39a120 test rename alter_table_online -> alter_table_online_debug
d560f19f813 separate online_alter_cache_data from binlog_cache_data
e968c475e7a put binlog_cache_data on a memroot
e6d69e3f952 always commit for non-trans engines
a7b11479979 savepoints
e9891c063ff don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
62fb6a20eb9 don't use start_consistent_snapshot
997aa8885b7 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
28cc0fcf5a4 don't do DROP SYSTEM VERSIONING online
d96ffbde842 set read_set early, before row reads
adbba1c0920 no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
6fb0d9dc0a3 online alter always uses ALGORITHM=COPY, LOCK=NONE
9da6f1f4687 remove handler::open_read_view()
63d43ff120b cleanup
338fe3c6962 support 'alter online table t1 page_checksum=0'
64c21c742b2 tests: move around, add new
1af3b635610 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
c4bb6280b2d ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
bf15bfb49c7 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
d61f467ef4e ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
2fdc269ee25 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
--- 8c8027970ba rpl: repack table_def
--- 7c25ddfd5ff Copy_field: add const to arguments
--- 40bed167bb2 rename tests
--- 2d837d42759 binlog_combinations.inc -> binlog_format_combinations.inc
--- 979f808883b cleanup: remove vcol_info->stored_in_db
--- 10561a10ed1 Fix recalculation of vcols in binlog_row_image=minimal
--- e7f8898393b cleanup: clarify ha_innobase::column_bitmaps_signal()
--- 74b94197eb2 allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

----

h2. Other notes

Engines:
InnoDB, MyISAM, Aria, Heap, CSV, RocksDB, Archive, Mroonga -- applies
MERGE, Spider, Blackhole, Sphinx -- skips online alter (doesn't hit debug sync point), probably because there is no real data
Columnstore -- skips online alter (doesn't increase Binlog_bytes_written, cannot test debug (!), it fails there, to be checked whether it's specific to the branch)
S3, OQGRAPH, FEDERATED, FEDERATEDX -- ALTER is prohibited in general
PERFORMANCE_SCHEMA -- cannot be ALTERed
(?) Conversion to S3 -- prohibited
(?) Sequences -- prohibited
(?) Connect -- prohibited

Elena Stepanova made changes - 2023-08-11 13:27

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [3663784d|https://github.com/MariaDB/server/commit/3663784d3d1981fe6df52d94c2b8369d84a3cc38]
Baseline: [2867894a (11.2)|https://github.com/MariaDB/server/commit/2867894ac6ca23fc2f82cfad2dd510351f597325]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(?) +Buildbot+
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=bb-11.2-oalter&revision=3663784d3d1981fe6df52d94c2b8369d84a3cc38&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (?) SAN tests ~~MDEV-30945~~ (MSAN)
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
3663784d3d1 ~~MDEV-31781~~ ALTER TABLE ENGINE=s3 fails
5f34e5ff5ac ~~MDEV-31777~~ ER_GET_ERRNO upon online alter on CONNECT table
63bedb02468 ~~MDEV-31631~~ Adding auto-increment to table with history online misbehaves
46b19c42a32 ~~MDEV-31776~~ Online ALTER reports the number of affected rows incorrectly
cfd1f427283 Rows_log_event: reorganize the class layout for a lower memory footprint
23ae9f1b2c6 ~~MDEV-31775~~ Server crash upon online alter on sequence
fade8a79479 review fixes WIP
b0484157ecd add to binlog_bytes_written for an initial description event
9b5dba149cc make a proper cleanup if online_alter_binlog is failed to create
62e596aa6a2 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
a3caa265b7a fix key detection on replica with extra columns
842cb51e81f ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
a31b2a52119 ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
8c774b12114 ~~MDEV-31646~~ preserve DMLs in case of online binlog fault
ca64ddcc709 ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
f34c7419cb8 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
bcb49750118 fix -Werror build
8b0101a853b follow-up ~~MDEV-30430~~: fix versioning.rpl
ff775c71ac1 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
dc7136481bf ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
1a062cc0ee1 Add const to get_foreign_key_list/get_parent_foreign_key_list
f6db56cff63 Add const to alloc-related thd methods
188c3f767e1 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
d75e4ea59a2 ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
f3d0a06c22f refactor unpack_row
abd73373329 unpack_row: unpack a correct number of fields
949a4f5f86a ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
ae477a9f842 unpack_row: set the correct fields in has_value_set for online alter
542e8caba74 ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
8959ba16ac6 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
157e0ceea56 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
ed3c6a4d062 ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
dd80527782e clean up Rows_log_event virtual methods
5404f2937fe ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
07191fb46d0 add partition test
afb459edbe8 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
cb4d928678f ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
d1fe3487449 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
dc4e59852d8 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
4f67b03c4f7 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
955c700e90b fix main.alter_table_{online,lock}
f434a196bda ~~MDEV-29069~~ follow-up: improve DEFAULT rules
cbe733b6b10 ~~MDEV-29069~~ follow-up: support partially usable keys
6eaf66f2d17 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
a379d5921c3 ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
26b11bb6f9c cleanup: cache the result of Rows_log_event::find_key()
ad19f3d708e set table->pos_in_table_list in online alter
79925e9b5f1 cleanup: remove rpl_group_info::get_table_data()
a8a67cdb8f4 cleanup: ifdefs
97a75c554d4 few rgi assertions. this can proof that rgi is always present
ec99d11cada rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
b2b030fd064 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
84f862c3f06 log_event.h: remove junk EOL spaces
71549936f83 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
eb9ac1b8518 Fix write_set too
4dfc48b9e0b ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
8e42878e550 rpl: check should go after defaults and vcols update
7331dfbac5c ~~MDEV-29067~~ Online alter ignores check constraint violation
cdaac40609d don't do ALTER IGNORE TABLE online
b89de5cf849 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
cfb0c7fe5d0 Do not ignore sql_mode when replicating
40eb328f0c2 Simplify rgi->get_table_data call
655d8764e68 reorder RPL_TABLE_LIST fields for better packing
0ef8bde91be ~~MDEV-29021~~ fixup
c37feb51ee1 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
7bf07c5d5c2 cleanup, remove dead code
229a4eaa2cf ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
0811905ad85 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
1c4e7ed3965 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
ced39a3cb1f remove redundant warnings in RBR and online alter
5004620d5cb cleanup: whitespace, etc
3618819bc0a ~~MDEV-28959~~ Online alter ignores strict table mode
17c0b8eba99 ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
c566c0c0f3f control Cache_flip_event_log lifetime with reference count
ee62df99aeb ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
517823bfbe1 test rename alter_table_online -> alter_table_online_debug
8dba420a547 separate online_alter_cache_data from binlog_cache_data
213cc14b65c put binlog_cache_data on a memroot
8718d110432 always commit for non-trans engines
55194c2277c savepoints
5064dc3bc83 don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
4198a9c04bc don't use start_consistent_snapshot
77bf7ac5ee6 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
a63684a1e92 don't do DROP SYSTEM VERSIONING online
307a4d33095 set read_set early, before row reads
2a18ae8e89b no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
512376f3e59 online alter always uses ALGORITHM=COPY, LOCK=NONE
c4ca1eadf54 remove handler::open_read_view()
9824910fb86 cleanup
87968ce5208 support 'alter online table t1 page_checksum=0'
84b03783860 tests: move around, add new
f7ff9b40118 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
20aae939c33 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
9ca93a9c574 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
ecf47bc2256 ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
a49e51b5db4 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
--- 0a7a45b42be rpl: repack table_def
--- 2e07d9c0def Copy_field: add const to arguments
--- 208f834397e rename tests
--- 1f9b60f2160 binlog_combinations.inc -> binlog_format_combinations.inc
--- eff7801152f cleanup: remove vcol_info->stored_in_db
--- efa160c3ad6 Fix recalculation of vcols in binlog_row_image=minimal
--- f1310b9d917 cleanup: clarify ha_innobase::column_bitmaps_signal()
--- 0d6ed7350be allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

----

h2. Other notes

Engines:
InnoDB, MyISAM, Aria, Heap, CSV, RocksDB, Archive, Mroonga -- applies
MERGE, Spider, Blackhole, Sphinx -- skips online alter (doesn't hit debug sync point), probably because there is no real data
Columnstore -- skips online alter (doesn't increase Binlog_bytes_written, cannot test debug (!), it fails there, to be checked whether it's specific to the branch)
S3, OQGRAPH, FEDERATED, FEDERATEDX -- ALTER is prohibited in general
PERFORMANCE_SCHEMA -- cannot be ALTERed
(!) Conversion to S3 -- fails, ~~MDEV-31781~~
(!) Sequences -- fails, ~~MDEV-31775~~ or maybe legacy ~~MDEV-31779~~
(!) Connect -- fails, ~~MDEV-31777~~ and a number of legacy (MDEV-30909, MDEV-30914, MDEV-30915, MDEV-30907)

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [c29ff60b|https://github.com/MariaDB/server/commit/c29ff60b2c847b21d1cb914ac38f49d238234b35]
Baseline: [e81fa345 (11.2)|https://github.com/MariaDB/server/commit/e81fa345020ec6a067583db6a7019d6404b26f93]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- Exceptions (limitations) need to be enumerated and documented, see ~~MDEV-16329~~
- New flag in old_mode: {{LOCK_ALTER_TABLE_COPY}}

*Testing*

(/) +Buildbot+
- (/) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (/) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (/) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=bb-11.2-oalter&revision=c29ff60b2c847b21d1cb914ac38f49d238234b35&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR+
- (?) GCOV for the patch
  default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (?) SAN tests
** (/) ASAN default big / nm + ps + view
** (/) MSAN default big / nm + ps + view
** (/) UBSAN default big / nm + ps + view
** (/) ASAN plugins big/nm
** (?) ASAN galera big/nm
{noformat}
galera.galera_gcache_recover_manytrx
The TLS connection was non-properly terminated.
{noformat}

(?) +Manual Testing+
- (/) By spec/exploratory
- (/) integration map traverse

(/) +Crash recovery+
- No new problems with stability observed
- testing of recovery correctness in comparison to binlog is limited due to a number of legacy issues in binlog replay

(/) +Replication+
- No new problems with server (primary or replica) stability observed
- testing of replicaton stability and correctness is limited due to a number of legacy issues in replication

(?) +Larger integration+
- (off) Galera
  outside the scope of this item, done in ~~MDEV-27986~~
- (/) Columnstore
  does not support online alter
- (/) MariaBackup
  no specific issues observed
- (off) Protocol
- (off) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (off) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (off) Galera rolling upgrade
  out of the scope of this item

(?) +Random+
- (?) RQG GCOV for the patch
- (/) RQG-100 ASAN
  50% tests fail due to unrelated issues
- (?) RQG-100 Release

(?) +Compatibility+
- (off) Windows
- (off) ARM
- (off) FreeBSD
- (?) i686

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+
Big DML events in concurrency with ALTER on tables without PK is a concern. old_mode is to be set for instances which can have such scenarios.

(/) +Updates/additions to non-MTR regression tests+
- various small ALTER-related changes
- syncpoint grammar extensions
- binlog replay improvements
- new old_mode in parameter presets and dynamic variables

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
c29ff60b2c8 fix cleenup??
00accd5e465 read_log_event: extract a shortcut with dedicated max_allowed_packet arg
1fa0e86f3cc ~~MDEV-31646~~ fixes
f5191233418 ~~MDEV-31812~~ Add switch to old_mode to disable non-locking ALTER
da81aa232c8 fix slave_exec_mode initialization
dd4e5d543d5 ~~MDEV-31804~~ Assertion `thd->m_transaction_psi == __null' fails
f4c5b3bc1ba Cleanup: make slave_exec_mode of its enum type and pack Log_event better
b6b0a830774 ~~MDEV-31838~~ Assertion fails upon replication online alter with MINIMAL row
a88019c0e42 ~~MDEV-31781~~ ALTER TABLE ENGINE=s3 fails
7e9ea6317e1 ~~MDEV-31777~~ ER_GET_ERRNO upon online alter on CONNECT table
0a32e10cb68 ~~MDEV-31631~~ Adding auto-increment to table with history online misbehaves
69c7c84583d ~~MDEV-31776~~ Online ALTER reports the number of affected rows incorrectly
0400e0014c7 Rows_log_event: reorganize the class layout for a lower memory footprint
ed44d2f69d0 ~~MDEV-31775~~ Server crash upon online alter on sequence
b749e1287f9 add to binlog_bytes_written for an initial description event
41f7852d9f1 make a proper cleanup if online_alter_binlog is failed to create
8750a2e9ee8 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
ebc53664028 fix key detection on replica with extra columns
6e5e24eac2a ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
df2b8694fef ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
ca32d2342df ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
d80cb40f132 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
7ed19179c69 fix -Werror build
c164dd95bd4 follow-up ~~MDEV-30430~~: fix versioning.rpl
73bbbc07719 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
8d5a9341781 ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
3b8525ac5ee Add const to get_foreign_key_list/get_parent_foreign_key_list
c3ea029832a Add const to alloc-related thd methods
d07bd4316b5 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
388d2039c5d ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
7f0b27a33ce refactor unpack_row
ed8a15d44b5 unpack_row: unpack a correct number of fields
03f806acb95 ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
d61761f2cb4 unpack_row: set the correct fields in has_value_set for online alter
b599911c92a ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
1a5b9fcc541 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
af106e27e27 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
5258c19894c ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
ab3b7b8e2dc clean up Rows_log_event virtual methods
057bb76d6bb ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
4819c13a496 add partition test
b609b9c6d10 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
c66d605c80e ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
ff0290047d9 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
ef1caa6263b ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
ad27cab7806 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
6b7a16a2f97 fix main.alter_table_{online,lock}
70e385bb6c9 ~~MDEV-29069~~ follow-up: improve DEFAULT rules
3694afb4e6e ~~MDEV-29069~~ follow-up: support partially usable keys
069a2063c44 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
23f7233fb4c ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
8292140d1d5 cleanup: cache the result of Rows_log_event::find_key()
39820b1ef87 set table->pos_in_table_list in online alter
0d8a4a8dd19 cleanup: remove rpl_group_info::get_table_data()
0495785dd4f cleanup: ifdefs
8be6788bd88 few rgi assertions. this can proof that rgi is always present
6b90a751c57 rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
f55f6916d2c ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
35401ae1e53 log_event.h: remove junk EOL spaces
a7e3bb4521d ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
ce7c344f861 Fix write_set too
ef10d88a43a ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
26c94583e42 rpl: check should go after defaults and vcols update
dd1bde0a035 ~~MDEV-29067~~ Online alter ignores check constraint violation
7b16df60410 don't do ALTER IGNORE TABLE online
c4d8107063b ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
295156d1a54 Do not ignore sql_mode when replicating
37fecbc237f Simplify rgi->get_table_data call
3ad859d37ae reorder RPL_TABLE_LIST fields for better packing
6df8322726c ~~MDEV-29021~~ fixup
4afa0b2e844 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
36764a6f278 cleanup, remove dead code
f950f4453fd ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
50ea1b9f3de ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
c61df2b4a10 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
c6d89f0a236 remove redundant warnings in RBR and online alter
e27314cf40b cleanup: whitespace, etc
ed186479fea ~~MDEV-28959~~ Online alter ignores strict table mode
43a364143ef ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
2a15af9ae9a control Cache_flip_event_log lifetime with reference count
7da5157220b ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
462bc39a120 test rename alter_table_online -> alter_table_online_debug
d560f19f813 separate online_alter_cache_data from binlog_cache_data
e968c475e7a put binlog_cache_data on a memroot
e6d69e3f952 always commit for non-trans engines
a7b11479979 savepoints
e9891c063ff don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
62fb6a20eb9 don't use start_consistent_snapshot
997aa8885b7 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
28cc0fcf5a4 don't do DROP SYSTEM VERSIONING online
d96ffbde842 set read_set early, before row reads
adbba1c0920 no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
6fb0d9dc0a3 online alter always uses ALGORITHM=COPY, LOCK=NONE
9da6f1f4687 remove handler::open_read_view()
63d43ff120b cleanup
338fe3c6962 support 'alter online table t1 page_checksum=0'
64c21c742b2 tests: move around, add new
1af3b635610 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
c4bb6280b2d ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
bf15bfb49c7 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
d61f467ef4e ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
2fdc269ee25 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
--- 8c8027970ba rpl: repack table_def
--- 7c25ddfd5ff Copy_field: add const to arguments
--- 40bed167bb2 rename tests
--- 2d837d42759 binlog_combinations.inc -> binlog_format_combinations.inc
--- 979f808883b cleanup: remove vcol_info->stored_in_db
--- 10561a10ed1 Fix recalculation of vcols in binlog_row_image=minimal
--- e7f8898393b cleanup: clarify ha_innobase::column_bitmaps_signal()
--- 74b94197eb2 allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

----

h2. Other notes

Engines:
InnoDB, MyISAM, Aria, Heap, CSV, RocksDB, Archive, Mroonga -- applies
MERGE, Spider, Blackhole, Sphinx -- skips online alter (doesn't hit debug sync point), probably because there is no real data
Columnstore -- skips online alter (doesn't increase Binlog_bytes_written, cannot test debug (!), it fails there, to be checked whether it's specific to the branch)
S3, OQGRAPH, FEDERATED, FEDERATEDX -- ALTER is prohibited in general
PERFORMANCE_SCHEMA -- cannot be ALTERed
(?) Conversion to S3 -- prohibited
(?) Sequences -- prohibited
(?) Connect -- prohibited

Elena Stepanova made changes - 2023-08-01 09:26

Status

Stalled [ 10000 ]

In Progress [ 3 ]

Elena Stepanova made changes - 2023-07-28 22:57

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [3663784d|https://github.com/MariaDB/server/commit/3663784d3d1981fe6df52d94c2b8369d84a3cc38]
Baseline: [2867894a (11.2)|https://github.com/MariaDB/server/commit/2867894ac6ca23fc2f82cfad2dd510351f597325]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(?) +Buildbot+
- (x) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (x) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (x) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=bb-11.2-oalter&revision=3663784d3d1981fe6df52d94c2b8369d84a3cc38&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (?) SAN tests ~~MDEV-30945~~ (MSAN)
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
3663784d3d1 ~~MDEV-31781~~ ALTER TABLE ENGINE=s3 fails
5f34e5ff5ac ~~MDEV-31777~~ ER_GET_ERRNO upon online alter on CONNECT table
63bedb02468 ~~MDEV-31631~~ Adding auto-increment to table with history online misbehaves
46b19c42a32 ~~MDEV-31776~~ Online ALTER reports the number of affected rows incorrectly
cfd1f427283 Rows_log_event: reorganize the class layout for a lower memory footprint
23ae9f1b2c6 ~~MDEV-31775~~ Server crash upon online alter on sequence
fade8a79479 review fixes WIP
b0484157ecd add to binlog_bytes_written for an initial description event
9b5dba149cc make a proper cleanup if online_alter_binlog is failed to create
62e596aa6a2 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
a3caa265b7a fix key detection on replica with extra columns
842cb51e81f ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
a31b2a52119 ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
8c774b12114 ~~MDEV-31646~~ preserve DMLs in case of online binlog fault
ca64ddcc709 ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
f34c7419cb8 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
bcb49750118 fix -Werror build
8b0101a853b follow-up ~~MDEV-30430~~: fix versioning.rpl
ff775c71ac1 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
dc7136481bf ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
1a062cc0ee1 Add const to get_foreign_key_list/get_parent_foreign_key_list
f6db56cff63 Add const to alloc-related thd methods
188c3f767e1 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
d75e4ea59a2 ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
f3d0a06c22f refactor unpack_row
abd73373329 unpack_row: unpack a correct number of fields
949a4f5f86a ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
ae477a9f842 unpack_row: set the correct fields in has_value_set for online alter
542e8caba74 ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
8959ba16ac6 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
157e0ceea56 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
ed3c6a4d062 ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
dd80527782e clean up Rows_log_event virtual methods
5404f2937fe ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
07191fb46d0 add partition test
afb459edbe8 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
cb4d928678f ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
d1fe3487449 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
dc4e59852d8 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
4f67b03c4f7 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
955c700e90b fix main.alter_table_{online,lock}
f434a196bda ~~MDEV-29069~~ follow-up: improve DEFAULT rules
cbe733b6b10 ~~MDEV-29069~~ follow-up: support partially usable keys
6eaf66f2d17 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
a379d5921c3 ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
26b11bb6f9c cleanup: cache the result of Rows_log_event::find_key()
ad19f3d708e set table->pos_in_table_list in online alter
79925e9b5f1 cleanup: remove rpl_group_info::get_table_data()
a8a67cdb8f4 cleanup: ifdefs
97a75c554d4 few rgi assertions. this can proof that rgi is always present
ec99d11cada rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
b2b030fd064 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
84f862c3f06 log_event.h: remove junk EOL spaces
71549936f83 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
eb9ac1b8518 Fix write_set too
4dfc48b9e0b ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
8e42878e550 rpl: check should go after defaults and vcols update
7331dfbac5c ~~MDEV-29067~~ Online alter ignores check constraint violation
cdaac40609d don't do ALTER IGNORE TABLE online
b89de5cf849 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
cfb0c7fe5d0 Do not ignore sql_mode when replicating
40eb328f0c2 Simplify rgi->get_table_data call
655d8764e68 reorder RPL_TABLE_LIST fields for better packing
0ef8bde91be ~~MDEV-29021~~ fixup
c37feb51ee1 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
7bf07c5d5c2 cleanup, remove dead code
229a4eaa2cf ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
0811905ad85 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
1c4e7ed3965 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
ced39a3cb1f remove redundant warnings in RBR and online alter
5004620d5cb cleanup: whitespace, etc
3618819bc0a ~~MDEV-28959~~ Online alter ignores strict table mode
17c0b8eba99 ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
c566c0c0f3f control Cache_flip_event_log lifetime with reference count
ee62df99aeb ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
517823bfbe1 test rename alter_table_online -> alter_table_online_debug
8dba420a547 separate online_alter_cache_data from binlog_cache_data
213cc14b65c put binlog_cache_data on a memroot
8718d110432 always commit for non-trans engines
55194c2277c savepoints
5064dc3bc83 don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
4198a9c04bc don't use start_consistent_snapshot
77bf7ac5ee6 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
a63684a1e92 don't do DROP SYSTEM VERSIONING online
307a4d33095 set read_set early, before row reads
2a18ae8e89b no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
512376f3e59 online alter always uses ALGORITHM=COPY, LOCK=NONE
c4ca1eadf54 remove handler::open_read_view()
9824910fb86 cleanup
87968ce5208 support 'alter online table t1 page_checksum=0'
84b03783860 tests: move around, add new
f7ff9b40118 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
20aae939c33 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
9ca93a9c574 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
ecf47bc2256 ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
a49e51b5db4 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
--- 0a7a45b42be rpl: repack table_def
--- 2e07d9c0def Copy_field: add const to arguments
--- 208f834397e rename tests
--- 1f9b60f2160 binlog_combinations.inc -> binlog_format_combinations.inc
--- eff7801152f cleanup: remove vcol_info->stored_in_db
--- efa160c3ad6 Fix recalculation of vcols in binlog_row_image=minimal
--- f1310b9d917 cleanup: clarify ha_innobase::column_bitmaps_signal()
--- 0d6ed7350be allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

----

h2. Other notes

Engines:
InnoDB, MyISAM, Aria, Heap, CSV, RocksDB, Archive, Mroonga -- applies
MERGE, Spider, Blackhole, Sphinx -- skips online alter (doesn't hit debug sync point), probably because there is no real data
Columnstore -- skips online alter (doesn't increase Binlog_bytes_written, cannot test debug (!), it fails there, to be checked whether it's specific to the branch)
S3, OQGRAPH, FEDERATED, FEDERATEDX -- ALTER is prohibited in general
PERFORMANCE_SCHEMA -- cannot be ALTERed
(!) Conversion to S3 -- fails, ~~MDEV-31781~~
(!) Sequences -- fails, ~~MDEV-31775~~ or maybe legacy ~~MDEV-31779~~
(!) Connect -- fails, ~~MDEV-31777~~ and a number of legacy (MDEV-30909, MDEV-30914, MDEV-30915, MDEV-30907)

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [3663784d|https://github.com/MariaDB/server/commit/3663784d3d1981fe6df52d94c2b8369d84a3cc38]
Baseline: [2867894a (11.2)|https://github.com/MariaDB/server/commit/2867894ac6ca23fc2f82cfad2dd510351f597325]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(?) +Buildbot+
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=bb-11.2-oalter&revision=3663784d3d1981fe6df52d94c2b8369d84a3cc38&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (?) SAN tests ~~MDEV-30945~~ (MSAN)
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
3663784d3d1 ~~MDEV-31781~~ ALTER TABLE ENGINE=s3 fails
5f34e5ff5ac ~~MDEV-31777~~ ER_GET_ERRNO upon online alter on CONNECT table
63bedb02468 ~~MDEV-31631~~ Adding auto-increment to table with history online misbehaves
46b19c42a32 ~~MDEV-31776~~ Online ALTER reports the number of affected rows incorrectly
cfd1f427283 Rows_log_event: reorganize the class layout for a lower memory footprint
23ae9f1b2c6 ~~MDEV-31775~~ Server crash upon online alter on sequence
fade8a79479 review fixes WIP
b0484157ecd add to binlog_bytes_written for an initial description event
9b5dba149cc make a proper cleanup if online_alter_binlog is failed to create
62e596aa6a2 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
a3caa265b7a fix key detection on replica with extra columns
842cb51e81f ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
a31b2a52119 ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
8c774b12114 ~~MDEV-31646~~ preserve DMLs in case of online binlog fault
ca64ddcc709 ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
f34c7419cb8 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
bcb49750118 fix -Werror build
8b0101a853b follow-up ~~MDEV-30430~~: fix versioning.rpl
ff775c71ac1 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
dc7136481bf ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
1a062cc0ee1 Add const to get_foreign_key_list/get_parent_foreign_key_list
f6db56cff63 Add const to alloc-related thd methods
188c3f767e1 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
d75e4ea59a2 ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
f3d0a06c22f refactor unpack_row
abd73373329 unpack_row: unpack a correct number of fields
949a4f5f86a ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
ae477a9f842 unpack_row: set the correct fields in has_value_set for online alter
542e8caba74 ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
8959ba16ac6 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
157e0ceea56 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
ed3c6a4d062 ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
dd80527782e clean up Rows_log_event virtual methods
5404f2937fe ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
07191fb46d0 add partition test
afb459edbe8 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
cb4d928678f ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
d1fe3487449 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
dc4e59852d8 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
4f67b03c4f7 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
955c700e90b fix main.alter_table_{online,lock}
f434a196bda ~~MDEV-29069~~ follow-up: improve DEFAULT rules
cbe733b6b10 ~~MDEV-29069~~ follow-up: support partially usable keys
6eaf66f2d17 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
a379d5921c3 ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
26b11bb6f9c cleanup: cache the result of Rows_log_event::find_key()
ad19f3d708e set table->pos_in_table_list in online alter
79925e9b5f1 cleanup: remove rpl_group_info::get_table_data()
a8a67cdb8f4 cleanup: ifdefs
97a75c554d4 few rgi assertions. this can proof that rgi is always present
ec99d11cada rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
b2b030fd064 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
84f862c3f06 log_event.h: remove junk EOL spaces
71549936f83 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
eb9ac1b8518 Fix write_set too
4dfc48b9e0b ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
8e42878e550 rpl: check should go after defaults and vcols update
7331dfbac5c ~~MDEV-29067~~ Online alter ignores check constraint violation
cdaac40609d don't do ALTER IGNORE TABLE online
b89de5cf849 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
cfb0c7fe5d0 Do not ignore sql_mode when replicating
40eb328f0c2 Simplify rgi->get_table_data call
655d8764e68 reorder RPL_TABLE_LIST fields for better packing
0ef8bde91be ~~MDEV-29021~~ fixup
c37feb51ee1 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
7bf07c5d5c2 cleanup, remove dead code
229a4eaa2cf ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
0811905ad85 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
1c4e7ed3965 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
ced39a3cb1f remove redundant warnings in RBR and online alter
5004620d5cb cleanup: whitespace, etc
3618819bc0a ~~MDEV-28959~~ Online alter ignores strict table mode
17c0b8eba99 ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
c566c0c0f3f control Cache_flip_event_log lifetime with reference count
ee62df99aeb ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
517823bfbe1 test rename alter_table_online -> alter_table_online_debug
8dba420a547 separate online_alter_cache_data from binlog_cache_data
213cc14b65c put binlog_cache_data on a memroot
8718d110432 always commit for non-trans engines
55194c2277c savepoints
5064dc3bc83 don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
4198a9c04bc don't use start_consistent_snapshot
77bf7ac5ee6 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
a63684a1e92 don't do DROP SYSTEM VERSIONING online
307a4d33095 set read_set early, before row reads
2a18ae8e89b no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
512376f3e59 online alter always uses ALGORITHM=COPY, LOCK=NONE
c4ca1eadf54 remove handler::open_read_view()
9824910fb86 cleanup
87968ce5208 support 'alter online table t1 page_checksum=0'
84b03783860 tests: move around, add new
f7ff9b40118 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
20aae939c33 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
9ca93a9c574 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
ecf47bc2256 ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
a49e51b5db4 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
--- 0a7a45b42be rpl: repack table_def
--- 2e07d9c0def Copy_field: add const to arguments
--- 208f834397e rename tests
--- 1f9b60f2160 binlog_combinations.inc -> binlog_format_combinations.inc
--- eff7801152f cleanup: remove vcol_info->stored_in_db
--- efa160c3ad6 Fix recalculation of vcols in binlog_row_image=minimal
--- f1310b9d917 cleanup: clarify ha_innobase::column_bitmaps_signal()
--- 0d6ed7350be allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

----

h2. Other notes

Engines:
InnoDB, MyISAM, Aria, Heap, CSV, RocksDB, Archive, Mroonga -- applies
MERGE, Spider, Blackhole, Sphinx -- skips online alter (doesn't hit debug sync point), probably because there is no real data
Columnstore -- skips online alter (doesn't increase Binlog_bytes_written, cannot test debug (!), it fails there, to be checked whether it's specific to the branch)
S3, OQGRAPH, FEDERATED, FEDERATEDX -- ALTER is prohibited in general
PERFORMANCE_SCHEMA -- cannot be ALTERed
(!) Conversion to S3 -- fails, ~~MDEV-31781~~
(!) Sequences -- fails, ~~MDEV-31775~~ or maybe legacy ~~MDEV-31779~~
(!) Connect -- fails, ~~MDEV-31777~~ and a number of legacy (MDEV-30909, MDEV-30914, MDEV-30915, MDEV-30907)

Elena Stepanova made changes - 2023-07-28 19:02

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [3663784d|https://github.com/MariaDB/server/commit/3663784d3d1981fe6df52d94c2b8369d84a3cc38]
Baseline: [2867894a (11.2)|https://github.com/MariaDB/server/commit/2867894ac6ca23fc2f82cfad2dd510351f597325]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(?) +Buildbot+
- (x) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (x) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=bb-11.2-oalter&revision=3663784d3d1981fe6df52d94c2b8369d84a3cc38&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (?) SAN tests ~~MDEV-30945~~ (MSAN)
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
3663784d3d1 ~~MDEV-31781~~ ALTER TABLE ENGINE=s3 fails
5f34e5ff5ac ~~MDEV-31777~~ ER_GET_ERRNO upon online alter on CONNECT table
63bedb02468 ~~MDEV-31631~~ Adding auto-increment to table with history online misbehaves
46b19c42a32 ~~MDEV-31776~~ Online ALTER reports the number of affected rows incorrectly
cfd1f427283 Rows_log_event: reorganize the class layout for a lower memory footprint
23ae9f1b2c6 ~~MDEV-31775~~ Server crash upon online alter on sequence
fade8a79479 review fixes WIP
b0484157ecd add to binlog_bytes_written for an initial description event
9b5dba149cc make a proper cleanup if online_alter_binlog is failed to create
62e596aa6a2 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
a3caa265b7a fix key detection on replica with extra columns
842cb51e81f ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
a31b2a52119 ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
8c774b12114 ~~MDEV-31646~~ preserve DMLs in case of online binlog fault
ca64ddcc709 ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
f34c7419cb8 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
bcb49750118 fix -Werror build
8b0101a853b follow-up ~~MDEV-30430~~: fix versioning.rpl
ff775c71ac1 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
dc7136481bf ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
1a062cc0ee1 Add const to get_foreign_key_list/get_parent_foreign_key_list
f6db56cff63 Add const to alloc-related thd methods
188c3f767e1 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
d75e4ea59a2 ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
f3d0a06c22f refactor unpack_row
abd73373329 unpack_row: unpack a correct number of fields
949a4f5f86a ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
ae477a9f842 unpack_row: set the correct fields in has_value_set for online alter
542e8caba74 ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
8959ba16ac6 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
157e0ceea56 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
ed3c6a4d062 ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
dd80527782e clean up Rows_log_event virtual methods
5404f2937fe ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
07191fb46d0 add partition test
afb459edbe8 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
cb4d928678f ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
d1fe3487449 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
dc4e59852d8 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
4f67b03c4f7 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
955c700e90b fix main.alter_table_{online,lock}
f434a196bda ~~MDEV-29069~~ follow-up: improve DEFAULT rules
cbe733b6b10 ~~MDEV-29069~~ follow-up: support partially usable keys
6eaf66f2d17 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
a379d5921c3 ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
26b11bb6f9c cleanup: cache the result of Rows_log_event::find_key()
ad19f3d708e set table->pos_in_table_list in online alter
79925e9b5f1 cleanup: remove rpl_group_info::get_table_data()
a8a67cdb8f4 cleanup: ifdefs
97a75c554d4 few rgi assertions. this can proof that rgi is always present
ec99d11cada rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
b2b030fd064 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
84f862c3f06 log_event.h: remove junk EOL spaces
71549936f83 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
eb9ac1b8518 Fix write_set too
4dfc48b9e0b ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
8e42878e550 rpl: check should go after defaults and vcols update
7331dfbac5c ~~MDEV-29067~~ Online alter ignores check constraint violation
cdaac40609d don't do ALTER IGNORE TABLE online
b89de5cf849 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
cfb0c7fe5d0 Do not ignore sql_mode when replicating
40eb328f0c2 Simplify rgi->get_table_data call
655d8764e68 reorder RPL_TABLE_LIST fields for better packing
0ef8bde91be ~~MDEV-29021~~ fixup
c37feb51ee1 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
7bf07c5d5c2 cleanup, remove dead code
229a4eaa2cf ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
0811905ad85 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
1c4e7ed3965 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
ced39a3cb1f remove redundant warnings in RBR and online alter
5004620d5cb cleanup: whitespace, etc
3618819bc0a ~~MDEV-28959~~ Online alter ignores strict table mode
17c0b8eba99 ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
c566c0c0f3f control Cache_flip_event_log lifetime with reference count
ee62df99aeb ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
517823bfbe1 test rename alter_table_online -> alter_table_online_debug
8dba420a547 separate online_alter_cache_data from binlog_cache_data
213cc14b65c put binlog_cache_data on a memroot
8718d110432 always commit for non-trans engines
55194c2277c savepoints
5064dc3bc83 don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
4198a9c04bc don't use start_consistent_snapshot
77bf7ac5ee6 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
a63684a1e92 don't do DROP SYSTEM VERSIONING online
307a4d33095 set read_set early, before row reads
2a18ae8e89b no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
512376f3e59 online alter always uses ALGORITHM=COPY, LOCK=NONE
c4ca1eadf54 remove handler::open_read_view()
9824910fb86 cleanup
87968ce5208 support 'alter online table t1 page_checksum=0'
84b03783860 tests: move around, add new
f7ff9b40118 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
20aae939c33 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
9ca93a9c574 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
ecf47bc2256 ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
a49e51b5db4 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
--- 0a7a45b42be rpl: repack table_def
--- 2e07d9c0def Copy_field: add const to arguments
--- 208f834397e rename tests
--- 1f9b60f2160 binlog_combinations.inc -> binlog_format_combinations.inc
--- eff7801152f cleanup: remove vcol_info->stored_in_db
--- efa160c3ad6 Fix recalculation of vcols in binlog_row_image=minimal
--- f1310b9d917 cleanup: clarify ha_innobase::column_bitmaps_signal()
--- 0d6ed7350be allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

----

h2. Other notes

Engines:
InnoDB, MyISAM, Aria, Heap, CSV, RocksDB, Archive, Mroonga -- applies
MERGE, Spider, Blackhole, Sphinx -- skips online alter (doesn't hit debug sync point), probably because there is no real data
Columnstore -- skips online alter (doesn't increase Binlog_bytes_written, cannot test debug (!), it fails there, to be checked whether it's specific to the branch)
S3, OQGRAPH, FEDERATED, FEDERATEDX -- ALTER is prohibited in general
PERFORMANCE_SCHEMA -- cannot be ALTERed
(!) Conversion to S3 -- fails, ~~MDEV-31781~~
(!) Sequences -- fails, ~~MDEV-31775~~ or maybe legacy ~~MDEV-31779~~
(!) Connect -- fails, ~~MDEV-31777~~ and a number of legacy (MDEV-30909, MDEV-30914, MDEV-30915, MDEV-30907)

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [3663784d|https://github.com/MariaDB/server/commit/3663784d3d1981fe6df52d94c2b8369d84a3cc38]
Baseline: [2867894a (11.2)|https://github.com/MariaDB/server/commit/2867894ac6ca23fc2f82cfad2dd510351f597325]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(?) +Buildbot+
- (x) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (x) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (x) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=bb-11.2-oalter&revision=3663784d3d1981fe6df52d94c2b8369d84a3cc38&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (?) SAN tests ~~MDEV-30945~~ (MSAN)
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
3663784d3d1 ~~MDEV-31781~~ ALTER TABLE ENGINE=s3 fails
5f34e5ff5ac ~~MDEV-31777~~ ER_GET_ERRNO upon online alter on CONNECT table
63bedb02468 ~~MDEV-31631~~ Adding auto-increment to table with history online misbehaves
46b19c42a32 ~~MDEV-31776~~ Online ALTER reports the number of affected rows incorrectly
cfd1f427283 Rows_log_event: reorganize the class layout for a lower memory footprint
23ae9f1b2c6 ~~MDEV-31775~~ Server crash upon online alter on sequence
fade8a79479 review fixes WIP
b0484157ecd add to binlog_bytes_written for an initial description event
9b5dba149cc make a proper cleanup if online_alter_binlog is failed to create
62e596aa6a2 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
a3caa265b7a fix key detection on replica with extra columns
842cb51e81f ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
a31b2a52119 ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
8c774b12114 ~~MDEV-31646~~ preserve DMLs in case of online binlog fault
ca64ddcc709 ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
f34c7419cb8 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
bcb49750118 fix -Werror build
8b0101a853b follow-up ~~MDEV-30430~~: fix versioning.rpl
ff775c71ac1 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
dc7136481bf ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
1a062cc0ee1 Add const to get_foreign_key_list/get_parent_foreign_key_list
f6db56cff63 Add const to alloc-related thd methods
188c3f767e1 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
d75e4ea59a2 ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
f3d0a06c22f refactor unpack_row
abd73373329 unpack_row: unpack a correct number of fields
949a4f5f86a ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
ae477a9f842 unpack_row: set the correct fields in has_value_set for online alter
542e8caba74 ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
8959ba16ac6 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
157e0ceea56 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
ed3c6a4d062 ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
dd80527782e clean up Rows_log_event virtual methods
5404f2937fe ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
07191fb46d0 add partition test
afb459edbe8 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
cb4d928678f ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
d1fe3487449 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
dc4e59852d8 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
4f67b03c4f7 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
955c700e90b fix main.alter_table_{online,lock}
f434a196bda ~~MDEV-29069~~ follow-up: improve DEFAULT rules
cbe733b6b10 ~~MDEV-29069~~ follow-up: support partially usable keys
6eaf66f2d17 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
a379d5921c3 ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
26b11bb6f9c cleanup: cache the result of Rows_log_event::find_key()
ad19f3d708e set table->pos_in_table_list in online alter
79925e9b5f1 cleanup: remove rpl_group_info::get_table_data()
a8a67cdb8f4 cleanup: ifdefs
97a75c554d4 few rgi assertions. this can proof that rgi is always present
ec99d11cada rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
b2b030fd064 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
84f862c3f06 log_event.h: remove junk EOL spaces
71549936f83 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
eb9ac1b8518 Fix write_set too
4dfc48b9e0b ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
8e42878e550 rpl: check should go after defaults and vcols update
7331dfbac5c ~~MDEV-29067~~ Online alter ignores check constraint violation
cdaac40609d don't do ALTER IGNORE TABLE online
b89de5cf849 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
cfb0c7fe5d0 Do not ignore sql_mode when replicating
40eb328f0c2 Simplify rgi->get_table_data call
655d8764e68 reorder RPL_TABLE_LIST fields for better packing
0ef8bde91be ~~MDEV-29021~~ fixup
c37feb51ee1 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
7bf07c5d5c2 cleanup, remove dead code
229a4eaa2cf ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
0811905ad85 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
1c4e7ed3965 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
ced39a3cb1f remove redundant warnings in RBR and online alter
5004620d5cb cleanup: whitespace, etc
3618819bc0a ~~MDEV-28959~~ Online alter ignores strict table mode
17c0b8eba99 ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
c566c0c0f3f control Cache_flip_event_log lifetime with reference count
ee62df99aeb ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
517823bfbe1 test rename alter_table_online -> alter_table_online_debug
8dba420a547 separate online_alter_cache_data from binlog_cache_data
213cc14b65c put binlog_cache_data on a memroot
8718d110432 always commit for non-trans engines
55194c2277c savepoints
5064dc3bc83 don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
4198a9c04bc don't use start_consistent_snapshot
77bf7ac5ee6 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
a63684a1e92 don't do DROP SYSTEM VERSIONING online
307a4d33095 set read_set early, before row reads
2a18ae8e89b no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
512376f3e59 online alter always uses ALGORITHM=COPY, LOCK=NONE
c4ca1eadf54 remove handler::open_read_view()
9824910fb86 cleanup
87968ce5208 support 'alter online table t1 page_checksum=0'
84b03783860 tests: move around, add new
f7ff9b40118 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
20aae939c33 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
9ca93a9c574 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
ecf47bc2256 ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
a49e51b5db4 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
--- 0a7a45b42be rpl: repack table_def
--- 2e07d9c0def Copy_field: add const to arguments
--- 208f834397e rename tests
--- 1f9b60f2160 binlog_combinations.inc -> binlog_format_combinations.inc
--- eff7801152f cleanup: remove vcol_info->stored_in_db
--- efa160c3ad6 Fix recalculation of vcols in binlog_row_image=minimal
--- f1310b9d917 cleanup: clarify ha_innobase::column_bitmaps_signal()
--- 0d6ed7350be allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

----

h2. Other notes

Engines:
InnoDB, MyISAM, Aria, Heap, CSV, RocksDB, Archive, Mroonga -- applies
MERGE, Spider, Blackhole, Sphinx -- skips online alter (doesn't hit debug sync point), probably because there is no real data
Columnstore -- skips online alter (doesn't increase Binlog_bytes_written, cannot test debug (!), it fails there, to be checked whether it's specific to the branch)
S3, OQGRAPH, FEDERATED, FEDERATEDX -- ALTER is prohibited in general
PERFORMANCE_SCHEMA -- cannot be ALTERed
(!) Conversion to S3 -- fails, ~~MDEV-31781~~
(!) Sequences -- fails, ~~MDEV-31775~~ or maybe legacy ~~MDEV-31779~~
(!) Connect -- fails, ~~MDEV-31777~~ and a number of legacy (MDEV-30909, MDEV-30914, MDEV-30915, MDEV-30907)

Elena Stepanova made changes - 2023-07-28 19:00

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [3663784d|https://github.com/MariaDB/server/commit/3663784d3d1981fe6df52d94c2b8369d84a3cc38]
Baseline: [2867894a (11.2)|https://github.com/MariaDB/server/commit/2867894ac6ca23fc2f82cfad2dd510351f597325]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(?) +Buildbot+
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=bb-11.2-oalter&revision=3663784d3d1981fe6df52d94c2b8369d84a3cc38&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (?) SAN tests ~~MDEV-30945~~ (MSAN)
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
3663784d3d1 ~~MDEV-31781~~ ALTER TABLE ENGINE=s3 fails
5f34e5ff5ac ~~MDEV-31777~~ ER_GET_ERRNO upon online alter on CONNECT table
63bedb02468 ~~MDEV-31631~~ Adding auto-increment to table with history online misbehaves
46b19c42a32 ~~MDEV-31776~~ Online ALTER reports the number of affected rows incorrectly
cfd1f427283 Rows_log_event: reorganize the class layout for a lower memory footprint
23ae9f1b2c6 ~~MDEV-31775~~ Server crash upon online alter on sequence
fade8a79479 review fixes WIP
b0484157ecd add to binlog_bytes_written for an initial description event
9b5dba149cc make a proper cleanup if online_alter_binlog is failed to create
62e596aa6a2 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
a3caa265b7a fix key detection on replica with extra columns
842cb51e81f ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
a31b2a52119 ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
8c774b12114 ~~MDEV-31646~~ preserve DMLs in case of online binlog fault
ca64ddcc709 ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
f34c7419cb8 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
bcb49750118 fix -Werror build
8b0101a853b follow-up ~~MDEV-30430~~: fix versioning.rpl
ff775c71ac1 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
dc7136481bf ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
1a062cc0ee1 Add const to get_foreign_key_list/get_parent_foreign_key_list
f6db56cff63 Add const to alloc-related thd methods
188c3f767e1 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
d75e4ea59a2 ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
f3d0a06c22f refactor unpack_row
abd73373329 unpack_row: unpack a correct number of fields
949a4f5f86a ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
ae477a9f842 unpack_row: set the correct fields in has_value_set for online alter
542e8caba74 ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
8959ba16ac6 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
157e0ceea56 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
ed3c6a4d062 ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
dd80527782e clean up Rows_log_event virtual methods
5404f2937fe ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
07191fb46d0 add partition test
afb459edbe8 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
cb4d928678f ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
d1fe3487449 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
dc4e59852d8 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
4f67b03c4f7 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
955c700e90b fix main.alter_table_{online,lock}
f434a196bda ~~MDEV-29069~~ follow-up: improve DEFAULT rules
cbe733b6b10 ~~MDEV-29069~~ follow-up: support partially usable keys
6eaf66f2d17 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
a379d5921c3 ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
26b11bb6f9c cleanup: cache the result of Rows_log_event::find_key()
ad19f3d708e set table->pos_in_table_list in online alter
79925e9b5f1 cleanup: remove rpl_group_info::get_table_data()
a8a67cdb8f4 cleanup: ifdefs
97a75c554d4 few rgi assertions. this can proof that rgi is always present
ec99d11cada rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
b2b030fd064 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
84f862c3f06 log_event.h: remove junk EOL spaces
71549936f83 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
eb9ac1b8518 Fix write_set too
4dfc48b9e0b ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
8e42878e550 rpl: check should go after defaults and vcols update
7331dfbac5c ~~MDEV-29067~~ Online alter ignores check constraint violation
cdaac40609d don't do ALTER IGNORE TABLE online
b89de5cf849 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
cfb0c7fe5d0 Do not ignore sql_mode when replicating
40eb328f0c2 Simplify rgi->get_table_data call
655d8764e68 reorder RPL_TABLE_LIST fields for better packing
0ef8bde91be ~~MDEV-29021~~ fixup
c37feb51ee1 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
7bf07c5d5c2 cleanup, remove dead code
229a4eaa2cf ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
0811905ad85 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
1c4e7ed3965 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
ced39a3cb1f remove redundant warnings in RBR and online alter
5004620d5cb cleanup: whitespace, etc
3618819bc0a ~~MDEV-28959~~ Online alter ignores strict table mode
17c0b8eba99 ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
c566c0c0f3f control Cache_flip_event_log lifetime with reference count
ee62df99aeb ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
517823bfbe1 test rename alter_table_online -> alter_table_online_debug
8dba420a547 separate online_alter_cache_data from binlog_cache_data
213cc14b65c put binlog_cache_data on a memroot
8718d110432 always commit for non-trans engines
55194c2277c savepoints
5064dc3bc83 don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
4198a9c04bc don't use start_consistent_snapshot
77bf7ac5ee6 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
a63684a1e92 don't do DROP SYSTEM VERSIONING online
307a4d33095 set read_set early, before row reads
2a18ae8e89b no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
512376f3e59 online alter always uses ALGORITHM=COPY, LOCK=NONE
c4ca1eadf54 remove handler::open_read_view()
9824910fb86 cleanup
87968ce5208 support 'alter online table t1 page_checksum=0'
84b03783860 tests: move around, add new
f7ff9b40118 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
20aae939c33 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
9ca93a9c574 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
ecf47bc2256 ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
a49e51b5db4 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
--- 0a7a45b42be rpl: repack table_def
--- 2e07d9c0def Copy_field: add const to arguments
--- 208f834397e rename tests
--- 1f9b60f2160 binlog_combinations.inc -> binlog_format_combinations.inc
--- eff7801152f cleanup: remove vcol_info->stored_in_db
--- efa160c3ad6 Fix recalculation of vcols in binlog_row_image=minimal
--- f1310b9d917 cleanup: clarify ha_innobase::column_bitmaps_signal()
--- 0d6ed7350be allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

----

h2. Other notes

Engines:
InnoDB, MyISAM, Aria, Heap, CSV, RocksDB, Archive, Mroonga -- applies
MERGE, Spider, Blackhole, Sphinx -- skips online alter (doesn't hit debug sync point), probably because there is no real data
Columnstore -- skips online alter (doesn't increase Binlog_bytes_written, cannot test debug (!), it fails there, to be checked whether it's specific to the branch)
S3, OQGRAPH, FEDERATED, FEDERATEDX -- ALTER is prohibited in general
PERFORMANCE_SCHEMA -- cannot be ALTERed
(!) Conversion to S3 -- fails, ~~MDEV-31781~~
(!) Sequences -- fails, ~~MDEV-31775~~ or maybe legacy ~~MDEV-31779~~
(!) Connect -- fails, ~~MDEV-31777~~ and a number of legacy (MDEV-30909, MDEV-30914, MDEV-30915, MDEV-30907)

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [3663784d|https://github.com/MariaDB/server/commit/3663784d3d1981fe6df52d94c2b8369d84a3cc38]
Baseline: [2867894a (11.2)|https://github.com/MariaDB/server/commit/2867894ac6ca23fc2f82cfad2dd510351f597325]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(?) +Buildbot+
- (x) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (x) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=bb-11.2-oalter&revision=3663784d3d1981fe6df52d94c2b8369d84a3cc38&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (?) SAN tests ~~MDEV-30945~~ (MSAN)
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
3663784d3d1 ~~MDEV-31781~~ ALTER TABLE ENGINE=s3 fails
5f34e5ff5ac ~~MDEV-31777~~ ER_GET_ERRNO upon online alter on CONNECT table
63bedb02468 ~~MDEV-31631~~ Adding auto-increment to table with history online misbehaves
46b19c42a32 ~~MDEV-31776~~ Online ALTER reports the number of affected rows incorrectly
cfd1f427283 Rows_log_event: reorganize the class layout for a lower memory footprint
23ae9f1b2c6 ~~MDEV-31775~~ Server crash upon online alter on sequence
fade8a79479 review fixes WIP
b0484157ecd add to binlog_bytes_written for an initial description event
9b5dba149cc make a proper cleanup if online_alter_binlog is failed to create
62e596aa6a2 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
a3caa265b7a fix key detection on replica with extra columns
842cb51e81f ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
a31b2a52119 ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
8c774b12114 ~~MDEV-31646~~ preserve DMLs in case of online binlog fault
ca64ddcc709 ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
f34c7419cb8 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
bcb49750118 fix -Werror build
8b0101a853b follow-up ~~MDEV-30430~~: fix versioning.rpl
ff775c71ac1 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
dc7136481bf ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
1a062cc0ee1 Add const to get_foreign_key_list/get_parent_foreign_key_list
f6db56cff63 Add const to alloc-related thd methods
188c3f767e1 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
d75e4ea59a2 ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
f3d0a06c22f refactor unpack_row
abd73373329 unpack_row: unpack a correct number of fields
949a4f5f86a ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
ae477a9f842 unpack_row: set the correct fields in has_value_set for online alter
542e8caba74 ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
8959ba16ac6 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
157e0ceea56 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
ed3c6a4d062 ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
dd80527782e clean up Rows_log_event virtual methods
5404f2937fe ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
07191fb46d0 add partition test
afb459edbe8 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
cb4d928678f ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
d1fe3487449 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
dc4e59852d8 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
4f67b03c4f7 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
955c700e90b fix main.alter_table_{online,lock}
f434a196bda ~~MDEV-29069~~ follow-up: improve DEFAULT rules
cbe733b6b10 ~~MDEV-29069~~ follow-up: support partially usable keys
6eaf66f2d17 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
a379d5921c3 ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
26b11bb6f9c cleanup: cache the result of Rows_log_event::find_key()
ad19f3d708e set table->pos_in_table_list in online alter
79925e9b5f1 cleanup: remove rpl_group_info::get_table_data()
a8a67cdb8f4 cleanup: ifdefs
97a75c554d4 few rgi assertions. this can proof that rgi is always present
ec99d11cada rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
b2b030fd064 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
84f862c3f06 log_event.h: remove junk EOL spaces
71549936f83 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
eb9ac1b8518 Fix write_set too
4dfc48b9e0b ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
8e42878e550 rpl: check should go after defaults and vcols update
7331dfbac5c ~~MDEV-29067~~ Online alter ignores check constraint violation
cdaac40609d don't do ALTER IGNORE TABLE online
b89de5cf849 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
cfb0c7fe5d0 Do not ignore sql_mode when replicating
40eb328f0c2 Simplify rgi->get_table_data call
655d8764e68 reorder RPL_TABLE_LIST fields for better packing
0ef8bde91be ~~MDEV-29021~~ fixup
c37feb51ee1 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
7bf07c5d5c2 cleanup, remove dead code
229a4eaa2cf ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
0811905ad85 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
1c4e7ed3965 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
ced39a3cb1f remove redundant warnings in RBR and online alter
5004620d5cb cleanup: whitespace, etc
3618819bc0a ~~MDEV-28959~~ Online alter ignores strict table mode
17c0b8eba99 ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
c566c0c0f3f control Cache_flip_event_log lifetime with reference count
ee62df99aeb ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
517823bfbe1 test rename alter_table_online -> alter_table_online_debug
8dba420a547 separate online_alter_cache_data from binlog_cache_data
213cc14b65c put binlog_cache_data on a memroot
8718d110432 always commit for non-trans engines
55194c2277c savepoints
5064dc3bc83 don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
4198a9c04bc don't use start_consistent_snapshot
77bf7ac5ee6 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
a63684a1e92 don't do DROP SYSTEM VERSIONING online
307a4d33095 set read_set early, before row reads
2a18ae8e89b no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
512376f3e59 online alter always uses ALGORITHM=COPY, LOCK=NONE
c4ca1eadf54 remove handler::open_read_view()
9824910fb86 cleanup
87968ce5208 support 'alter online table t1 page_checksum=0'
84b03783860 tests: move around, add new
f7ff9b40118 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
20aae939c33 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
9ca93a9c574 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
ecf47bc2256 ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
a49e51b5db4 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
--- 0a7a45b42be rpl: repack table_def
--- 2e07d9c0def Copy_field: add const to arguments
--- 208f834397e rename tests
--- 1f9b60f2160 binlog_combinations.inc -> binlog_format_combinations.inc
--- eff7801152f cleanup: remove vcol_info->stored_in_db
--- efa160c3ad6 Fix recalculation of vcols in binlog_row_image=minimal
--- f1310b9d917 cleanup: clarify ha_innobase::column_bitmaps_signal()
--- 0d6ed7350be allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

----

h2. Other notes

Engines:
InnoDB, MyISAM, Aria, Heap, CSV, RocksDB, Archive, Mroonga -- applies
MERGE, Spider, Blackhole, Sphinx -- skips online alter (doesn't hit debug sync point), probably because there is no real data
Columnstore -- skips online alter (doesn't increase Binlog_bytes_written, cannot test debug (!), it fails there, to be checked whether it's specific to the branch)
S3, OQGRAPH, FEDERATED, FEDERATEDX -- ALTER is prohibited in general
PERFORMANCE_SCHEMA -- cannot be ALTERed
(!) Conversion to S3 -- fails, ~~MDEV-31781~~
(!) Sequences -- fails, ~~MDEV-31775~~ or maybe legacy ~~MDEV-31779~~
(!) Connect -- fails, ~~MDEV-31777~~ and a number of legacy (MDEV-30909, MDEV-30914, MDEV-30915, MDEV-30907)

Elena Stepanova made changes - 2023-07-28 18:49

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [3663784d|https://github.com/MariaDB/server/commit/3663784d3d1981fe6df52d94c2b8369d84a3cc38]
Baseline: [2867894a (11.2)|https://github.com/MariaDB/server/commit/2867894ac6ca23fc2f82cfad2dd510351f597325]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(?) +Buildbot
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=bb-11.2-oalter&revision=3663784d3d1981fe6df52d94c2b8369d84a3cc38&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (?) SAN tests ~~MDEV-30945~~ (MSAN)
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
3663784d3d1 ~~MDEV-31781~~ ALTER TABLE ENGINE=s3 fails
5f34e5ff5ac ~~MDEV-31777~~ ER_GET_ERRNO upon online alter on CONNECT table
63bedb02468 ~~MDEV-31631~~ Adding auto-increment to table with history online misbehaves
46b19c42a32 ~~MDEV-31776~~ Online ALTER reports the number of affected rows incorrectly
cfd1f427283 Rows_log_event: reorganize the class layout for a lower memory footprint
23ae9f1b2c6 ~~MDEV-31775~~ Server crash upon online alter on sequence
fade8a79479 review fixes WIP
b0484157ecd add to binlog_bytes_written for an initial description event
9b5dba149cc make a proper cleanup if online_alter_binlog is failed to create
62e596aa6a2 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
a3caa265b7a fix key detection on replica with extra columns
842cb51e81f ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
a31b2a52119 ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
8c774b12114 ~~MDEV-31646~~ preserve DMLs in case of online binlog fault
ca64ddcc709 ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
f34c7419cb8 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
bcb49750118 fix -Werror build
8b0101a853b follow-up ~~MDEV-30430~~: fix versioning.rpl
ff775c71ac1 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
dc7136481bf ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
1a062cc0ee1 Add const to get_foreign_key_list/get_parent_foreign_key_list
f6db56cff63 Add const to alloc-related thd methods
188c3f767e1 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
d75e4ea59a2 ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
f3d0a06c22f refactor unpack_row
abd73373329 unpack_row: unpack a correct number of fields
949a4f5f86a ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
ae477a9f842 unpack_row: set the correct fields in has_value_set for online alter
542e8caba74 ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
8959ba16ac6 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
157e0ceea56 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
ed3c6a4d062 ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
dd80527782e clean up Rows_log_event virtual methods
5404f2937fe ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
07191fb46d0 add partition test
afb459edbe8 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
cb4d928678f ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
d1fe3487449 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
dc4e59852d8 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
4f67b03c4f7 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
955c700e90b fix main.alter_table_{online,lock}
f434a196bda ~~MDEV-29069~~ follow-up: improve DEFAULT rules
cbe733b6b10 ~~MDEV-29069~~ follow-up: support partially usable keys
6eaf66f2d17 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
a379d5921c3 ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
26b11bb6f9c cleanup: cache the result of Rows_log_event::find_key()
ad19f3d708e set table->pos_in_table_list in online alter
79925e9b5f1 cleanup: remove rpl_group_info::get_table_data()
a8a67cdb8f4 cleanup: ifdefs
97a75c554d4 few rgi assertions. this can proof that rgi is always present
ec99d11cada rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
b2b030fd064 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
84f862c3f06 log_event.h: remove junk EOL spaces
71549936f83 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
eb9ac1b8518 Fix write_set too
4dfc48b9e0b ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
8e42878e550 rpl: check should go after defaults and vcols update
7331dfbac5c ~~MDEV-29067~~ Online alter ignores check constraint violation
cdaac40609d don't do ALTER IGNORE TABLE online
b89de5cf849 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
cfb0c7fe5d0 Do not ignore sql_mode when replicating
40eb328f0c2 Simplify rgi->get_table_data call
655d8764e68 reorder RPL_TABLE_LIST fields for better packing
0ef8bde91be ~~MDEV-29021~~ fixup
c37feb51ee1 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
7bf07c5d5c2 cleanup, remove dead code
229a4eaa2cf ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
0811905ad85 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
1c4e7ed3965 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
ced39a3cb1f remove redundant warnings in RBR and online alter
5004620d5cb cleanup: whitespace, etc
3618819bc0a ~~MDEV-28959~~ Online alter ignores strict table mode
17c0b8eba99 ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
c566c0c0f3f control Cache_flip_event_log lifetime with reference count
ee62df99aeb ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
517823bfbe1 test rename alter_table_online -> alter_table_online_debug
8dba420a547 separate online_alter_cache_data from binlog_cache_data
213cc14b65c put binlog_cache_data on a memroot
8718d110432 always commit for non-trans engines
55194c2277c savepoints
5064dc3bc83 don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
4198a9c04bc don't use start_consistent_snapshot
77bf7ac5ee6 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
a63684a1e92 don't do DROP SYSTEM VERSIONING online
307a4d33095 set read_set early, before row reads
2a18ae8e89b no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
512376f3e59 online alter always uses ALGORITHM=COPY, LOCK=NONE
c4ca1eadf54 remove handler::open_read_view()
9824910fb86 cleanup
87968ce5208 support 'alter online table t1 page_checksum=0'
84b03783860 tests: move around, add new
f7ff9b40118 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
20aae939c33 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
9ca93a9c574 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
ecf47bc2256 ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
a49e51b5db4 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
--- 0a7a45b42be rpl: repack table_def
--- 2e07d9c0def Copy_field: add const to arguments
--- 208f834397e rename tests
--- 1f9b60f2160 binlog_combinations.inc -> binlog_format_combinations.inc
--- eff7801152f cleanup: remove vcol_info->stored_in_db
--- efa160c3ad6 Fix recalculation of vcols in binlog_row_image=minimal
--- f1310b9d917 cleanup: clarify ha_innobase::column_bitmaps_signal()
--- 0d6ed7350be allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

----

h2. Other notes

Engines:
InnoDB, MyISAM, Aria, Heap, CSV, RocksDB, Archive, Mroonga -- applies
MERGE, Spider, Blackhole, Sphinx -- skips online alter (doesn't hit debug sync point), probably because there is no real data
Columnstore -- skips online alter (doesn't increase Binlog_bytes_written, cannot test debug (!), it fails there, to be checked whether it's specific to the branch)
S3, OQGRAPH, FEDERATED, FEDERATEDX -- ALTER is prohibited in general
PERFORMANCE_SCHEMA -- cannot be ALTERed
(!) Conversion to S3 -- fails, ~~MDEV-31781~~
(!) Sequences -- fails, ~~MDEV-31775~~ or maybe legacy ~~MDEV-31779~~
(!) Connect -- fails, ~~MDEV-31777~~ and a number of legacy (MDEV-30909, MDEV-30914, MDEV-30915, MDEV-30907)

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [3663784d|https://github.com/MariaDB/server/commit/3663784d3d1981fe6df52d94c2b8369d84a3cc38]
Baseline: [2867894a (11.2)|https://github.com/MariaDB/server/commit/2867894ac6ca23fc2f82cfad2dd510351f597325]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(?) +Buildbot+
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=bb-11.2-oalter&revision=3663784d3d1981fe6df52d94c2b8369d84a3cc38&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (?) SAN tests ~~MDEV-30945~~ (MSAN)
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
3663784d3d1 ~~MDEV-31781~~ ALTER TABLE ENGINE=s3 fails
5f34e5ff5ac ~~MDEV-31777~~ ER_GET_ERRNO upon online alter on CONNECT table
63bedb02468 ~~MDEV-31631~~ Adding auto-increment to table with history online misbehaves
46b19c42a32 ~~MDEV-31776~~ Online ALTER reports the number of affected rows incorrectly
cfd1f427283 Rows_log_event: reorganize the class layout for a lower memory footprint
23ae9f1b2c6 ~~MDEV-31775~~ Server crash upon online alter on sequence
fade8a79479 review fixes WIP
b0484157ecd add to binlog_bytes_written for an initial description event
9b5dba149cc make a proper cleanup if online_alter_binlog is failed to create
62e596aa6a2 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
a3caa265b7a fix key detection on replica with extra columns
842cb51e81f ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
a31b2a52119 ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
8c774b12114 ~~MDEV-31646~~ preserve DMLs in case of online binlog fault
ca64ddcc709 ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
f34c7419cb8 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
bcb49750118 fix -Werror build
8b0101a853b follow-up ~~MDEV-30430~~: fix versioning.rpl
ff775c71ac1 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
dc7136481bf ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
1a062cc0ee1 Add const to get_foreign_key_list/get_parent_foreign_key_list
f6db56cff63 Add const to alloc-related thd methods
188c3f767e1 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
d75e4ea59a2 ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
f3d0a06c22f refactor unpack_row
abd73373329 unpack_row: unpack a correct number of fields
949a4f5f86a ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
ae477a9f842 unpack_row: set the correct fields in has_value_set for online alter
542e8caba74 ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
8959ba16ac6 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
157e0ceea56 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
ed3c6a4d062 ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
dd80527782e clean up Rows_log_event virtual methods
5404f2937fe ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
07191fb46d0 add partition test
afb459edbe8 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
cb4d928678f ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
d1fe3487449 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
dc4e59852d8 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
4f67b03c4f7 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
955c700e90b fix main.alter_table_{online,lock}
f434a196bda ~~MDEV-29069~~ follow-up: improve DEFAULT rules
cbe733b6b10 ~~MDEV-29069~~ follow-up: support partially usable keys
6eaf66f2d17 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
a379d5921c3 ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
26b11bb6f9c cleanup: cache the result of Rows_log_event::find_key()
ad19f3d708e set table->pos_in_table_list in online alter
79925e9b5f1 cleanup: remove rpl_group_info::get_table_data()
a8a67cdb8f4 cleanup: ifdefs
97a75c554d4 few rgi assertions. this can proof that rgi is always present
ec99d11cada rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
b2b030fd064 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
84f862c3f06 log_event.h: remove junk EOL spaces
71549936f83 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
eb9ac1b8518 Fix write_set too
4dfc48b9e0b ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
8e42878e550 rpl: check should go after defaults and vcols update
7331dfbac5c ~~MDEV-29067~~ Online alter ignores check constraint violation
cdaac40609d don't do ALTER IGNORE TABLE online
b89de5cf849 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
cfb0c7fe5d0 Do not ignore sql_mode when replicating
40eb328f0c2 Simplify rgi->get_table_data call
655d8764e68 reorder RPL_TABLE_LIST fields for better packing
0ef8bde91be ~~MDEV-29021~~ fixup
c37feb51ee1 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
7bf07c5d5c2 cleanup, remove dead code
229a4eaa2cf ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
0811905ad85 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
1c4e7ed3965 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
ced39a3cb1f remove redundant warnings in RBR and online alter
5004620d5cb cleanup: whitespace, etc
3618819bc0a ~~MDEV-28959~~ Online alter ignores strict table mode
17c0b8eba99 ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
c566c0c0f3f control Cache_flip_event_log lifetime with reference count
ee62df99aeb ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
517823bfbe1 test rename alter_table_online -> alter_table_online_debug
8dba420a547 separate online_alter_cache_data from binlog_cache_data
213cc14b65c put binlog_cache_data on a memroot
8718d110432 always commit for non-trans engines
55194c2277c savepoints
5064dc3bc83 don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
4198a9c04bc don't use start_consistent_snapshot
77bf7ac5ee6 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
a63684a1e92 don't do DROP SYSTEM VERSIONING online
307a4d33095 set read_set early, before row reads
2a18ae8e89b no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
512376f3e59 online alter always uses ALGORITHM=COPY, LOCK=NONE
c4ca1eadf54 remove handler::open_read_view()
9824910fb86 cleanup
87968ce5208 support 'alter online table t1 page_checksum=0'
84b03783860 tests: move around, add new
f7ff9b40118 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
20aae939c33 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
9ca93a9c574 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
ecf47bc2256 ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
a49e51b5db4 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
--- 0a7a45b42be rpl: repack table_def
--- 2e07d9c0def Copy_field: add const to arguments
--- 208f834397e rename tests
--- 1f9b60f2160 binlog_combinations.inc -> binlog_format_combinations.inc
--- eff7801152f cleanup: remove vcol_info->stored_in_db
--- efa160c3ad6 Fix recalculation of vcols in binlog_row_image=minimal
--- f1310b9d917 cleanup: clarify ha_innobase::column_bitmaps_signal()
--- 0d6ed7350be allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

----

h2. Other notes

Engines:
InnoDB, MyISAM, Aria, Heap, CSV, RocksDB, Archive, Mroonga -- applies
MERGE, Spider, Blackhole, Sphinx -- skips online alter (doesn't hit debug sync point), probably because there is no real data
Columnstore -- skips online alter (doesn't increase Binlog_bytes_written, cannot test debug (!), it fails there, to be checked whether it's specific to the branch)
S3, OQGRAPH, FEDERATED, FEDERATEDX -- ALTER is prohibited in general
PERFORMANCE_SCHEMA -- cannot be ALTERed
(!) Conversion to S3 -- fails, ~~MDEV-31781~~
(!) Sequences -- fails, ~~MDEV-31775~~ or maybe legacy ~~MDEV-31779~~
(!) Connect -- fails, ~~MDEV-31777~~ and a number of legacy (MDEV-30909, MDEV-30914, MDEV-30915, MDEV-30907)

Elena Stepanova made changes - 2023-07-28 18:37

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [62e596aa6|https://github.com/MariaDB/server/commit/62e596aa6a279bafbecc91d82d990f540adcc31f]
Baseline: [2867894a (11.2)|https://github.com/MariaDB/server/commit/2867894ac6ca23fc2f82cfad2dd510351f597325]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(?) +Buildbot (62e596aa6)
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=62e596aa6a279bafbecc91d82d990f540adcc31f&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR (62e596aa6)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (?) SAN tests ~~MDEV-30945~~ (MSAN)
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
62e596aa6a2 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
a3caa265b7a fix key detection on replica with extra columns
842cb51e81f ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
a31b2a52119 ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
8c774b12114 ~~MDEV-31646~~ preserve DMLs in case of online binlog fault
ca64ddcc709 ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
f34c7419cb8 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
bcb49750118 fix -Werror build
8b0101a853b follow-up ~~MDEV-30430~~: fix versioning.rpl
ff775c71ac1 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
dc7136481bf ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
1a062cc0ee1 Add const to get_foreign_key_list/get_parent_foreign_key_list
f6db56cff63 Add const to alloc-related thd methods
188c3f767e1 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
d75e4ea59a2 ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
f3d0a06c22f refactor unpack_row
abd73373329 unpack_row: unpack a correct number of fields
949a4f5f86a ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
ae477a9f842 unpack_row: set the correct fields in has_value_set for online alter
542e8caba74 ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
8959ba16ac6 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
157e0ceea56 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
ed3c6a4d062 ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
dd80527782e clean up Rows_log_event virtual methods
5404f2937fe ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
07191fb46d0 add partition test
afb459edbe8 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
cb4d928678f ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
d1fe3487449 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
dc4e59852d8 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
4f67b03c4f7 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
955c700e90b fix main.alter_table_{online,lock}
f434a196bda ~~MDEV-29069~~ follow-up: improve DEFAULT rules
cbe733b6b10 ~~MDEV-29069~~ follow-up: support partially usable keys
6eaf66f2d17 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
a379d5921c3 ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
26b11bb6f9c cleanup: cache the result of Rows_log_event::find_key()
ad19f3d708e set table->pos_in_table_list in online alter
79925e9b5f1 cleanup: remove rpl_group_info::get_table_data()
a8a67cdb8f4 cleanup: ifdefs
97a75c554d4 few rgi assertions. this can proof that rgi is always present
ec99d11cada rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
b2b030fd064 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
84f862c3f06 log_event.h: remove junk EOL spaces
71549936f83 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
eb9ac1b8518 Fix write_set too
4dfc48b9e0b ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
8e42878e550 rpl: check should go after defaults and vcols update
7331dfbac5c ~~MDEV-29067~~ Online alter ignores check constraint violation
cdaac40609d don't do ALTER IGNORE TABLE online
b89de5cf849 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
cfb0c7fe5d0 Do not ignore sql_mode when replicating
40eb328f0c2 Simplify rgi->get_table_data call
655d8764e68 reorder RPL_TABLE_LIST fields for better packing
0ef8bde91be ~~MDEV-29021~~ fixup
c37feb51ee1 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
7bf07c5d5c2 cleanup, remove dead code
229a4eaa2cf ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
0811905ad85 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
1c4e7ed3965 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
ced39a3cb1f remove redundant warnings in RBR and online alter
5004620d5cb cleanup: whitespace, etc
3618819bc0a ~~MDEV-28959~~ Online alter ignores strict table mode
17c0b8eba99 ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
c566c0c0f3f control Cache_flip_event_log lifetime with reference count
ee62df99aeb ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
517823bfbe1 test rename alter_table_online -> alter_table_online_debug
8dba420a547 separate online_alter_cache_data from binlog_cache_data
213cc14b65c put binlog_cache_data on a memroot
8718d110432 always commit for non-trans engines
55194c2277c savepoints
5064dc3bc83 don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
4198a9c04bc don't use start_consistent_snapshot
77bf7ac5ee6 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
a63684a1e92 don't do DROP SYSTEM VERSIONING online
307a4d33095 set read_set early, before row reads
2a18ae8e89b no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
512376f3e59 online alter always uses ALGORITHM=COPY, LOCK=NONE
c4ca1eadf54 remove handler::open_read_view()
9824910fb86 cleanup
87968ce5208 support 'alter online table t1 page_checksum=0'
84b03783860 tests: move around, add new
f7ff9b40118 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
20aae939c33 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
9ca93a9c574 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
ecf47bc2256 ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
a49e51b5db4 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
0a7a45b42be rpl: repack table_def
2e07d9c0def Copy_field: add const to arguments
208f834397e rename tests
1f9b60f2160 binlog_combinations.inc -> binlog_format_combinations.inc
eff7801152f cleanup: remove vcol_info->stored_in_db
efa160c3ad6 Fix recalculation of vcols in binlog_row_image=minimal
f1310b9d917 cleanup: clarify ha_innobase::column_bitmaps_signal()
0d6ed7350be allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

----

h2. Other notes

Engines:
InnoDB, MyISAM, Aria, Heap, CSV, RocksDB, Archive, Mroonga -- applies
MERGE, Spider, Blackhole, Sphinx -- skips online alter (doesn't hit debug sync point), probably because there is no real data
Columnstore -- skips online alter (doesn't increase Binlog_bytes_written, cannot test debug (!), it fails there, to be checked whether it's specific to the branch)
S3, OQGRAPH, FEDERATED, FEDERATEDX -- ALTER is prohibited in general
PERFORMANCE_SCHEMA -- cannot be ALTERed
(!) Conversion to S3 -- fails, ~~MDEV-31781~~
(!) Sequences -- fails, ~~MDEV-31775~~ or maybe legacy ~~MDEV-31779~~
(!) Connect -- fails, ~~MDEV-31777~~ and a number of legacy (MDEV-30909, MDEV-30914, MDEV-30915, MDEV-30907)

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [3663784d|https://github.com/MariaDB/server/commit/3663784d3d1981fe6df52d94c2b8369d84a3cc38]
Baseline: [2867894a (11.2)|https://github.com/MariaDB/server/commit/2867894ac6ca23fc2f82cfad2dd510351f597325]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(?) +Buildbot
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=bb-11.2-oalter&revision=3663784d3d1981fe6df52d94c2b8369d84a3cc38&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (?) SAN tests ~~MDEV-30945~~ (MSAN)
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
3663784d3d1 ~~MDEV-31781~~ ALTER TABLE ENGINE=s3 fails
5f34e5ff5ac ~~MDEV-31777~~ ER_GET_ERRNO upon online alter on CONNECT table
63bedb02468 ~~MDEV-31631~~ Adding auto-increment to table with history online misbehaves
46b19c42a32 ~~MDEV-31776~~ Online ALTER reports the number of affected rows incorrectly
cfd1f427283 Rows_log_event: reorganize the class layout for a lower memory footprint
23ae9f1b2c6 ~~MDEV-31775~~ Server crash upon online alter on sequence
fade8a79479 review fixes WIP
b0484157ecd add to binlog_bytes_written for an initial description event
9b5dba149cc make a proper cleanup if online_alter_binlog is failed to create
62e596aa6a2 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
a3caa265b7a fix key detection on replica with extra columns
842cb51e81f ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
a31b2a52119 ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
8c774b12114 ~~MDEV-31646~~ preserve DMLs in case of online binlog fault
ca64ddcc709 ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
f34c7419cb8 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
bcb49750118 fix -Werror build
8b0101a853b follow-up ~~MDEV-30430~~: fix versioning.rpl
ff775c71ac1 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
dc7136481bf ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
1a062cc0ee1 Add const to get_foreign_key_list/get_parent_foreign_key_list
f6db56cff63 Add const to alloc-related thd methods
188c3f767e1 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
d75e4ea59a2 ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
f3d0a06c22f refactor unpack_row
abd73373329 unpack_row: unpack a correct number of fields
949a4f5f86a ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
ae477a9f842 unpack_row: set the correct fields in has_value_set for online alter
542e8caba74 ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
8959ba16ac6 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
157e0ceea56 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
ed3c6a4d062 ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
dd80527782e clean up Rows_log_event virtual methods
5404f2937fe ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
07191fb46d0 add partition test
afb459edbe8 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
cb4d928678f ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
d1fe3487449 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
dc4e59852d8 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
4f67b03c4f7 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
955c700e90b fix main.alter_table_{online,lock}
f434a196bda ~~MDEV-29069~~ follow-up: improve DEFAULT rules
cbe733b6b10 ~~MDEV-29069~~ follow-up: support partially usable keys
6eaf66f2d17 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
a379d5921c3 ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
26b11bb6f9c cleanup: cache the result of Rows_log_event::find_key()
ad19f3d708e set table->pos_in_table_list in online alter
79925e9b5f1 cleanup: remove rpl_group_info::get_table_data()
a8a67cdb8f4 cleanup: ifdefs
97a75c554d4 few rgi assertions. this can proof that rgi is always present
ec99d11cada rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
b2b030fd064 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
84f862c3f06 log_event.h: remove junk EOL spaces
71549936f83 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
eb9ac1b8518 Fix write_set too
4dfc48b9e0b ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
8e42878e550 rpl: check should go after defaults and vcols update
7331dfbac5c ~~MDEV-29067~~ Online alter ignores check constraint violation
cdaac40609d don't do ALTER IGNORE TABLE online
b89de5cf849 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
cfb0c7fe5d0 Do not ignore sql_mode when replicating
40eb328f0c2 Simplify rgi->get_table_data call
655d8764e68 reorder RPL_TABLE_LIST fields for better packing
0ef8bde91be ~~MDEV-29021~~ fixup
c37feb51ee1 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
7bf07c5d5c2 cleanup, remove dead code
229a4eaa2cf ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
0811905ad85 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
1c4e7ed3965 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
ced39a3cb1f remove redundant warnings in RBR and online alter
5004620d5cb cleanup: whitespace, etc
3618819bc0a ~~MDEV-28959~~ Online alter ignores strict table mode
17c0b8eba99 ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
c566c0c0f3f control Cache_flip_event_log lifetime with reference count
ee62df99aeb ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
517823bfbe1 test rename alter_table_online -> alter_table_online_debug
8dba420a547 separate online_alter_cache_data from binlog_cache_data
213cc14b65c put binlog_cache_data on a memroot
8718d110432 always commit for non-trans engines
55194c2277c savepoints
5064dc3bc83 don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
4198a9c04bc don't use start_consistent_snapshot
77bf7ac5ee6 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
a63684a1e92 don't do DROP SYSTEM VERSIONING online
307a4d33095 set read_set early, before row reads
2a18ae8e89b no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
512376f3e59 online alter always uses ALGORITHM=COPY, LOCK=NONE
c4ca1eadf54 remove handler::open_read_view()
9824910fb86 cleanup
87968ce5208 support 'alter online table t1 page_checksum=0'
84b03783860 tests: move around, add new
f7ff9b40118 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
20aae939c33 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
9ca93a9c574 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
ecf47bc2256 ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
a49e51b5db4 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
--- 0a7a45b42be rpl: repack table_def
--- 2e07d9c0def Copy_field: add const to arguments
--- 208f834397e rename tests
--- 1f9b60f2160 binlog_combinations.inc -> binlog_format_combinations.inc
--- eff7801152f cleanup: remove vcol_info->stored_in_db
--- efa160c3ad6 Fix recalculation of vcols in binlog_row_image=minimal
--- f1310b9d917 cleanup: clarify ha_innobase::column_bitmaps_signal()
--- 0d6ed7350be allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

----

h2. Other notes

Engines:
InnoDB, MyISAM, Aria, Heap, CSV, RocksDB, Archive, Mroonga -- applies
MERGE, Spider, Blackhole, Sphinx -- skips online alter (doesn't hit debug sync point), probably because there is no real data
Columnstore -- skips online alter (doesn't increase Binlog_bytes_written, cannot test debug (!), it fails there, to be checked whether it's specific to the branch)
S3, OQGRAPH, FEDERATED, FEDERATEDX -- ALTER is prohibited in general
PERFORMANCE_SCHEMA -- cannot be ALTERed
(!) Conversion to S3 -- fails, ~~MDEV-31781~~
(!) Sequences -- fails, ~~MDEV-31775~~ or maybe legacy ~~MDEV-31779~~
(!) Connect -- fails, ~~MDEV-31777~~ and a number of legacy (MDEV-30909, MDEV-30914, MDEV-30915, MDEV-30907)

Elena Stepanova made changes - 2023-07-26 23:07

Comment

[ New branch for testing: bb-10.10-~~MDEV-16329~~ ]

Elena Stepanova made changes - 2023-07-26 23:06

Comment

[ Currently the branch to test is preview-10.10-~~MDEV-16329~~-online-alter ]

Elena Stepanova made changes - 2023-07-26 23:06

Comment

[ The above is outdated, now branch information is updated directly in the description. ]

Elena Stepanova made changes - 2023-07-26 23:06

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [62e596aa6|https://github.com/MariaDB/server/commit/62e596aa6a279bafbecc91d82d990f540adcc31f]
Baseline: [2867894a (11.2)|https://github.com/MariaDB/server/commit/2867894ac6ca23fc2f82cfad2dd510351f597325]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(?) +Buildbot (62e596aa6)
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=62e596aa6a279bafbecc91d82d990f540adcc31f&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR (62e596aa6)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (?) SAN tests ~~MDEV-30945~~ (MSAN)
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
62e596aa6a2 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
a3caa265b7a fix key detection on replica with extra columns
842cb51e81f ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
a31b2a52119 ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
8c774b12114 ~~MDEV-31646~~ preserve DMLs in case of online binlog fault
ca64ddcc709 ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
f34c7419cb8 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
bcb49750118 fix -Werror build
8b0101a853b follow-up ~~MDEV-30430~~: fix versioning.rpl
ff775c71ac1 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
dc7136481bf ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
1a062cc0ee1 Add const to get_foreign_key_list/get_parent_foreign_key_list
f6db56cff63 Add const to alloc-related thd methods
188c3f767e1 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
d75e4ea59a2 ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
f3d0a06c22f refactor unpack_row
abd73373329 unpack_row: unpack a correct number of fields
949a4f5f86a ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
ae477a9f842 unpack_row: set the correct fields in has_value_set for online alter
542e8caba74 ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
8959ba16ac6 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
157e0ceea56 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
ed3c6a4d062 ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
dd80527782e clean up Rows_log_event virtual methods
5404f2937fe ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
07191fb46d0 add partition test
afb459edbe8 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
cb4d928678f ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
d1fe3487449 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
dc4e59852d8 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
4f67b03c4f7 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
955c700e90b fix main.alter_table_{online,lock}
f434a196bda ~~MDEV-29069~~ follow-up: improve DEFAULT rules
cbe733b6b10 ~~MDEV-29069~~ follow-up: support partially usable keys
6eaf66f2d17 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
a379d5921c3 ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
26b11bb6f9c cleanup: cache the result of Rows_log_event::find_key()
ad19f3d708e set table->pos_in_table_list in online alter
79925e9b5f1 cleanup: remove rpl_group_info::get_table_data()
a8a67cdb8f4 cleanup: ifdefs
97a75c554d4 few rgi assertions. this can proof that rgi is always present
ec99d11cada rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
b2b030fd064 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
84f862c3f06 log_event.h: remove junk EOL spaces
71549936f83 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
eb9ac1b8518 Fix write_set too
4dfc48b9e0b ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
8e42878e550 rpl: check should go after defaults and vcols update
7331dfbac5c ~~MDEV-29067~~ Online alter ignores check constraint violation
cdaac40609d don't do ALTER IGNORE TABLE online
b89de5cf849 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
cfb0c7fe5d0 Do not ignore sql_mode when replicating
40eb328f0c2 Simplify rgi->get_table_data call
655d8764e68 reorder RPL_TABLE_LIST fields for better packing
0ef8bde91be ~~MDEV-29021~~ fixup
c37feb51ee1 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
7bf07c5d5c2 cleanup, remove dead code
229a4eaa2cf ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
0811905ad85 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
1c4e7ed3965 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
ced39a3cb1f remove redundant warnings in RBR and online alter
5004620d5cb cleanup: whitespace, etc
3618819bc0a ~~MDEV-28959~~ Online alter ignores strict table mode
17c0b8eba99 ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
c566c0c0f3f control Cache_flip_event_log lifetime with reference count
ee62df99aeb ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
517823bfbe1 test rename alter_table_online -> alter_table_online_debug
8dba420a547 separate online_alter_cache_data from binlog_cache_data
213cc14b65c put binlog_cache_data on a memroot
8718d110432 always commit for non-trans engines
55194c2277c savepoints
5064dc3bc83 don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
4198a9c04bc don't use start_consistent_snapshot
77bf7ac5ee6 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
a63684a1e92 don't do DROP SYSTEM VERSIONING online
307a4d33095 set read_set early, before row reads
2a18ae8e89b no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
512376f3e59 online alter always uses ALGORITHM=COPY, LOCK=NONE
c4ca1eadf54 remove handler::open_read_view()
9824910fb86 cleanup
87968ce5208 support 'alter online table t1 page_checksum=0'
84b03783860 tests: move around, add new
f7ff9b40118 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
20aae939c33 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
9ca93a9c574 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
ecf47bc2256 ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
a49e51b5db4 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
0a7a45b42be rpl: repack table_def
2e07d9c0def Copy_field: add const to arguments
208f834397e rename tests
1f9b60f2160 binlog_combinations.inc -> binlog_format_combinations.inc
eff7801152f cleanup: remove vcol_info->stored_in_db
efa160c3ad6 Fix recalculation of vcols in binlog_row_image=minimal
f1310b9d917 cleanup: clarify ha_innobase::column_bitmaps_signal()
0d6ed7350be allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

----

h2. Other notes

Engines:
InnoDB, MyISAM, Aria, Heap, CSV -- applies
MERGE -- skips online alter (doesn't hit debug sync point), probably because there is no real data
Columnstore -- skips online alter (doesn't increase Binlog_bytes_written, cannot test debug (!), it fails there, to be checked whether it's specific to the branch)
OQGRAPH -- ALTER is prohibited in general
PERFORMANCE_SCHEMA -- cannot be ALTERed

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [62e596aa6|https://github.com/MariaDB/server/commit/62e596aa6a279bafbecc91d82d990f540adcc31f]
Baseline: [2867894a (11.2)|https://github.com/MariaDB/server/commit/2867894ac6ca23fc2f82cfad2dd510351f597325]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(?) +Buildbot (62e596aa6)
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=62e596aa6a279bafbecc91d82d990f540adcc31f&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR (62e596aa6)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (?) SAN tests ~~MDEV-30945~~ (MSAN)
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
62e596aa6a2 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
a3caa265b7a fix key detection on replica with extra columns
842cb51e81f ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
a31b2a52119 ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
8c774b12114 ~~MDEV-31646~~ preserve DMLs in case of online binlog fault
ca64ddcc709 ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
f34c7419cb8 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
bcb49750118 fix -Werror build
8b0101a853b follow-up ~~MDEV-30430~~: fix versioning.rpl
ff775c71ac1 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
dc7136481bf ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
1a062cc0ee1 Add const to get_foreign_key_list/get_parent_foreign_key_list
f6db56cff63 Add const to alloc-related thd methods
188c3f767e1 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
d75e4ea59a2 ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
f3d0a06c22f refactor unpack_row
abd73373329 unpack_row: unpack a correct number of fields
949a4f5f86a ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
ae477a9f842 unpack_row: set the correct fields in has_value_set for online alter
542e8caba74 ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
8959ba16ac6 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
157e0ceea56 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
ed3c6a4d062 ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
dd80527782e clean up Rows_log_event virtual methods
5404f2937fe ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
07191fb46d0 add partition test
afb459edbe8 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
cb4d928678f ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
d1fe3487449 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
dc4e59852d8 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
4f67b03c4f7 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
955c700e90b fix main.alter_table_{online,lock}
f434a196bda ~~MDEV-29069~~ follow-up: improve DEFAULT rules
cbe733b6b10 ~~MDEV-29069~~ follow-up: support partially usable keys
6eaf66f2d17 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
a379d5921c3 ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
26b11bb6f9c cleanup: cache the result of Rows_log_event::find_key()
ad19f3d708e set table->pos_in_table_list in online alter
79925e9b5f1 cleanup: remove rpl_group_info::get_table_data()
a8a67cdb8f4 cleanup: ifdefs
97a75c554d4 few rgi assertions. this can proof that rgi is always present
ec99d11cada rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
b2b030fd064 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
84f862c3f06 log_event.h: remove junk EOL spaces
71549936f83 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
eb9ac1b8518 Fix write_set too
4dfc48b9e0b ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
8e42878e550 rpl: check should go after defaults and vcols update
7331dfbac5c ~~MDEV-29067~~ Online alter ignores check constraint violation
cdaac40609d don't do ALTER IGNORE TABLE online
b89de5cf849 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
cfb0c7fe5d0 Do not ignore sql_mode when replicating
40eb328f0c2 Simplify rgi->get_table_data call
655d8764e68 reorder RPL_TABLE_LIST fields for better packing
0ef8bde91be ~~MDEV-29021~~ fixup
c37feb51ee1 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
7bf07c5d5c2 cleanup, remove dead code
229a4eaa2cf ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
0811905ad85 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
1c4e7ed3965 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
ced39a3cb1f remove redundant warnings in RBR and online alter
5004620d5cb cleanup: whitespace, etc
3618819bc0a ~~MDEV-28959~~ Online alter ignores strict table mode
17c0b8eba99 ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
c566c0c0f3f control Cache_flip_event_log lifetime with reference count
ee62df99aeb ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
517823bfbe1 test rename alter_table_online -> alter_table_online_debug
8dba420a547 separate online_alter_cache_data from binlog_cache_data
213cc14b65c put binlog_cache_data on a memroot
8718d110432 always commit for non-trans engines
55194c2277c savepoints
5064dc3bc83 don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
4198a9c04bc don't use start_consistent_snapshot
77bf7ac5ee6 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
a63684a1e92 don't do DROP SYSTEM VERSIONING online
307a4d33095 set read_set early, before row reads
2a18ae8e89b no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
512376f3e59 online alter always uses ALGORITHM=COPY, LOCK=NONE
c4ca1eadf54 remove handler::open_read_view()
9824910fb86 cleanup
87968ce5208 support 'alter online table t1 page_checksum=0'
84b03783860 tests: move around, add new
f7ff9b40118 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
20aae939c33 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
9ca93a9c574 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
ecf47bc2256 ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
a49e51b5db4 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
0a7a45b42be rpl: repack table_def
2e07d9c0def Copy_field: add const to arguments
208f834397e rename tests
1f9b60f2160 binlog_combinations.inc -> binlog_format_combinations.inc
eff7801152f cleanup: remove vcol_info->stored_in_db
efa160c3ad6 Fix recalculation of vcols in binlog_row_image=minimal
f1310b9d917 cleanup: clarify ha_innobase::column_bitmaps_signal()
0d6ed7350be allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

----

h2. Other notes

Engines:
InnoDB, MyISAM, Aria, Heap, CSV, RocksDB, Archive, Mroonga -- applies
MERGE, Spider, Blackhole, Sphinx -- skips online alter (doesn't hit debug sync point), probably because there is no real data
Columnstore -- skips online alter (doesn't increase Binlog_bytes_written, cannot test debug (!), it fails there, to be checked whether it's specific to the branch)
S3, OQGRAPH, FEDERATED, FEDERATEDX -- ALTER is prohibited in general
PERFORMANCE_SCHEMA -- cannot be ALTERed
(!) Conversion to S3 -- fails, ~~MDEV-31781~~
(!) Sequences -- fails, ~~MDEV-31775~~ or maybe legacy ~~MDEV-31779~~
(!) Connect -- fails, ~~MDEV-31777~~ and a number of legacy (MDEV-30909, MDEV-30914, MDEV-30915, MDEV-30907)

Elena Stepanova made changes - 2023-07-26 22:45

Link

This issue relates to ~~MDEV-31781~~ [ ~~MDEV-31781~~ ]

Elena Stepanova made changes - 2023-07-25 22:52

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [62e596aa6|https://github.com/MariaDB/server/commit/62e596aa6a279bafbecc91d82d990f540adcc31f]
Baseline: [2867894a (11.2)|https://github.com/MariaDB/server/commit/2867894ac6ca23fc2f82cfad2dd510351f597325]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(?) +Buildbot (62e596aa6)
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=62e596aa6a279bafbecc91d82d990f540adcc31f&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR (62e596aa6)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (?) SAN tests ~~MDEV-30945~~ (MSAN)
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
62e596aa6a2 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
a3caa265b7a fix key detection on replica with extra columns
842cb51e81f ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
a31b2a52119 ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
8c774b12114 ~~MDEV-31646~~ preserve DMLs in case of online binlog fault
ca64ddcc709 ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
f34c7419cb8 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
bcb49750118 fix -Werror build
8b0101a853b follow-up ~~MDEV-30430~~: fix versioning.rpl
ff775c71ac1 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
dc7136481bf ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
1a062cc0ee1 Add const to get_foreign_key_list/get_parent_foreign_key_list
f6db56cff63 Add const to alloc-related thd methods
188c3f767e1 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
d75e4ea59a2 ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
f3d0a06c22f refactor unpack_row
abd73373329 unpack_row: unpack a correct number of fields
949a4f5f86a ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
ae477a9f842 unpack_row: set the correct fields in has_value_set for online alter
542e8caba74 ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
8959ba16ac6 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
157e0ceea56 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
ed3c6a4d062 ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
dd80527782e clean up Rows_log_event virtual methods
5404f2937fe ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
07191fb46d0 add partition test
afb459edbe8 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
cb4d928678f ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
d1fe3487449 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
dc4e59852d8 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
4f67b03c4f7 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
955c700e90b fix main.alter_table_{online,lock}
f434a196bda ~~MDEV-29069~~ follow-up: improve DEFAULT rules
cbe733b6b10 ~~MDEV-29069~~ follow-up: support partially usable keys
6eaf66f2d17 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
a379d5921c3 ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
26b11bb6f9c cleanup: cache the result of Rows_log_event::find_key()
ad19f3d708e set table->pos_in_table_list in online alter
79925e9b5f1 cleanup: remove rpl_group_info::get_table_data()
a8a67cdb8f4 cleanup: ifdefs
97a75c554d4 few rgi assertions. this can proof that rgi is always present
ec99d11cada rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
b2b030fd064 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
84f862c3f06 log_event.h: remove junk EOL spaces
71549936f83 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
eb9ac1b8518 Fix write_set too
4dfc48b9e0b ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
8e42878e550 rpl: check should go after defaults and vcols update
7331dfbac5c ~~MDEV-29067~~ Online alter ignores check constraint violation
cdaac40609d don't do ALTER IGNORE TABLE online
b89de5cf849 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
cfb0c7fe5d0 Do not ignore sql_mode when replicating
40eb328f0c2 Simplify rgi->get_table_data call
655d8764e68 reorder RPL_TABLE_LIST fields for better packing
0ef8bde91be ~~MDEV-29021~~ fixup
c37feb51ee1 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
7bf07c5d5c2 cleanup, remove dead code
229a4eaa2cf ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
0811905ad85 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
1c4e7ed3965 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
ced39a3cb1f remove redundant warnings in RBR and online alter
5004620d5cb cleanup: whitespace, etc
3618819bc0a ~~MDEV-28959~~ Online alter ignores strict table mode
17c0b8eba99 ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
c566c0c0f3f control Cache_flip_event_log lifetime with reference count
ee62df99aeb ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
517823bfbe1 test rename alter_table_online -> alter_table_online_debug
8dba420a547 separate online_alter_cache_data from binlog_cache_data
213cc14b65c put binlog_cache_data on a memroot
8718d110432 always commit for non-trans engines
55194c2277c savepoints
5064dc3bc83 don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
4198a9c04bc don't use start_consistent_snapshot
77bf7ac5ee6 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
a63684a1e92 don't do DROP SYSTEM VERSIONING online
307a4d33095 set read_set early, before row reads
2a18ae8e89b no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
512376f3e59 online alter always uses ALGORITHM=COPY, LOCK=NONE
c4ca1eadf54 remove handler::open_read_view()
9824910fb86 cleanup
87968ce5208 support 'alter online table t1 page_checksum=0'
84b03783860 tests: move around, add new
f7ff9b40118 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
20aae939c33 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
9ca93a9c574 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
ecf47bc2256 ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
a49e51b5db4 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
0a7a45b42be rpl: repack table_def
2e07d9c0def Copy_field: add const to arguments
208f834397e rename tests
1f9b60f2160 binlog_combinations.inc -> binlog_format_combinations.inc
eff7801152f cleanup: remove vcol_info->stored_in_db
efa160c3ad6 Fix recalculation of vcols in binlog_row_image=minimal
f1310b9d917 cleanup: clarify ha_innobase::column_bitmaps_signal()
0d6ed7350be allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [62e596aa6|https://github.com/MariaDB/server/commit/62e596aa6a279bafbecc91d82d990f540adcc31f]
Baseline: [2867894a (11.2)|https://github.com/MariaDB/server/commit/2867894ac6ca23fc2f82cfad2dd510351f597325]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(?) +Buildbot (62e596aa6)
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=62e596aa6a279bafbecc91d82d990f540adcc31f&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR (62e596aa6)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (?) SAN tests ~~MDEV-30945~~ (MSAN)
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
62e596aa6a2 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
a3caa265b7a fix key detection on replica with extra columns
842cb51e81f ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
a31b2a52119 ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
8c774b12114 ~~MDEV-31646~~ preserve DMLs in case of online binlog fault
ca64ddcc709 ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
f34c7419cb8 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
bcb49750118 fix -Werror build
8b0101a853b follow-up ~~MDEV-30430~~: fix versioning.rpl
ff775c71ac1 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
dc7136481bf ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
1a062cc0ee1 Add const to get_foreign_key_list/get_parent_foreign_key_list
f6db56cff63 Add const to alloc-related thd methods
188c3f767e1 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
d75e4ea59a2 ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
f3d0a06c22f refactor unpack_row
abd73373329 unpack_row: unpack a correct number of fields
949a4f5f86a ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
ae477a9f842 unpack_row: set the correct fields in has_value_set for online alter
542e8caba74 ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
8959ba16ac6 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
157e0ceea56 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
ed3c6a4d062 ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
dd80527782e clean up Rows_log_event virtual methods
5404f2937fe ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
07191fb46d0 add partition test
afb459edbe8 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
cb4d928678f ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
d1fe3487449 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
dc4e59852d8 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
4f67b03c4f7 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
955c700e90b fix main.alter_table_{online,lock}
f434a196bda ~~MDEV-29069~~ follow-up: improve DEFAULT rules
cbe733b6b10 ~~MDEV-29069~~ follow-up: support partially usable keys
6eaf66f2d17 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
a379d5921c3 ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
26b11bb6f9c cleanup: cache the result of Rows_log_event::find_key()
ad19f3d708e set table->pos_in_table_list in online alter
79925e9b5f1 cleanup: remove rpl_group_info::get_table_data()
a8a67cdb8f4 cleanup: ifdefs
97a75c554d4 few rgi assertions. this can proof that rgi is always present
ec99d11cada rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
b2b030fd064 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
84f862c3f06 log_event.h: remove junk EOL spaces
71549936f83 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
eb9ac1b8518 Fix write_set too
4dfc48b9e0b ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
8e42878e550 rpl: check should go after defaults and vcols update
7331dfbac5c ~~MDEV-29067~~ Online alter ignores check constraint violation
cdaac40609d don't do ALTER IGNORE TABLE online
b89de5cf849 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
cfb0c7fe5d0 Do not ignore sql_mode when replicating
40eb328f0c2 Simplify rgi->get_table_data call
655d8764e68 reorder RPL_TABLE_LIST fields for better packing
0ef8bde91be ~~MDEV-29021~~ fixup
c37feb51ee1 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
7bf07c5d5c2 cleanup, remove dead code
229a4eaa2cf ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
0811905ad85 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
1c4e7ed3965 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
ced39a3cb1f remove redundant warnings in RBR and online alter
5004620d5cb cleanup: whitespace, etc
3618819bc0a ~~MDEV-28959~~ Online alter ignores strict table mode
17c0b8eba99 ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
c566c0c0f3f control Cache_flip_event_log lifetime with reference count
ee62df99aeb ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
517823bfbe1 test rename alter_table_online -> alter_table_online_debug
8dba420a547 separate online_alter_cache_data from binlog_cache_data
213cc14b65c put binlog_cache_data on a memroot
8718d110432 always commit for non-trans engines
55194c2277c savepoints
5064dc3bc83 don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
4198a9c04bc don't use start_consistent_snapshot
77bf7ac5ee6 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
a63684a1e92 don't do DROP SYSTEM VERSIONING online
307a4d33095 set read_set early, before row reads
2a18ae8e89b no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
512376f3e59 online alter always uses ALGORITHM=COPY, LOCK=NONE
c4ca1eadf54 remove handler::open_read_view()
9824910fb86 cleanup
87968ce5208 support 'alter online table t1 page_checksum=0'
84b03783860 tests: move around, add new
f7ff9b40118 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
20aae939c33 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
9ca93a9c574 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
ecf47bc2256 ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
a49e51b5db4 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
0a7a45b42be rpl: repack table_def
2e07d9c0def Copy_field: add const to arguments
208f834397e rename tests
1f9b60f2160 binlog_combinations.inc -> binlog_format_combinations.inc
eff7801152f cleanup: remove vcol_info->stored_in_db
efa160c3ad6 Fix recalculation of vcols in binlog_row_image=minimal
f1310b9d917 cleanup: clarify ha_innobase::column_bitmaps_signal()
0d6ed7350be allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

----

h2. Other notes

Engines:
InnoDB, MyISAM, Aria, Heap, CSV -- applies
MERGE -- skips online alter (doesn't hit debug sync point), probably because there is no real data
Columnstore -- skips online alter (doesn't increase Binlog_bytes_written, cannot test debug (!), it fails there, to be checked whether it's specific to the branch)
OQGRAPH -- ALTER is prohibited in general
PERFORMANCE_SCHEMA -- cannot be ALTERed

Elena Stepanova made changes - 2023-07-25 19:00

Link

This issue relates to ~~MDEV-31775~~ [ ~~MDEV-31775~~ ]

Elena Stepanova made changes - 2023-07-24 15:29

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [62e596aa6|https://github.com/MariaDB/server/commit/62e596aa6a279bafbecc91d82d990f540adcc31f]
Baseline: [2867894a (11.2)|https://github.com/MariaDB/server/commit/2867894ac6ca23fc2f82cfad2dd510351f597325]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(x) +Buildbot (62e596aa6)+ : ~~MDEV-30987~~ ~~MDEV-30945~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=62e596aa6a279bafbecc91d82d990f540adcc31f&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR (62e596aa6)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (?) SAN tests ~~MDEV-30945~~ (MSAN)
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
62e596aa6a2 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
a3caa265b7a fix key detection on replica with extra columns
842cb51e81f ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
a31b2a52119 ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
8c774b12114 ~~MDEV-31646~~ preserve DMLs in case of online binlog fault
ca64ddcc709 ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
f34c7419cb8 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
bcb49750118 fix -Werror build
8b0101a853b follow-up ~~MDEV-30430~~: fix versioning.rpl
ff775c71ac1 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
dc7136481bf ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
1a062cc0ee1 Add const to get_foreign_key_list/get_parent_foreign_key_list
f6db56cff63 Add const to alloc-related thd methods
188c3f767e1 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
d75e4ea59a2 ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
f3d0a06c22f refactor unpack_row
abd73373329 unpack_row: unpack a correct number of fields
949a4f5f86a ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
ae477a9f842 unpack_row: set the correct fields in has_value_set for online alter
542e8caba74 ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
8959ba16ac6 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
157e0ceea56 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
ed3c6a4d062 ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
dd80527782e clean up Rows_log_event virtual methods
5404f2937fe ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
07191fb46d0 add partition test
afb459edbe8 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
cb4d928678f ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
d1fe3487449 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
dc4e59852d8 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
4f67b03c4f7 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
955c700e90b fix main.alter_table_{online,lock}
f434a196bda ~~MDEV-29069~~ follow-up: improve DEFAULT rules
cbe733b6b10 ~~MDEV-29069~~ follow-up: support partially usable keys
6eaf66f2d17 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
a379d5921c3 ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
26b11bb6f9c cleanup: cache the result of Rows_log_event::find_key()
ad19f3d708e set table->pos_in_table_list in online alter
79925e9b5f1 cleanup: remove rpl_group_info::get_table_data()
a8a67cdb8f4 cleanup: ifdefs
97a75c554d4 few rgi assertions. this can proof that rgi is always present
ec99d11cada rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
b2b030fd064 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
84f862c3f06 log_event.h: remove junk EOL spaces
71549936f83 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
eb9ac1b8518 Fix write_set too
4dfc48b9e0b ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
8e42878e550 rpl: check should go after defaults and vcols update
7331dfbac5c ~~MDEV-29067~~ Online alter ignores check constraint violation
cdaac40609d don't do ALTER IGNORE TABLE online
b89de5cf849 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
cfb0c7fe5d0 Do not ignore sql_mode when replicating
40eb328f0c2 Simplify rgi->get_table_data call
655d8764e68 reorder RPL_TABLE_LIST fields for better packing
0ef8bde91be ~~MDEV-29021~~ fixup
c37feb51ee1 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
7bf07c5d5c2 cleanup, remove dead code
229a4eaa2cf ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
0811905ad85 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
1c4e7ed3965 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
ced39a3cb1f remove redundant warnings in RBR and online alter
5004620d5cb cleanup: whitespace, etc
3618819bc0a ~~MDEV-28959~~ Online alter ignores strict table mode
17c0b8eba99 ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
c566c0c0f3f control Cache_flip_event_log lifetime with reference count
ee62df99aeb ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
517823bfbe1 test rename alter_table_online -> alter_table_online_debug
8dba420a547 separate online_alter_cache_data from binlog_cache_data
213cc14b65c put binlog_cache_data on a memroot
8718d110432 always commit for non-trans engines
55194c2277c savepoints
5064dc3bc83 don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
4198a9c04bc don't use start_consistent_snapshot
77bf7ac5ee6 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
a63684a1e92 don't do DROP SYSTEM VERSIONING online
307a4d33095 set read_set early, before row reads
2a18ae8e89b no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
512376f3e59 online alter always uses ALGORITHM=COPY, LOCK=NONE
c4ca1eadf54 remove handler::open_read_view()
9824910fb86 cleanup
87968ce5208 support 'alter online table t1 page_checksum=0'
84b03783860 tests: move around, add new
f7ff9b40118 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
20aae939c33 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
9ca93a9c574 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
ecf47bc2256 ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
a49e51b5db4 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
0a7a45b42be rpl: repack table_def
2e07d9c0def Copy_field: add const to arguments
208f834397e rename tests
1f9b60f2160 binlog_combinations.inc -> binlog_format_combinations.inc
eff7801152f cleanup: remove vcol_info->stored_in_db
efa160c3ad6 Fix recalculation of vcols in binlog_row_image=minimal
f1310b9d917 cleanup: clarify ha_innobase::column_bitmaps_signal()
0d6ed7350be allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [62e596aa6|https://github.com/MariaDB/server/commit/62e596aa6a279bafbecc91d82d990f540adcc31f]
Baseline: [2867894a (11.2)|https://github.com/MariaDB/server/commit/2867894ac6ca23fc2f82cfad2dd510351f597325]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(?) +Buildbot (62e596aa6)
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=62e596aa6a279bafbecc91d82d990f540adcc31f&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR (62e596aa6)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (?) SAN tests ~~MDEV-30945~~ (MSAN)
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
62e596aa6a2 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
a3caa265b7a fix key detection on replica with extra columns
842cb51e81f ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
a31b2a52119 ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
8c774b12114 ~~MDEV-31646~~ preserve DMLs in case of online binlog fault
ca64ddcc709 ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
f34c7419cb8 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
bcb49750118 fix -Werror build
8b0101a853b follow-up ~~MDEV-30430~~: fix versioning.rpl
ff775c71ac1 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
dc7136481bf ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
1a062cc0ee1 Add const to get_foreign_key_list/get_parent_foreign_key_list
f6db56cff63 Add const to alloc-related thd methods
188c3f767e1 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
d75e4ea59a2 ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
f3d0a06c22f refactor unpack_row
abd73373329 unpack_row: unpack a correct number of fields
949a4f5f86a ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
ae477a9f842 unpack_row: set the correct fields in has_value_set for online alter
542e8caba74 ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
8959ba16ac6 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
157e0ceea56 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
ed3c6a4d062 ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
dd80527782e clean up Rows_log_event virtual methods
5404f2937fe ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
07191fb46d0 add partition test
afb459edbe8 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
cb4d928678f ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
d1fe3487449 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
dc4e59852d8 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
4f67b03c4f7 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
955c700e90b fix main.alter_table_{online,lock}
f434a196bda ~~MDEV-29069~~ follow-up: improve DEFAULT rules
cbe733b6b10 ~~MDEV-29069~~ follow-up: support partially usable keys
6eaf66f2d17 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
a379d5921c3 ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
26b11bb6f9c cleanup: cache the result of Rows_log_event::find_key()
ad19f3d708e set table->pos_in_table_list in online alter
79925e9b5f1 cleanup: remove rpl_group_info::get_table_data()
a8a67cdb8f4 cleanup: ifdefs
97a75c554d4 few rgi assertions. this can proof that rgi is always present
ec99d11cada rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
b2b030fd064 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
84f862c3f06 log_event.h: remove junk EOL spaces
71549936f83 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
eb9ac1b8518 Fix write_set too
4dfc48b9e0b ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
8e42878e550 rpl: check should go after defaults and vcols update
7331dfbac5c ~~MDEV-29067~~ Online alter ignores check constraint violation
cdaac40609d don't do ALTER IGNORE TABLE online
b89de5cf849 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
cfb0c7fe5d0 Do not ignore sql_mode when replicating
40eb328f0c2 Simplify rgi->get_table_data call
655d8764e68 reorder RPL_TABLE_LIST fields for better packing
0ef8bde91be ~~MDEV-29021~~ fixup
c37feb51ee1 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
7bf07c5d5c2 cleanup, remove dead code
229a4eaa2cf ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
0811905ad85 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
1c4e7ed3965 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
ced39a3cb1f remove redundant warnings in RBR and online alter
5004620d5cb cleanup: whitespace, etc
3618819bc0a ~~MDEV-28959~~ Online alter ignores strict table mode
17c0b8eba99 ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
c566c0c0f3f control Cache_flip_event_log lifetime with reference count
ee62df99aeb ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
517823bfbe1 test rename alter_table_online -> alter_table_online_debug
8dba420a547 separate online_alter_cache_data from binlog_cache_data
213cc14b65c put binlog_cache_data on a memroot
8718d110432 always commit for non-trans engines
55194c2277c savepoints
5064dc3bc83 don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
4198a9c04bc don't use start_consistent_snapshot
77bf7ac5ee6 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
a63684a1e92 don't do DROP SYSTEM VERSIONING online
307a4d33095 set read_set early, before row reads
2a18ae8e89b no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
512376f3e59 online alter always uses ALGORITHM=COPY, LOCK=NONE
c4ca1eadf54 remove handler::open_read_view()
9824910fb86 cleanup
87968ce5208 support 'alter online table t1 page_checksum=0'
84b03783860 tests: move around, add new
f7ff9b40118 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
20aae939c33 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
9ca93a9c574 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
ecf47bc2256 ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
a49e51b5db4 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
0a7a45b42be rpl: repack table_def
2e07d9c0def Copy_field: add const to arguments
208f834397e rename tests
1f9b60f2160 binlog_combinations.inc -> binlog_format_combinations.inc
eff7801152f cleanup: remove vcol_info->stored_in_db
efa160c3ad6 Fix recalculation of vcols in binlog_row_image=minimal
f1310b9d917 cleanup: clarify ha_innobase::column_bitmaps_signal()
0d6ed7350be allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

Elena Stepanova made changes - 2023-07-24 15:17

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(x) +Buildbot (e599b8b4)+ : ~~MDEV-30987~~ ~~MDEV-30945~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR (e599b8b4)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (x) SAN tests ~~MDEV-30945~~ (MSAN)
** (x) default big / nm + ps
** (off) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
e599b8b4586 ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
7b2e11bd42d ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
e1d1e23b457 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
f9b33ac5703 fix Run-Time Check Failure on Windows
fdd53f6a1e9 fix build
28eaae3acc9 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
01d630af134 fix main.query_cache_innodb and main.alter_table_{online,lock}
6e3f6df7ddc ~~MDEV-29069~~ follow-up: improve DEFAULT rules
7a2138ed897 ~~MDEV-29069~~ follow-up: support partially usable keys
0442a784eba ~~MDEV-29069~~ follow-up: allow non-deterministic DEFAULTs
9a21cbb11aa ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
0a18001b076 cleanup: cache the result of Rows_log_event::find_key()
6c51e27abb0 set table->pos_in_table_list in online alter
40ab76b1d16 cleanup: remove rpl_group_info::get_table_data()
0c1cddfc67b cleanup: ifdefs
fdd0c9e08d8 few rgi assertions. this can proof that rgi is always present
8876e53b845 rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
76cd3bfaad6 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
4af400cf84f log_event.h: remove junk EOL spaces
8bd015193f4 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
3c8111a276f Fix write_set too
148c1ee7dec ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
ea6fb5c2dc5 rpl: check should go after defaults and vcols update
b39b4553229 ~~MDEV-29067~~ Online alter ignores check constraint violation
607ad8d4e7a don't do ALTER IGNORE TABLE online
47ca9f78799 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
48b6d9bff94 Do not ignore sql_mode when replicating
82eea293ba0 Simplify rgi->get_table_data call
4ee10b52bff reorder RPL_TABLE_LIST fields for better packing
2f02da22cd1 ~~MDEV-29021~~ fixup
ed325ef3d44 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
083b6897932 cleanup, remove dead code
88384b40023 ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
63bcd28f492 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
d4d714f0e90 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
571870ae85b remove redundant warnings in RBR and online alter
cce37c33136 cleanup: whitespace, etc
2ecbc7fb5b2 ~~MDEV-28959~~ Online alter ignores strict table mode
f59f1446c0c ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
2cce964647c control Cache_flip_event_log lifetime with reference count
02e80e0bda4 ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
ce3c9945b12 test rename alter_table_online -> alter_table_online_debug
e5de0743266 separate online_alter_cache_data from binlog_cache_data
eac5bd2ed4f put binlog_cache_data on a memroot
054e429ef2a always commit for non-trans engines
63fab8bc99f savepoints
b0168a567ce don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
54b5394ef73 don't use start_consistent_snapshot
a645fcddc4f don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
741bdcf64a8 don't do DROP SYSTEM VERSIONING online
eb1411714d4 set read_set early, before row reads
4a5b5249c4c no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
bae5f6a68de online alter always uses ALGORITHM=COPY, LOCK=NONE
b42d2abcb6a remove handler::open_read_view()
a5eea2ad2f0 cleanup
cdd4d4a54e6 support 'alter online table t1 page_checksum=0'
256242f5d34 tests: move around, add new
b7d5ed4e87d ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
d7c841f5ca3 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
d10c2710ba2 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
3459c92ebab ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
3d1f5ae09c6 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
aec6f2a96b3 rpl: repack table_def
5e8fb8b3768 Copy_field: add const to arguments
7c5e90ba17b rename tests
9a16e6380f4 binlog_combinations.inc -> binlog_format_combinations.inc
5d4f1104e2a cleanup: remove vcol_info->stored_in_db
8a4c485f3d7 Fix recalculation of vcols in binlog_row_image=minimal
829267dc91d cleanup: clarify ha_innobase::column_bitmaps_signal()
32b56d79e00 allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.2-oalter|https://github.com/MariaDB/server/tree/bb-11.2-oalter]

Last commit in testing: [62e596aa6|https://github.com/MariaDB/server/commit/62e596aa6a279bafbecc91d82d990f540adcc31f]
Baseline: [2867894a (11.2)|https://github.com/MariaDB/server/commit/2867894ac6ca23fc2f82cfad2dd510351f597325]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(x) +Buildbot (62e596aa6)+ : ~~MDEV-30987~~ ~~MDEV-30945~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.2-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.2-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=62e596aa6a279bafbecc91d82d990f540adcc31f&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR (62e596aa6)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (?) SAN tests ~~MDEV-30945~~ (MSAN)
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
62e596aa6a2 ~~MDEV-31755~~ Replica's DML event deadlocks wit online alter table
a3caa265b7a fix key detection on replica with extra columns
842cb51e81f ~~MDEV-31677~~ Assertion failed upon online ALTER with binlog_row_image=NOBLOB
a31b2a52119 ~~MDEV-31646~~ untie from max_allowed_packet and opt_binlog_rows_event_max_size
8c774b12114 ~~MDEV-31646~~ preserve DMLs in case of online binlog fault
ca64ddcc709 ~~MDEV-31646~~ Online alter applies binlog cache limit to cache writes
f34c7419cb8 ~~MDEV-31601~~ Some ALTER TABLEs fail ... with a wrong error message
bcb49750118 fix -Werror build
8b0101a853b follow-up ~~MDEV-30430~~: fix versioning.rpl
ff775c71ac1 fix main.mysql57_virtual, main.alter_table, innodb.alter_algorithm
dc7136481bf ~~MDEV-30984~~ Online ALTER table is denied with non-informative error messages
1a062cc0ee1 Add const to get_foreign_key_list/get_parent_foreign_key_list
f6db56cff63 Add const to alloc-related thd methods
188c3f767e1 ~~MDEV-30987~~ main.alter_table_online times out with view-protocol
d75e4ea59a2 ~~MDEV-31059~~ "Slave SQL" errors upon concurrent DML and erroneous ALTER
f3d0a06c22f refactor unpack_row
abd73373329 unpack_row: unpack a correct number of fields
949a4f5f86a ~~MDEV-31058~~ ER_KEY_NOT_FOUND upon concurrent CHANGE column autoinc and DML
ae477a9f842 unpack_row: set the correct fields in has_value_set for online alter
542e8caba74 ~~MDEV-30949~~ Direct leak in binlog_online_alter_end_trans
8959ba16ac6 ~~MDEV-31043~~ ER_KEY_NOT_FOUND upon concurrent ALTER and transaction
157e0ceea56 ~~MDEV-31033~~ ER_KEY_NOT_FOUND upon online COPY ALTER on a partitioned table
ed3c6a4d062 ~~MDEV-30945~~ RPL tests are failing with MSAN use-of-uninitialized-value
dd80527782e clean up Rows_log_event virtual methods
5404f2937fe ~~MDEV-30891~~ Assertion `!table->versioned(VERS_TRX_ID)' failed
07191fb46d0 add partition test
afb459edbe8 ~~MDEV-30985~~ Replica stops with error on ALTER ONLINE with Geometry Types
cb4d928678f ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
d1fe3487449 ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
dc4e59852d8 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
4f67b03c4f7 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
955c700e90b fix main.alter_table_{online,lock}
f434a196bda ~~MDEV-29069~~ follow-up: improve DEFAULT rules
cbe733b6b10 ~~MDEV-29069~~ follow-up: support partially usable keys
6eaf66f2d17 ~~MDEV-29069~~ follow-up: allow deterministic DEFAULTs
a379d5921c3 ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
26b11bb6f9c cleanup: cache the result of Rows_log_event::find_key()
ad19f3d708e set table->pos_in_table_list in online alter
79925e9b5f1 cleanup: remove rpl_group_info::get_table_data()
a8a67cdb8f4 cleanup: ifdefs
97a75c554d4 few rgi assertions. this can proof that rgi is always present
ec99d11cada rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
b2b030fd064 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
84f862c3f06 log_event.h: remove junk EOL spaces
71549936f83 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
eb9ac1b8518 Fix write_set too
4dfc48b9e0b ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
8e42878e550 rpl: check should go after defaults and vcols update
7331dfbac5c ~~MDEV-29067~~ Online alter ignores check constraint violation
cdaac40609d don't do ALTER IGNORE TABLE online
b89de5cf849 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
cfb0c7fe5d0 Do not ignore sql_mode when replicating
40eb328f0c2 Simplify rgi->get_table_data call
655d8764e68 reorder RPL_TABLE_LIST fields for better packing
0ef8bde91be ~~MDEV-29021~~ fixup
c37feb51ee1 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
7bf07c5d5c2 cleanup, remove dead code
229a4eaa2cf ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
0811905ad85 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
1c4e7ed3965 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
ced39a3cb1f remove redundant warnings in RBR and online alter
5004620d5cb cleanup: whitespace, etc
3618819bc0a ~~MDEV-28959~~ Online alter ignores strict table mode
17c0b8eba99 ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
c566c0c0f3f control Cache_flip_event_log lifetime with reference count
ee62df99aeb ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
517823bfbe1 test rename alter_table_online -> alter_table_online_debug
8dba420a547 separate online_alter_cache_data from binlog_cache_data
213cc14b65c put binlog_cache_data on a memroot
8718d110432 always commit for non-trans engines
55194c2277c savepoints
5064dc3bc83 don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
4198a9c04bc don't use start_consistent_snapshot
77bf7ac5ee6 don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
a63684a1e92 don't do DROP SYSTEM VERSIONING online
307a4d33095 set read_set early, before row reads
2a18ae8e89b no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
512376f3e59 online alter always uses ALGORITHM=COPY, LOCK=NONE
c4ca1eadf54 remove handler::open_read_view()
9824910fb86 cleanup
87968ce5208 support 'alter online table t1 page_checksum=0'
84b03783860 tests: move around, add new
f7ff9b40118 ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
20aae939c33 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
9ca93a9c574 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
ecf47bc2256 ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
a49e51b5db4 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
0a7a45b42be rpl: repack table_def
2e07d9c0def Copy_field: add const to arguments
208f834397e rename tests
1f9b60f2160 binlog_combinations.inc -> binlog_format_combinations.inc
eff7801152f cleanup: remove vcol_info->stored_in_db
efa160c3ad6 Fix recalculation of vcols in binlog_row_image=minimal
f1310b9d917 cleanup: clarify ha_innobase::column_bitmaps_signal()
0d6ed7350be allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

Elena Stepanova made changes - 2023-07-04 19:25

Link

This issue relates to MDEV-31624 [ MDEV-31624 ]

Elena Stepanova made changes - 2023-06-27 16:11

Link

This issue relates to ~~MDEV-31563~~ [ ~~MDEV-31563~~ ]

Ralf Gebhardt made changes - 2023-05-30 08:18

Fix Version/s		11.2 [ 28603 ]
Fix Version/s	11.1 [ 28549 ]

Elena Stepanova made changes - 2023-05-03 00:23

Link

This issue relates to ~~MDEV-31172~~ [ ~~MDEV-31172~~ ]

Elena Stepanova made changes - 2023-04-26 23:29

Link

This issue relates to ~~MDEV-31136~~ [ ~~MDEV-31136~~ ]

Elena Stepanova made changes - 2023-04-25 19:05

Link

This issue relates to ~~MDEV-31128~~ [ ~~MDEV-31128~~ ]

Elena Stepanova made changes - 2023-04-17 14:18

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(x) +Buildbot (e599b8b4)+ : ~~MDEV-30987~~ ~~MDEV-30945~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR (e599b8b4)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (x) SAN tests ~~MDEV-30945~~ (MSAN)
** (x) default big / nm + ps
** (off) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
e599b8b4586 ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
7b2e11bd42d ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
e1d1e23b457 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
f9b33ac5703 fix Run-Time Check Failure on Windows
fdd53f6a1e9 fix build
28eaae3acc9 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
01d630af134 fix main.query_cache_innodb and main.alter_table_{online,lock}
6e3f6df7ddc ~~MDEV-29069~~ follow-up: improve DEFAULT rules
7a2138ed897 ~~MDEV-29069~~ follow-up: support partially usable keys
0442a784eba ~~MDEV-29069~~ follow-up: allow non-deterministic DEFAULTs
9a21cbb11aa ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
0a18001b076 cleanup: cache the result of Rows_log_event::find_key()
6c51e27abb0 set table->pos_in_table_list in online alter
40ab76b1d16 cleanup: remove rpl_group_info::get_table_data()
0c1cddfc67b cleanup: ifdefs
fdd0c9e08d8 few rgi assertions. this can proof that rgi is always present
8876e53b845 rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
76cd3bfaad6 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
4af400cf84f log_event.h: remove junk EOL spaces
8bd015193f4 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
3c8111a276f Fix write_set too
148c1ee7dec ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
ea6fb5c2dc5 rpl: check should go after defaults and vcols update
b39b4553229 ~~MDEV-29067~~ Online alter ignores check constraint violation
607ad8d4e7a don't do ALTER IGNORE TABLE online
47ca9f78799 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
48b6d9bff94 Do not ignore sql_mode when replicating
82eea293ba0 Simplify rgi->get_table_data call
4ee10b52bff reorder RPL_TABLE_LIST fields for better packing
2f02da22cd1 ~~MDEV-29021~~ fixup
ed325ef3d44 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
083b6897932 cleanup, remove dead code
88384b40023 ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
63bcd28f492 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
d4d714f0e90 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
571870ae85b remove redundant warnings in RBR and online alter
cce37c33136 cleanup: whitespace, etc
2ecbc7fb5b2 ~~MDEV-28959~~ Online alter ignores strict table mode
f59f1446c0c ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
2cce964647c control Cache_flip_event_log lifetime with reference count
02e80e0bda4 ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
ce3c9945b12 test rename alter_table_online -> alter_table_online_debug
e5de0743266 separate online_alter_cache_data from binlog_cache_data
eac5bd2ed4f put binlog_cache_data on a memroot
054e429ef2a always commit for non-trans engines
63fab8bc99f savepoints
b0168a567ce don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
54b5394ef73 don't use start_consistent_snapshot
a645fcddc4f don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
741bdcf64a8 don't do DROP SYSTEM VERSIONING online
eb1411714d4 set read_set early, before row reads
4a5b5249c4c no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
bae5f6a68de online alter always uses ALGORITHM=COPY, LOCK=NONE
b42d2abcb6a remove handler::open_read_view()
a5eea2ad2f0 cleanup
cdd4d4a54e6 support 'alter online table t1 page_checksum=0'
256242f5d34 tests: move around, add new
b7d5ed4e87d ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
d7c841f5ca3 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
d10c2710ba2 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
3459c92ebab ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
3d1f5ae09c6 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
aec6f2a96b3 rpl: repack table_def
5e8fb8b3768 Copy_field: add const to arguments
7c5e90ba17b rename tests
9a16e6380f4 binlog_combinations.inc -> binlog_format_combinations.inc
5d4f1104e2a cleanup: remove vcol_info->stored_in_db
8a4c485f3d7 Fix recalculation of vcols in binlog_row_image=minimal
829267dc91d cleanup: clarify ha_innobase::column_bitmaps_signal()
32b56d79e00 allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

Extra info: ~~MDEV-11424~~ - InnoDB operations which can/cannot be instant

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(x) +Buildbot (e599b8b4)+ : ~~MDEV-30987~~ ~~MDEV-30945~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR (e599b8b4)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (x) SAN tests ~~MDEV-30945~~ (MSAN)
** (x) default big / nm + ps
** (off) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
e599b8b4586 ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
7b2e11bd42d ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
e1d1e23b457 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
f9b33ac5703 fix Run-Time Check Failure on Windows
fdd53f6a1e9 fix build
28eaae3acc9 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
01d630af134 fix main.query_cache_innodb and main.alter_table_{online,lock}
6e3f6df7ddc ~~MDEV-29069~~ follow-up: improve DEFAULT rules
7a2138ed897 ~~MDEV-29069~~ follow-up: support partially usable keys
0442a784eba ~~MDEV-29069~~ follow-up: allow non-deterministic DEFAULTs
9a21cbb11aa ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
0a18001b076 cleanup: cache the result of Rows_log_event::find_key()
6c51e27abb0 set table->pos_in_table_list in online alter
40ab76b1d16 cleanup: remove rpl_group_info::get_table_data()
0c1cddfc67b cleanup: ifdefs
fdd0c9e08d8 few rgi assertions. this can proof that rgi is always present
8876e53b845 rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
76cd3bfaad6 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
4af400cf84f log_event.h: remove junk EOL spaces
8bd015193f4 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
3c8111a276f Fix write_set too
148c1ee7dec ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
ea6fb5c2dc5 rpl: check should go after defaults and vcols update
b39b4553229 ~~MDEV-29067~~ Online alter ignores check constraint violation
607ad8d4e7a don't do ALTER IGNORE TABLE online
47ca9f78799 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
48b6d9bff94 Do not ignore sql_mode when replicating
82eea293ba0 Simplify rgi->get_table_data call
4ee10b52bff reorder RPL_TABLE_LIST fields for better packing
2f02da22cd1 ~~MDEV-29021~~ fixup
ed325ef3d44 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
083b6897932 cleanup, remove dead code
88384b40023 ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
63bcd28f492 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
d4d714f0e90 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
571870ae85b remove redundant warnings in RBR and online alter
cce37c33136 cleanup: whitespace, etc
2ecbc7fb5b2 ~~MDEV-28959~~ Online alter ignores strict table mode
f59f1446c0c ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
2cce964647c control Cache_flip_event_log lifetime with reference count
02e80e0bda4 ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
ce3c9945b12 test rename alter_table_online -> alter_table_online_debug
e5de0743266 separate online_alter_cache_data from binlog_cache_data
eac5bd2ed4f put binlog_cache_data on a memroot
054e429ef2a always commit for non-trans engines
63fab8bc99f savepoints
b0168a567ce don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
54b5394ef73 don't use start_consistent_snapshot
a645fcddc4f don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
741bdcf64a8 don't do DROP SYSTEM VERSIONING online
eb1411714d4 set read_set early, before row reads
4a5b5249c4c no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
bae5f6a68de online alter always uses ALGORITHM=COPY, LOCK=NONE
b42d2abcb6a remove handler::open_read_view()
a5eea2ad2f0 cleanup
cdd4d4a54e6 support 'alter online table t1 page_checksum=0'
256242f5d34 tests: move around, add new
b7d5ed4e87d ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
d7c841f5ca3 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
d10c2710ba2 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
3459c92ebab ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
3d1f5ae09c6 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
aec6f2a96b3 rpl: repack table_def
5e8fb8b3768 Copy_field: add const to arguments
7c5e90ba17b rename tests
9a16e6380f4 binlog_combinations.inc -> binlog_format_combinations.inc
5d4f1104e2a cleanup: remove vcol_info->stored_in_db
8a4c485f3d7 Fix recalculation of vcols in binlog_row_image=minimal
829267dc91d cleanup: clarify ha_innobase::column_bitmaps_signal()
32b56d79e00 allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

Elena Stepanova made changes - 2023-04-16 11:42

Link

This issue relates to ~~MDEV-31059~~ [ ~~MDEV-31059~~ ]

Elena Stepanova made changes - 2023-04-15 19:03

Link

This issue relates to ~~MDEV-31058~~ [ ~~MDEV-31058~~ ]

Elena Stepanova made changes - 2023-04-12 22:53

Link

This issue relates to ~~MDEV-31043~~ [ ~~MDEV-31043~~ ]

Elena Stepanova made changes - 2023-04-12 14:21

Link

This issue relates to ~~MDEV-30983~~ [ ~~MDEV-30983~~ ]

Elena Stepanova made changes - 2023-04-10 16:51

Link

This issue relates to ~~MDEV-31033~~ [ ~~MDEV-31033~~ ]

Elena Stepanova made changes - 2023-04-03 21:57

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(x) +Buildbot (e599b8b4)+ : ~~MDEV-30987~~ ~~MDEV-30945~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR (e599b8b4)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (x) SAN tests ~~MDEV-30945~~ (MSAN)
** (x) default big / nm + ps
** (off) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
e599b8b4586 ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
7b2e11bd42d ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
e1d1e23b457 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
f9b33ac5703 fix Run-Time Check Failure on Windows
fdd53f6a1e9 fix build
28eaae3acc9 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
01d630af134 fix main.query_cache_innodb and main.alter_table_{online,lock}
6e3f6df7ddc ~~MDEV-29069~~ follow-up: improve DEFAULT rules
7a2138ed897 ~~MDEV-29069~~ follow-up: support partially usable keys
0442a784eba ~~MDEV-29069~~ follow-up: allow non-deterministic DEFAULTs
9a21cbb11aa ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
0a18001b076 cleanup: cache the result of Rows_log_event::find_key()
6c51e27abb0 set table->pos_in_table_list in online alter
40ab76b1d16 cleanup: remove rpl_group_info::get_table_data()
0c1cddfc67b cleanup: ifdefs
fdd0c9e08d8 few rgi assertions. this can proof that rgi is always present
8876e53b845 rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
76cd3bfaad6 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
4af400cf84f log_event.h: remove junk EOL spaces
8bd015193f4 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
3c8111a276f Fix write_set too
148c1ee7dec ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
ea6fb5c2dc5 rpl: check should go after defaults and vcols update
b39b4553229 ~~MDEV-29067~~ Online alter ignores check constraint violation
607ad8d4e7a don't do ALTER IGNORE TABLE online
47ca9f78799 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
48b6d9bff94 Do not ignore sql_mode when replicating
82eea293ba0 Simplify rgi->get_table_data call
4ee10b52bff reorder RPL_TABLE_LIST fields for better packing
2f02da22cd1 ~~MDEV-29021~~ fixup
ed325ef3d44 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
083b6897932 cleanup, remove dead code
88384b40023 ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
63bcd28f492 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
d4d714f0e90 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
571870ae85b remove redundant warnings in RBR and online alter
cce37c33136 cleanup: whitespace, etc
2ecbc7fb5b2 ~~MDEV-28959~~ Online alter ignores strict table mode
f59f1446c0c ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
2cce964647c control Cache_flip_event_log lifetime with reference count
02e80e0bda4 ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
ce3c9945b12 test rename alter_table_online -> alter_table_online_debug
e5de0743266 separate online_alter_cache_data from binlog_cache_data
eac5bd2ed4f put binlog_cache_data on a memroot
054e429ef2a always commit for non-trans engines
63fab8bc99f savepoints
b0168a567ce don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
54b5394ef73 don't use start_consistent_snapshot
a645fcddc4f don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
741bdcf64a8 don't do DROP SYSTEM VERSIONING online
eb1411714d4 set read_set early, before row reads
4a5b5249c4c no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
bae5f6a68de online alter always uses ALGORITHM=COPY, LOCK=NONE
b42d2abcb6a remove handler::open_read_view()
a5eea2ad2f0 cleanup
cdd4d4a54e6 support 'alter online table t1 page_checksum=0'
256242f5d34 tests: move around, add new
b7d5ed4e87d ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
d7c841f5ca3 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
d10c2710ba2 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
3459c92ebab ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
3d1f5ae09c6 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
aec6f2a96b3 rpl: repack table_def
5e8fb8b3768 Copy_field: add const to arguments
7c5e90ba17b rename tests
9a16e6380f4 binlog_combinations.inc -> binlog_format_combinations.inc
5d4f1104e2a cleanup: remove vcol_info->stored_in_db
8a4c485f3d7 Fix recalculation of vcols in binlog_row_image=minimal
829267dc91d cleanup: clarify ha_innobase::column_bitmaps_signal()
32b56d79e00 allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(x) +Buildbot (e599b8b4)+ : ~~MDEV-30987~~ ~~MDEV-30945~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR (e599b8b4)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (x) SAN tests ~~MDEV-30945~~ (MSAN)
** (x) default big / nm + ps
** (off) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}.
_Should be run without debug-sync or on a release build, otherwise too many low-level debug tests fail because they are not hitting expected sync points._

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
e599b8b4586 ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
7b2e11bd42d ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
e1d1e23b457 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
f9b33ac5703 fix Run-Time Check Failure on Windows
fdd53f6a1e9 fix build
28eaae3acc9 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
01d630af134 fix main.query_cache_innodb and main.alter_table_{online,lock}
6e3f6df7ddc ~~MDEV-29069~~ follow-up: improve DEFAULT rules
7a2138ed897 ~~MDEV-29069~~ follow-up: support partially usable keys
0442a784eba ~~MDEV-29069~~ follow-up: allow non-deterministic DEFAULTs
9a21cbb11aa ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
0a18001b076 cleanup: cache the result of Rows_log_event::find_key()
6c51e27abb0 set table->pos_in_table_list in online alter
40ab76b1d16 cleanup: remove rpl_group_info::get_table_data()
0c1cddfc67b cleanup: ifdefs
fdd0c9e08d8 few rgi assertions. this can proof that rgi is always present
8876e53b845 rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
76cd3bfaad6 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
4af400cf84f log_event.h: remove junk EOL spaces
8bd015193f4 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
3c8111a276f Fix write_set too
148c1ee7dec ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
ea6fb5c2dc5 rpl: check should go after defaults and vcols update
b39b4553229 ~~MDEV-29067~~ Online alter ignores check constraint violation
607ad8d4e7a don't do ALTER IGNORE TABLE online
47ca9f78799 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
48b6d9bff94 Do not ignore sql_mode when replicating
82eea293ba0 Simplify rgi->get_table_data call
4ee10b52bff reorder RPL_TABLE_LIST fields for better packing
2f02da22cd1 ~~MDEV-29021~~ fixup
ed325ef3d44 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
083b6897932 cleanup, remove dead code
88384b40023 ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
63bcd28f492 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
d4d714f0e90 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
571870ae85b remove redundant warnings in RBR and online alter
cce37c33136 cleanup: whitespace, etc
2ecbc7fb5b2 ~~MDEV-28959~~ Online alter ignores strict table mode
f59f1446c0c ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
2cce964647c control Cache_flip_event_log lifetime with reference count
02e80e0bda4 ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
ce3c9945b12 test rename alter_table_online -> alter_table_online_debug
e5de0743266 separate online_alter_cache_data from binlog_cache_data
eac5bd2ed4f put binlog_cache_data on a memroot
054e429ef2a always commit for non-trans engines
63fab8bc99f savepoints
b0168a567ce don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
54b5394ef73 don't use start_consistent_snapshot
a645fcddc4f don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
741bdcf64a8 don't do DROP SYSTEM VERSIONING online
eb1411714d4 set read_set early, before row reads
4a5b5249c4c no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
bae5f6a68de online alter always uses ALGORITHM=COPY, LOCK=NONE
b42d2abcb6a remove handler::open_read_view()
a5eea2ad2f0 cleanup
cdd4d4a54e6 support 'alter online table t1 page_checksum=0'
256242f5d34 tests: move around, add new
b7d5ed4e87d ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
d7c841f5ca3 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
d10c2710ba2 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
3459c92ebab ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
3d1f5ae09c6 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
aec6f2a96b3 rpl: repack table_def
5e8fb8b3768 Copy_field: add const to arguments
7c5e90ba17b rename tests
9a16e6380f4 binlog_combinations.inc -> binlog_format_combinations.inc
5d4f1104e2a cleanup: remove vcol_info->stored_in_db
8a4c485f3d7 Fix recalculation of vcols in binlog_row_image=minimal
829267dc91d cleanup: clarify ha_innobase::column_bitmaps_signal()
32b56d79e00 allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

Elena Stepanova made changes - 2023-04-03 19:54

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(x) +Buildbot (e599b8b4)+ : ~~MDEV-30987~~ ~~MDEV-30945~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR (e599b8b4)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (x) SAN tests ~~MDEV-30945~~ (MSAN)
** (x) default big / nm + ps
** (off) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
e599b8b4586 ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
7b2e11bd42d ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
e1d1e23b457 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
f9b33ac5703 fix Run-Time Check Failure on Windows
fdd53f6a1e9 fix build
28eaae3acc9 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
01d630af134 fix main.query_cache_innodb and main.alter_table_{online,lock}
6e3f6df7ddc ~~MDEV-29069~~ follow-up: improve DEFAULT rules
7a2138ed897 ~~MDEV-29069~~ follow-up: support partially usable keys
0442a784eba ~~MDEV-29069~~ follow-up: allow non-deterministic DEFAULTs
9a21cbb11aa ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
0a18001b076 cleanup: cache the result of Rows_log_event::find_key()
6c51e27abb0 set table->pos_in_table_list in online alter
40ab76b1d16 cleanup: remove rpl_group_info::get_table_data()
0c1cddfc67b cleanup: ifdefs
fdd0c9e08d8 few rgi assertions. this can proof that rgi is always present
8876e53b845 rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
76cd3bfaad6 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
4af400cf84f log_event.h: remove junk EOL spaces
8bd015193f4 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
3c8111a276f Fix write_set too
148c1ee7dec ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
ea6fb5c2dc5 rpl: check should go after defaults and vcols update
b39b4553229 ~~MDEV-29067~~ Online alter ignores check constraint violation
607ad8d4e7a don't do ALTER IGNORE TABLE online
47ca9f78799 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
48b6d9bff94 Do not ignore sql_mode when replicating
82eea293ba0 Simplify rgi->get_table_data call
4ee10b52bff reorder RPL_TABLE_LIST fields for better packing
2f02da22cd1 ~~MDEV-29021~~ fixup
ed325ef3d44 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
083b6897932 cleanup, remove dead code
88384b40023 ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
63bcd28f492 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
d4d714f0e90 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
571870ae85b remove redundant warnings in RBR and online alter
cce37c33136 cleanup: whitespace, etc
2ecbc7fb5b2 ~~MDEV-28959~~ Online alter ignores strict table mode
f59f1446c0c ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
2cce964647c control Cache_flip_event_log lifetime with reference count
02e80e0bda4 ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
ce3c9945b12 test rename alter_table_online -> alter_table_online_debug
e5de0743266 separate online_alter_cache_data from binlog_cache_data
eac5bd2ed4f put binlog_cache_data on a memroot
054e429ef2a always commit for non-trans engines
63fab8bc99f savepoints
b0168a567ce don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
54b5394ef73 don't use start_consistent_snapshot
a645fcddc4f don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
741bdcf64a8 don't do DROP SYSTEM VERSIONING online
eb1411714d4 set read_set early, before row reads
4a5b5249c4c no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
bae5f6a68de online alter always uses ALGORITHM=COPY, LOCK=NONE
b42d2abcb6a remove handler::open_read_view()
a5eea2ad2f0 cleanup
cdd4d4a54e6 support 'alter online table t1 page_checksum=0'
256242f5d34 tests: move around, add new
b7d5ed4e87d ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
d7c841f5ca3 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
d10c2710ba2 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
3459c92ebab ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
3d1f5ae09c6 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
aec6f2a96b3 rpl: repack table_def
5e8fb8b3768 Copy_field: add const to arguments
7c5e90ba17b rename tests
9a16e6380f4 binlog_combinations.inc -> binlog_format_combinations.inc
5d4f1104e2a cleanup: remove vcol_info->stored_in_db
8a4c485f3d7 Fix recalculation of vcols in binlog_row_image=minimal
829267dc91d cleanup: clarify ha_innobase::column_bitmaps_signal()
32b56d79e00 allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(x) +Buildbot (e599b8b4)+ : ~~MDEV-30987~~ ~~MDEV-30945~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR (e599b8b4)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (x) SAN tests ~~MDEV-30945~~ (MSAN)
** (x) default big / nm + ps
** (off) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+
- MTR with {{alter-algorithm=copy}}

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
e599b8b4586 ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
7b2e11bd42d ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
e1d1e23b457 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
f9b33ac5703 fix Run-Time Check Failure on Windows
fdd53f6a1e9 fix build
28eaae3acc9 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
01d630af134 fix main.query_cache_innodb and main.alter_table_{online,lock}
6e3f6df7ddc ~~MDEV-29069~~ follow-up: improve DEFAULT rules
7a2138ed897 ~~MDEV-29069~~ follow-up: support partially usable keys
0442a784eba ~~MDEV-29069~~ follow-up: allow non-deterministic DEFAULTs
9a21cbb11aa ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
0a18001b076 cleanup: cache the result of Rows_log_event::find_key()
6c51e27abb0 set table->pos_in_table_list in online alter
40ab76b1d16 cleanup: remove rpl_group_info::get_table_data()
0c1cddfc67b cleanup: ifdefs
fdd0c9e08d8 few rgi assertions. this can proof that rgi is always present
8876e53b845 rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
76cd3bfaad6 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
4af400cf84f log_event.h: remove junk EOL spaces
8bd015193f4 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
3c8111a276f Fix write_set too
148c1ee7dec ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
ea6fb5c2dc5 rpl: check should go after defaults and vcols update
b39b4553229 ~~MDEV-29067~~ Online alter ignores check constraint violation
607ad8d4e7a don't do ALTER IGNORE TABLE online
47ca9f78799 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
48b6d9bff94 Do not ignore sql_mode when replicating
82eea293ba0 Simplify rgi->get_table_data call
4ee10b52bff reorder RPL_TABLE_LIST fields for better packing
2f02da22cd1 ~~MDEV-29021~~ fixup
ed325ef3d44 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
083b6897932 cleanup, remove dead code
88384b40023 ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
63bcd28f492 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
d4d714f0e90 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
571870ae85b remove redundant warnings in RBR and online alter
cce37c33136 cleanup: whitespace, etc
2ecbc7fb5b2 ~~MDEV-28959~~ Online alter ignores strict table mode
f59f1446c0c ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
2cce964647c control Cache_flip_event_log lifetime with reference count
02e80e0bda4 ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
ce3c9945b12 test rename alter_table_online -> alter_table_online_debug
e5de0743266 separate online_alter_cache_data from binlog_cache_data
eac5bd2ed4f put binlog_cache_data on a memroot
054e429ef2a always commit for non-trans engines
63fab8bc99f savepoints
b0168a567ce don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
54b5394ef73 don't use start_consistent_snapshot
a645fcddc4f don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
741bdcf64a8 don't do DROP SYSTEM VERSIONING online
eb1411714d4 set read_set early, before row reads
4a5b5249c4c no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
bae5f6a68de online alter always uses ALGORITHM=COPY, LOCK=NONE
b42d2abcb6a remove handler::open_read_view()
a5eea2ad2f0 cleanup
cdd4d4a54e6 support 'alter online table t1 page_checksum=0'
256242f5d34 tests: move around, add new
b7d5ed4e87d ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
d7c841f5ca3 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
d10c2710ba2 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
3459c92ebab ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
3d1f5ae09c6 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
aec6f2a96b3 rpl: repack table_def
5e8fb8b3768 Copy_field: add const to arguments
7c5e90ba17b rename tests
9a16e6380f4 binlog_combinations.inc -> binlog_format_combinations.inc
5d4f1104e2a cleanup: remove vcol_info->stored_in_db
8a4c485f3d7 Fix recalculation of vcols in binlog_row_image=minimal
829267dc91d cleanup: clarify ha_innobase::column_bitmaps_signal()
32b56d79e00 allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

Elena Stepanova made changes - 2023-04-03 19:48

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(x) +Buildbot (e599b8b4)+ : ~~MDEV-30987~~ ~~MDEV-30945~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR (e599b8b4)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (x) SAN tests ~~MDEV-30945~~ (MSAN)
** (x) default big / nm + ps
** (off) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(x) +Buildbot (e599b8b4)+ : ~~MDEV-30987~~ ~~MDEV-30945~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR (e599b8b4)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (x) SAN tests ~~MDEV-30945~~ (MSAN)
** (x) default big / nm + ps
** (off) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
e599b8b4586 ~~MDEV-30924~~ Server crashes in MYSQL_LOG::is_open upon ALTER vs FUNCTION
7b2e11bd42d ~~MDEV-30925~~ Assertion failed in translog_write_record in ONLINE ALTER + Aria
e1d1e23b457 ~~MDEV-30902~~ Server crash in LEX::first_lists_tables_same
f9b33ac5703 fix Run-Time Check Failure on Windows
fdd53f6a1e9 fix build
28eaae3acc9 ~~MDEV-29068~~ Cascade foreign key updates do not apply in online alter
01d630af134 fix main.query_cache_innodb and main.alter_table_{online,lock}
6e3f6df7ddc ~~MDEV-29069~~ follow-up: improve DEFAULT rules
7a2138ed897 ~~MDEV-29069~~ follow-up: support partially usable keys
0442a784eba ~~MDEV-29069~~ follow-up: allow non-deterministic DEFAULTs
9a21cbb11aa ~~MDEV-29069~~ ER_KEY_NOT_FOUND on online autoinc addition + concurrent DELETE
0a18001b076 cleanup: cache the result of Rows_log_event::find_key()
6c51e27abb0 set table->pos_in_table_list in online alter
40ab76b1d16 cleanup: remove rpl_group_info::get_table_data()
0c1cddfc67b cleanup: ifdefs
fdd0c9e08d8 few rgi assertions. this can proof that rgi is always present
8876e53b845 rename rpl/rpl_alter_instant -> rpl/rpl_alter_innodb
76cd3bfaad6 ~~MDEV-29038~~ XA assertions failing in binlog_rollback and binlog_commit
4af400cf84f log_event.h: remove junk EOL spaces
8bd015193f4 ~~MDEV-29013~~ ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique
3c8111a276f Fix write_set too
148c1ee7dec ~~MDEV-28816~~ Assertion `wsrep_thd_is_applying(thd)' failed in int wsrep_ignored_error_code(Log_event*, int)
ea6fb5c2dc5 rpl: check should go after defaults and vcols update
b39b4553229 ~~MDEV-29067~~ Online alter ignores check constraint violation
607ad8d4e7a don't do ALTER IGNORE TABLE online
47ca9f78799 ~~MDEV-29021~~ add test case from ~~MDEV-29013~~
48b6d9bff94 Do not ignore sql_mode when replicating
82eea293ba0 Simplify rgi->get_table_data call
4ee10b52bff reorder RPL_TABLE_LIST fields for better packing
2f02da22cd1 ~~MDEV-29021~~ fixup
ed325ef3d44 ~~MDEV-29021~~ ALTER TABLE fails when a stored virtual column is dropped+added
083b6897932 cleanup, remove dead code
88384b40023 ~~MDEV-28943~~ Online alter fails under LOCK TABLE with ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
63bcd28f492 ~~MDEV-28930~~ ALTER TABLE Deadlocks with parallel TL_WRITE
d4d714f0e90 ~~MDEV-28967~~ Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog`
571870ae85b remove redundant warnings in RBR and online alter
cce37c33136 cleanup: whitespace, etc
2ecbc7fb5b2 ~~MDEV-28959~~ Online alter ignores strict table mode
f59f1446c0c ~~MDEV-28944~~ XA assertions failing in binlog_rollback and binlog_commit
2cce964647c control Cache_flip_event_log lifetime with reference count
02e80e0bda4 ~~MDEV-28771~~ Assertion `table->in_use&&tdc->flushed' failed after ALTER
ce3c9945b12 test rename alter_table_online -> alter_table_online_debug
e5de0743266 separate online_alter_cache_data from binlog_cache_data
eac5bd2ed4f put binlog_cache_data on a memroot
054e429ef2a always commit for non-trans engines
63fab8bc99f savepoints
b0168a567ce don't crash if ALTER TABLE fails and a long transaction blocks MDL upgrade
54b5394ef73 don't use start_consistent_snapshot
a645fcddc4f don't copy stmt IO_CACHE to trx IO_CACHE at the stmt end
741bdcf64a8 don't do DROP SYSTEM VERSIONING online
eb1411714d4 set read_set early, before row reads
4a5b5249c4c no ALTER TABLE should return ER_NO_DEFAULT_FOR_FIELD
bae5f6a68de online alter always uses ALGORITHM=COPY, LOCK=NONE
b42d2abcb6a remove handler::open_read_view()
a5eea2ad2f0 cleanup
cdd4d4a54e6 support 'alter online table t1 page_checksum=0'
256242f5d34 tests: move around, add new
b7d5ed4e87d ~~MDEV-16329~~ [5/5] ALTER ONLINE TABLE
d7c841f5ca3 ~~MDEV-16329~~ [4/5] Refactor MYSQL_BIN_LOG: extract Event_log ancestor
d10c2710ba2 ~~MDEV-16329~~ [3/5] use binlog_cache_data directly in most places
3459c92ebab ~~MDEV-16329~~ [2/5] refactor binlog and cache_mngr
3d1f5ae09c6 ~~MDEV-16329~~ [1/5] add THD::binlog_get_cache_mngr
aec6f2a96b3 rpl: repack table_def
5e8fb8b3768 Copy_field: add const to arguments
7c5e90ba17b rename tests
9a16e6380f4 binlog_combinations.inc -> binlog_format_combinations.inc
5d4f1104e2a cleanup: remove vcol_info->stored_in_db
8a4c485f3d7 Fix recalculation of vcols in binlog_row_image=minimal
829267dc91d cleanup: clarify ha_innobase::column_bitmaps_signal()
32b56d79e00 allow random_bytes() in virtual columns
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

Elena Stepanova made changes - 2023-04-03 19:46

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.

*Testing*

(x) +Buildbot (e599b8b4)+ : ~~MDEV-30987~~ ~~MDEV-30945~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR (e599b8b4)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (x) SAN tests ~~MDEV-30945~~ (MSAN)
** (x) default big / nm + ps
** (off) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

- Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.
- (!) Exceptions (limitations) need to be enumerated and documented

*Testing*

(x) +Buildbot (e599b8b4)+ : ~~MDEV-30987~~ ~~MDEV-30945~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR (e599b8b4)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (x) SAN tests ~~MDEV-30945~~ (MSAN)
** (x) default big / nm + ps
** (off) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

Elena Stepanova made changes - 2023-04-03 19:45

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.

*Testing*

(x) +Buildbot (e599b8b4)+ : ~~MDEV-30987~~ ~~MDEV-30945~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR (e599b8b4)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (x) SAN tests ~~MDEV-30945~~ (MSAN)
** (x) default big / nm + ps
** (off) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(?) +Replication+
- (?) RBR
- (?) SBR

(?) +Larger integration+
- (?) Galera
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.

*Testing*

(x) +Buildbot (e599b8b4)+ : ~~MDEV-30987~~ ~~MDEV-30945~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR (e599b8b4)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (x) SAN tests ~~MDEV-30945~~ (MSAN)
** (x) default big / nm + ps
** (off) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(off) +Replication+
Outside the scope of this item, done in ~~MDEV-28809~~
- (off) RBR
- (off) SBR

(?) +Larger integration+
- (off) Galera
outside the scope of this item, done in ~~MDEV-27986~~
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

Elena Stepanova made changes - 2023-04-03 19:43

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.

*Testing*

(x) +Buildbot (e599b8b4)+ : ~~MDEV-30987~~ ~~MDEV-30945~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +MTR (e599b8b4)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]

- (x) SAN tests ~~MDEV-30945~~ (MSAN)
** (/) default big / nm + ps
** (off) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(?) +Replication+
- (?) RBR
- (?) SBR

(?) +Larger integration+
- (?) Galera
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.

*Testing*

(x) +Buildbot (e599b8b4)+ : ~~MDEV-30987~~ ~~MDEV-30945~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +Extra MTR (e599b8b4)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]
- (x) SAN tests ~~MDEV-30945~~ (MSAN)
** (x) default big / nm + ps
** (off) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(?) +Replication+
- (?) RBR
- (?) SBR

(?) +Larger integration+
- (?) Galera
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

Elena Stepanova made changes - 2023-04-03 19:42

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.

*Testing*

(x) +Buildbot (e599b8b4)+ : ~~MDEV-30987~~ ~~MDEV-30945~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +MTR (e599b8b4)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]

- (x) SAN tests ~~MDEV-30945~~ (MSAN)
** (/) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(?) +Replication+
- (?) RBR
- (?) SBR

(?) +Larger integration+
- (?) Galera
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.

*Testing*

(x) +Buildbot (e599b8b4)+ : ~~MDEV-30987~~ ~~MDEV-30945~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +MTR (e599b8b4)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]

- (x) SAN tests ~~MDEV-30945~~ (MSAN)
** (/) default big / nm + ps
** (off) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(?) +Replication+
- (?) RBR
- (?) SBR

(?) +Larger integration+
- (?) Galera
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

Elena Stepanova made changes - 2023-04-03 19:40

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.

*Testing*

(x) +Buildbot (e599b8b4)+ : ~~MDEV-30987~~ ~~MDEV-30945~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +MTR (e599b8b4)+
- (?) GCOV for the patch
- (x) SAN tests ~~MDEV-30945~~ (MSAN)
** (/) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(?) +Replication+
- (?) RBR
- (?) SBR

(?) +Larger integration+
- (?) Galera
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.

*Testing*

(x) +Buildbot (e599b8b4)+ : ~~MDEV-30987~~ ~~MDEV-30945~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +MTR (e599b8b4)+
- (?) GCOV for the patch
default set + galera set, nm-big run: Lines with zero coverage (58/1850) [^missings-mtr-e599b8b4.txt]

- (x) SAN tests ~~MDEV-30945~~ (MSAN)
** (/) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(?) +Replication+
- (?) RBR
- (?) SBR

(?) +Larger integration+
- (?) Galera
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

Elena Stepanova made changes - 2023-04-03 19:39

Attachment		missings-mtr-e599b8b4.txt [ 69370 ]
Attachment		coverage-mtr-e599b8b4.txt [ 69371 ]
Attachment		oalter-e599b8b4.diff [ 69372 ]

Elena Stepanova made changes - 2023-04-03 18:09

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.

*Testing*

(x) +Buildbot+ : ~~MDEV-30987~~ ~~MDEV-30945~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +MTR+
- (?) GCOV for the patch
- (?) SAN tests
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(?) +Replication+
- (?) RBR
- (?) SBR

(?) +Larger integration+
- (?) Galera
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.

*Testing*

(x) +Buildbot (e599b8b4)+ : ~~MDEV-30987~~ ~~MDEV-30945~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +MTR (e599b8b4)+
- (?) GCOV for the patch
- (x) SAN tests ~~MDEV-30945~~ (MSAN)
** (/) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(?) +Replication+
- (?) RBR
- (?) SBR

(?) +Larger integration+
- (?) Galera
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

Elena Stepanova made changes - 2023-04-02 18:08

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.

*Testing*

(x) +Buildbot+ : ~~MDEV-30987~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +MTR+
- (?) GCOV for the patch
- (?) SAN tests
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(?) +Replication+
- (?) RBR
- (?) SBR

(?) +Larger integration+
- (?) Galera
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.

*Testing*

(x) +Buildbot+ : ~~MDEV-30987~~ ~~MDEV-30945~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +MTR+
- (?) GCOV for the patch
- (?) SAN tests
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(?) +Replication+
- (?) RBR
- (?) SBR

(?) +Larger integration+
- (?) Galera
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

Elena Stepanova made changes - 2023-04-02 17:59

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.

*Testing*

(x) +Buildbot+
- ~~MDEV-30987~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +MTR+
- (?) GCOV for the patch
- (?) SAN tests
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(?) +Replication+
- (?) RBR
- (?) SBR

(?) +Larger integration+
- (?) Galera
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.

*Testing*

(x) +Buildbot+ : ~~MDEV-30987~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +MTR+
- (?) GCOV for the patch
- (?) SAN tests
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(?) +Replication+
- (?) RBR
- (?) SBR

(?) +Larger integration+
- (?) Galera
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

Elena Stepanova made changes - 2023-04-02 17:59

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.

*Testing*

(?) +Buildbot+
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +MTR+
- (?) GCOV for the patch
- (?) SAN tests
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(?) +Replication+
- (?) RBR
- (?) SBR

(?) +Larger integration+
- (?) Galera
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.

*Testing*

(x) +Buildbot+
- ~~MDEV-30987~~
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +MTR+
- (?) GCOV for the patch
- (?) SAN tests
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(?) +Replication+
- (?) RBR
- (?) SBR

(?) +Larger integration+
- (?) Galera
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

Elena Stepanova made changes - 2023-04-02 17:58

Link

This issue relates to ~~MDEV-30987~~ [ ~~MDEV-30987~~ ]

Elena Stepanova made changes - 2023-04-02 16:40

Description

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.

*Testing*

(?) +Buildbot+
- (?) [old BB - <branch>|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB - <branch>|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +MTR+
- (?) GCOV for the patch
- (?) SAN tests
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(?) +Replication+
- (?) RBR
- (?) SBR

(?) +Larger integration+
- (?) Galera
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished

*JIRA, Info*

~~MDEV-16329~~ Engine-independent online ALTER TABLE

*Github*

Branch: [bb-11.0-oalter|https://github.com/MariaDB/server/tree/bb-11.0-oalter]

Last commit in testing: [e599b8b4|https://github.com/MariaDB/server/commit/e599b8b45867cd24720a9ecd4ba28500f579c5a4]
Baseline: [799f7595 (still 11.0)|https://github.com/MariaDB/server/commit/799f75953f5b362736a2992fd5d92a14db94bb20]

(?) +Specification/documentation notes+

Despite all implications in ~~MDEV-16329~~, {color:red}*online {{ALTER TABLE}} is not the same as {{ALTER ONLINE TABLE}}*{color}. The explicit syntax is
{code:sql}
ALTER TABLE .. ALGORITHM=COPY, LOCK=NONE
{code}
In many cases it should happen automatically, on {{ALTER TABLE}} without ALGORITHM/LOCK specification.

*Testing*

(?) +Buildbot+
- (?) [old BB|http://buildbot.mariadb.net/buildbot/grid?category=main&branch=bb-11.0-oalter]
- (?) [new BB|https://buildbot.mariadb.org/#/grid?branch=bb-11.0-oalter]
- (?) Cross-reference for the commit in testing: [cross-reference|http://buildbot.askmonty.org/ci/reports/cross_reference#branch=&revision=e599b8b45867cd24720a9ecd4ba28500f579c5a4&platform=&fail_name=&fail_variant=&fail_info_full=&typ=&info=&dt=&limit=1000&fail_info_short=]

(?) +Code review notes+

(?) +MTR+
- (?) GCOV for the patch
- (?) SAN tests
** (?) default big / nm + ps
** (?) plugins big/nm

(?) +Manual Testing+
- (?) By spec/exploratory
- (?) integration map traverse

(?) +Crash recovery+

(?) +Replication+
- (?) RBR
- (?) SBR

(?) +Larger integration+
- (?) Galera
- (?) Columnstore
- (?) MariaBackup
- (?) Protocol
- (?) Connectors

(?) +Upgrade+
- (?) Live upgrade
- (?) Dump upgrade
- (?) Package upgrade
- (?) MariaBackup upgrade
- (?) OM => NS replication upgrade
- (?) Galera rolling upgrade

(?) +Random+
- (?) RQG GCOV for the patch
- (?) RQG-100 ASAN
- (?) RQG-100 Release

(?) +Compatibility+
- (?) Windows
- (?) ARM
- (?) FreeBSD

(?) +Misc attention points+

(?) +Performance considerations+

(?) +Updates/additions to non-MTR regression tests+

*Branch commits* ({{feature / \-\-\- unrelated}})
{noformat}
{noformat}

----
Legend:
(?) Some work still needs to be done on the item (continue testing, re-check after fixing, etc.)
(off) No work expected on the item
(x) The item is a problem which is currently considered a blocker for the feature
(flag) The item cannot be worked on for the time being
(!) Attention point but not a blocker for the feature (low-prio bug, documentation point, unfinished check, etc.)
(/) Work on the item is considered finished