Details
-
Task
-
Status: Open (View Workflow)
-
Major
-
Resolution: Unresolved
-
None
-
None
Description
Problem
A replica applying row-based events (Delete_rows / Update_rows) to a DuckDB table stops with:
[ERROR] Slave SQL: Could not execute Delete\_rows\_v1 event on table test.invoice; Storage engine DUCKDB of the table {{test}}.{{invoice}} doesn't have this option, Error\_code: 1031; handler error HA\_ERR\_WRONG\_COMMAND
|
Root cause: Rows_log_event::find_key() unconditionally selects a unique PK for the row lookup (the HA_NOSAME branch does not consult index_flags()), and use_pk_position() is false for DuckDB, so find_row() calls ha_index_read_map() which is a HA_ERR_WRONG_COMMAND stub in the DuckDB handler. rnd_pos() is a stub as well, and there is no fallback to a table scan in this code path.
Reproduced with a two-server MTR setup (master-slave.inc, binlog_format=MIXED): the master survives DELETE FROM (with the MDEV-40957 fix), but the replica SQL thread aborts with error 1031 on the first Delete_rows event.
Proposed fix: positioning stubs on the replica (read-free apply)
find_row() unpacks the event's before-image into record[0] before performing the lookup; the lookup merely overwrites record[0] with the row read from the table. If the positioning call is a no-op returning success, record[0] keeps the before-image from the event, which is sufficient for the DuckDB handler:
- DELETE: ha_delete_row(record[0]) identifies the row by PK taken from the before-image;
- UPDATE: the server saves record[0] to record[1], unpacks the after-image into record[0], and ha_update_row(record[1], record[0]) gets the old PK and the complete new row.
Implementation sketch (engine-side only, no server changes):
Add HA_PRIMARY_KEY_REQUIRED_FOR_POSITION to ha_duckdb::table_flags() so that use_pk_position() becomes true and find_row() takes the ha_rnd_pos_by_record(record[0]) path instead of ha_index_read_map().
Override ha_duckdb::rnd_pos_by_record() to return 0 without touching record[0], guarded so that the fake success is only returned in the replication applier context (rgi_slave); any other caller (filesort, multi-table UPDATE/DELETE, window functions) must keep getting HA_ERR_WRONG_COMMAND to avoid silent data corruption.
This mirrors what AliSQL does server-side for its DuckDB engine: a dedicated ROW_LOOKUP_DUCKDB algorithm whose do_duckdb_update() unpacks the before-image and calls do_apply_row() directly, with index_read_map()/rnd_pos() remaining stubs. The stub-based approach achieves the same without patching sql/ (similar in spirit to RocksDB read-free replication).
Constraints and trade-offs
- binlog_row_image=FULL is required: with MINIMAL the after-image is incomplete and the delete+insert update path would destroy non-updated columns. The engine must detect an incomplete row image and fail the apply with a clear error.
- A primary key is required (duckdb_require_primary_key is already ON by default).
- Semantics become idempotent rather than strict: a DELETE for a non-existent row silently affects 0 rows, an UPDATE re-inserts the row; master/replica drift is no longer detected by the applier. This matches RocksDB read-free replication behaviour and must be documented.
Test
Extend the replication MTR test from MDEV-40957 (two servers, DuckDB table with UUID PK, INSERT + DELETE FROM under MIXED and ROW binlog formats) so that both master and replica stay healthy and the replica row counts match the master.
Attachments
Issue Links
- relates to
-
MDEV-40957 Crash with DELETE FROM in DuckDB table and incorrect mixed DML processing
-
- Needs Feedback
-