# MDEV-39143 — Replicate binary JSON from MySQL to MariaDB — Design ## 1. Summary Enable a MariaDB **slave** to apply row events from a MySQL **master** that contain JSON columns, both as **full documents** and as **partial (diff) updates**. The feature reuses two existing capabilities — the slave's row-event type-conversion path and the read-only `mysql_json` plugin — and adds a new path for MySQL's partial-JSON update events. Direction: MySQL master → MariaDB slave, **ROW** binlog format. (STATEMENT format already replicates JSON as SQL text.) ## 2. Background MySQL and MariaDB both have support for JSON data but they store JSON documents differently: MySQL employs a binary JSON format while MariaDB stores JSON documents as text. - **MySQL's binary JSON (JSONB)**: MySQL's JSON column type stores documents in a binary encoding rather than as text. It is designed to avoid re-parsing text on every read. Its main features are as follows: it is self-describing, meaning that no separate schema is needed to decode a document; it allows direct member/element lookup without a full scan and it features sorted keys for efficient binary search. The actual grammar is in the source (sql-common/json_binary.h), which MySQL does mirror publicly via its Doxygen build. And it's the same format used for a single value inside a Json_diff in MySQL's partial-JSON replication protocol — a diff doesn't invent its own value encoding, it just carries one JSONB value per REPLACE/INSERT operation. - **MariaDB's JSON.** MariaDB does not have a distinct storage type for JSON data. Instead, a `JSON` column is `longtext CHARACTER SET utf8mb4` with an implicit `CHECK(json_valid(...))`, recognised by a family of `Type_handler`s over `Field_blob` (`Type_handler_json_common::is_json_type_handler()`). Documents are stored and re-validated as text; there is no parsed or binary on-disk form. This feature is based on two existing MariaDB mechanisms: mysql_json plugin and the slave conversion table. The myslq_json plugin provides MariaDB read-only support for MySQL JSON. With the slave comversion table a slave can convert data types when applying row events from the master. - **The `mysql_json` plugin** (`plugin/type_mysql_json/`) supplies a read-only `Type_handler_mysql_json` for type 245, so the server can name and decode such a column without a native handler. It cannot be used to create a column from SQL. `Field_mysql_json::val_str()` decodes a JSONB value to text via `parse_mysql_json_value()`, covering the full type set of the previous paragraph including OPAQUE; a value it cannot parse yields an empty string and `ER_UNKNOWN_ERROR`, since the plugin has no way to repair a corrupt document. Before this feature, its only caller was `.frm` open for an on-disk MySQL table (`sql/table.cc:2593`); it was not wired into replication. - **The slave conversion table.** When a row event's column type differs from the slave's, `unpack_row()` cannot write directly into the target field. `table_def::create_conversion_table()` instead builds a temporary `TABLE` typed to the master's columns; unpacking writes into that table, and `convert_field()` copies from it into the real field via `Copy_field`, which performs the type conversion. This mechanism predates this feature and is unchanged by it (§4.1); only diff application (§4.2) required new code, since a diff is not a value the copy step can consume. ## 3. Scope In scope: full-document JSON via `WRITE_ROWS`/`UPDATE_ROWS`/`DELETE_ROWS` (event types 30/31/32), and partial JSON via `PARTIAL_UPDATE_ROWS_EVENT` (39), the latter essential per the original requirement. Direction is MySQL master to MariaDB slave; the reverse direction is not addressed. Out of scope: STATEMENT format, which already replicates JSON as SQL text; and non-JSON heterogeneous type conversions, whose existing behaviour is unchanged. ## 4. Architecture — two independent paths This feature is implemented by two independent code paths: a full-document path is executed for a INSERT, DELETE and non-partial UPDATE of a JSON column, and a partial UPDATE path is followed when a partial UPDATE is performed on a JSON column. Both code paths share the `mysql_json` binary→text decoder and MariaDB's JSON engine. ### 4.1 Full-document path (INSERT / DELETE / non-partial UPDATE) The changes for the full-document path are very simple and small as the existing machinery for converting tables can be used. Only two edits enable the entire existing conv-table machinery: 1. **`table_def::field_type_handler()`** (`rpl_utility_server.cc`): map binlog type `245` → the `MYSQL_JSON` plugin handler (mirror `table.cc`). Today it returns `NULL` → `SLAVE_FIELD_UNKNOWN_TYPE`. 2. **`Field_longstr::rpl_conv_type_from()`** (`rpl_utility_server.cc`): accept a `type_handler_mysql_json` source. Today it falls through to `CONV_TYPE_IMPOSSIBLE` because the plugin handler is not in the accepted-source list. Return: - **`CONV_TYPE_VARIANT`** when the slave column is native JSON (`is_json_type_handler(type_handler())`), and - **`CONV_TYPE_SUBSET_TO_SUPERSET`** when the slave column is a plain string/blob (longtext/varchar). Supporting plumbing (needed because 245 isn't a enum_field_types member — see the MYSQL_TYPE_JSON_MYSQL note below) 3. calc_field_size() (rpl_utility.cc:35) — a case MYSQL_TYPE_JSON_MYSQL: alongside MYSQL_TYPE_BLOB etc., so the unpack code knows how many bytes a JSON column's raw value occupies (length-prefixed like a blob). Also changed the switch to cast to (uint), since 245 isn't a legal enum_field_types case label. 4. Metadata parsing in table_def::table_def() (rpl_utility.cc) — a matching case reading the one metadata byte (the pack length) that a JSON column carries in the binlog's type-definition block, same (uint) cast for the same reason. 5. The MYSQL_TYPE_JSON_MYSQL constant (rpl_utility.h) — #define MYSQL_TYPE_JSON_MYSQL 245, since 245 has no name in enum_field_types and collides with MYSQL_TYPE_VIRTUAL in MariaDB's own pre-10.2 .frm encoding. Used by all of the above. 6. The missing-plugin diagnostic (rpl_utility_server.cc) — when field_type_handler() returns NULL because mysql_json isn't installed, give_compatibility_error() now names the plugin and the INSTALL SONAME needed, instead of "unknown field type 245". Everything downstream already exists: - `create_conversion_table()` → `conv_table->add(handler, metadata, target)` → `Type_handler_mysql_json::make_conversion_table_field()` builds a `Field_mysql_json` (adopts the target charset). - `convert_field()` → `Copy_field::set(result, conv, save=TRUE)`; the JSON/longtext target has `BLOB_FLAG`, so `do_save_blob` (`field_conv.cc`) calls `from_field->val_str()` → `Field_mysql_json::val_str()` **decodes JSONB → text** and stores it. No new copy code needed. ### 4.2 Partial path (`PARTIAL_UPDATE_ROWS_EVENT`) The partial path is executed at the slave when it encounters a MySQL JSON column update specified as a diff (PARTIAL_UPDATE_ROWS_EVENT). The MariaDB slave decodes binary JSON diff vector specifying the changes to the JSON document and employs a sequence of JSON_REPLACE, JSON_REMOVE and JSON_INSERT or JSON_ARRAY_INSERT operations to update the JSON column value. MySQL logs partial JSON updates as a **distinct event type** (39), and only the UPDATE **after-image** can carry diffs. The partial path requires code changes in three plaves, which all reside on the unpack side. **1. Accept the event.** `read_log_event_no_checksum()` builds an ordinary `Update_rows_log_event` for type 39 — the event structure is identical, only the payload differs. The case must sit inside the `#if defined(HAVE_REPLICATION)` region, because that constructor exists only there; otherwise the embedded server fails to build. `LOG_EVENT_IS_UPDATE_ROW()` recognises the type, and `Rows_log_event::is_partial_json_update()` reports it, so `unpack_current_row()` can pass ```c is_partial_json_update() && cols == &m_cols_ai ``` as `unpack_row()`'s `is_partial_json_after_image`. Gating on `m_cols_ai` is what keeps the before-image — which has no header and never carries diffs — on the ordinary path. **2. Read the after-image header.** When that flag is set, `unpack_row()` reads `value_options` as a net field length and, if `PARTIAL_JSON_UPDATES` is set, takes `(json_column_count() + 7) / 8` bytes of `partial_bits` before the null bits. `json_column_count()` counts JSON columns in the **master's** type array. While walking columns, a JSON column whose `partial_bits` bit is set is routed to the diff path; every other column unpacks normally. The bitmap is per row, so the same column can be a full value in one row and a diff in the next. **3. Decode and apply in one pass.** `apply_partial_json_column()` replaces the usual unpack-then-`convert_field()` for that column, because a diff is not a final value and there is nothing to copy out of the conversion table. It: - decodes the diff vector with `read_mysql_json_diff_vector()`, advancing the caller's read cursor past it; - reads the base document from the **before-image** in `record[1]`, reached by temporarily shifting the field (`move_field_offset()`) — the matched row is already there at unpack time, which is what makes the single pass possible; - folds the diffs in order through MariaDB's own JSON functions, each result feeding the next: `REPLACE` → `JSON_REPLACE`, `REMOVE` → `JSON_REMOVE`, and `INSERT` → `JSON_INSERT` or `JSON_ARRAY_INSERT` chosen from the last leg of the path (§8.2); - decodes each diff's binary-JSON payload by storing it into the conv-table `Field_mysql_json` and reading it back as text, wrapped in `JSON_COMPACT()` so it is inserted as a JSON value rather than a quoted string; - stores the resulting text into the slave field. The Items are built on a private `THD::free_list` released per column, so their buffers do not accumulate to the end of the statement. ## 5. Conversion gating The decision of whether a conversion is allowed to happen (or conversion gating) is driven entirely by the `enum_conv_type` returned in §4.1 (see `is_conversion_ok`): - **JSON → JSON**: `CONV_TYPE_VARIANT` ⇒ always allowed, **no `slave_type_conversions` needed**. Rationale: same logical type, lossless representation change; matches the temporal-precision `VARIANT` precedent. - **JSON → plain string/blob**: `CONV_TYPE_SUBSET_TO_SUPERSET` ⇒ requires `slave_type_conversions=ALL_NON_LOSSY`; preserves the strict-mode contract. ## 6. Wire formats (reference) In addition to MySQL's binary JSON document format (§2), MySQL defines two wire formats for partial JSON replication: one for the PARTIAL_UPDATE_ROWS_EVENT after-image header, and another for the Json_diff_vector and its per-diff layout. Both formats are specified in prose by MySQL [WL#2955 "RBR replication of partial JSON updates"](https://dev.mysql.com/worklog/task/?id=2955), §4 F2 (the `shared-image`: `value_options` + `partial_columns`) and §4 F3 (the diff vector and the per-diff layout). The formats were verified against MySQL 9.2 sources (`sql/rpl_record.cc` `start_partial_bit_reader()`, `sql-common/json_diff.cc` `read_binary()` and `read_json_diff()`), which still match the 8.0-era worklog. ### 6.1 Notation `net_field_length` is MySQL's variable-length integer (`safe_net_field_length_ll()` in `sql-common/pack.c`): a first byte below 251 *is* the value; 252 introduces a 2-byte value, 253 a 3-byte value, 254 an 8-byte value; 251 means SQL NULL and does not occur in these fields. So every length below is one byte in the common case and silently wider for large documents — it must be read with the bounds-checked helper, never assumed to be one byte. Multi-byte integers are little-endian. ### 6.2 Partial UPDATE after-image Only the **after-image** of a `PARTIAL_UPDATE_ROWS_EVENT` (type 39) carries this header. The before-image, and every image of every other event type, is unchanged from ordinary row format — which is why `unpack_row()` takes an explicit `is_partial_json_after_image` flag rather than deciding for itself. ``` [ value_options : net_field_length ] bitmap; only bit 0 (PARTIAL_JSON_UPDATES) is defined [ partial_bits : ceil(json_column_count/8) ] present only if that bit is set [ null_bits : ceil(image_cols/8) ] ordinary row-image null bits [ column values ... ] ``` Three details decide whether the row parses correctly: - **`partial_bits` precedes `null_bits`.** Getting the order or the size wrong does not fail cleanly — it shifts every subsequent field and the row decodes into nonsense. - **`partial_bits` is sized from the number of JSON columns in the _source_ table**, not from the columns present in this row image and not from the slave's table. A JSON column that is absent from the image, or NULL, or missing on the slave, still occupies its bit. Hence `table_def::json_column_count()` counts over the master's type array. - **`value_options` is absent entirely** when the event is not type 39. MySQL rejects a value greater than 1 as a corrupt event, since no other bit is defined. A column's value is a diff vector rather than a full value if and only if all of: the event is type 39, this is the after-image, `value_options & PARTIAL_JSON_UPDATES`, the column's source type is JSON, and its `partial_bits` bit is set. Any JSON column failing the last test carries a normal full value in the same row — the bitmap is per row, so a column can alternate between the two across consecutive rows of one event. ### 6.3 `Json_diff_vector` The payload of one partial JSON column. ``` [ length : 4 bytes ] byte count of the diffs that follow; excludes itself [ diff_1 ] … [ diff_N ] ``` The diff **count is not stored**: decoding reads diffs until exactly `length` bytes are consumed. A vector is therefore self-delimiting, and a corrupt `length` is the one error that cannot be detected from the vector alone — it must be bounds-checked against the end of the row image by the caller. Note the asymmetry: this outer length is a plain 4-byte integer (`uint4korr`), while every length *inside* a diff is a `net_field_length`. ### 6.4 Each diff ``` [ operation : 1 byte ] 0=REPLACE, 1=INSERT, 2=REMOVE [ path_length : net_field_length ] [ path : path_length bytes ] JSON path, as TEXT if operation != REMOVE: [ value_length : net_field_length ] [ value : value_length bytes ] MySQL binary JSON (JSONB) ``` - **`value_length` and `value` are omitted if and only if the operation is REMOVE.** There is no placeholder — the next diff starts immediately after the path. - **The path is text**, in the same syntax MariaDB's `json_path_setup()` accepts, so it needs no translation to be *parsed*. It is always a single location: no wildcards, no ranges. (Resolution is a separate matter — see §8.2 on `[last-N]`.) - **The value is binary JSONB**, not text, and must be decoded through the `mysql_json` plugin exactly like a full document. - **`operation` is validated against the known three.** MySQL treats anything else as a corrupt event. - **The operations are not independent**: they apply in order to the result of the previous one, so array indices in a later path refer to the array as earlier diffs have already left it. ### 6.5 Worked example Real bytes from `std_data/mdev39143_partial_json_insert.000001`, the after-image of `UPDATE t1 SET j=JSON_SET(j,'$.arr[7]',99) WHERE id=2` on `t1(id INT PRIMARY KEY, j JSON)`: ``` 01 value_options = PARTIAL_JSON_UPDATES 01 partial_bits: 1 JSON column -> 1 byte; bit 0 set, so column j is a diff 00 null_bits: 2 columns -> 1 byte; neither is NULL 02 00 00 00 column id = 2 0e 00 00 00 diff vector: 14 bytes of diffs follow 01 operation = INSERT 08 path_length = 8 24 2e 61 72 72 5b 37 5d path = "$.arr[7]" 03 value_length = 3 05 63 00 JSONB: type 0x05 = int16, value 0x0063 = 99 ``` The diff bytes total 1 + 1 + 8 + 1 + 3 = 14, matching the vector length, so the vector ends exactly where the next column would begin. ## 7. Code change points | Area | File | Change | |---|---|---| | Type code constant | `sql/rpl_utility.h` | `MYSQL_TYPE_JSON_MYSQL 245`; not in `mysql_com.h`, see §8.4 | | Source type recognition | `sql/rpl_utility_server.cc` | binlog type 245 → `MYSQL_JSON` plugin handler | | Conversion acceptance | `sql/rpl_utility_server.cc` | accept a `mysql_json` source; VARIANT (JSON target) / SUBSET (other) | | Field sizing and metadata | `sql/rpl_utility.cc` | treat 245 as a blob; one metadata byte | | JSON column count | `sql/rpl_utility.h` | `table_def::json_column_count()` | | Missing-plugin diagnostic | `sql/rpl_utility_server.cc` | in `give_compatibility_error()`, name the plugin instead of "unknown field type 245" | | `.frm` path | `sql/table.cc` | use the shared constant | | Event 39 acceptance | `sql/log_event.cc`, `log_event.h` | build an `Update_rows_log_event`, inside the `HAVE_REPLICATION` guard; `LOG_EVENT_IS_UPDATE_ROW()`, `is_partial_json_update()`, `unpack_current_row()` | | After-image header | `sql/rpl_record.cc` | `unpack_row()` reads `value_options` / `partial_bits`, routes per row and column | | Diff decode | `sql/rpl_record.cc` | `read_mysql_json_diff_vector()` and `Mysql_json_diff`, file-static | | Diff apply | `sql/rpl_record.cc` | `apply_partial_json_column()` | | From-the-end guard | `sql/rpl_record.cc` | refuse rather than diverge silently (§8.2) | | Tooling (secondary, not done) | `mysqlbinlog` | print event 39 | `sql/log_event_server.cc` is **not** touched: the diff is applied during unpack, not in the Update apply path. Reused unchanged: `Field_mysql_json` (plugin), `make_conversion_table_field`, `create_conversion_table`, `do_save_blob` (full-document path only), `json_lib`, `Item_func_json_*`. ## 8. Edge cases & constraints Status keys: **[done]** implemented and covered by a test; **[impl]** implemented, not yet covered by a test; **[open]** not implemented; **[verify]** needs checking. ### 8.1 Value decoding (full documents and diff payloads) - **JSONB type coverage** — `Field_mysql_json::val_str()` handles the whole format: objects/arrays (small and large), literals, all int widths, double, utf8mb4 strings, and OPAQUE (temporal → quoted string, decimal → string, anything else → `"base64:typeN:…"`). Reused unchanged. **[done]** - **Large documents** — the large-object/array variants use 4-byte element-count and size fields; a blob-typed column value is length-prefixed per its pack length. `calc_field_size()` handles type 245 as a blob. **[impl]** - **Charset** — MySQL JSON is utf8mb4; MariaDB JSON is `longtext CHARACTER SET utf8mb4`, so the conversion field adopts the target charset and no transcoding is required. **[impl]** - **JSON null vs SQL NULL** — a JSON `null` *inside* a document is a value and survives decoding; an SQL NULL column is signalled by the row image's null bits and never reaches the decoder. **[impl]** - **Key-order normalisation** — MySQL stores object keys sorted by length then by code point; decoding to text preserves that order rather than the order the author wrote. Round-tripping a document through replication can therefore reorder keys relative to the source text. Semantically irrelevant for JSON, but visible in a `SELECT` and therefore in `.result` files. **[impl]** - **Double/float text representation** — MySQL and MariaDB may render the same binary double with a different number of significant digits. Any mismatch shows up as a value difference on the slave, not an error. **[verify]** ### 8.2 Partial updates - **Per-row dual mode** — `partial_bits` is per row, so the same JSON column can arrive as a full value in one row and as a diff in the next. Unpacking branches per row and per column. **[impl]** - **`json_column_count()` comes from the master's type array**, not the slave table: the bitmap has one bit per JSON column of the *source* table, whether or not that column is in the row image or exists on the slave. **[impl]** - **Before-image is always full** — only an UPDATE after-image can carry diffs, which is why `unpack_row()` takes `is_partial_json_after_image` and the caller passes it only for `m_cols_ai`. A before-image has no `value_options` header at all; misreading one would desynchronise the whole row. **[done]** - **Sequential application** — diffs apply in order, each result feeding the next; they are not independent and cannot be reordered or parallelised. **[impl]** - **INSERT vs ARRAY_INSERT** — MySQL `sql-common/json_diff.h` and WL#2955 §4 F3 specify that an `INSERT` diff behaves like `JSON_ARRAY_INSERT` when the path names an array cell and like `JSON_INSERT` when it names an object member; MySQL's own `mysqlbinlog` prints them that way. The last leg of the path now selects the function. On its own this changes no observable outcome, because MySQL only emits an array-cell `INSERT` when the cell does *not* exist, and for those paths MariaDB's two functions currently agree — it is a prerequisite for the `[last-N]` fix below, after which the choice becomes decisive. **[done]** - **`[last-N]` index resolution differs** — the one that actually corrupts data. MySQL resolves an out-of-range `[last-N]` by clamping to 0 and *prepending*; MariaDB treats it as past the end and *appends*. Verified on MySQL 9.2 against a recorded binlog: `JSON_SET(j,'$.arr[last-9]',77)` on `[1,2,3]` gives `[77,1,2,3]` on the source and `[1,2,3,77]` on the slave. The divergence is narrower than it first appears. `[last]`, `[last-1]` and `[last-2]` resolve identically on both servers for `JSON_EXTRACT`, `JSON_REPLACE` and `JSON_REMOVE`, so in-range paths — including in REPLACE and REMOVE diffs — are unaffected. Only `JSON_ARRAY_INSERT` mishandles `[last-N]`: it inserts at `last-2` as if the index were 1 rather than 0 on a three-element array, inconsistent with the other functions, and appends instead of clamping when the index goes negative. That looks like a pre-existing MariaDB bug in `JSON_ARRAY_INSERT` reachable from plain SQL, independent of replication, and probably belongs in its own MDEV rather than here. Covered, and documented as divergent, by `rpl_json_partial_insert`. **[open]** - **REJECTED vs ERROR** — MySQL distinguishes a diff that cannot be applied because the document does not match (missing path, insert target already present) from a corrupt diff. The former leaves the value unchanged and lets replication continue — it signals the slave is already out of sync; only the latter stops the SQL thread. The implementation currently treats every failure as an error. **[open]** - **NULL transitions** — a column going NULL → non-NULL or back is logged as a full value, never as a diff. A diff arriving against a NULL before-image therefore means the slave is out of sync; the code detects this but reports it with a placeholder `ER_UNKNOWN_ERROR`. **[open]** - **Diff vector framing** — the 4-byte length counts only the diffs that follow, and the diff *count* is not stored; decoding runs until the byte budget is exhausted. Every inner length is bounds-checked against the vector's end so a corrupt length cannot read past the row image. **[impl]** ### 8.3 Target column compatibility - **JSON → JSON**: `CONV_TYPE_VARIANT`; allowed with no `slave_type_conversions` setting, on the temporal-precision precedent. **[done]** - **JSON → longtext/varchar**: `CONV_TYPE_SUBSET_TO_SUPERSET`; requires `slave_type_conversions=ALL_NON_LOSSY`. **[impl]** - **JSON → anything else**: no conversion; the existing incompatible-table-def path reports it. **[impl]** - **`CHECK(json_valid(...))`** on the MariaDB column is evaluated on store, so a decoded document that is somehow invalid fails at the field rather than silently landing. **[verify]** - **Generated columns and functional indexes** that read the JSON column must be recomputed after a diff is applied, not after the raw unpack — the diff path bypasses `convert_field()`, so anything hooked to the normal store path needs checking. **[verify]** ### 8.4 Build and portability constraints These were discovered during implementation and constrain where code may live. - **Two `mysql_com.h` headers.** The server's `include/mysql_com.h` and Connector/C's `libmariadb/include/mariadb_com.h` share the include guard `_mysql_com_h`, so only one is ever seen. Client-side compiles get the Connector/C one, which declares `MYSQL_TYPE_JSON=245` while the server enum does not. Anything added to the server header is invisible to shared `sql/` code compiled in a client context — which is why `MYSQL_TYPE_JSON_MYSQL` lives in `sql/rpl_utility.h`. **[done]** - **245 is overloaded.** `include/mysql_com.h` defines `MYSQL_TYPE_VIRTUAL 245` for pre-10.2 `.frm` virtual columns. The same byte means JSON only in a MySQL `.frm` or row event; context decides. Hence a distinct constant name. **[done]** - **`mysqlbinlog` compiles server sources.** `client/mysqlbinlog.cc` `#include`s `log_event.cc` and `rpl_utility.cc` as source, but not `rpl_record.cc`. Code those files touch must compile without the server, and helpers private to `rpl_record.cc` must not be declared in a header `log_event.h` pulls in. **[done]** - **`HAVE_REPLICATION`.** `Update_rows_log_event`'s buffer constructor exists only under that macro, so the `PARTIAL_UPDATE_ROWS_EVENT` case in `read_log_event_no_checksum()` must sit inside the guarded region or the embedded server (`libmysqld`) fails to build. **[done]** ### 8.5 Operational - **Plugin requirement.** The `mysql_json` plugin must be installed on the slave. `field_type_handler()` resolves it by name and returns NULL if absent; `give_compatibility_error()` then reports a message naming the plugin and the `INSTALL SONAME` needed, rather than "unknown field type 245". **[done]** - **Hard-require vs auto-load** is still undecided; today the DBA installs it manually and the test does so explicitly. **[open]** - **Distinct diagnostics** are wanted for: missing plugin **[done]**, corrupt JSONB **[open]**, corrupt diff vector (`HA_ERR_CORRUPT_EVENT`) **[impl]**, REJECTED **[open]**, and conversion disallowed because non-lossy conversions are off **[impl]**. ## 9. Test plan ### 9.1 Method All tests follow the `suite/rpl/rpl_from_mysql80.test` pattern: a binlog recorded once on a real MySQL server is checked into `std_data`, the MariaDB master is stopped and its `master-bin.000001` replaced with that file, and the slave pulls and applies it over the ordinary protocol. No MySQL server runs during the test; both servers are MariaDB, and what is under test is the slave apply path. Expected values are MySQL's own, read from the source table when the binlog was recorded. Recording needs a MySQL server with `binlog_row_value_options=PARTIAL_JSON` and `binlog_format=ROW`. Both existing fixtures were recorded on MySQL 9.2, so that is the `Json_diff` encoding actually exercised. ### 9.2 Existing | Test | Fixture | Covers | |---|---|---| | `rpl.rpl_json_partial_update` | `mdev39143_partial_json.000001` | full-document INSERT, then REPLACE and REMOVE diffs across several events on one row | | `rpl.rpl_json_partial_insert` | `mdev39143_partial_json_insert.000001` | INSERT diffs: object member, array append; REPLACE and REMOVE; and the refusal of a from-the-end array INSERT (§8.2) | | `main.mysql_upgrade_mysql_json` | — | pre-existing; guards the `.frm` path that shares the type-code constant | | `rpl.rpl_from_mysql80` | `mdev35643_mysql_80_binlog.000001` | pre-existing MySQL 8.0 fixture; its `PARTIAL_UPDATE_ROWS_EVENT` now applies via this feature instead of erroring | Both new tests install `type_mysql_json` on the slave and uninstall it in teardown — otherwise `mysql.plugin` differs from the pre-test state and MTR's `check-testcase` fails. ### 9.3 Not yet written - **`rpl_from_mysql80.test` is fixed**, and getting it to pass exposed a real bug: the partial UPDATE's diff value is 150 bytes, needing a multi-byte JSONB length varint, and `apply_partial_json_column()` decoded diff values via `Field::store()`, which validated the raw JSONB bytes as utf8mb4 and rejected any structural byte ≥ 0x80 — invisible until a value exceeded 127 bytes. Fixed by decoding through `Field_blob::set_ptr()` instead, the same charset-agnostic mechanism the full-document unpack path already uses. The fixture's *initial* INSERT is still forced to STATEMENT format by a now-stale workaround baked into the recording itself ("missing support for the JSON type in row events" — no longer true); removing it needs re-recording on a MySQL 8.0 server, not yet done. - Full-document breadth: OPAQUE values, large documents, unicode, JSON null vs SQL NULL, and a longtext target with the `slave_type_conversions` gating. - Partial breadth: multiple JSON columns, per-row dual mode (full value and diff for the same column in consecutive rows), NULL transitions. - Failure paths: corrupt diff vector and exotic paths, best injected with `BINLOG ''` rather than a recording; REJECTED once it is implemented. Requires `have_type_mysql_json`, `have_binlog_format_row`, `have_innodb`.