[MDEV-33291] Slave SQL: Could not execute Delete_rows_v1/Update_rows_v1 event; Can't find record, Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND or HA_ERR_NO_PARTITION_FOUND - Jira

Details

Type: Bug
Status: Open (View Workflow)
Priority: Critical
Resolution: Unresolved
Affects Version/s: 10.6, 10.11, 11.0(EOL), 11.1(EOL), 11.2(EOL), 11.3(EOL), 11.4
Fix Version/s: 10.6, 10.11
Component/s: Replication, Storage Engine - InnoDB
Labels:
- affects-tests
- regression-10.6

Description

While the testcase does not immediately look significant (we're deleting from mysql.innodb_table_stats) note that this is ~~likely~~ just an artifact, i.e. the issue can ~~likely~~ be reproduced in other ways without deleting from innodb_table_stats. There are number of bugs open that look somewhat related (linked), but they are on older versions (except ~~MDEV-33022~~ which is in combination with Galera), and this issue could be reproduced on 10.6+ only.

# Requires standard master/slave setup

CREATE TABLE t (c VARCHAR(2000) BINARY CHARACTER SET 'utf8') ENGINE=InnoDB;

ALTER TABLE t ADD UNIQUE (c);

SELECT c FROM t;

DELETE FROM mysql.innodb_table_stats;

Leads to (on slave):

11.4.0 8bf9f21855606ddeb0fe90fb1e70165adbec6746 (Debug, Slave)
2024-01-22 15:37:12 6 [Note] Slave SQL thread initialized, starting replication in log 'FIRST' at position 4, relay log './relaylog.000001' position: 4; GTID position ''
2024-01-22 15:37:15 69 [ERROR] mariadbd: Can't find record in 'innodb_table_stats'
2024-01-22 15:37:15 69 [Warning] Slave SQL: Could not execute Delete_rows_v1 event on table mysql.innodb_table_stats; Can't find record in 'innodb_table_stats', Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND; the event's master log binlog.000001, end_log_pos 0, Gtid 0-1-6, Internal MariaDB error code: 1032

Present in 10.6+, opt+dbg builds

Attachments

Issue Links

relates to

MDEV-33359 Inconsistent MariaDB error codes 0, 1045, 2003 when using RESET MASTER and init_connect

Open

MDEV-33426 Assertion `status_var.local_memory_used == 0 || !debug_assert_on_not_freed_memory' failed in THD::~THD from handle_slave_sql on slave upon INSERT to TEMPORARY Aria table, Memory not freed: -50616

In Testing

MDEV-16710 Slave SQL: Could not execute Update_rows_v1 event with RocksDB and triggers

Open

MDEV-27135 error code: 1032 on random slaves

Open

MDEV-33022 Galera: Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND;

Closed

Activity

Ascending order - Click to sort in descending order

View 28 older comments

Roel Van de Paar added a comment - 2024-02-07 22:02 - edited

Also observed as a result of RESET MASTER:

[ERROR] Error reading packet from server: binlog truncated in the middle of event; consider out of disk space on master; the first event '.' at 4, the last event read from 'binlog.000001' at 1839, the last byte read from 'binlog.000001' at 1858. (server_errno=1236)

As well as

[ERROR] Error reading packet from server: The binlog on the master is missing the GTID 0-2-130 requested by the slave (even though both a prior and a subsequent sequence number does exist), and GTID strict mode is enabled (server_errno=1236)

Interesting here is that the in both cases master binlog issues were detected.

Roel Van de Paar added a comment - 2024-02-07 22:02 - edited Also observed as a result of RESET MASTER : [ERROR] Error reading packet from server: binlog truncated in the middle of event; consider out of disk space on master; the first event '.' at 4, the last event read from 'binlog.000001' at 1839, the last byte read from 'binlog.000001' at 1858. (server_errno=1236) As well as [ERROR] Error reading packet from server: The binlog on the master is missing the GTID 0-2-130 requested by the slave (even though both a prior and a subsequent sequence number does exist), and GTID strict mode is enabled (server_errno=1236) Interesting here is that the in both cases master binlog issues were detected.

Andrei Elkin added a comment - 2024-02-08 16:49

elenst,

is it the next solution, in which a statement would be converted to ROW because it uses a stat table?

Right, just 'next' is also an initial one .

So I've made my cycle back to a.

Andrei Elkin added a comment - 2024-02-08 16:49 elenst , is it the next solution, in which a statement would be converted to ROW because it uses a stat table? Right, just 'next' is also an initial one . So I've made my cycle back to a.

Andrei Elkin added a comment - 2024-02-08 17:05

[ERROR] Error reading packet from server: binlog truncated ...

because of RESET MASTER must have been _fixed addressed with --~~slave-connections-needed-for-purge~~.

Andrei Elkin added a comment - 2024-02-08 17:05 [ERROR] Error reading packet from server: binlog truncated ... because of RESET MASTER must have been fixed addressed with -- slave-connections-needed-for-purge .

Andrei Elkin added a comment - 2024-02-09 08:55

The executive summary is:
1. ignore on slave events that target mysql."stats" table
2. the statements that read from such tables 'must be detected as UNSAFE'.

Andrei Elkin added a comment - 2024-02-09 08:55 The executive summary is: 1. ignore on slave events that target mysql."stats" table 2. the statements that read from such tables ' must be detected as UNSAFE '.

Roel Van de Paar added a comment - 2025-03-25 03:50

Arbritary, but possibly related. Noticed this in a testrun.

SET SESSION max_error_count=-1;

CREATE TABLE t1 (a INT);

BINLOG ' SOgWTg8CAAAAbgAAAHIAAAAAAAQANS42LjMtbTUtZGVidWctbG9nAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAABI6BZOEzgNAAgAEgAEBAQEEgAAVgAEGggAAAAICAgCAAAAAAVAYI8=';

binlog 'bBf2ZBMBAAAANAAAAHUkAAAAAHEAAAAAAAEABHRlc3QAAnQxAAQDDw8IBP0C4h0AaTGFIg==bBf2ZBgBAAAASAAAAL0kAAAAAHEAAAAAAAEABP//8I + kAAABAGIBAGWuv1VNCQAAAPBuWwAAAQBiAQBlrr9VTQkAAADxS9Lu';

CS 12.0.0 c3f21762e9db30c4a5dd1e9ac676dbdafa303d4a (Optimized) Build 19/03/2025

2025-03-25 14:45:29 4 [ERROR] mariadbd: Can't find record in 't1'

2025-03-25 14:45:29 4 [ERROR]  BINLOG_BASE64_EVENT: Could not execute Update_rows_v1 event on table test.t1; handler error HA_ERR_END_OF_FILE; the event's master log FIRST, end_log_pos 9405, Internal MariaDB error code: 1032

Roel Van de Paar added a comment - 2025-03-25 03:50 Arbritary, but possibly related. Noticed this in a testrun. SET SESSION max_error_count=-1; CREATE TABLE t1 (a INT ); BINLOG ' SOgWTg8CAAAAbgAAAHIAAAAAAAQANS42LjMtbTUtZGVidWctbG9nAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAABI6BZOEzgNAAgAEgAEBAQEEgAAVgAEGggAAAAICAgCAAAAAAVAYI8=' ; binlog 'bBf2ZBMBAAAANAAAAHUkAAAAAHEAAAAAAAEABHRlc3QAAnQxAAQDDw8IBP0C4h0AaTGFIg==bBf2ZBgBAAAASAAAAL0kAAAAAHEAAAAAAAEABP//8I + kAAABAGIBAGWuv1VNCQAAAPBuWwAAAQBiAQBlrr9VTQkAAADxS9Lu' ; CS 12.0.0 c3f21762e9db30c4a5dd1e9ac676dbdafa303d4a (Optimized) Build 19/03/2025 2025-03-25 14:45:29 4 [ERROR] mariadbd: Can't find record in 't1' 2025-03-25 14:45:29 4 [ERROR] BINLOG_BASE64_EVENT: Could not execute Update_rows_v1 event on table test.t1; handler error HA_ERR_END_OF_FILE; the event's master log FIRST, end_log_pos 9405, Internal MariaDB error code: 1032

MariaDB Server

Slave SQL: Could not execute Delete_rows_v1/Update_rows_v1 event; Can't find record, Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND or HA_ERR_NO_PARTITION_FOUND

Details

Description

Attachments

Issue Links

Activity

People

Dates

Git Integration