Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-33291

Slave SQL: Could not execute Delete_rows_v1/Update_rows_v1 event; Can't find record, Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND or HA_ERR_NO_PARTITION_FOUND

Details

    Description

      While the testcase does not immediately look significant (we're deleting from mysql.innodb_table_stats) note that this is likely just an artifact, i.e. the issue can likely be reproduced in other ways without deleting from innodb_table_stats. There are number of bugs open that look somewhat related (linked), but they are on older versions (except MDEV-33022 which is in combination with Galera), and this issue could be reproduced on 10.6+ only.

      # Requires standard master/slave setup
      CREATE TABLE t (c VARCHAR(2000) BINARY CHARACTER SET 'utf8') ENGINE=InnoDB;
      ALTER TABLE t ADD UNIQUE (c);
      SELECT c FROM t;
      DELETE FROM mysql.innodb_table_stats;
      

      Leads to (on slave):

      11.4.0 8bf9f21855606ddeb0fe90fb1e70165adbec6746 (Debug, Slave)

      2024-01-22 15:37:12 6 [Note] Slave SQL thread initialized, starting replication in log 'FIRST' at position 4, relay log './relaylog.000001' position: 4; GTID position ''
      2024-01-22 15:37:15 69 [ERROR] mariadbd: Can't find record in 'innodb_table_stats'
      2024-01-22 15:37:15 69 [Warning] Slave SQL: Could not execute Delete_rows_v1 event on table mysql.innodb_table_stats; Can't find record in 'innodb_table_stats', Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND; the event's master log binlog.000001, end_log_pos 0, Gtid 0-1-6, Internal MariaDB error code: 1032
      

      Present in 10.6+, opt+dbg builds

      Attachments

        Issue Links

          Activity

            Being the target table matters in your example

            That's because these are just examples, they are expected to be extrapolated properly when you think of a common solution. But you are suggesting a solution each time based on one example only, not on a general problem, that's why I provide the next one trying to explain why the suggestion won't work.

            update t1 set a = 10 where a not in (select count(*) from mysql.innodb_table_stats);
            which must be detected as UNSAFE and binlogged in ROW format which you test demonstrates it does not do.

            Why is that?
            In general (not taking into account specific tables involved), this statement is deterministic, it's not converted to row automatically and has no reason to. Unlike the previous one with limit.
            Or is it the next solution, in which a statement would be converted to ROW because it uses a stat table? That is,
            a) ignore replicating changes to stat tables +
            b) convert everything else that uses stat tables to RBR?

            elenst Elena Stepanova added a comment - Being the target table matters in your example That's because these are just examples, they are expected to be extrapolated properly when you think of a common solution. But you are suggesting a solution each time based on one example only, not on a general problem, that's why I provide the next one trying to explain why the suggestion won't work. update t1 set a = 10 where a not in (select count(*) from mysql.innodb_table_stats); which must be detected as UNSAFE and binlogged in ROW format which you test demonstrates it does not do. Why is that? In general (not taking into account specific tables involved), this statement is deterministic, it's not converted to row automatically and has no reason to. Unlike the previous one with limit . Or is it the next solution, in which a statement would be converted to ROW because it uses a stat table? That is, a) ignore replicating changes to stat tables + b) convert everything else that uses stat tables to RBR?
            Roel Roel Van de Paar added a comment - - edited

            Also observed as a result of RESET MASTER:

            [ERROR] Error reading packet from server: binlog truncated in the middle of event; consider out of disk space on master; the first event '.' at 4, the last event read from 'binlog.000001' at 1839, the last byte read from 'binlog.000001' at 1858. (server_errno=1236)
            

            As well as

            [ERROR] Error reading packet from server: The binlog on the master is missing the GTID 0-2-130 requested by the slave (even though both a prior and a subsequent sequence number does exist), and GTID strict mode is enabled (server_errno=1236)
            

            Interesting here is that the in both cases master binlog issues were detected.

            Roel Roel Van de Paar added a comment - - edited Also observed as a result of RESET MASTER : [ERROR] Error reading packet from server: binlog truncated in the middle of event; consider out of disk space on master; the first event '.' at 4, the last event read from 'binlog.000001' at 1839, the last byte read from 'binlog.000001' at 1858. (server_errno=1236) As well as [ERROR] Error reading packet from server: The binlog on the master is missing the GTID 0-2-130 requested by the slave (even though both a prior and a subsequent sequence number does exist), and GTID strict mode is enabled (server_errno=1236) Interesting here is that the in both cases master binlog issues were detected.
            Elkin Andrei Elkin added a comment -

            elenst,

            is it the next solution, in which a statement would be converted to ROW because it uses a stat table?

            Right, just 'next' is also an initial one .

            So I've made my cycle back to a.

            Elkin Andrei Elkin added a comment - elenst , is it the next solution, in which a statement would be converted to ROW because it uses a stat table? Right, just 'next' is also an initial one . So I've made my cycle back to a.
            Elkin Andrei Elkin added a comment -

            [ERROR] Error reading packet from server: binlog truncated ...

            because of RESET MASTER must have been fixed addressed with --slave-connections-needed-for-purge.

            Elkin Andrei Elkin added a comment - [ERROR] Error reading packet from server: binlog truncated ... because of RESET MASTER must have been fixed addressed with -- slave-connections-needed-for-purge .
            Elkin Andrei Elkin added a comment -

            The executive summary is:
            1. ignore on slave events that target mysql."stats" table
            2. the statements that read from such tables 'must be detected as UNSAFE'.

            Elkin Andrei Elkin added a comment - The executive summary is: 1. ignore on slave events that target mysql."stats" table 2. the statements that read from such tables ' must be detected as UNSAFE '.

            People

              Elkin Andrei Elkin
              Roel Roel Van de Paar
              Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.