[MDEV-21686] Server with full disk hangs on SHOW SLAVE STATUS Created: 2020-02-08  Updated: 2023-06-06  Resolved: 2023-06-06

Status: Closed
Project: MariaDB Server
Component/s: Galera
Affects Version/s: None
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Michaël de groot Assignee: Jan Lindström
Resolution: Not a Bug Votes: 1
Labels: None


 Description   

This issue occurred due to disk being full on the slave side. Please decide for yourself if it is needed to fix this.

I am in a migration process, with a cluster replicating from another cluster using asynchronous replication. I have this configuration on the slave side:

replicate_rewrite_db=app->app_prd
replicate_do_db=app_prd
server_id=0 # (this produces a warning in the error log, server rewrites it to 1)
server_audit=FORCE_PLUS_PERMANENT
server_audit_logging = OFF
server_audit_events=connect,query_ddl,query_dml,query_dcl
server_audit_output_type = syslog

The PROCESSLIST of the server shows this:

MariaDB [(none)]> show processlist;
+------+-------------+-----------------+-----------+-----------+-------+----------------------------------------+-------------------+----------+
| Id   | User        | Host            | db        | Command   | Time  | State                                  | Info              | Progress |
+------+-------------+-----------------+-----------+-----------+-------+----------------------------------------+-------------------+----------+
|    1 | system user |                 | NULL      | Sleep     | 19599 | Committed 61092                        | NULL              |    0.000 |
|    2 | system user |                 | NULL      | Sleep     | 51649 | WSREP aborter idle                     | NULL              |    0.000 |
|    3 | system user |                 | NULL      | Daemon    |  NULL | InnoDB purge worker                    | NULL              |    0.000 |
|    4 | system user |                 | NULL      | Daemon    |  NULL | InnoDB purge worker                    | NULL              |    0.000 |
|    5 | system user |                 | NULL      | Daemon    |  NULL | InnoDB purge worker                    | NULL              |    0.000 |
|    6 | system user |                 | NULL      | Daemon    |  NULL | InnoDB purge coordinator               | NULL              |    0.000 |
|    7 | system user |                 | NULL      | Daemon    |  NULL | InnoDB shutdown handler                | NULL              |    0.000 |
|   11 | system user |                 | NULL      | Sleep     | 51648 | wsrep applier idle                     | NULL              |    0.000 |
|   12 | system user |                 | NULL      | Sleep     | 19599 | Committed 61091                        | NULL              |    0.000 |
|   13 | system user |                 | NULL      | Sleep     | 51648 | wsrep applier idle                     | NULL              |    0.000 |
|  483 | monitor     | 10.4.0.18:32790 | NULL      | Sleep     |     1 |                                        | NULL              |    0.000 |
|  484 | monitor     | 10.4.0.13:57144 | NULL      | Sleep     |     3 |                                        | NULL              |    0.000 |
|  500 | monitor     | 10.4.0.7:45908  | NULL      | Sleep     |     3 |                                        | NULL              |    0.000 |
| 2583 | root        | localhost       | regus_prd | Killed    |   707 | Init                                   | show slave status |    0.000 |
| 2590 | system user |                 | NULL      | Slave_IO  |   841 | Queueing master event to the relay log | NULL              |    0.000 |
| 2591 | system user |                 | NULL      | Slave_SQL |   733 | Closing tables                         | NULL              |    0.000 |
| 2605 | root        | localhost       | NULL      | Killed    |   620 | Init                                   | show slave status |    0.000 |
| 2608 | root        | localhost       | NULL      | Query     |     0 | Init                                   | show processlist  |    0.000 |
+------+-------------+-----------------+-----------+-----------+-------+----------------------------------------+-------------------+----------+
18 rows in set (0.000 sec)

The InnoDB status shows this:

------------
TRANSACTIONS
------------
Trx id counter 2835757
Purge done for trx's n:o < 2835757 undo n:o < 0 state: running but idle
History list length 40
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 421687012229744, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421687012225528, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421687012221312, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 2835752, ACTIVE (PREPARED) 808 sec
4 lock struct(s), heap size 1136, 2 row lock(s), undo log entries 3
MySQL thread id 2591, OS thread handle 140209058854656, query id 114359 Closing tables
---TRANSACTION 421687012212880, not started
0 lock struct(s), heap size 1136, 0 row lock(s)

The other galera nodes have no hanging of transactions.



 Comments   
Comment by Jan Lindström [ 2023-06-06 ]

If disk is really full I do not see a way to handle that gracefully.

Generated at Thu Feb 08 09:09:02 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.