[MDEV-18287] Status threads_running show wrong value since 10.3 Created: 2019-01-17  Updated: 2019-11-21  Resolved: 2019-11-21

Status: Closed
Project: MariaDB Server
Component/s: Server
Affects Version/s: 10.3, 10.4
Fix Version/s: 10.5.0

Type: Bug Priority: Major
Reporter: Oli Sennhauser Assignee: Sergey Vojtovich
Resolution: Fixed Votes: 0
Labels: monitoring, status, threads, threads_running
Environment:

Linux, n.a.


Issue Links:
Blocks
blocks MDEV-14505 Threads_running becomes scalability b... Closed
Relates
relates to MDEV-16264 Implement a common work queue for Inn... Closed

 Description   

SHOW GLOBAL STATUS LIKE 'threads_running' was a good indicator for the current running workload in a database.

Sine MariaDB 10.3 it is showing higher values than before (MariaDB 10.2 and lower and competitive products).

10.2.14-MariaDB-log MariaDB Server

SELECT id, user, command, state FROM information_schema.processlist;
+----+-------------+-------------+------------------------------------------------------------------+
| id | user        | command     | state                                                            |
+----+-------------+-------------+------------------------------------------------------------------+
| 21 | root        | Query       | Filling schema table                                             |
| 10 | replication | Binlog Dump | Master has sent all binlog to slave; waiting for binlog to be up |
|  5 | system user | Daemon      | InnoDB shutdown handler                                          |
|  4 | system user | Daemon      | InnoDB purge worker                                              |
|  2 | system user | Daemon      | InnoDB purge worker                                              |
|  1 | system user | Daemon      | InnoDB purge worker                                              |
|  3 | system user | Daemon      | InnoDB purge coordinator                                         |
+----+-------------+-------------+------------------------------------------------------------------+
7 rows in set (0.00 sec)

SHOW GLOBAL STATUS LIKE 'threads_running';
+-----------------+-------+
| Variable_name   | Value |
+-----------------+-------+
| Threads_running | 2     |
+-----------------+-------+

10.3.12-MariaDB-log MariaDB Server

SELECT id, user, command, state FROM information_schema.processlist;
+----+-------------+---------+--------------------------+
| id | user        | command | state                    |
+----+-------------+---------+--------------------------+
| 10 | root        | Query   | Filling schema table     |
|  5 | system user | Daemon  | InnoDB shutdown handler  |
|  2 | system user | Daemon  | InnoDB purge coordinator |
|  4 | system user | Daemon  | InnoDB purge worker      |
|  3 | system user | Daemon  | InnoDB purge worker      |
|  1 | system user | Daemon  | InnoDB purge worker      |
+----+-------------+---------+--------------------------+
6 rows in set (0.001 sec)

SHOW GLOBAL STATUS LIKE 'threads_running';
+-----------------+-------+
| Variable_name   | Value |
+-----------------+-------+
| Threads_running | 6     |
+-----------------+-------+

It seems like now it is showing all Threads from Processlist including Daemon threads. This was not the case before. The Daemon threads are currently not doing anything on my system so they should not be counted in.

Impact: This could influence/confuse all monitoring solutions for MariaDB and will show a virtual higher workload compared to previous versions (bad image) and pseudo-require more CPU resource allocation for newer releases.

Possibly was introduced with this fix: https://jira.mariadb.org/browse/MDEV-14505



 Comments   
Comment by Elena Stepanova [ 2019-01-17 ]

It was indeed done in the scope of MDEV-14505, and technically it works as documented:

all threads having a command not equal to COM_SLEEP are included

I'll leave it to svoj to further comment on it.

Comment by Oli Sennhauser [ 2019-01-18 ]

I thought about this tonight as well an I came to a similar conclusion. So possibly the problem is not threads_running but that daemon threads should be in command COM_SLEEP when they are not active. This would also make the value of threads_running better to estimate system load from the database point of view. So threads_runnings was showing the wrong number all the time up to now.

And I realise again I should read more documentation and not remember how it was earlier...

Comment by Sergey Vojtovich [ 2019-01-18 ]

If disregarding sleeping background threads works, it should be relatively easy to implement.

Comment by Sergey Vojtovich [ 2019-11-21 ]

Must've been fixed in 10.5 by MDEV-16264.

Generated at Thu Feb 08 08:42:56 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.