[MDEV-27441] FR: provide more detailed monitoring regarding temp table usage Created: 2022-01-07  Updated: 2023-11-30

Status: Open
Project: MariaDB Server
Component/s: None
Fix Version/s: None

Type: New Feature Priority: Minor
Reporter: Hartmut Holzgraefe Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None

Issue Links:
Relates

 Description   

Request:

  • monitor the usage of the tmp space, trigger alerts when it crosses certain threshold
  • find the query that's using the tmp space

Original description:

MariaDB currently only provides status counters for Created_tmp_tables and Created_tmp_disk_tables.

PostgreSQL for example provides more insight into this by tracking:

  • total number of bytes written to temporary table files per database

See temp_bytes on https://www.postgresql.org/docs/9.2/monitoring-stats.html#PG-STAT-DATABASE-VIEW

  • temporary file blocks read and written per statement

See temp_blks_read and temp_blks on https://www.postgresql.org/docs/9.0/pgstatstatements.html#PGSTATSTATEMENTS-COLUMNS



 Comments   
Comment by Sergei Golubchik [ 2022-04-27 ]

There're more stats than Created_tmp_tables and Created_tmp_disk_tables:

MariaDB [test]> show status like '%_tmp_%';
+-------------------------+-------+
| Variable_name           | Value |
+-------------------------+-------+
| Created_tmp_disk_tables | 0     |
| Created_tmp_files       | 4     |
| Created_tmp_tables      | 2     |
| Handler_tmp_delete      | 0     |
| Handler_tmp_update      | 0     |
| Handler_tmp_write       | 4     |
| Rows_tmp_read           | 4     |
+-------------------------+-------+
7 rows in set (0.002 sec)

would that be sufficient?

Comment by Sergei Golubchik [ 2022-05-16 ]

to monitor the space we need to take into account both temp tables and temp files.

Tables are easier, they're files on disk, one can see file sizes, one can see what query created what tables (as every #sql table name as a connection id as part of the file name). A sum of all writes into a temp table is wrong, there can be updates and deletes, no every write adds to the disk use.

Files are somewhat difficult as they don't have directory entries, only an open file descriptor. And they can be big and eat up a lot of disk space

Comment by Michael Widenius [ 2022-09-29 ]

What should be monitored are:

  • Temporary files created by filesort and replication buffers and
    sorting in engines as part of alter table, repair table, optimize table.
    As (almost?) all server temporary files are using the IO_CACHE interface,
    the 2 first are relatively easy to do, the engine temporary files a bit harder.
  • On disk internal temporary tables (MyISAM or Aria). These are relatively
    easy to handle.
  • Explicitly created temporary tables (CREATE TEMPORARY TABLE).
    These are hard to do as this require a change in the storage engine
    interface to store status information into the corresponding THD.
    We could 'as a temporary solution' get this information from the file
    system. As the temporary tables includes the session id, it is easy to
    attach these to each user.
Generated at Thu Feb 08 09:52:56 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.