[MDEV-20492] Information schema query producing many disk temp table Created: 2019-09-04  Updated: 2022-11-22  Resolved: 2019-11-25

Status: Closed
Project: MariaDB Server
Component/s: Information Schema
Affects Version/s: 10.3.16
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: VAROQUI Stephane Assignee: Oleksandr Byelkin
Resolution: Not a Bug Votes: 0
Labels: None

Issue Links:
Relates
relates to MDEV-19 Memory tables: VARCHAR and BLOB support Stalled
relates to MDEV-30071 Huge dentry cache after upgrading to ... Open

 Description   

On slow SATA disk our monitoring API try to compute CRC32 of DDL using information schema

Such trivial queries piling in processlist up to 40s , and so guessing a table lock is hold for the same duration .

Profiling show the time is spend in removing temp tables

| Removing tmp table     | 7.341188 |
 
 
| Created_tmp_disk_tables | 132   |
 
 
MariaDB [(none)]> SELECT CRC32(b.column_name) + CRC32(b.column_type)+CRC32(COALESCE(b.is_nullable,''))+CRC32(COALESCE(b.CHARACTER_SET_NAME,''))+ CRC32(COALESCE(b.COLLATION_NAME,''))+ CRC32(COALESCE(b.COLUMN_DEFAULT,''))+CRC32(COALESCE(c.CONSTRAINT_NAME,''))+CRC32(COALESCE(c.ORDINAL_POSITION,'')) as CRC  FROM information_schema.COLUMNS b left join information_schema.KEY_COLUMN_USAGE c ON b.table_schema=c.table_schema  AND  b.table_name=c.table_name where b.table_schema='wpmixrgen' AND  b.table_name='wp_comments';
+-------------+
| CRC         |
+-------------+
| 10559682178 |
| 13586448768 |
| 15640549970 |
| 17764828815 |
| 18237274403 |
| 18860079141 |
| 11228629995 |
| 11446859360 |
| 14178433118 |
| 15610854266 |
| 19381875266 |
| 17733767415 |
| 18959763247 |
| 17370551431 |
| 16025333695 |
+-------------+
15 rows in set (0.019 sec)
 
MariaDB [(none)]> show status like '%tmp%';
+-------------------------+-------+
| Variable_name           | Value |
+-------------------------+-------+
| Created_tmp_disk_tables | 144   |

Found a workaround by using show create table instead but i d like to understand why information_schema produce multiple on disk temp tables , any pointer would be appreciate , thanks i advance .

Found that changing join level from 6 to 2 change from BNLH to (flat, BNL ) and reduce the number of temp table per 2 . Still so few rows should not produce disk temp table but memory instead no ?



 Comments   
Comment by Oleksandr Byelkin [ 2019-10-11 ]

Where is bug here?
maybe increasing memory limit for temporary files can help?

Comment by VAROQUI Stephane [ 2019-10-11 ]

Hi Oleksander, why a 6 records query on IS produce 10 disk temp table the memory use should be less than a 1Kb

Comment by VAROQUI Stephane [ 2019-10-11 ]

The issue is that most monitoring tools will not really look at what mariadb does behind the hood and clients may simply switch to MysQL 8.0 using innodb table dictionary without even understand that the monyog is just killing their disk !

Comment by VAROQUI Stephane [ 2019-10-11 ]

I not sure about what is thae case here but why do i receive such jira ticket

"Anyways, additional question would your pro (or osc relase) version allow a real time/live migration from MariaDB 10.2 to MySQL 7/8?"

Comment by VAROQUI Stephane [ 2019-10-11 ]

I don't like to hear this , and i'd like to help founding why this happen

Comment by Sergei Golubchik [ 2019-10-14 ]

My first guess would be that the I_S code does not realize you're selecting data only for one table from I_S.KEY_COLUMN_USAGE.
Try to use inner join instead of left join. Or even propagate constants manually, like in:

SELECT ...  FROM information_schema.COLUMNS b JOIN information_schema.KEY_COLUMN_USAGE c WHERE c.table_schema='wpmixrgen' AND c.table_name='wp_comments' AND b.table_schema='wpmixrgen' AND  b.table_name='wp_comments';

Just to see if that helps, I'm not saying you should do it in production.

Comment by VAROQUI Stephane [ 2019-10-15 ]

Hi Sergei, good to hear from you !

You guys make my day with insert returning

The issue can be narrow to data type of COLUMNS metadata:

COLUMN_DEFAULT longtext YES   NULL  
COLUMN_TYPE longtext NO      
GENERATION_EXPRESSION longtext YES   NULL  

While data type choice can be questioned for those columns : can ENUM definition of longtext may exeed varchar max ? , i can't imagine a default or generated exxpression so long

Still we get on simple query:

show status like '%tmp%';
+-------------------------+-------+
| Variable_name           | Value |
+-------------------------+-------+
| Created_tmp_disk_tables | 71    |
 
MariaDB [(none)]> SELECT  CRC32(b.column_type) FROM information_schema.COLUMNS b  where b.table_schema='wpmixrgen' AND  b.table_name='wp_comments';
+----------------------+
| CRC32(b.column_type) |
+----------------------+
|           2848410689 |
 
MariaDB [(none)]> show status like '%tmp%';
+-------------------------+-------+
| Variable_name           | Value |
+-------------------------+-------+
| Created_tmp_disk_tables | 72    |

here NOT using

  • order by,
  • no join
  • removed CRC32 and still get tmp table

Where does disk temp table comes in ?

Tx svar

Thanks stephane

Comment by VAROQUI Stephane [ 2019-10-15 ]

By creating same table in aria instead of memory the disk temporay table is not created

Comment by Sergei Golubchik [ 2019-10-16 ]

on-disk temporary table is created to store the values of `column_type` column. because MEMORY tables cannot have BLOB columns, as you know.

Comment by Sergei Golubchik [ 2019-11-25 ]

This is not a bug, everything works as intended.

We want to optimize that in the future and not create temp tables in this case. But still, it's not a bug.

Comment by VAROQUI Stephane [ 2019-11-25 ]

Thanks i did not know every request to information_schema produce extra memory table as middle man resultset i was just thinking the data can flow from memory to thread result buffer directly, if not the case of course that's not a bug

Comment by VAROQUI Stephane [ 2019-11-25 ]

Also Monty told me once, aria was now used for temporary tables to avoid touching the disks, was just curious to notice it's not case for IS as well as this make monitoring from IS almost a no go !

Comment by Sergei Golubchik [ 2019-11-25 ]

Yes, every request to information_schema produces a temporary table. If all requested columns are supported by the MEMORY engine then the table is created using MEMORY engine. But if there are blob columns, the engine will be Aria.

Aria temporary tables don't necessarily go to disk, as far as I understand. They can stay in the page cache and then these pages can be discarded when a table is dropped without ever being written to disk. But if the page cache will need to free some pages, it might write temp table pages to disk.

Generated at Thu Feb 08 08:59:53 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.