[MDEV-20968] Possible performance issue when querying DB metadata (constraints, fkeys) Created: 2019-11-04  Updated: 2020-09-29  Resolved: 2019-12-24

Status: Closed
Project: MariaDB Server
Component/s: Information Schema
Affects Version/s: None
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Miroslav Stanik Assignee: Unassigned
Resolution: Incomplete Votes: 0
Labels: need_feedback


 Description   

When running queries to load constraints, there is a possible performance issue. it takes a long time to execute and load data from system tables:

How to reproduce:
1. create 100 databases, each db contains 1000 tables, each table contains 6 columns.
2. execute these queries:

SELECT CONSTRAINT_NAME, TABLE_NAME, CONSTRAINT_TYPE FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS;
	SELECT
		CONSTRAINT_NAME,
		COLUMN_NAME,
		TABLE_NAME,
		TABLE_SCHEMA,
		REFERENCED_COLUMN_NAME,
		REFERENCED_TABLE_NAME,
		REFERENCED_TABLE_SCHEMA
	FROM INFORMATION_SCHEMA.KEY_COLUMN_USAGE;

it takes ages to finish.

procs to generate env:

CREATE DEFINER = `root`@`localhost` PROCEDURE db1.gendbs  ()  LANGUAGE SQL DETERMINISTIC CONTAINS SQL SQL SECURITY DEFINER BEGIN
   DECLARE x     INT;
   DECLARE str   VARCHAR(255);
 
   SET x = 0;
   SET str = '';
 
  loop_label:
   LOOP
      IF x > 99
      THEN
         LEAVE loop_label;
      END IF;
 
      SET x = x + 1;
 
      SET str = CONCAT('CREATE DATABASE gendb',x);
      
      EXECUTE IMMEDIATE             str;
      
      
   END LOOP;
END;
 
 
CREATE DEFINER = `root`@`localhost` PROCEDURE db1.gentables  ()  LANGUAGE SQL DETERMINISTIC CONTAINS SQL SQL SECURITY DEFINER BEGIN
   DECLARE x      INT;
   DECLARE y      INT;
   DECLARE str    VARCHAR(255);
   DECLARE str2   VARCHAR(255);
 
   SET x = 0;
   SET y =0;
   SET str = '';
   SET str2 =
          ' ( id          INT NOT NULL,  name        CHAR(50) NOT NULL,  address     CHAR(50),  city        CHAR(50),  state       CHAR(25), zip_code    CHAR(10) )';
 
  OUTER_LOOP:
   LOOP
      IF y > 99
      THEN
         LEAVE OUTER_LOOP;
      END IF;
 
      SET y = y + 1;
 
     NESTED_LOOP:
      LOOP
         IF x >999
         THEN
            LEAVE NESTED_LOOP;
         END IF;
 
         SET x = x + 1;
 
 
 
         SET str = CONCAT('CREATE table ','gendb',y,'.','employee', x, str2);
 
         EXECUTE IMMEDIATE          str;
      END LOOP;
       SET x =0;
   END LOOP;
END;



 Comments   
Comment by Elena Stepanova [ 2019-11-05 ]

Which version is it about?

Comment by Miroslav Stanik [ 2019-11-05 ]

version 10.4.8 but reproducible also with previous versions

Comment by Elena Stepanova [ 2019-11-18 ]

So, are you comparing the performance with some other version or server, where it works considerably faster? "Takes a long time" is subjective matter. Opening 100,000 tables surely won't be fast, and how long exactly it takes depends a lot on the environment.

Comment by Dominic Watson [ 2020-09-29 ]

While it is subjective, it is problematic. On a MariaDB RDS server with ~9000 tables, the following query was sometimes taking over 20m:

SELECT table_schema, table_name FROM information_schema.key_column_usage WHERE referenced_table_schema='db_name' AND referenced_table_name='table_name'

This query was running as part of Percona Online Schema Change script to determine how to handle foreign key changes. We've upgraded hardware and that has improved the situation, but the query is still taking over 60 seconds.

There is a resolved ticket for MySQL on this subject that might be useful: https://bugs.mysql.com/bug.php?id=98238

We are on 10.3.8.

Generated at Thu Feb 08 09:03:35 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.