[MDEV-15675] encryption.innodb_encryption failed in buildbot with timeout - Jira

Details

Type: Bug
Status: Closed (View Workflow)
Priority: Critical
Resolution: Fixed
Affects Version/s: 10.2(EOL), 10.3(EOL), 10.4(EOL), 10.5, 10.6
Fix Version/s: 10.2.41, 10.3.32, 10.4.22, 10.5.13, 10.6.5
Component/s: Encryption, Storage Engine - InnoDB, Tests
Labels:
None

Description

http://buildbot.askmonty.org/buildbot/builders/kvm-rpm-centos74-amd64-debug/builds/190/steps/mtr/logs/stdio

encryption.innodb_encryption 'innodb,undo3' w4 [ fail ]

        Test ended at 2018-03-23 00:13:48

CURRENT_TEST: encryption.innodb_encryption

--- /usr/share/mysql-test/suite/encryption/r/innodb_encryption.result	2018-03-22 20:37:55.000000000 +0000

+++ /dev/shm/var/4/log/innodb_encryption.reject	2018-03-23 00:13:47.701287503 +0000

@@ -33,6 +33,14 @@

 # since threads are off tables should remain unencrypted

 SET GLOBAL innodb_encrypt_tables = on;

 # Wait 15s to check that nothing gets encrypted

+Timeout in wait_condition.inc for SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_VERSION <> 0;

+Id	User	Host	db	Command	Time	State	Info	Progress

+1	system user		NULL	Daemon	NULL	InnoDB purge worker	NULL	0.000

+2	system user		NULL	Daemon	NULL	InnoDB purge coordinator	NULL	0.000

+3	system user		NULL	Daemon	NULL	InnoDB purge worker	NULL	0.000

+4	system user		NULL	Daemon	NULL	InnoDB purge worker	NULL	0.000

+5	system user		NULL	Daemon	NULL	InnoDB shutdown handler	NULL	0.000

+9	root	localhost	test	Query	0	init	show full processlist	0.000

 SELECT NAME FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_VERSION = 0

 AND NAME NOT LIKE 'innodb_undo%' AND NAME NOT LIKE 'mysql/innodb_%_stats';

 NAME

mysqltest: Result length mismatch

Attachments

Activity

Ascending order - Click to sort in descending order

Elena Stepanova added a comment - 2018-09-15 13:02

Still happens on current 10.2:
http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/14259/steps/mtr_nm/logs/stdio

Elena Stepanova added a comment - 2018-09-15 13:02 Still happens on current 10.2: http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/14259/steps/mtr_nm/logs/stdio

Alice Sherepa added a comment - 2020-01-09 15:00 - edited

recent failure on 10.2:

encryption.innodb_encryption 'innodb,undo3' w1 [ fail ]

        Test ended at 2020-01-09 15:47:44

CURRENT_TEST: encryption.innodb_encryption

--- /git/10.2/mysql-test/suite/encryption/r/innodb_encryption.result	2019-09-13 10:22:13.673678951 +0200

+++ /git/10.2/mysql-test/suite/encryption/r/innodb_encryption.reject	2020-01-09 15:47:44.320783522 +0100

@@ -34,6 +34,14 @@

 # since threads are off tables should remain unencrypted

 SET GLOBAL innodb_encrypt_tables = on;

 # Wait 15s to check that nothing gets encrypted

+Timeout in wait_condition.inc for SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_VERSION <> 0;

+Id	User	Host	db	Command	Time	State	Info	Progress

+1	system user		NULL	Daemon	NULL	InnoDB purge worker	NULL	0.000

+2	system user		NULL	Daemon	NULL	InnoDB purge worker	NULL	0.000

+3	system user		NULL	Daemon	NULL	InnoDB purge coordinator	NULL	0.000

+4	system user		NULL	Daemon	NULL	InnoDB purge worker	NULL	0.000

+5	system user		NULL	Daemon	NULL	InnoDB shutdown handler	NULL	0.000

+9	root	localhost	test	Query	0	init	show full processlist	0.000

 SELECT NAME FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_VERSION = 0

 AND NAME NOT LIKE 'innodb_undo%' AND NAME NOT LIKE 'mysql/innodb_%_stats';

 NAME

mysqltest: Result length mismatch

10.2 8317f77ccc490c94f2f43d2e6 http://buildbot.askmonty.org/buildbot/builders/mac-1012-bintar/builds/6463/steps/test/logs/stdio

Alice Sherepa added a comment - 2020-01-09 15:00 - edited recent failure on 10.2: encryption.innodb_encryption 'innodb,undo3' w1 [ fail ] Test ended at 2020-01-09 15:47:44 CURRENT_TEST: encryption.innodb_encryption --- /git/10.2/mysql-test/suite/encryption/r/innodb_encryption.result 2019-09-13 10:22:13.673678951 +0200 +++ /git/10.2/mysql-test/suite/encryption/r/innodb_encryption.reject 2020-01-09 15:47:44.320783522 +0100 @@ -34,6 +34,14 @@ # since threads are off tables should remain unencrypted SET GLOBAL innodb_encrypt_tables = on; # Wait 15s to check that nothing gets encrypted +Timeout in wait_condition.inc for SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_VERSION <> 0; +Id User Host db Command Time State Info Progress +1 system user NULL Daemon NULL InnoDB purge worker NULL 0.000 +2 system user NULL Daemon NULL InnoDB purge worker NULL 0.000 +3 system user NULL Daemon NULL InnoDB purge coordinator NULL 0.000 +4 system user NULL Daemon NULL InnoDB purge worker NULL 0.000 +5 system user NULL Daemon NULL InnoDB shutdown handler NULL 0.000 +9 root localhost test Query 0 init show full processlist 0.000 SELECT NAME FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_VERSION = 0 AND NAME NOT LIKE 'innodb_undo%' AND NAME NOT LIKE 'mysql/innodb_%_stats'; NAME mysqltest: Result length mismatch 10.2 8317f77ccc490c94f2f43d2e6 http://buildbot.askmonty.org/buildbot/builders/mac-1012-bintar/builds/6463/steps/test/logs/stdio

Alice Sherepa added a comment - 2020-11-20 10:14

https://buildbot.askmonty.org/buildbot/builders/kvm-fulltest/builds/26768/steps/mtr_ps/logs/stdio

encryption.innodb_encryption 'innodb,undo3' w3 [ fail ]

        Test ended at 2020-11-17 13:18:08

CURRENT_TEST: encryption.innodb_encryption

--- /mnt/buildbot/build/mariadb-10.5.9/mysql-test/suite/encryption/r/innodb_encryption.result	2020-11-17 11:19:50.000000000 -0500

+++ /mnt/buildbot/build/mariadb-10.5.9/mysql-test/suite/encryption/r/innodb_encryption.reject	2020-11-17 13:18:08.621356124 -0500

@@ -34,6 +34,9 @@

 # since threads are off tables should remain unencrypted

 SET GLOBAL innodb_encrypt_tables = on;

 # Wait 15s to check that nothing gets encrypted

+Timeout in wait_condition.inc for SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_VERSION <> 0;

+Id	User	Host	db	Command	Time	State	Info	Progress

+4	root	localhost	test	Execute	0	starting	show full processlist	0.000

 SELECT NAME FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_VERSION = 0

 AND NAME NOT LIKE 'innodb_undo%' AND NAME NOT LIKE 'mysql/innodb_%_stats' AND NAME NOT LIKE 'mysql/transaction_registry';

 NAME

mysqltest: Result length mismatch

Alice Sherepa added a comment - 2020-11-20 10:14 https://buildbot.askmonty.org/buildbot/builders/kvm-fulltest/builds/26768/steps/mtr_ps/logs/stdio encryption.innodb_encryption 'innodb,undo3' w3 [ fail ] Test ended at 2020-11-17 13:18:08 CURRENT_TEST: encryption.innodb_encryption --- /mnt/buildbot/build/mariadb-10.5.9/mysql-test/suite/encryption/r/innodb_encryption.result 2020-11-17 11:19:50.000000000 -0500 +++ /mnt/buildbot/build/mariadb-10.5.9/mysql-test/suite/encryption/r/innodb_encryption.reject 2020-11-17 13:18:08.621356124 -0500 @@ -34,6 +34,9 @@ # since threads are off tables should remain unencrypted SET GLOBAL innodb_encrypt_tables = on; # Wait 15s to check that nothing gets encrypted +Timeout in wait_condition.inc for SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_VERSION <> 0; +Id User Host db Command Time State Info Progress +4 root localhost test Execute 0 starting show full processlist 0.000 SELECT NAME FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_VERSION = 0 AND NAME NOT LIKE 'innodb_undo%' AND NAME NOT LIKE 'mysql/innodb_%_stats' AND NAME NOT LIKE 'mysql/transaction_registry'; NAME mysqltest: Result length mismatch

Elena Stepanova added a comment - 2021-07-06 12:21

The test has been failing regularly in buildbot for a long time.

Elena Stepanova added a comment - 2021-07-06 12:21 The test has been failing regularly in buildbot for a long time.

Thirunarayanan Balathandayuthapani added a comment - 2021-07-13 09:35

This issue could be solved by ~~MDEV-25998~~ patch

Thirunarayanan Balathandayuthapani added a comment - 2021-07-13 09:35 This issue could be solved by MDEV-25998 patch

Thirunarayanan Balathandayuthapani added a comment - 2021-09-17 12:50

diff --git a/mysql-test/suite/encryption/r/innodb_encryption.result b/mysql-test/suite/encryption/r/innodb_encryption.result

index 75614c6b88c..22b4c3a5ad6 100644

--- a/mysql-test/suite/encryption/r/innodb_encryption.result

+++ b/mysql-test/suite/encryption/r/innodb_encryption.result

@@ -19,7 +19,7 @@ innodb_system

 # Success!

 # Now turn off encryption and wait for threads to decrypt everything

 SET GLOBAL innodb_encrypt_tables = off;

-# Wait max 10 min for key encryption threads to encrypt all spaces

+# Wait max 10 min for key encryption threads to decrypt all spaces

 SELECT NAME FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_VERSION = 0

 AND NAME NOT LIKE 'innodb_undo%' AND NAME NOT LIKE 'mysql/innodb_%_stats';

 NAME

diff --git a/mysql-test/suite/encryption/t/innodb_encryption.test b/mysql-test/suite/encryption/t/innodb_encryption.test

index cd844af0649..303e74fdba4 100644

--- a/mysql-test/suite/encryption/t/innodb_encryption.test

+++ b/mysql-test/suite/encryption/t/innodb_encryption.test

@@ -14,7 +14,9 @@ SHOW VARIABLES LIKE 'innodb_encrypt%';

 SET GLOBAL innodb_encrypt_tables = ON;

---let $tables_count= `select count(*) + 1 from information_schema.tables where engine = 'InnoDB'`

+let $undo_count= `select @@global.innodb_undo_tablespaces`;

+--let $tables_count= `select count(*) + 1 + $undo_count from information_schema.tables where engine = 'InnoDB'`

 --echo # Wait max 10 min for key encryption threads to encrypt all spaces

 --let $wait_timeout= 600

@@ -33,7 +35,7 @@ AND NAME NOT LIKE 'innodb_undo%' AND NAME NOT LIKE 'mysql/innodb_%_stats';

 --echo # Now turn off encryption and wait for threads to decrypt everything

 SET GLOBAL innodb_encrypt_tables = off;

---echo # Wait max 10 min for key encryption threads to encrypt all spaces

 --let $wait_timeout= 600

@@ -33,7 +35,7 @@ AND NAME NOT LIKE 'innodb_undo%' AND NAME NOT LIKE 'mysql/innodb_%_stats';

 --echo # Now turn off encryption and wait for threads to decrypt everything

 SET GLOBAL innodb_encrypt_tables = off;

---echo # Wait max 10 min for key encryption threads to encrypt all spaces

+--echo # Wait max 10 min for key encryption threads to decrypt all spaces

 --let $wait_timeout= 600

 --let $wait_condition=SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_VERSION <> 0;

 --source include/wait_condition.inc

Above patch fixes the test case failure. It fails to count the number of undo tablespaces for encrypting the tablespace

Thirunarayanan Balathandayuthapani added a comment - 2021-09-17 12:50 diff --git a/mysql-test/suite/encryption/r/innodb_encryption.result b/mysql-test/suite/encryption/r/innodb_encryption.result index 75614c6b88c..22b4c3a5ad6 100644 --- a/mysql-test/suite/encryption/r/innodb_encryption.result +++ b/mysql-test/suite/encryption/r/innodb_encryption.result @@ -19,7 +19,7 @@ innodb_system # Success! # Now turn off encryption and wait for threads to decrypt everything SET GLOBAL innodb_encrypt_tables = off; -# Wait max 10 min for key encryption threads to encrypt all spaces +# Wait max 10 min for key encryption threads to decrypt all spaces SELECT NAME FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_VERSION = 0 AND NAME NOT LIKE 'innodb_undo%' AND NAME NOT LIKE 'mysql/innodb_%_stats'; NAME diff --git a/mysql-test/suite/encryption/t/innodb_encryption.test b/mysql-test/suite/encryption/t/innodb_encryption.test index cd844af0649..303e74fdba4 100644 --- a/mysql-test/suite/encryption/t/innodb_encryption.test +++ b/mysql-test/suite/encryption/t/innodb_encryption.test @@ -14,7 +14,9 @@ SHOW VARIABLES LIKE 'innodb_encrypt%'; SET GLOBAL innodb_encrypt_tables = ON; ---let $tables_count= `select count(*) + 1 from information_schema.tables where engine = 'InnoDB'` +let $undo_count= `select @@global.innodb_undo_tablespaces`; + +--let $tables_count= `select count(*) + 1 + $undo_count from information_schema.tables where engine = 'InnoDB'` --echo # Wait max 10 min for key encryption threads to encrypt all spaces --let $wait_timeout= 600 @@ -33,7 +35,7 @@ AND NAME NOT LIKE 'innodb_undo%' AND NAME NOT LIKE 'mysql/innodb_%_stats'; --echo # Now turn off encryption and wait for threads to decrypt everything SET GLOBAL innodb_encrypt_tables = off; ---echo # Wait max 10 min for key encryption threads to encrypt all spaces --let $wait_timeout= 600 @@ -33,7 +35,7 @@ AND NAME NOT LIKE 'innodb_undo%' AND NAME NOT LIKE 'mysql/innodb_%_stats'; --echo # Now turn off encryption and wait for threads to decrypt everything SET GLOBAL innodb_encrypt_tables = off; ---echo # Wait max 10 min for key encryption threads to encrypt all spaces +--echo # Wait max 10 min for key encryption threads to decrypt all spaces --let $wait_timeout= 600 --let $wait_condition=SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_VERSION <> 0; --source include/wait_condition.inc Above patch fixes the test case failure. It fails to count the number of undo tablespaces for encrypting the tablespace

Marko Mäkelä added a comment - 2021-09-17 16:08

Nice work! OK to push.

Marko Mäkelä added a comment - 2021-09-17 16:08 Nice work! OK to push.

People

Assignee:: Thirunarayanan Balathandayuthapani

Reporter:: Elena Stepanova

Votes:: 0 Vote for this issue

Watchers:: 4 Start watching this issue

Dates

Created:: 2018-03-26 15:18

Updated:: 2021-09-21 10:58

Resolved:: 2021-09-17 16:37

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.

MariaDB Server