[MDEV-7093] Instance naming in spider tests does not correlate with buildbot Created: 2014-11-12  Updated: 2023-06-07

Status: Open
Project: MariaDB Server
Component/s: OTHER, Storage Engine - Spider
Affects Version/s: 10.0
Fix Version/s: 10.4, 10.5, 10.6

Type: Bug Priority: Minor
Reporter: Elena Stepanova Assignee: Yuchen Pei
Resolution: Unresolved Votes: 0
Labels: buildbot, spider, tests

Issue Links:
Relates
relates to MDEV-7069 Fix buildbot failures in main server ... Stalled
Sprint: 2017-02

 Description   

Currently mysqld instances in spider tests are named like this:

[mysqld.1.1]
...
[mysqld.2.1]
...

So, when tests are run in buildbot, the error log files are created as

mysqld.1.1.err[.X] 
mysqld.2.1.err[.Y]
...

(X and Y can appear in case of parallel mode, it's irrelevant to the problem).

But buildbot code only looks for these patterns:

mysqld.%d.err
mysqld.%d.err.%d

So, it ignores error logs related to Spider tests, which makes it difficult or impossible to analyze their failures.

I suppose it's not easy to get buildbot code changed, but we should be able to change the spider test config files. E.g. I ran a quick experiment with this change, the test suite still passed:

--- storage/spider/mysql-test/spider/my.cnf	2013-12-25 09:13:15 +0000
+++ storage/spider/mysql-test/spider/my.cnf	2014-11-12 14:10:10 +0000
@@ -1,7 +1,7 @@
 # Use default setting for mysqld processes
 !include include/default_mysqld.cnf
 
-[mysqld.1.1]
+[mysqld.11]
 log-bin=                  master-bin
 loose_handlersocket_port=                20000
 loose_handlersocket_port_wr=             20001
@@ -18,7 +18,7 @@
 open_files_limit=                        4096
 loose_partition=                         1
 
-[mysqld.2.1]
+[mysqld.21]
 loose_handlersocket_port=                20002
 loose_handlersocket_port_wr=             20003
 loose_handlersocket_threads=             2
@@ -33,7 +33,7 @@
 loose_handlersocket_close_table_interval=2
 open_files_limit=                        4096
 
-[mysqld.2.2]
+[mysqld.22]
 loose_handlersocket_port=                20004
 loose_handlersocket_port_wr=             20005
 loose_handlersocket_threads=             2
@@ -48,18 +48,18 @@
 loose_handlersocket_close_table_interval=2
 open_files_limit=                        4096
 
-[mysqld.2.3]
-
-[mysqld.3.1]
-loose_partition=                         1
-
-[mysqld.3.2]
-loose_partition=                         1
-
-[mysqld.3.3]
-loose_partition=                         1
-
-[mysqld.4.1]
+[mysqld.23]
+
+[mysqld.31]
+loose_partition=                         1
+
+[mysqld.32]
+loose_partition=                         1
+
+[mysqld.33]
+loose_partition=                         1
+
+[mysqld.41]
 loose_partition=                         1
 
 
@@ -69,10 +69,10 @@
 USE_HA_TEST=              1
 USE_GENERAL_LOG=          1
 USE_REPLICATION=          1
-MASTER_1_MYPORT=          @mysqld.1.1.port
+MASTER_1_MYPORT=          @mysqld.11.port
 MASTER_1_HSRPORT=         20000
 MASTER_1_HSWPORT=         20001
-MASTER_1_MYSOCK=          @mysqld.1.1.socket
+MASTER_1_MYSOCK=          @mysqld.11.socket
 MASTER_1_ENGINE_TYPE=     Spider
 #MASTER_1_ENGINE_TYPE=     MyISAM
 MASTER_1_ENGINE=          ENGINE=Spider
@@ -80,67 +80,67 @@
 MASTER_1_ENGINE2=         ENGINE=MyISAM
 MASTER_1_CHARSET2=        DEFAULT CHARSET=utf8
 MASTER_1_CHARSET3=        DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
-SLAVE1_1_MYPORT=          @mysqld.4.1.port
-SLAVE1_1_MYSOCK=          @mysqld.4.1.socket
+SLAVE1_1_MYPORT=          @mysqld.41.port
+SLAVE1_1_MYSOCK=          @mysqld.41.socket
 SLAVE1_1_ENGINE_TYPE=     MyISAM
 SLAVE1_1_ENGINE=          ENGINE=MyISAM
 SLAVE1_1_CHARSET=         DEFAULT CHARSET=utf8
 USE_CHILD_GROUP2=         1
 OUTPUT_CHILD_GROUP2=      0
-CHILD2_1_MYPORT=          @mysqld.2.1.port
+CHILD2_1_MYPORT=          @mysqld.21.port
 CHILD2_1_HSRPORT=         20002
 CHILD2_1_HSWPORT=         20003
-CHILD2_1_MYSOCK=          @mysqld.2.1.socket
+CHILD2_1_MYSOCK=          @mysqld.21.socket
 CHILD2_1_ENGINE_TYPE=     InnoDB
 CHILD2_1_ENGINE=          ENGINE=InnoDB
 CHILD2_1_CHARSET=         DEFAULT CHARSET=utf8
 CHILD2_1_CHARSET2=        DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
-CHILD2_2_MYPORT=          @mysqld.2.2.port
+CHILD2_2_MYPORT=          @mysqld.22.port
 CHILD2_2_HSRPORT=         20004
 CHILD2_2_HSWPORT=         20005
-CHILD2_2_MYSOCK=          @mysqld.2.2.socket
+CHILD2_2_MYSOCK=          @mysqld.22.socket
 CHILD2_2_ENGINE_TYPE=     InnoDB
 CHILD2_2_ENGINE=          ENGINE=InnoDB
 CHILD2_2_CHARSET=         DEFAULT CHARSET=utf8
-CHILD2_3_MYPORT=          @mysqld.2.3.port
-CHILD2_3_MYSOCK=          @mysqld.2.3.socket
+CHILD2_3_MYPORT=          @mysqld.23.port
+CHILD2_3_MYSOCK=          @mysqld.23.socket
 CHILD2_3_ENGINE_TYPE=     InnoDB
 CHILD2_3_ENGINE=          ENGINE=InnoDB
 CHILD2_3_CHARSET=         DEFAULT CHARSET=utf8
-CHILD2_1_FT_MYPORT=       @mysqld.2.1.port
-CHILD2_1_FT_MYSOCK=       @mysqld.2.1.socket
+CHILD2_1_FT_MYPORT=       @mysqld.21.port
+CHILD2_1_FT_MYSOCK=       @mysqld.21.socket
 CHILD2_1_FT_ENGINE_TYPE=  MyISAM
 CHILD2_1_FT_ENGINE=       ENGINE=MyISAM
 CHILD2_1_FT_CHARSET=      DEFAULT CHARSET=utf8
-CHILD2_2_FT_MYPORT=       @mysqld.2.2.port
-CHILD2_2_FT_MYSOCK=       @mysqld.2.2.socket
+CHILD2_2_FT_MYPORT=       @mysqld.22.port
+CHILD2_2_FT_MYSOCK=       @mysqld.22.socket
 CHILD2_2_FT_ENGINE_TYPE=  MyISAM
 CHILD2_2_FT_ENGINE=       ENGINE=MyISAM
 CHILD2_2_FT_CHARSET=      DEFAULT CHARSET=utf8
-CHILD2_1_GM_MYPORT=       @mysqld.2.1.port
-CHILD2_1_GM_MYSOCK=       @mysqld.2.1.socket
+CHILD2_1_GM_MYPORT=       @mysqld.21.port
+CHILD2_1_GM_MYSOCK=       @mysqld.21.socket
 CHILD2_1_GM_ENGINE_TYPE=  MyISAM
 CHILD2_1_GM_ENGINE=       ENGINE=MyISAM
 CHILD2_1_GM_CHARSET=      DEFAULT CHARSET=utf8
-CHILD2_2_GM_MYPORT=       @mysqld.2.2.port
-CHILD2_2_GM_MYSOCK=       @mysqld.2.2.socket
+CHILD2_2_GM_MYPORT=       @mysqld.22.port
+CHILD2_2_GM_MYSOCK=       @mysqld.22.socket
 CHILD2_2_GM_ENGINE_TYPE=  MyISAM
 CHILD2_2_GM_ENGINE=       ENGINE=MyISAM
 CHILD2_2_GM_CHARSET=      DEFAULT CHARSET=utf8
 USE_CHILD_GROUP3=         1
 OUTPUT_CHILD_GROUP3=      0
-CHILD3_1_MYPORT=          @mysqld.3.1.port
-CHILD3_1_MYSOCK=          @mysqld.3.1.socket
+CHILD3_1_MYPORT=          @mysqld.31.port
+CHILD3_1_MYSOCK=          @mysqld.31.socket
 CHILD3_1_ENGINE_TYPE=     InnoDB
 CHILD3_1_ENGINE=          ENGINE=InnoDB
 CHILD3_1_CHARSET=         DEFAULT CHARSET=utf8
-CHILD3_2_MYPORT=          @mysqld.3.2.port
-CHILD3_2_MYSOCK=          @mysqld.3.2.socket
+CHILD3_2_MYPORT=          @mysqld.32.port
+CHILD3_2_MYSOCK=          @mysqld.32.socket
 CHILD3_2_ENGINE_TYPE=     InnoDB
 CHILD3_2_ENGINE=          ENGINE=InnoDB
 CHILD3_2_CHARSET=         DEFAULT CHARSET=utf8
-CHILD3_3_MYPORT=          @mysqld.3.3.port
-CHILD3_3_MYSOCK=          @mysqld.3.3.socket
+CHILD3_3_MYPORT=          @mysqld.33.port
+CHILD3_3_MYSOCK=          @mysqld.33.socket
 CHILD3_3_ENGINE_TYPE=     InnoDB
 CHILD3_3_ENGINE=          ENGINE=InnoDB
 CHILD3_3_CHARSET=         DEFAULT CHARSET=utf8

Please consider doing something similar, to make the final error log to match the buildbot pattern.



 Comments   
Comment by Nayuta Yanagisawa (Inactive) [ 2021-08-13 ]

Hi elenst! Is this issue is still valide?

Comment by Sergei Golubchik [ 2021-08-14 ]

Yes, apparently it is:

$ find storage/spider/mysql-test -name '*.cnf'|xargs grep '\[mysqld\.[0-9]'|head
storage/spider/mysql-test/spider/bugfix/t/slave_transaction_retry_errors_5digit.cnf:[mysqld.4.1]
storage/spider/mysql-test/spider/bugfix/my_3_3.cnf:[mysqld.3.3]
storage/spider/mysql-test/spider/bugfix/my_2_3.cnf:[mysqld.2.3]
storage/spider/mysql-test/spider/bugfix/my_3_2.cnf:[mysqld.3.2]
storage/spider/mysql-test/spider/bugfix/my_4_1.cnf:[mysqld.4.1]
storage/spider/mysql-test/spider/bugfix/my_1_1.cnf:[mysqld.1.1]
storage/spider/mysql-test/spider/bugfix/my_2_1.cnf:[mysqld.2.1]
storage/spider/mysql-test/spider/bugfix/my_3_1.cnf:[mysqld.3.1]
storage/spider/mysql-test/spider/bugfix/my_2_2.cnf:[mysqld.2.2]
storage/spider/mysql-test/spider/my.cnf:[mysqld.1.1]

Generated at Thu Feb 08 07:16:57 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.