Details
Blocker Description
Replace the black list of unstable tests introduced in MDEV-10604 and maintained since then with a list of tests which distributions can run in their build/packaging process.
Since the goal of running MTR tests during 3-rd party build process is not to test MariaDB code but only to ensure that the build is functional, there is no need to run thousands of MariaDB tests and combinations. So, the new collection will be much shorter and more static than the previous unstable-tests list.
Criteria:
- separate components (plugins, clients, scripts) should be represented by at least one test when possible;
- every suite should be represented by at least one test, except for those which aren't run regularly and thus aren't maintained properly;
- the tests shouldn't have any sporadic failures on main maintenance branches in the last half of a year (*);
- the tests can't require debug build, windows, or a special configuration not normally present on build machines;
- out of the remaining candidates, the priority is given to those with higher line coverage and/or more generic ones (judging by name) and/or more those which closer relate to the system/environment (judging by name).
(*) Once the list is created, this condition won't be explicitly re-checked before the release, as it was done for unstable-tests. Instead, we should have an buildbot step which would run these tests, and its failure should be "fatal" and require an immediate action.
|
Draft list for 10.2 (first edition) |
archive.archive
|
archive.archive_gis
|
archive.partition_archive
|
binlog.binlog_base64_flag
|
binlog.binlog_database
|
binlog.binlog_innodb
|
binlog.binlog_parallel_replication_marks_stm_mix
|
binlog.binlog_row_mix_innodb_myisam
|
binlog.flashback
|
binlog_encryption.encrypted_slave
|
connect.bin
|
connect.bson
|
connect.dbf
|
connect.dir
|
connect.endian
|
connect.general
|
connect.json
|
connect.mysql
|
connect.secure_file_priv
|
connect.tbl
|
csv.csv
|
disks.disks
|
encryption.encryption_force
|
encryption.innodb_encryption_tables
|
encryption.tempfiles_encrypted
|
federated.federated
|
federated.federatedx
|
gcol.gcol_select_innodb
|
handler.interface
|
heap.heap
|
innodb.innodb
|
innodb.autoinc_persist
|
innodb.innodb_defrag_binlog
|
innodb.innodb_mysql
|
innodb.monitor
|
innodb.purge
|
innodb.table_flags
|
innodb.xa_recovery
|
innodb_fts.fulltext
|
innodb_gis.geometry
|
innodb_gis.rtree
|
innodb_zip.innodb-zip
|
innodb_zip.page_size
|
json.json_no_table
|
main-test_sql_discovery.create
|
main.blackhole
|
main.bootstrap
|
main.compress
|
main.connect
|
main.ctype_collate
|
main.ctype_utf8
|
main.default
|
main.dyncol
|
main.fulltext
|
main.function_defaults
|
main.gis
|
main.grant
|
main.handlersocket
|
main.information_schema
|
main.innodb_ext_key
|
main.log_tables
|
main.lowercase_fs_off
|
main.myisam
|
main.mysql_client_test
|
main.mysql_protocols
|
main.mysql_upgrade
|
main.mysqladmin
|
main.mysqlbinlog
|
main.mysqlcheck
|
main.mysqld--defaults-file
|
main.mysqldump
|
main.mysqlhotcopy_myisam
|
main.mysqlshow
|
main.mysqlslap
|
main.mysqltest
|
main.parser
|
main.partition
|
main.perror
|
main.plugin
|
main.plugin_auth
|
main.pool_of_threads
|
main.ps
|
main.repair
|
main.shutdown
|
main.sp
|
main.ssl
|
main.ssl_compress
|
main.stat_tables
|
main.statistics
|
main.subselect
|
main.symlink
|
main.temp_table
|
main.timezone
|
main.type_timestamp_hires
|
main.user_var
|
main.userstat
|
main.variables
|
main.view
|
main.win
|
main.xa
|
maria.maria
|
mariabackup.full_backup
|
metadata_lock_info.table_metadata_lock
|
mroonga/storage.variable_version
|
mroonga/wrapper.count_star
|
multi_source.multisource
|
oqgraph.general-innodb
|
parts.rpl_partition
|
perfschema.selects
|
plugins.auth_ed25519
|
plugins.cracklib_password_check
|
plugins.dialog
|
plugins.fulltext_plugin
|
plugins.locales
|
plugins.pam_cleartext
|
plugins.processlist
|
plugins.qc_info
|
plugins.server_audit
|
plugins.simple_password_check
|
plugins.sql_error_log
|
plugins.two_password_validations
|
plugins.unix_socket
|
query_response_time.basic
|
rocksdb.rocksdb
|
roles.definer
|
rpl.rpl_gtid_basic
|
rpl.rpl_relayrotate
|
rpl.rpl_row_blob_innodb
|
rpl.rpl_semi_sync_event
|
rpl.rpl_sp
|
rpl.rpl_stm_binlog_max_cache_size
|
rpl.rpl_switch_stm_row_mixed
|
sequence.simple
|
spider.basic_sql
|
spider.ha
|
sql_discovery.simple
|
sys_vars.sysvars_aria
|
sys_vars.sysvars_server_notembedded
|
vcol.vcol_syntax
|
wsrep.variables
|
On my machine it takes ~3 min on a non-debug 10.2 build, with MTR running in shm with --parallel=4.
|
10.3 change over 10.2 list |
--- a/mysql-test/collections/smoke_test
|
+++ b/mysql-test/collections/smoke_test
|
@@ -48,7 +48,8 @@ innodb_gis.rtree
|
innodb_zip.innodb-zip
|
innodb_zip.page_size
|
json.json_no_table
|
-main-test_sql_discovery.create
|
+# Doesn't work in 10.3+ due to MDEV-25384
|
+# main-test_sql_discovery.create
|
main.blackhole
|
main.bootstrap
|
main.compress
|
@@ -145,13 +146,22 @@ sys_vars.sysvars_server_notembedded
|
wsrep.variables
|
"
|
|
+tests_103="
|
+compat/oracle.binlog_ptr_mysqlbinlog
|
+compat/oracle.sp-package
|
+compat/maxdb.rpl_mariadb_timestamp
|
+sql_sequence.mysqldump
|
+versioning.simple
|
+versioning.trx_id
|
+"
|
+
|
tests_to_run=""
|
|
while read -r test
|
do
|
test=`echo $test | sed -e s/\#.*//`
|
tests_to_run="$tests_to_run $test"
|
-done <<< $(echo "$tests_102")
|
+done <<< $(echo "$tests_102 $tests_103")
|
|
echo $tests_to_run
|
|
|
10.4 additions to 10.3 list |
diff --git a/mysql-test/collections/smoke_test b/mysql-test/collections/smoke_test
|
index 445e0f6d529..986d9856ac2 100755
|
--- a/mysql-test/collections/smoke_test
|
+++ b/mysql-test/collections/smoke_test
|
@@ -154,6 +154,10 @@ sql_sequence.mysqldump
|
versioning.simple
|
versioning.trx_id
|
"
|
+tests_104="
|
+period.versioning
|
+plugins.multiauth
|
+"
|
|
tests_to_run=""
|
|
@@ -161,7 +165,7 @@ while read -r test
|
do
|
test=`echo $test | sed -e s/\#.*//`
|
tests_to_run="$tests_to_run $test"
|
-done <<< $(echo "$tests_102 $tests_103")
|
+done <<< $(echo "$tests_102 $tests_103 $tests_104")
|
|
echo $tests_to_run
|
|
10.5 additions to 10.4 list |
diff --git a/mysql-test/collections/smoke_test b/mysql-test/collections/smoke_test
|
index 986d9856ac2..2391f4f6057 100755
|
--- a/mysql-test/collections/smoke_test
|
+++ b/mysql-test/collections/smoke_test
|
@@ -159,13 +159,18 @@ period.versioning
|
plugins.multiauth
|
"
|
|
+tests_105="
|
+client.mariadb-conv
|
+innodb_i_s.innodb_sys_tables
|
+"
|
+
|
tests_to_run=""
|
|
while read -r test
|
do
|
test=`echo $test | sed -e s/\#.*//`
|
tests_to_run="$tests_to_run $test"
|
-done <<< $(echo "$tests_102 $tests_103 $tests_104")
|
+done <<< $(echo "$tests_102 $tests_103 $tests_104 $tests_105")
|
|
echo $tests_to_run
|
In the current edition, the list is implemented as a shell script under mysql-test/collections/
It runs MTR with the pre-defined list of tests and optionally with other MTR parameters passed from the command line.
One disadvantage of having it this way (and same would be if it was a plain text list) is that i out-of-source builds it will be a bit awkward to call, because collections remain in the sourcedir, together with MTR tests.
For example, in the old buildbot deb package builds, it would be something like
cd builddir/mysql-test
|
../../mysql-test/collections/smoke_test <other options>
|
Attachments
Issue Links
- relates to
-
MDEV-7069 Fix buildbot failures in main server trees
-
- Stalled
-
-
MDEV-10605 main.create_delayed failed in buildbot with a timeout
-
- Open
-
Activity
| Link | This issue relates to MDEV-10605 [ MDEV-10605 ] |
| Priority | Major [ 3 ] | Blocker [ 1 ] |
| Status | Open [ 1 ] | In Progress [ 3 ] |
| Status | In Progress [ 3 ] | Stalled [ 10000 ] |
| Description |
Replace the black list of unstable tests introduced in Since the goal of running MTR tests during 3-rd party build process is not to test MariaDB *code* but only to ensure that the build is functional, there is no need to run thousands of MariaDB tests and combinations. So, the new collection will be much shorter and more static than the previous unstable-tests list. |
Replace the black list of unstable tests introduced in Since the goal of running MTR tests during 3-rd party build process is not to test MariaDB *code* but only to ensure that the build is functional, there is no need to run thousands of MariaDB tests and combinations. So, the new collection will be much shorter and more static than the previous unstable-tests list. Criteria: - separate components (plugins, clients, scripts) should be represented by at least one test when possible; - every suite should be represented by at least one test, except for those which aren't run regularly and thus aren't maintained properly; - the tests shouldn't have any sporadic failures on main maintenance branches in the last half of a year \(\*\); - the tests can't require debug build, windows, or a special configuration not normally present on build machines; - out of the remaining candidates, the priority is given to those with higher line coverage and/or more generic ones (judging by name) and/or more those which closer relate to the system/environment (judging by name). \(\*\) _Once the list is created, this condition won't be explicitly re-checked before the release, as it was done for unstable-tests. Instead, we should have an buildbot step which would run these tests, and its failure should be "fatal" and require an immediate action._ {noformat:title=Draft list for 10.2 (first edition)} archive.archive archive.archive_gis archive.partition_archive binlog.binlog_base64_flag binlog.binlog_database binlog.binlog_innodb binlog.binlog_parallel_replication_marks_stm_mix binlog.binlog_row_mix_innodb_myisam binlog.flashback binlog_encryption.encrypted_slave connect.bin connect.bson connect.dbf connect.dir connect.endian connect.general connect.json connect.mysql connect.secure_file_priv connect.tbl csv.csv disks.disks encryption.encryption_force encryption.innodb_encryption_tables encryption.tempfiles_encrypted federated.federated federated.federatedx gcol.gcol_select_innodb handler.interface heap.heap innodb.innodb innodb.autoinc_persist innodb.innodb_defrag_binlog innodb.innodb_mysql innodb.monitor innodb.purge innodb.table_flags innodb.xa_recovery innodb_fts.fulltext innodb_gis.geometry innodb_gis.rtree innodb_zip.innodb-zip innodb_zip.page_size json.json_no_table main-test_sql_discovery.create main.blackhole main.bootstrap main.compress main.connect main.ctype_collate main.ctype_utf8 main.default main.dyncol main.fulltext main.function_defaults main.gis main.grant main.handlersocket main.information_schema main.innodb_ext_key main.log_tables main.lowercase_fs_off main.myisam main.mysql_client_test main.mysql_protocols main.mysql_upgrade main.mysqladmin main.mysqlbinlog main.mysqlcheck main.mysqld--defaults-file main.mysqldump main.mysqlhotcopy_myisam main.mysqlshow main.mysqlslap main.mysqltest main.parser main.partition main.perror main.plugin main.plugin_auth main.pool_of_threads main.ps main.repair main.shutdown main.sp main.ssl main.ssl_compress main.stat_tables main.statistics main.subselect main.symlink main.temp_table main.timezone main.type_timestamp_hires main.user_var main.userstat main.variables main.view main.win main.xa maria.maria mariabackup.full_backup metadata_lock_info.table_metadata_lock mroonga/storage.variable_version mroonga/wrapper.count_star multi_source.multisource oqgraph.general-innodb parts.rpl_partition perfschema.selects plugins.auth_ed25519 plugins.cracklib_password_check plugins.dialog plugins.fulltext_plugin plugins.locales plugins.pam_cleartext plugins.processlist plugins.qc_info plugins.server_audit plugins.simple_password_check plugins.sql_error_log plugins.two_password_validations plugins.unix_socket query_response_time.basic rocksdb.rocksdb roles.definer rpl.rpl_gtid_basic rpl.rpl_relayrotate rpl.rpl_row_blob_innodb rpl.rpl_semi_sync_event rpl.rpl_sp rpl.rpl_stm_binlog_max_cache_size rpl.rpl_switch_stm_row_mixed sequence.simple spider.basic_sql spider.ha sql_discovery.simple sys_vars.sysvars_aria sys_vars.sysvars_server_notembedded vcol.vcol_syntax wsrep.variables {noformat} On my machine it takes ~3 min on a non-debug 10.2 build, with MTR running in shm with {{--parallel=4}}. |
| Description |
Replace the black list of unstable tests introduced in Since the goal of running MTR tests during 3-rd party build process is not to test MariaDB *code* but only to ensure that the build is functional, there is no need to run thousands of MariaDB tests and combinations. So, the new collection will be much shorter and more static than the previous unstable-tests list. Criteria: - separate components (plugins, clients, scripts) should be represented by at least one test when possible; - every suite should be represented by at least one test, except for those which aren't run regularly and thus aren't maintained properly; - the tests shouldn't have any sporadic failures on main maintenance branches in the last half of a year \(\*\); - the tests can't require debug build, windows, or a special configuration not normally present on build machines; - out of the remaining candidates, the priority is given to those with higher line coverage and/or more generic ones (judging by name) and/or more those which closer relate to the system/environment (judging by name). \(\*\) _Once the list is created, this condition won't be explicitly re-checked before the release, as it was done for unstable-tests. Instead, we should have an buildbot step which would run these tests, and its failure should be "fatal" and require an immediate action._ {noformat:title=Draft list for 10.2 (first edition)} archive.archive archive.archive_gis archive.partition_archive binlog.binlog_base64_flag binlog.binlog_database binlog.binlog_innodb binlog.binlog_parallel_replication_marks_stm_mix binlog.binlog_row_mix_innodb_myisam binlog.flashback binlog_encryption.encrypted_slave connect.bin connect.bson connect.dbf connect.dir connect.endian connect.general connect.json connect.mysql connect.secure_file_priv connect.tbl csv.csv disks.disks encryption.encryption_force encryption.innodb_encryption_tables encryption.tempfiles_encrypted federated.federated federated.federatedx gcol.gcol_select_innodb handler.interface heap.heap innodb.innodb innodb.autoinc_persist innodb.innodb_defrag_binlog innodb.innodb_mysql innodb.monitor innodb.purge innodb.table_flags innodb.xa_recovery innodb_fts.fulltext innodb_gis.geometry innodb_gis.rtree innodb_zip.innodb-zip innodb_zip.page_size json.json_no_table main-test_sql_discovery.create main.blackhole main.bootstrap main.compress main.connect main.ctype_collate main.ctype_utf8 main.default main.dyncol main.fulltext main.function_defaults main.gis main.grant main.handlersocket main.information_schema main.innodb_ext_key main.log_tables main.lowercase_fs_off main.myisam main.mysql_client_test main.mysql_protocols main.mysql_upgrade main.mysqladmin main.mysqlbinlog main.mysqlcheck main.mysqld--defaults-file main.mysqldump main.mysqlhotcopy_myisam main.mysqlshow main.mysqlslap main.mysqltest main.parser main.partition main.perror main.plugin main.plugin_auth main.pool_of_threads main.ps main.repair main.shutdown main.sp main.ssl main.ssl_compress main.stat_tables main.statistics main.subselect main.symlink main.temp_table main.timezone main.type_timestamp_hires main.user_var main.userstat main.variables main.view main.win main.xa maria.maria mariabackup.full_backup metadata_lock_info.table_metadata_lock mroonga/storage.variable_version mroonga/wrapper.count_star multi_source.multisource oqgraph.general-innodb parts.rpl_partition perfschema.selects plugins.auth_ed25519 plugins.cracklib_password_check plugins.dialog plugins.fulltext_plugin plugins.locales plugins.pam_cleartext plugins.processlist plugins.qc_info plugins.server_audit plugins.simple_password_check plugins.sql_error_log plugins.two_password_validations plugins.unix_socket query_response_time.basic rocksdb.rocksdb roles.definer rpl.rpl_gtid_basic rpl.rpl_relayrotate rpl.rpl_row_blob_innodb rpl.rpl_semi_sync_event rpl.rpl_sp rpl.rpl_stm_binlog_max_cache_size rpl.rpl_switch_stm_row_mixed sequence.simple spider.basic_sql spider.ha sql_discovery.simple sys_vars.sysvars_aria sys_vars.sysvars_server_notembedded vcol.vcol_syntax wsrep.variables {noformat} On my machine it takes ~3 min on a non-debug 10.2 build, with MTR running in shm with {{--parallel=4}}. |
Replace the black list of unstable tests introduced in Since the goal of running MTR tests during 3-rd party build process is not to test MariaDB *code* but only to ensure that the build is functional, there is no need to run thousands of MariaDB tests and combinations. So, the new collection will be much shorter and more static than the previous unstable-tests list. Criteria: - separate components (plugins, clients, scripts) should be represented by at least one test when possible; - every suite should be represented by at least one test, except for those which aren't run regularly and thus aren't maintained properly; - the tests shouldn't have any sporadic failures on main maintenance branches in the last half of a year \(\*\); - the tests can't require debug build, windows, or a special configuration not normally present on build machines; - out of the remaining candidates, the priority is given to those with higher line coverage and/or more generic ones (judging by name) and/or more those which closer relate to the system/environment (judging by name). \(\*\) _Once the list is created, this condition won't be explicitly re-checked before the release, as it was done for unstable-tests. Instead, we should have an buildbot step which would run these tests, and its failure should be "fatal" and require an immediate action._ {noformat:title=Draft list for 10.2 (first edition)} archive.archive archive.archive_gis archive.partition_archive binlog.binlog_base64_flag binlog.binlog_database binlog.binlog_innodb binlog.binlog_parallel_replication_marks_stm_mix binlog.binlog_row_mix_innodb_myisam binlog.flashback binlog_encryption.encrypted_slave connect.bin connect.bson connect.dbf connect.dir connect.endian connect.general connect.json connect.mysql connect.secure_file_priv connect.tbl csv.csv disks.disks encryption.encryption_force encryption.innodb_encryption_tables encryption.tempfiles_encrypted federated.federated federated.federatedx gcol.gcol_select_innodb handler.interface heap.heap innodb.innodb innodb.autoinc_persist innodb.innodb_defrag_binlog innodb.innodb_mysql innodb.monitor innodb.purge innodb.table_flags innodb.xa_recovery innodb_fts.fulltext innodb_gis.geometry innodb_gis.rtree innodb_zip.innodb-zip innodb_zip.page_size json.json_no_table main-test_sql_discovery.create main.blackhole main.bootstrap main.compress main.connect main.ctype_collate main.ctype_utf8 main.default main.dyncol main.fulltext main.function_defaults main.gis main.grant main.handlersocket main.information_schema main.innodb_ext_key main.log_tables main.lowercase_fs_off main.myisam main.mysql_client_test main.mysql_protocols main.mysql_upgrade main.mysqladmin main.mysqlbinlog main.mysqlcheck main.mysqld--defaults-file main.mysqldump main.mysqlhotcopy_myisam main.mysqlshow main.mysqlslap main.mysqltest main.parser main.partition main.perror main.plugin main.plugin_auth main.pool_of_threads main.ps main.repair main.shutdown main.sp main.ssl main.ssl_compress main.stat_tables main.statistics main.subselect main.symlink main.temp_table main.timezone main.type_timestamp_hires main.user_var main.userstat main.variables main.view main.win main.xa maria.maria mariabackup.full_backup metadata_lock_info.table_metadata_lock mroonga/storage.variable_version mroonga/wrapper.count_star multi_source.multisource oqgraph.general-innodb parts.rpl_partition perfschema.selects plugins.auth_ed25519 plugins.cracklib_password_check plugins.dialog plugins.fulltext_plugin plugins.locales plugins.pam_cleartext plugins.processlist plugins.qc_info plugins.server_audit plugins.simple_password_check plugins.sql_error_log plugins.two_password_validations plugins.unix_socket query_response_time.basic rocksdb.rocksdb roles.definer rpl.rpl_gtid_basic rpl.rpl_relayrotate rpl.rpl_row_blob_innodb rpl.rpl_semi_sync_event rpl.rpl_sp rpl.rpl_stm_binlog_max_cache_size rpl.rpl_switch_stm_row_mixed sequence.simple spider.basic_sql spider.ha sql_discovery.simple sys_vars.sysvars_aria sys_vars.sysvars_server_notembedded vcol.vcol_syntax wsrep.variables {noformat} On my machine it takes ~3 min on a non-debug 10.2 build, with MTR running in shm with {{--parallel=4}}. {noformat:title=10.3 change over 10.2 list} --- a/mysql-test/collections/smoke_test +++ b/mysql-test/collections/smoke_test @@ -48,7 +48,8 @@ innodb_gis.rtree innodb_zip.innodb-zip innodb_zip.page_size json.json_no_table -main-test_sql_discovery.create +# Doesn't work in 10.3+ due to MDEV-25384 +# main-test_sql_discovery.create main.blackhole main.bootstrap main.compress @@ -145,13 +146,22 @@ sys_vars.sysvars_server_notembedded wsrep.variables " +tests_103=" +compat/oracle.binlog_ptr_mysqlbinlog +compat/oracle.sp-package +compat/maxdb.rpl_mariadb_timestamp +sql_sequence.mysqldump +versioning.simple +versioning.trx_id +" + tests_to_run="" while read -r test do test=`echo $test | sed -e s/\#.*//` tests_to_run="$tests_to_run $test" -done <<< $(echo "$tests_102") +done <<< $(echo "$tests_102 $tests_103") echo $tests_to_run {noformat} |
| Description |
Replace the black list of unstable tests introduced in Since the goal of running MTR tests during 3-rd party build process is not to test MariaDB *code* but only to ensure that the build is functional, there is no need to run thousands of MariaDB tests and combinations. So, the new collection will be much shorter and more static than the previous unstable-tests list. Criteria: - separate components (plugins, clients, scripts) should be represented by at least one test when possible; - every suite should be represented by at least one test, except for those which aren't run regularly and thus aren't maintained properly; - the tests shouldn't have any sporadic failures on main maintenance branches in the last half of a year \(\*\); - the tests can't require debug build, windows, or a special configuration not normally present on build machines; - out of the remaining candidates, the priority is given to those with higher line coverage and/or more generic ones (judging by name) and/or more those which closer relate to the system/environment (judging by name). \(\*\) _Once the list is created, this condition won't be explicitly re-checked before the release, as it was done for unstable-tests. Instead, we should have an buildbot step which would run these tests, and its failure should be "fatal" and require an immediate action._ {noformat:title=Draft list for 10.2 (first edition)} archive.archive archive.archive_gis archive.partition_archive binlog.binlog_base64_flag binlog.binlog_database binlog.binlog_innodb binlog.binlog_parallel_replication_marks_stm_mix binlog.binlog_row_mix_innodb_myisam binlog.flashback binlog_encryption.encrypted_slave connect.bin connect.bson connect.dbf connect.dir connect.endian connect.general connect.json connect.mysql connect.secure_file_priv connect.tbl csv.csv disks.disks encryption.encryption_force encryption.innodb_encryption_tables encryption.tempfiles_encrypted federated.federated federated.federatedx gcol.gcol_select_innodb handler.interface heap.heap innodb.innodb innodb.autoinc_persist innodb.innodb_defrag_binlog innodb.innodb_mysql innodb.monitor innodb.purge innodb.table_flags innodb.xa_recovery innodb_fts.fulltext innodb_gis.geometry innodb_gis.rtree innodb_zip.innodb-zip innodb_zip.page_size json.json_no_table main-test_sql_discovery.create main.blackhole main.bootstrap main.compress main.connect main.ctype_collate main.ctype_utf8 main.default main.dyncol main.fulltext main.function_defaults main.gis main.grant main.handlersocket main.information_schema main.innodb_ext_key main.log_tables main.lowercase_fs_off main.myisam main.mysql_client_test main.mysql_protocols main.mysql_upgrade main.mysqladmin main.mysqlbinlog main.mysqlcheck main.mysqld--defaults-file main.mysqldump main.mysqlhotcopy_myisam main.mysqlshow main.mysqlslap main.mysqltest main.parser main.partition main.perror main.plugin main.plugin_auth main.pool_of_threads main.ps main.repair main.shutdown main.sp main.ssl main.ssl_compress main.stat_tables main.statistics main.subselect main.symlink main.temp_table main.timezone main.type_timestamp_hires main.user_var main.userstat main.variables main.view main.win main.xa maria.maria mariabackup.full_backup metadata_lock_info.table_metadata_lock mroonga/storage.variable_version mroonga/wrapper.count_star multi_source.multisource oqgraph.general-innodb parts.rpl_partition perfschema.selects plugins.auth_ed25519 plugins.cracklib_password_check plugins.dialog plugins.fulltext_plugin plugins.locales plugins.pam_cleartext plugins.processlist plugins.qc_info plugins.server_audit plugins.simple_password_check plugins.sql_error_log plugins.two_password_validations plugins.unix_socket query_response_time.basic rocksdb.rocksdb roles.definer rpl.rpl_gtid_basic rpl.rpl_relayrotate rpl.rpl_row_blob_innodb rpl.rpl_semi_sync_event rpl.rpl_sp rpl.rpl_stm_binlog_max_cache_size rpl.rpl_switch_stm_row_mixed sequence.simple spider.basic_sql spider.ha sql_discovery.simple sys_vars.sysvars_aria sys_vars.sysvars_server_notembedded vcol.vcol_syntax wsrep.variables {noformat} On my machine it takes ~3 min on a non-debug 10.2 build, with MTR running in shm with {{--parallel=4}}. {noformat:title=10.3 change over 10.2 list} --- a/mysql-test/collections/smoke_test +++ b/mysql-test/collections/smoke_test @@ -48,7 +48,8 @@ innodb_gis.rtree innodb_zip.innodb-zip innodb_zip.page_size json.json_no_table -main-test_sql_discovery.create +# Doesn't work in 10.3+ due to MDEV-25384 +# main-test_sql_discovery.create main.blackhole main.bootstrap main.compress @@ -145,13 +146,22 @@ sys_vars.sysvars_server_notembedded wsrep.variables " +tests_103=" +compat/oracle.binlog_ptr_mysqlbinlog +compat/oracle.sp-package +compat/maxdb.rpl_mariadb_timestamp +sql_sequence.mysqldump +versioning.simple +versioning.trx_id +" + tests_to_run="" while read -r test do test=`echo $test | sed -e s/\#.*//` tests_to_run="$tests_to_run $test" -done <<< $(echo "$tests_102") +done <<< $(echo "$tests_102 $tests_103") echo $tests_to_run {noformat} |
Replace the black list of unstable tests introduced in Since the goal of running MTR tests during 3-rd party build process is not to test MariaDB *code* but only to ensure that the build is functional, there is no need to run thousands of MariaDB tests and combinations. So, the new collection will be much shorter and more static than the previous unstable-tests list. Criteria: - separate components (plugins, clients, scripts) should be represented by at least one test when possible; - every suite should be represented by at least one test, except for those which aren't run regularly and thus aren't maintained properly; - the tests shouldn't have any sporadic failures on main maintenance branches in the last half of a year \(\*\); - the tests can't require debug build, windows, or a special configuration not normally present on build machines; - out of the remaining candidates, the priority is given to those with higher line coverage and/or more generic ones (judging by name) and/or more those which closer relate to the system/environment (judging by name). \(\*\) _Once the list is created, this condition won't be explicitly re-checked before the release, as it was done for unstable-tests. Instead, we should have an buildbot step which would run these tests, and its failure should be "fatal" and require an immediate action._ {noformat:title=Draft list for 10.2 (first edition)} archive.archive archive.archive_gis archive.partition_archive binlog.binlog_base64_flag binlog.binlog_database binlog.binlog_innodb binlog.binlog_parallel_replication_marks_stm_mix binlog.binlog_row_mix_innodb_myisam binlog.flashback binlog_encryption.encrypted_slave connect.bin connect.bson connect.dbf connect.dir connect.endian connect.general connect.json connect.mysql connect.secure_file_priv connect.tbl csv.csv disks.disks encryption.encryption_force encryption.innodb_encryption_tables encryption.tempfiles_encrypted federated.federated federated.federatedx gcol.gcol_select_innodb handler.interface heap.heap innodb.innodb innodb.autoinc_persist innodb.innodb_defrag_binlog innodb.innodb_mysql innodb.monitor innodb.purge innodb.table_flags innodb.xa_recovery innodb_fts.fulltext innodb_gis.geometry innodb_gis.rtree innodb_zip.innodb-zip innodb_zip.page_size json.json_no_table main-test_sql_discovery.create main.blackhole main.bootstrap main.compress main.connect main.ctype_collate main.ctype_utf8 main.default main.dyncol main.fulltext main.function_defaults main.gis main.grant main.handlersocket main.information_schema main.innodb_ext_key main.log_tables main.lowercase_fs_off main.myisam main.mysql_client_test main.mysql_protocols main.mysql_upgrade main.mysqladmin main.mysqlbinlog main.mysqlcheck main.mysqld--defaults-file main.mysqldump main.mysqlhotcopy_myisam main.mysqlshow main.mysqlslap main.mysqltest main.parser main.partition main.perror main.plugin main.plugin_auth main.pool_of_threads main.ps main.repair main.shutdown main.sp main.ssl main.ssl_compress main.stat_tables main.statistics main.subselect main.symlink main.temp_table main.timezone main.type_timestamp_hires main.user_var main.userstat main.variables main.view main.win main.xa maria.maria mariabackup.full_backup metadata_lock_info.table_metadata_lock mroonga/storage.variable_version mroonga/wrapper.count_star multi_source.multisource oqgraph.general-innodb parts.rpl_partition perfschema.selects plugins.auth_ed25519 plugins.cracklib_password_check plugins.dialog plugins.fulltext_plugin plugins.locales plugins.pam_cleartext plugins.processlist plugins.qc_info plugins.server_audit plugins.simple_password_check plugins.sql_error_log plugins.two_password_validations plugins.unix_socket query_response_time.basic rocksdb.rocksdb roles.definer rpl.rpl_gtid_basic rpl.rpl_relayrotate rpl.rpl_row_blob_innodb rpl.rpl_semi_sync_event rpl.rpl_sp rpl.rpl_stm_binlog_max_cache_size rpl.rpl_switch_stm_row_mixed sequence.simple spider.basic_sql spider.ha sql_discovery.simple sys_vars.sysvars_aria sys_vars.sysvars_server_notembedded vcol.vcol_syntax wsrep.variables {noformat} On my machine it takes ~3 min on a non-debug 10.2 build, with MTR running in shm with {{--parallel=4}}. {noformat:title=10.3 change over 10.2 list} --- a/mysql-test/collections/smoke_test +++ b/mysql-test/collections/smoke_test @@ -48,7 +48,8 @@ innodb_gis.rtree innodb_zip.innodb-zip innodb_zip.page_size json.json_no_table -main-test_sql_discovery.create +# Doesn't work in 10.3+ due to MDEV-25384 +# main-test_sql_discovery.create main.blackhole main.bootstrap main.compress @@ -145,13 +146,22 @@ sys_vars.sysvars_server_notembedded wsrep.variables " +tests_103=" +compat/oracle.binlog_ptr_mysqlbinlog +compat/oracle.sp-package +compat/maxdb.rpl_mariadb_timestamp +sql_sequence.mysqldump +versioning.simple +versioning.trx_id +" + tests_to_run="" while read -r test do test=`echo $test | sed -e s/\#.*//` tests_to_run="$tests_to_run $test" -done <<< $(echo "$tests_102") +done <<< $(echo "$tests_102 $tests_103") echo $tests_to_run {noformat} {noformat:title=10.4 additions to 10.3 list} diff --git a/mysql-test/collections/smoke_test b/mysql-test/collections/smoke_test index 445e0f6d529..986d9856ac2 100755 --- a/mysql-test/collections/smoke_test +++ b/mysql-test/collections/smoke_test @@ -154,6 +154,10 @@ sql_sequence.mysqldump versioning.simple versioning.trx_id " +tests_104=" +period.versioning +plugins.multiauth +" tests_to_run="" @@ -161,7 +165,7 @@ while read -r test do test=`echo $test | sed -e s/\#.*//` tests_to_run="$tests_to_run $test" -done <<< $(echo "$tests_102 $tests_103") +done <<< $(echo "$tests_102 $tests_103 $tests_104") echo $tests_to_run {noformat} {noformat:title=10.5 additions to 10.4 list} diff --git a/mysql-test/collections/smoke_test b/mysql-test/collections/smoke_test index 986d9856ac2..2391f4f6057 100755 --- a/mysql-test/collections/smoke_test +++ b/mysql-test/collections/smoke_test @@ -159,13 +159,18 @@ period.versioning plugins.multiauth " +tests_105=" +client.mariadb-conv +innodb_i_s.innodb_sys_tables +" + tests_to_run="" while read -r test do test=`echo $test | sed -e s/\#.*//` tests_to_run="$tests_to_run $test" -done <<< $(echo "$tests_102 $tests_103 $tests_104") +done <<< $(echo "$tests_102 $tests_103 $tests_104 $tests_105") echo $tests_to_run {noformat} In the current edition, the list is implemented as a shell script under mysql-test/collections/ It runs MTR with the pre-defined list of tests and optionally with other MTR parameters passed from the command line. One disadvantage of having it this way (and same would be if it was a plain text list) is that i out-of-source builds it will be a bit awkward to call, because collections remain in the sourcedir, together with MTR tests. For example, in the old buildbot deb package builds, it would be something like {noformat} cd builddir/mysql-test ../../mysql-test/collections/smoke_test <other options> {noformat} |
| Assignee | Elena Stepanova [ elenst ] | Otto Kekäläinen [ otto ] |
| Status | Stalled [ 10000 ] | In Review [ 10002 ] |
| Assignee | Otto Kekäläinen [ otto ] | Elena Stepanova [ elenst ] |
| Status | In Review [ 10002 ] | Stalled [ 10000 ] |
| Status | Stalled [ 10000 ] | In Progress [ 3 ] |
| issue.field.resolutiondate | 2021-04-22 06:02:39.0 | 2021-04-22 06:02:39.04 |
| Fix Version/s | 10.2.38 [ 25207 ] | |
| Fix Version/s | 10.3.29 [ 25206 ] | |
| Fix Version/s | 10.4.19 [ 25205 ] | |
| Fix Version/s | 10.5.10 [ 25204 ] | |
| Fix Version/s | 10.6.0 [ 24431 ] | |
| Fix Version/s | 10.2 [ 14601 ] | |
| Fix Version/s | 10.3 [ 22126 ] | |
| Fix Version/s | 10.4 [ 22408 ] | |
| Fix Version/s | 10.5 [ 23123 ] | |
| Resolution | Fixed [ 1 ] | |
| Status | In Progress [ 3 ] | Closed [ 6 ] |
| Workflow | MariaDB v3 [ 120581 ] | MariaDB v4 [ 134420 ] |