Details
Blocker Description
Replace the black list of unstable tests introduced in MDEV-10604 and maintained since then with a list of tests which distributions can run in their build/packaging process.
Since the goal of running MTR tests during 3-rd party build process is not to test MariaDB code but only to ensure that the build is functional, there is no need to run thousands of MariaDB tests and combinations. So, the new collection will be much shorter and more static than the previous unstable-tests list.
Criteria:
- separate components (plugins, clients, scripts) should be represented by at least one test when possible;
- every suite should be represented by at least one test, except for those which aren't run regularly and thus aren't maintained properly;
- the tests shouldn't have any sporadic failures on main maintenance branches in the last half of a year (*);
- the tests can't require debug build, windows, or a special configuration not normally present on build machines;
- out of the remaining candidates, the priority is given to those with higher line coverage and/or more generic ones (judging by name) and/or more those which closer relate to the system/environment (judging by name).
(*) Once the list is created, this condition won't be explicitly re-checked before the release, as it was done for unstable-tests. Instead, we should have an buildbot step which would run these tests, and its failure should be "fatal" and require an immediate action.
|
Draft list for 10.2 (first edition) |
archive.archive
|
archive.archive_gis
|
archive.partition_archive
|
binlog.binlog_base64_flag
|
binlog.binlog_database
|
binlog.binlog_innodb
|
binlog.binlog_parallel_replication_marks_stm_mix
|
binlog.binlog_row_mix_innodb_myisam
|
binlog.flashback
|
binlog_encryption.encrypted_slave
|
connect.bin
|
connect.bson
|
connect.dbf
|
connect.dir
|
connect.endian
|
connect.general
|
connect.json
|
connect.mysql
|
connect.secure_file_priv
|
connect.tbl
|
csv.csv
|
disks.disks
|
encryption.encryption_force
|
encryption.innodb_encryption_tables
|
encryption.tempfiles_encrypted
|
federated.federated
|
federated.federatedx
|
gcol.gcol_select_innodb
|
handler.interface
|
heap.heap
|
innodb.innodb
|
innodb.autoinc_persist
|
innodb.innodb_defrag_binlog
|
innodb.innodb_mysql
|
innodb.monitor
|
innodb.purge
|
innodb.table_flags
|
innodb.xa_recovery
|
innodb_fts.fulltext
|
innodb_gis.geometry
|
innodb_gis.rtree
|
innodb_zip.innodb-zip
|
innodb_zip.page_size
|
json.json_no_table
|
main-test_sql_discovery.create
|
main.blackhole
|
main.bootstrap
|
main.compress
|
main.connect
|
main.ctype_collate
|
main.ctype_utf8
|
main.default
|
main.dyncol
|
main.fulltext
|
main.function_defaults
|
main.gis
|
main.grant
|
main.handlersocket
|
main.information_schema
|
main.innodb_ext_key
|
main.log_tables
|
main.lowercase_fs_off
|
main.myisam
|
main.mysql_client_test
|
main.mysql_protocols
|
main.mysql_upgrade
|
main.mysqladmin
|
main.mysqlbinlog
|
main.mysqlcheck
|
main.mysqld--defaults-file
|
main.mysqldump
|
main.mysqlhotcopy_myisam
|
main.mysqlshow
|
main.mysqlslap
|
main.mysqltest
|
main.parser
|
main.partition
|
main.perror
|
main.plugin
|
main.plugin_auth
|
main.pool_of_threads
|
main.ps
|
main.repair
|
main.shutdown
|
main.sp
|
main.ssl
|
main.ssl_compress
|
main.stat_tables
|
main.statistics
|
main.subselect
|
main.symlink
|
main.temp_table
|
main.timezone
|
main.type_timestamp_hires
|
main.user_var
|
main.userstat
|
main.variables
|
main.view
|
main.win
|
main.xa
|
maria.maria
|
mariabackup.full_backup
|
metadata_lock_info.table_metadata_lock
|
mroonga/storage.variable_version
|
mroonga/wrapper.count_star
|
multi_source.multisource
|
oqgraph.general-innodb
|
parts.rpl_partition
|
perfschema.selects
|
plugins.auth_ed25519
|
plugins.cracklib_password_check
|
plugins.dialog
|
plugins.fulltext_plugin
|
plugins.locales
|
plugins.pam_cleartext
|
plugins.processlist
|
plugins.qc_info
|
plugins.server_audit
|
plugins.simple_password_check
|
plugins.sql_error_log
|
plugins.two_password_validations
|
plugins.unix_socket
|
query_response_time.basic
|
rocksdb.rocksdb
|
roles.definer
|
rpl.rpl_gtid_basic
|
rpl.rpl_relayrotate
|
rpl.rpl_row_blob_innodb
|
rpl.rpl_semi_sync_event
|
rpl.rpl_sp
|
rpl.rpl_stm_binlog_max_cache_size
|
rpl.rpl_switch_stm_row_mixed
|
sequence.simple
|
spider.basic_sql
|
spider.ha
|
sql_discovery.simple
|
sys_vars.sysvars_aria
|
sys_vars.sysvars_server_notembedded
|
vcol.vcol_syntax
|
wsrep.variables
|
On my machine it takes ~3 min on a non-debug 10.2 build, with MTR running in shm with --parallel=4.
|
10.3 change over 10.2 list |
--- a/mysql-test/collections/smoke_test
|
+++ b/mysql-test/collections/smoke_test
|
@@ -48,7 +48,8 @@ innodb_gis.rtree
|
innodb_zip.innodb-zip
|
innodb_zip.page_size
|
json.json_no_table
|
-main-test_sql_discovery.create
|
+# Doesn't work in 10.3+ due to MDEV-25384
|
+# main-test_sql_discovery.create
|
main.blackhole
|
main.bootstrap
|
main.compress
|
@@ -145,13 +146,22 @@ sys_vars.sysvars_server_notembedded
|
wsrep.variables
|
"
|
|
+tests_103="
|
+compat/oracle.binlog_ptr_mysqlbinlog
|
+compat/oracle.sp-package
|
+compat/maxdb.rpl_mariadb_timestamp
|
+sql_sequence.mysqldump
|
+versioning.simple
|
+versioning.trx_id
|
+"
|
+
|
tests_to_run=""
|
|
while read -r test
|
do
|
test=`echo $test | sed -e s/\#.*//`
|
tests_to_run="$tests_to_run $test"
|
-done <<< $(echo "$tests_102")
|
+done <<< $(echo "$tests_102 $tests_103")
|
|
echo $tests_to_run
|
|
|
10.4 additions to 10.3 list |
diff --git a/mysql-test/collections/smoke_test b/mysql-test/collections/smoke_test
|
index 445e0f6d529..986d9856ac2 100755
|
--- a/mysql-test/collections/smoke_test
|
+++ b/mysql-test/collections/smoke_test
|
@@ -154,6 +154,10 @@ sql_sequence.mysqldump
|
versioning.simple
|
versioning.trx_id
|
"
|
+tests_104="
|
+period.versioning
|
+plugins.multiauth
|
+"
|
|
tests_to_run=""
|
|
@@ -161,7 +165,7 @@ while read -r test
|
do
|
test=`echo $test | sed -e s/\#.*//`
|
tests_to_run="$tests_to_run $test"
|
-done <<< $(echo "$tests_102 $tests_103")
|
+done <<< $(echo "$tests_102 $tests_103 $tests_104")
|
|
echo $tests_to_run
|
|
10.5 additions to 10.4 list |
diff --git a/mysql-test/collections/smoke_test b/mysql-test/collections/smoke_test
|
index 986d9856ac2..2391f4f6057 100755
|
--- a/mysql-test/collections/smoke_test
|
+++ b/mysql-test/collections/smoke_test
|
@@ -159,13 +159,18 @@ period.versioning
|
plugins.multiauth
|
"
|
|
+tests_105="
|
+client.mariadb-conv
|
+innodb_i_s.innodb_sys_tables
|
+"
|
+
|
tests_to_run=""
|
|
while read -r test
|
do
|
test=`echo $test | sed -e s/\#.*//`
|
tests_to_run="$tests_to_run $test"
|
-done <<< $(echo "$tests_102 $tests_103 $tests_104")
|
+done <<< $(echo "$tests_102 $tests_103 $tests_104 $tests_105")
|
|
echo $tests_to_run
|
In the current edition, the list is implemented as a shell script under mysql-test/collections/
It runs MTR with the pre-defined list of tests and optionally with other MTR parameters passed from the command line.
One disadvantage of having it this way (and same would be if it was a plain text list) is that i out-of-source builds it will be a bit awkward to call, because collections remain in the sourcedir, together with MTR tests.
For example, in the old buildbot deb package builds, it would be something like
cd builddir/mysql-test
|
../../mysql-test/collections/smoke_test <other options>
|
Attachments
Issue Links
- relates to
-
MDEV-7069 Fix buildbot failures in main server trees
-
- Stalled
-
-
MDEV-10605 main.create_delayed failed in buildbot with a timeout
-
- Open
-