Details

    • Task
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • N/A
    • Tests
    • None

    Description

      This is an umbrella task to cover rolling effort of fixing (mostly) sporadic test failures in the community buildbot affecting pull requests CI.

      Similar MDEV-7069 has not been updated since 2017 and is bloated with issues, which may be irrelevant for this effort.

      Attachments

        Issue Links

          Activity

            elenst, very nice points. I thought of some of them.

            * development trees break tests all the time and spoil the statistics

            I'm not too concerned about it. In fact I do want to include development branches, especially branches of pull requests like "refs/pull/3943/head".

            * I suppose you want to limit statistics to a certain period

            Yes, previous month. That is March as of now.

            * there are some builders in the new buildbot which always fail, massively

            At this point I want only "Required" builders, those that do branch protection. Once they're good enough, we can add more builders, e.g. all that build pull requests.

            * there are certain components (galera is the first one that comes to mind) which will likely lead the statistics

            I don't recall things like this being a problem on "Required" builders, so I guess they are not built there? If they are, well, we'll see what we can do about it.

            * sometimes even a main branch gets a broken commit where tests fail massively

            I'm not worried about it too much yet. I aim to perform some manual verifications anyway.

            All in all I'd loved to see:
            1. test name
            2. number of times test was executed (if possible)
            3. number of times test has failed
            4. percentage of failures (for convenience)

            Conditions:
            1. builder is a "Required" builder
            2. period is last month
            3. all trees included
            4. no other special filtering atm

            Order: number of times test has failed DESC

            Limit: 100 (we may pick 10, but it is good to see broader picture)

            svoj Sergey Vojtovich added a comment - elenst , very nice points. I thought of some of them. * development trees break tests all the time and spoil the statistics I'm not too concerned about it. In fact I do want to include development branches, especially branches of pull requests like "refs/pull/3943/head". * I suppose you want to limit statistics to a certain period Yes, previous month. That is March as of now. * there are some builders in the new buildbot which always fail, massively At this point I want only "Required" builders, those that do branch protection. Once they're good enough, we can add more builders, e.g. all that build pull requests. * there are certain components (galera is the first one that comes to mind) which will likely lead the statistics I don't recall things like this being a problem on "Required" builders, so I guess they are not built there? If they are, well, we'll see what we can do about it. * sometimes even a main branch gets a broken commit where tests fail massively I'm not worried about it too much yet. I aim to perform some manual verifications anyway. All in all I'd loved to see: 1. test name 2. number of times test was executed (if possible) 3. number of times test has failed 4. percentage of failures (for convenience) Conditions: 1. builder is a "Required" builder 2. period is last month 3. all trees included 4. no other special filtering atm Order: number of times test has failed DESC Limit: 100 (we may pick 10, but it is good to see broader picture)

            In fact I do want to include development branches
            ...
            all trees included

            As you wish, but I have to warn you that it will likely make the statistics completely useless. People push half-baked stuff, WiP, tests not updated, server not starting, and so on. The counts will have nothing to do with tests and everything to do with the quality of branches.

            At this point I want only "Required" builders

            I need the list of builder names. It's a moving target, I have no idea which are required for branch protection now.

            2. number of times test was executed (if possible)

            Not possible. Buildbots only store MTR failures, not MTR tests, unless the foundation developed it themselves.

            4. percentage of failures (for convenience)

            Percentage from what? If it's from the number of runs, as said above, not possible.

            So, if I get the list of builders, I can run the query. Alternatively, given that you only want the new buildbot builders, probably Razvan can run the query or give you access to the new buildbot's database. I initially thought it was about results from two buildbots, then I keep the database which combines the data from both of them, but for branch protection ones you can query the new bb's database directly. I don't mind though, I can still do it on mine.

            elenst Elena Stepanova added a comment - In fact I do want to include development branches ... all trees included As you wish, but I have to warn you that it will likely make the statistics completely useless. People push half-baked stuff, WiP, tests not updated, server not starting, and so on. The counts will have nothing to do with tests and everything to do with the quality of branches. At this point I want only "Required" builders I need the list of builder names. It's a moving target, I have no idea which are required for branch protection now. 2. number of times test was executed (if possible) Not possible. Buildbots only store MTR failures , not MTR tests, unless the foundation developed it themselves. 4. percentage of failures (for convenience) Percentage from what? If it's from the number of runs, as said above, not possible. So, if I get the list of builders, I can run the query. Alternatively, given that you only want the new buildbot builders, probably Razvan can run the query or give you access to the new buildbot's database. I initially thought it was about results from two buildbots, then I keep the database which combines the data from both of them, but for branch protection ones you can query the new bb's database directly. I don't mind though, I can still do it on mine.

            As you wish, but I have to warn you that it will likely make the statistics completely useless.

            I'm almost convinced. I'm just concerned that main trees may not have enough runs for sampling.

            Builders list:
            amd64-debian-11-msan-clang-16
            amd64-ubuntu-2004-debug
            amd64-ubuntu-2204-debug-ps
            amd64-windows

            svoj Sergey Vojtovich added a comment - As you wish, but I have to warn you that it will likely make the statistics completely useless. I'm almost convinced. I'm just concerned that main trees may not have enough runs for sampling. Builders list: amd64-debian-11-msan-clang-16 amd64-ubuntu-2004-debug amd64-ubuntu-2204-debug-ps amd64-windows

            I'm just concerned that main trees may not have enough runs for sampling.

            You may be right about it, I didn't realize there would be so few builders, I used to do it for the old buildbot where all its builders counted. It was quite enough for sampling.
            I've done both, main-only and all trees. In fact, I've done six, the other dimension is time: March only, from the beginning of April, from the beginning of March.

            Main branches, March

            +---------------------------------------------------------------+---+---------------------+---------------------+
            | test_name                                                     | c | min(dt)             | max(dt)             |
            +---------------------------------------------------------------+---+---------------------+---------------------+
            | main.lock_sync                                                | 4 | 2025-03-11 23:50:24 | 2025-03-26 13:05:56 |
            | mariabackup.incremental_compressed                            | 4 | 2025-03-15 11:17:19 | 2025-03-26 14:52:35 |
            | mariabackup.partial                                           | 3 | 2025-03-19 00:23:51 | 2025-03-24 11:43:31 |
            | atomic.alter_table                                            | 1 | 2025-03-05 19:27:56 | 2025-03-05 19:27:56 |
            | atomic.create_table                                           | 1 | 2025-03-05 15:01:41 | 2025-03-05 15:01:41 |
            | binlog_encryption.encrypted_master_switch_to_unencrypted_gtid | 1 | 2025-03-19 20:14:43 | 2025-03-19 20:14:43 |
            | encryption.innodb-encryption-alter                            | 1 | 2025-03-21 17:02:31 | 2025-03-21 17:02:31 |
            | innodb.doublewrite_debug                                      | 1 | 2025-03-31 09:45:13 | 2025-03-31 09:45:13 |
            | innodb.temp_truncate_freed                                    | 1 | 2025-03-29 11:34:15 | 2025-03-29 11:34:15 |
            | main.mysql-interactive                                        | 1 | 2025-03-22 03:17:54 | 2025-03-22 03:17:54 |
            | main.mysql_upgrade-28915                                      | 1 | 2025-03-18 12:13:49 | 2025-03-18 12:13:49 |
            | main.query_cache_debug                                        | 1 | 2025-03-14 15:09:47 | 2025-03-14 15:09:47 |
            | main.vector2                                                  | 1 | 2025-03-06 02:12:59 | 2025-03-06 02:12:59 |
            +---------------------------------------------------------------+---+---------------------+---------------------+
            13 rows
            

            All branches, March

            +--------------------------------------------------------------+----+---------------------+---------------------+
            | test_name                                                    | c  | min(dt)             | max(dt)             |
            +--------------------------------------------------------------+----+---------------------+---------------------+
            | versioning.foreign                                           | 85 | 2025-03-02 12:32:33 | 2025-03-30 15:44:30 |
            | main.timezone                                                | 75 | 2025-03-30 03:53:14 | 2025-03-31 21:59:00 |
            | innodb.doublewrite                                           | 51 | 2025-03-06 04:44:18 | 2025-03-29 11:27:28 |
            | innodb.insert_into_empty_notembedded                         | 45 | 2025-03-28 14:15:00 | 2025-03-28 14:33:40 |
            | versioning.delete                                            | 44 | 2025-03-02 12:32:33 | 2025-03-30 15:35:02 |
            | binlog_encryption.encrypted_slave                            | 40 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 |
            | main.mysqldump-system                                        | 39 | 2025-03-11 10:29:17 | 2025-03-26 13:59:44 |
            | versioning.rpl                                               | 39 | 2025-03-02 15:55:01 | 2025-03-30 15:44:30 |
            | main.mysql_upgrade                                           | 31 | 2025-03-02 13:22:14 | 2025-03-31 18:42:35 |
            | binlog_encryption.rpl_gtid_basic                             | 30 | 2025-03-06 09:17:06 | 2025-03-26 14:47:54 |
            | federated.federated                                          | 30 | 2025-03-04 12:01:23 | 2025-03-12 07:57:33 |
            | mariabackup.incremental_compressed                           | 29 | 2025-03-07 15:25:09 | 2025-03-30 11:58:34 |
            | innodb_fts.misc_1                                            | 28 | 2025-03-02 15:55:01 | 2025-03-30 15:44:30 |
            | binlog_encryption.encrypted_master                           | 27 | 2025-03-06 09:17:06 | 2025-03-26 14:47:54 |
            | rpl.rpl_mysqldump_gtid_slave_pos                             | 27 | 2025-03-28 14:15:00 | 2025-03-28 14:33:40 |
            | main.lock_sync                                               | 24 | 2025-03-05 14:36:26 | 2025-03-31 12:29:04 |
            | main.mysql-interactive                                       | 24 | 2025-03-03 01:17:22 | 2025-03-30 15:35:02 |
            | atomic.alter_table                                           | 22 | 2025-03-03 04:04:53 | 2025-03-28 10:33:50 |
            | main.mysqldump                                               | 21 | 2025-03-05 11:49:53 | 2025-03-31 10:03:15 |
            | mariabackup.partial                                          | 20 | 2025-03-07 15:06:24 | 2025-03-31 16:43:59 |
            | binlog_encryption.rpl_sync                                   | 19 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 |
            | plugins.thread_pool_server_audit                             | 19 | 2025-03-02 14:18:31 | 2025-03-31 07:28:47 |
            | rpl.rpl_gtid_crash                                           | 19 | 2025-03-11 02:07:47 | 2025-03-31 11:48:34 |
            | rpl.rpl_parallel_optimistic_xa_lsu_off                       | 19 | 2025-03-11 02:07:47 | 2025-03-26 11:34:05 |
            | s3.mysqldump                                                 | 19 | 2025-03-03 04:47:39 | 2025-03-26 11:45:10 |
            | binlog_encryption.rpl_binlog_errors                          | 18 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 |
            | binlog_encryption.rpl_relayrotate                            | 18 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 |
            | rpl.parallel_backup_xa_debug                                 | 18 | 2025-03-01 01:24:54 | 2025-03-30 09:25:59 |
            | rpl.rpl_read_only                                            | 18 | 2025-03-11 02:07:47 | 2025-03-31 17:23:39 |
            | rpl.rpl_xa                                                   | 18 | 2025-03-11 02:07:47 | 2025-03-26 11:34:05 |
            | type_cursor.type_sys_refcursor-type_row-debug                | 18 | 2025-03-17 14:42:41 | 2025-03-18 10:48:12 |
            | binlog_encryption.rpl_parallel_analyze_table_hang            | 17 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 |
            | mariabackup.full_backup                                      | 17 | 2025-03-11 02:07:47 | 2025-03-26 12:38:55 |
            | binlog_encryption.rpl_special_charset                        | 16 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 |
            | binlog_encryption.rpl_sporadic_master                        | 16 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 |
            | main.opt_hints_join_order                                    | 16 | 2025-03-04 11:57:06 | 2025-03-13 14:52:01 |
            | binlog_encryption.rpl_corruption                             | 15 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 |
            | encryption.doublewrite_debug                                 | 15 | 2025-03-07 15:07:04 | 2025-03-28 10:38:05 |
            | innodb.innodb-64k-crash                                      | 15 | 2025-03-25 13:04:20 | 2025-03-28 14:54:46 |
            | innodb.undo_upgrade                                          | 15 | 2025-03-11 02:07:47 | 2025-03-12 02:51:27 |
            | rpl.rpl_xa_gtid_pos_auto_engine                              | 15 | 2025-03-11 02:07:47 | 2025-03-26 11:34:05 |
            | binlog_encryption.rpl_loaddata_local                         | 14 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 |
            | binlog_encryption.rpl_parallel_deadlock_corrupt_binlog       | 14 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 |
            | binlog_encryption.binlog_xa_recover                          | 13 | 2025-03-11 02:07:47 | 2025-03-19 06:32:29 |
            | binlog_encryption.rpl_incident                               | 13 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 |
            | parts.partition_alter4_innodb                                | 13 | 2025-03-03 11:11:02 | 2025-03-31 12:43:12 |
            | binlog_encryption.rpl_parallel_show_binlog_events_purge_logs | 12 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 |
            | binlog_encryption.rpl_skip_replication                       | 12 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 |
            | connect.alter                                                | 12 | 2025-03-04 12:01:27 | 2025-03-12 07:59:25 |
            | mariabackup.unsupported_redo                                 | 12 | 2025-03-07 15:06:24 | 2025-03-26 08:59:09 |
            | rpl.rpl_do_grant                                             | 12 | 2025-03-02 13:25:18 | 2025-03-13 15:02:07 |
            | type_cursor.type_sys_refcursor-do-debug                      | 12 | 2025-03-17 14:42:41 | 2025-03-17 19:27:40 |
            | type_cursor.type_sys_refcursor-sf-debug                      | 12 | 2025-03-17 14:42:41 | 2025-03-17 19:27:40 |
            | binlog_encryption.binlog_incident                            | 11 | 2025-03-18 09:17:10 | 2025-03-19 06:32:29 |
            | binlog_encryption.binlog_row_annotate                        | 11 | 2025-03-18 09:17:10 | 2025-03-19 09:17:45 |
            | binlog_encryption.rpl_init_slave_errors                      | 11 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 |
            | binlog_encryption.rpl_parallel_innodb_lock_conflict          | 11 | 2025-03-01 07:12:19 | 2025-03-26 14:47:54 |
            | innodb_gis.rtree_purge                                       | 11 | 2025-03-05 11:49:37 | 2025-03-25 17:27:36 |
            | binlog_encryption.rpl_parallel_domain                        | 10 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 |
            | compat/oracle.ora_outer_join                                 | 10 | 2025-03-19 14:37:16 | 2025-03-31 12:40:31 |
            | compat/oracle.sp-sys_refcursor                               | 10 | 2025-03-15 16:22:04 | 2025-03-17 16:42:05 |
            | federated.federatedx                                         | 10 | 2025-03-26 11:34:05 | 2025-03-28 14:33:40 |
            | federated.federated_server                                   | 10 | 2025-03-04 12:01:23 | 2025-03-12 07:57:17 |
            | rpl.parallel_backup_lsu_off                                  | 10 | 2025-03-11 02:07:47 | 2025-03-26 11:34:05 |
            | rpl.parallel_backup_slave_binlog_off                         | 10 | 2025-03-11 02:07:47 | 2025-03-26 11:34:05 |
            | rpl.rpl_parallel_innodb_lock_conflict                        | 10 | 2025-03-03 03:59:55 | 2025-03-21 20:22:56 |
            | rpl.rpl_parallel_multi_domain_xa                             | 10 | 2025-03-11 02:07:47 | 2025-03-28 15:03:11 |
            | rpl.rpl_xa_survive_disconnect_lsu_off                        | 10 | 2025-03-11 02:07:47 | 2025-03-26 11:34:05 |
            | spider/bugfix.mdev_27902                                     | 10 | 2025-03-04 12:01:27 | 2025-03-12 07:57:33 |
            | spider/bugfix.mdev_28683                                     | 10 | 2025-03-04 12:01:27 | 2025-03-12 07:57:33 |
            | spider/bugfix.mdev_29421                                     | 10 | 2025-03-04 12:01:27 | 2025-03-12 07:57:33 |
            | spider/bugfix.mdev_31475                                     | 10 | 2025-03-04 12:01:27 | 2025-03-12 07:57:33 |
            | spider/bugfix.mdev_32492                                     | 10 | 2025-03-04 12:01:27 | 2025-03-12 07:57:33 |
            | spider/bugfix.mdev_34541                                     | 10 | 2025-03-04 12:01:27 | 2025-03-12 07:57:33 |
            | spider/bugfix.mdev_34555                                     | 10 | 2025-03-04 12:01:27 | 2025-03-12 07:57:33 |
            | spider/bugfix.mdev_34636                                     | 10 | 2025-03-04 12:01:27 | 2025-03-12 07:57:33 |
            | spider/bugfix.self_reference                                 | 10 | 2025-03-04 12:01:27 | 2025-03-12 07:57:33 |
            | spider/bugfix.self_reference_multi                           | 10 | 2025-03-04 12:01:27 | 2025-03-12 07:57:33 |
            | binlog.binlog_checkpoint_flush_logs                          |  9 | 2025-03-11 02:07:47 | 2025-03-12 02:52:39 |
            | binlog.binlog_commit_by_rotate_atomic                        |  9 | 2025-03-11 02:07:47 | 2025-03-12 02:52:39 |
            | binlog.binlog_dmls_on_tmp_tables_readonly                    |  9 | 2025-03-31 17:22:28 | 2025-03-31 17:23:39 |
            | binlog.binlog_empty_xa_prepared                              |  9 | 2025-03-11 02:07:47 | 2025-03-12 02:52:39 |
            | binlog.binlog_mdev342                                        |  9 | 2025-03-11 02:07:47 | 2025-03-12 02:52:39 |
            | binlog.binlog_recover_async_rollback_trx                     |  9 | 2025-03-11 02:07:47 | 2025-03-12 02:52:39 |
            | binlog.binlog_truncate_active_log                            |  9 | 2025-03-11 02:07:47 | 2025-03-12 02:52:39 |
            | binlog.binlog_xa_prepared_disconnect                         |  9 | 2025-03-11 02:07:47 | 2025-03-12 02:52:39 |
            | binlog.binlog_xa_recover                                     |  9 | 2025-03-11 02:07:47 | 2025-03-12 02:52:39 |
            | binlog_encryption.rpl_parallel_free_deferred_event           |  9 | 2025-03-06 09:17:06 | 2025-03-26 14:47:54 |
            | binlog_encryption.rpl_parallel_incorrect_relay_pos           |  9 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 |
            | binlog_encryption.rpl_temporal_format_default_to_default     |  9 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 |
            | innodb_fts.innodb_fts_misc_1                                 |  9 | 2025-03-02 15:55:01 | 2025-03-30 15:44:30 |
            | innodb_fts.misc_debug                                        |  9 | 2025-03-05 11:57:13 | 2025-03-05 13:29:15 |
            | main.loadxml                                                 |  9 | 2025-03-28 14:15:00 | 2025-03-28 14:33:40 |
            | main.mariadb-dump-debug                                      |  9 | 2025-03-28 14:15:00 | 2025-03-28 14:33:40 |
            | main.mysqldump-compat                                        |  9 | 2025-03-28 14:15:00 | 2025-03-28 14:33:40 |
            | plugins.sql_error_log_withdbinfo                             |  9 | 2025-03-02 14:18:31 | 2025-03-31 07:28:47 |
            | rpl.parallel_backup                                          |  9 | 2025-03-11 02:07:47 | 2025-03-12 02:52:39 |
            | rpl.rpl_parallel_optimistic_xa                               |  9 | 2025-03-11 02:07:47 | 2025-03-12 02:51:27 |
            | rpl.rpl_perfschema_connect_config                            |  9 | 2025-03-28 14:40:32 | 2025-03-31 17:23:39 |
            | rpl.rpl_xa_empty_transaction                                 |  9 | 2025-03-11 02:07:47 | 2025-03-12 02:51:27 |
            +--------------------------------------------------------------+----+---------------------+---------------------+
            100 rows
            

            Main branches, April

            +--------------------------------------------+---+---------------------+---------------------+
            | test_name                                  | c | min(dt)             | max(dt)             |
            +--------------------------------------------+---+---------------------+---------------------+
            | innodb.doublewrite                         | 4 | 2025-04-02 12:11:57 | 2025-04-19 17:53:54 |
            | innodb.temp_truncate_freed                 | 3 | 2025-04-14 16:08:23 | 2025-04-19 17:53:54 |
            | atomic.create_table                        | 2 | 2025-04-18 10:09:22 | 2025-04-19 13:37:54 |
            | main.lock_sync                             | 2 | 2025-04-18 23:04:57 | 2025-04-19 17:55:51 |
            | main.mysql_upgrade-28915                   | 2 | 2025-04-16 20:25:47 | 2025-04-19 17:57:26 |
            | main.timezone                              | 2 | 2025-04-15 11:50:34 | 2025-04-15 11:53:41 |
            | atomic.alter_table                         | 1 | 2025-04-02 06:49:06 | 2025-04-02 06:49:06 |
            | atomic.drop_table                          | 1 | 2025-04-19 13:37:54 | 2025-04-19 13:37:54 |
            | encryption.innochecksum                    | 1 | 2025-04-19 09:23:59 | 2025-04-19 09:23:59 |
            | innodb.doublewrite_debug                   | 1 | 2025-04-03 13:39:25 | 2025-04-03 13:39:25 |
            | innodb.innodb-64k                          | 1 | 2025-04-10 16:20:37 | 2025-04-10 16:20:37 |
            | innodb.innodb-lock-inherit-read_commited   | 1 | 2025-04-15 11:47:24 | 2025-04-15 11:47:24 |
            | innodb.innodb_stats_table_flag_auto_recalc | 1 | 2025-04-17 07:05:30 | 2025-04-17 07:05:30 |
            | innodb.instant_alter_crash                 | 1 | 2025-04-16 20:21:32 | 2025-04-16 20:21:32 |
            | main.mysql_upgrade                         | 1 | 2025-04-18 10:09:22 | 2025-04-18 10:09:22 |
            | main.rowid_filter_innodb                   | 1 | 2025-04-03 13:39:25 | 2025-04-03 13:39:25 |
            | mariabackup.partial                        | 1 | 2025-04-01 23:59:40 | 2025-04-01 23:59:40 |
            | rpl.rpl_gtid_crash                         | 1 | 2025-04-15 11:47:24 | 2025-04-15 11:47:24 |
            | rpl.rpl_parallel_multi_domain_xa           | 1 | 2025-04-14 07:45:53 | 2025-04-14 07:45:53 |
            | rpl.rpl_row_create_table                   | 1 | 2025-04-19 09:21:21 | 2025-04-19 09:21:21 |
            +--------------------------------------------+---+---------------------+---------------------+
            20 rows
            

            All branches, April

            +-----------------------------------------------------+-----+---------------------+---------------------+
            | test_name                                           | c   | min(dt)             | max(dt)             |
            +-----------------------------------------------------+-----+---------------------+---------------------+
            | main.timezone                                       | 209 | 2025-04-01 03:20:45 | 2025-04-19 13:20:06 |
            | versioning.update                                   |  40 | 2025-04-17 17:47:33 | 2025-04-17 21:04:39 |
            | main.mysql_upgrade                                  |  38 | 2025-04-03 10:45:27 | 2025-04-18 19:28:10 |
            | innodb.doublewrite                                  |  34 | 2025-04-01 03:22:55 | 2025-04-20 10:58:51 |
            | main.lock_sync                                      |  34 | 2025-04-02 07:23:05 | 2025-04-19 17:55:51 |
            | main.mysqld--help                                   |  29 | 2025-04-06 14:53:55 | 2025-04-16 06:04:05 |
            | versioning.foreign                                  |  24 | 2025-04-06 11:00:42 | 2025-04-06 11:05:00 |
            | perfschema.table_schema                             |  20 | 2025-04-06 14:53:55 | 2025-04-06 17:50:55 |
            | versioning.rpl                                      |  19 | 2025-04-06 11:00:42 | 2025-04-06 11:05:00 |
            | compat/oracle.exception                             |  18 | 2025-04-06 14:54:11 | 2025-04-07 16:00:35 |
            | rpl.rpl_xa                                          |  18 | 2025-04-17 17:49:40 | 2025-04-17 21:04:39 |
            | rpl.rpl_xa_gtid_pos_auto_engine                     |  18 | 2025-04-17 17:49:40 | 2025-04-17 21:04:39 |
            | sys_vars.character_set_results_basic                |  18 | 2025-04-06 14:54:11 | 2025-04-07 16:00:35 |
            | mariabackup.encrypted_page_corruption               |  17 | 2025-04-16 05:56:06 | 2025-04-16 06:04:52 |
            | innodb.temp_truncate_freed                          |  16 | 2025-04-08 07:32:28 | 2025-04-19 18:08:41 |
            | main.alter_table                                    |  16 | 2025-04-06 14:53:55 | 2025-04-06 17:44:02 |
            | main.cte_nonrecursive                               |  16 | 2025-04-02 11:13:44 | 2025-04-08 21:57:41 |
            | main.order_by                                       |  16 | 2025-04-14 14:44:23 | 2025-04-17 17:51:52 |
            | main.view                                           |  15 | 2025-04-02 11:13:44 | 2025-04-18 10:47:23 |
            | mariabackup.incremental_backup                      |  15 | 2025-04-16 05:56:06 | 2025-04-16 06:04:52 |
            | mariabackup.incremental_encrypted                   |  15 | 2025-04-16 05:56:06 | 2025-04-16 06:04:52 |
            | main.derived_view                                   |  13 | 2025-04-02 11:13:44 | 2025-04-15 19:01:07 |
            | innodb.innodb-system-table-view                     |  12 | 2025-04-16 11:04:16 | 2025-04-16 11:09:07 |
            | main.analyze_format_json                            |  12 | 2025-04-14 14:44:23 | 2025-04-17 07:22:24 |
            | main.explain_json                                   |  12 | 2025-04-14 14:44:23 | 2025-04-17 07:22:24 |
            | main.show_explain_json                              |  12 | 2025-04-14 14:44:23 | 2025-04-17 07:29:29 |
            | main.subselect4                                     |  12 | 2025-04-14 14:44:23 | 2025-04-17 07:22:24 |
            | main.subselect_cache                                |  12 | 2025-04-14 14:44:23 | 2025-04-17 07:22:24 |
            | main.subselect_sj_jcl6                              |  12 | 2025-04-08 01:17:51 | 2025-04-10 22:39:57 |
            | vcol.wrong_arena                                    |  12 | 2025-04-03 10:42:52 | 2025-04-08 10:09:14 |
            | binlog_encryption.encrypted_master                  |  11 | 2025-04-14 23:07:12 | 2025-04-17 21:26:07 |
            | atomic.alter_table                                  |   9 | 2025-04-02 06:49:06 | 2025-04-18 18:18:02 |
            | binlog_encryption.rpl_init_slave_errors             |   9 | 2025-04-03 13:48:18 | 2025-04-03 13:51:36 |
            | innodb_fts.misc_1                                   |   9 | 2025-04-06 11:01:59 | 2025-04-06 11:05:00 |
            | main.mysql_client_test                              |   9 | 2025-04-03 10:45:27 | 2025-04-18 10:47:23 |
            | perfschema.ortho_iter                               |   9 | 2025-04-02 11:13:44 | 2025-04-14 20:05:10 |
            | roles.show_grants_replicated                        |   9 | 2025-04-03 10:45:27 | 2025-04-08 10:09:14 |
            | rpl.rpl_binlog_compress                             |   9 | 2025-04-14 23:07:12 | 2025-04-14 23:09:40 |
            | rpl.rpl_create_drop_role                            |   9 | 2025-04-03 10:45:27 | 2025-04-08 10:09:14 |
            | rpl.rpl_do_grant                                    |   9 | 2025-04-03 10:45:27 | 2025-04-08 10:09:14 |
            | sequence.group_by                                   |   9 | 2025-04-01 04:56:12 | 2025-04-01 06:34:25 |
            | encryption.doublewrite_debug                        |   8 | 2025-04-16 05:56:06 | 2025-04-16 06:04:52 |
            | main.group_by                                       |   8 | 2025-04-15 03:04:07 | 2025-04-17 17:51:52 |
            | main.opt_hint_timeout                               |   8 | 2025-04-08 10:08:54 | 2025-04-08 10:30:30 |
            | main.partition_explicit_prune                       |   8 | 2025-04-06 14:53:55 | 2025-04-06 16:41:33 |
            | main.subselect_sj                                   |   8 | 2025-04-08 01:17:51 | 2025-04-08 21:57:41 |
            | main.union                                          |   8 | 2025-04-02 11:13:44 | 2025-04-02 11:21:06 |
            | main.xa                                             |   8 | 2025-04-11 06:53:10 | 2025-04-11 07:12:05 |
            | parts.partition_alter4_innodb                       |   8 | 2025-04-03 10:34:28 | 2025-04-19 13:19:17 |
            | vcol.partition                                      |   8 | 2025-04-17 17:47:33 | 2025-04-17 21:04:39 |
            | encryption.innodb-encryption-alter                  |   7 | 2025-04-02 07:22:40 | 2025-04-19 18:08:41 |
            | innodb.fk_col_alter                                 |   7 | 2025-04-09 13:34:36 | 2025-04-10 06:39:55 |
            | innodb.foreign_null                                 |   7 | 2025-04-09 13:34:36 | 2025-04-10 06:39:55 |
            | main.connect                                        |   7 | 2025-04-02 12:45:14 | 2025-04-08 10:09:14 |
            | main.func_json                                      |   7 | 2025-04-03 10:45:27 | 2025-04-10 02:42:04 |
            | main.grant4                                         |   7 | 2025-04-03 10:45:27 | 2025-04-09 08:17:36 |
            | main.mysql_upgrade-28915                            |   7 | 2025-04-01 19:40:45 | 2025-04-19 17:57:26 |
            | main.ps_error                                       |   7 | 2025-04-04 00:17:32 | 2025-04-19 18:19:18 |
            | mariabackup.full_backup                             |   7 | 2025-04-16 05:56:06 | 2025-04-16 06:04:52 |
            | sql_sequence.grant                                  |   7 | 2025-04-03 15:45:28 | 2025-04-17 10:36:42 |
            | sys_vars.sysvars_innodb                             |   7 | 2025-04-16 07:17:19 | 2025-04-19 05:59:08 |
            | binlog.binlog_invalid_read_in_rotate                |   6 | 2025-04-17 21:26:07 | 2025-04-17 21:26:07 |
            | binlog_encryption.rpl_gtid_basic                    |   6 | 2025-04-17 21:26:07 | 2025-04-17 21:26:07 |
            | federated.federatedx_create_handlers                |   6 | 2025-04-02 11:13:53 | 2025-04-02 11:18:40 |
            | innodb_zip.prefix_index_liftedlimit                 |   6 | 2025-04-17 17:49:40 | 2025-04-17 21:04:39 |
            | main.derived                                        |   6 | 2025-04-04 02:29:45 | 2025-04-04 04:15:44 |
            | main.grant5                                         |   6 | 2025-04-03 10:42:52 | 2025-04-08 10:09:14 |
            | main.ssl_7937                                       |   6 | 2025-04-02 12:45:14 | 2025-04-02 12:47:27 |
            | mariabackup.small_ibd                               |   6 | 2025-04-16 05:56:06 | 2025-04-16 06:04:52 |
            | roles.acl_statistics                                |   6 | 2025-04-03 10:42:52 | 2025-04-08 10:09:14 |
            | spider/bugfix.mdev_20502                            |   6 | 2025-04-01 03:54:27 | 2025-04-01 06:25:01 |
            | sql_sequence.check                                  |   6 | 2025-04-07 06:42:01 | 2025-04-07 07:59:20 |
            | sql_sequence.other                                  |   6 | 2025-04-18 10:23:18 | 2025-04-18 10:47:23 |
            | sys_vars.innodb_purge_rseg_truncate_frequency_basic |   6 | 2025-04-17 17:49:40 | 2025-04-17 21:04:39 |
            | sys_vars.max_sort_length_func                       |   6 | 2025-04-17 17:49:40 | 2025-04-17 21:04:39 |
            | atomic.create_table                                 |   5 | 2025-04-08 17:13:20 | 2025-04-19 13:37:54 |
            | binlog_encryption.rpl_skip_replication              |   5 | 2025-04-01 19:35:31 | 2025-04-17 21:26:07 |
            | main.set_password                                   |   5 | 2025-04-02 12:45:14 | 2025-04-08 10:09:14 |
            | main.show_analyze_json                              |   5 | 2025-04-02 12:45:41 | 2025-04-17 07:29:29 |
            | compat/oracle.ora_outer_join                        |   4 | 2025-04-13 10:53:13 | 2025-04-13 13:20:24 |
            | innodb.alter_varchar_change                         |   4 | 2025-04-16 11:04:16 | 2025-04-16 11:09:07 |
            | innodb.innodb-lock-inherit-read_commited            |   4 | 2025-04-07 19:45:25 | 2025-04-19 09:11:55 |
            | innodb.innodb_buffer_pool_resize_temporary          |   4 | 2025-04-11 04:42:54 | 2025-04-11 04:46:15 |
            | innodb.innodb_ctype_ldml                            |   4 | 2025-04-09 07:56:52 | 2025-04-09 08:01:55 |
            | innodb.instant_alter_debug                          |   4 | 2025-04-03 06:56:29 | 2025-04-03 07:00:53 |
            | innodb.instant_alter_index_rename                   |   4 | 2025-04-16 11:04:16 | 2025-04-16 11:09:07 |
            | main.commit_1innodb                                 |   4 | 2025-04-17 05:07:03 | 2025-04-17 05:12:20 |
            | main.ctype_gbk                                      |   4 | 2025-04-17 17:47:33 | 2025-04-17 17:51:52 |
            | main.ctype_ldml                                     |   4 | 2025-04-10 13:10:13 | 2025-04-10 13:31:13 |
            | main.func_extract                                   |   4 | 2025-04-15 15:44:22 | 2025-04-15 15:48:08 |
            | main.mariadb-import                                 |   4 | 2025-04-10 12:39:10 | 2025-04-10 12:44:29 |
            | main.mysqltest                                      |   4 | 2025-04-10 13:10:13 | 2025-04-10 13:31:13 |
            | main.opt_hints_join_order                           |   4 | 2025-04-07 18:19:50 | 2025-04-07 18:23:08 |
            | main.ps_ddl                                         |   4 | 2025-04-09 07:56:52 | 2025-04-09 08:01:55 |
            | main.ps_show_log                                    |   4 | 2025-04-06 14:53:55 | 2025-04-06 14:58:19 |
            | main.show_check                                     |   4 | 2025-04-09 07:56:52 | 2025-04-09 08:01:55 |
            | main.sp-anchor-row-type-table                       |   4 | 2025-04-17 14:26:24 | 2025-04-17 14:31:14 |
            | main.sp-row                                         |   4 | 2025-04-17 14:26:24 | 2025-04-17 14:31:14 |
            | mariabackup.partial                                 |   4 | 2025-04-01 08:19:39 | 2025-04-15 10:05:51 |
            | mariabackup.partial_exclude                         |   4 | 2025-04-16 05:56:06 | 2025-04-16 06:04:52 |
            +-----------------------------------------------------+-----+---------------------+---------------------+
            100 rows
            

            Main branches, from the beginning of March

            +---------------------------------------------------------------+---+---------------------+---------------------+
            | test_name                                                     | c | min(dt)             | max(dt)             |
            +---------------------------------------------------------------+---+---------------------+---------------------+
            | main.lock_sync                                                | 6 | 2025-03-11 23:50:24 | 2025-04-19 17:55:51 |
            | innodb.doublewrite                                            | 4 | 2025-04-02 12:11:57 | 2025-04-19 17:53:54 |
            | innodb.temp_truncate_freed                                    | 4 | 2025-03-29 11:34:15 | 2025-04-19 17:53:54 |
            | mariabackup.incremental_compressed                            | 4 | 2025-03-15 11:17:19 | 2025-03-26 14:52:35 |
            | mariabackup.partial                                           | 4 | 2025-03-19 00:23:51 | 2025-04-01 23:59:40 |
            | atomic.create_table                                           | 3 | 2025-03-05 15:01:41 | 2025-04-19 13:37:54 |
            | main.mysql_upgrade-28915                                      | 3 | 2025-03-18 12:13:49 | 2025-04-19 17:57:26 |
            | atomic.alter_table                                            | 2 | 2025-03-05 19:27:56 | 2025-04-02 06:49:06 |
            | innodb.doublewrite_debug                                      | 2 | 2025-03-31 09:45:13 | 2025-04-03 13:39:25 |
            | main.timezone                                                 | 2 | 2025-04-15 11:50:34 | 2025-04-15 11:53:41 |
            | atomic.drop_table                                             | 1 | 2025-04-19 13:37:54 | 2025-04-19 13:37:54 |
            | binlog_encryption.encrypted_master_switch_to_unencrypted_gtid | 1 | 2025-03-19 20:14:43 | 2025-03-19 20:14:43 |
            | encryption.innochecksum                                       | 1 | 2025-04-19 09:23:59 | 2025-04-19 09:23:59 |
            | encryption.innodb-encryption-alter                            | 1 | 2025-03-21 17:02:31 | 2025-03-21 17:02:31 |
            | innodb.innodb-64k                                             | 1 | 2025-04-10 16:20:37 | 2025-04-10 16:20:37 |
            | innodb.innodb-lock-inherit-read_commited                      | 1 | 2025-04-15 11:47:24 | 2025-04-15 11:47:24 |
            | innodb.innodb_stats_table_flag_auto_recalc                    | 1 | 2025-04-17 07:05:30 | 2025-04-17 07:05:30 |
            | innodb.instant_alter_crash                                    | 1 | 2025-04-16 20:21:32 | 2025-04-16 20:21:32 |
            | main.mysql-interactive                                        | 1 | 2025-03-22 03:17:54 | 2025-03-22 03:17:54 |
            | main.mysql_upgrade                                            | 1 | 2025-04-18 10:09:22 | 2025-04-18 10:09:22 |
            | main.query_cache_debug                                        | 1 | 2025-03-14 15:09:47 | 2025-03-14 15:09:47 |
            | main.rowid_filter_innodb                                      | 1 | 2025-04-03 13:39:25 | 2025-04-03 13:39:25 |
            | main.vector2                                                  | 1 | 2025-03-06 02:12:59 | 2025-03-06 02:12:59 |
            | rpl.rpl_gtid_crash                                            | 1 | 2025-04-15 11:47:24 | 2025-04-15 11:47:24 |
            | rpl.rpl_parallel_multi_domain_xa                              | 1 | 2025-04-14 07:45:53 | 2025-04-14 07:45:53 |
            | rpl.rpl_row_create_table                                      | 1 | 2025-04-19 09:21:21 | 2025-04-19 09:21:21 |
            +---------------------------------------------------------------+---+---------------------+---------------------+
            26 rows
            

            All branches, from the beginning of March

            +--------------------------------------------------------------+-----+---------------------+---------------------+
            | test_name                                                    | c   | min(dt)             | max(dt)             |
            +--------------------------------------------------------------+-----+---------------------+---------------------+
            | main.timezone                                                | 284 | 2025-03-30 03:53:14 | 2025-04-19 13:20:06 |
            | versioning.foreign                                           | 109 | 2025-03-02 12:32:33 | 2025-04-06 11:05:00 |
            | innodb.doublewrite                                           |  85 | 2025-03-06 04:44:18 | 2025-04-20 10:58:51 |
            | main.mysql_upgrade                                           |  69 | 2025-03-02 13:22:14 | 2025-04-18 19:28:10 |
            | main.lock_sync                                               |  58 | 2025-03-05 14:36:26 | 2025-04-19 17:55:51 |
            | versioning.rpl                                               |  58 | 2025-03-02 15:55:01 | 2025-04-06 11:05:00 |
            | innodb.insert_into_empty_notembedded                         |  45 | 2025-03-28 14:15:00 | 2025-03-28 14:33:40 |
            | versioning.delete                                            |  44 | 2025-03-02 12:32:33 | 2025-03-30 15:35:02 |
            | main.mysqldump-system                                        |  42 | 2025-03-11 10:29:17 | 2025-04-08 10:09:14 |
            | binlog_encryption.encrypted_slave                            |  40 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 |
            | versioning.update                                            |  40 | 2025-04-17 17:47:33 | 2025-04-17 21:04:39 |
            | binlog_encryption.encrypted_master                           |  38 | 2025-03-06 09:17:06 | 2025-04-17 21:26:07 |
            | innodb_fts.misc_1                                            |  37 | 2025-03-02 15:55:01 | 2025-04-06 11:05:00 |
            | main.mysqld--help                                            |  37 | 2025-03-19 11:19:21 | 2025-04-16 06:04:05 |
            | binlog_encryption.rpl_gtid_basic                             |  36 | 2025-03-06 09:17:06 | 2025-04-17 21:26:07 |
            | rpl.rpl_xa                                                   |  36 | 2025-03-11 02:07:47 | 2025-04-17 21:04:39 |
            | rpl.rpl_xa_gtid_pos_auto_engine                              |  33 | 2025-03-11 02:07:47 | 2025-04-17 21:04:39 |
            | atomic.alter_table                                           |  31 | 2025-03-03 04:04:53 | 2025-04-18 18:18:02 |
            | federated.federated                                          |  30 | 2025-03-04 12:01:23 | 2025-03-12 07:57:33 |
            | mariabackup.incremental_compressed                           |  29 | 2025-03-07 15:25:09 | 2025-03-30 11:58:34 |
            | perfschema.table_schema                                      |  28 | 2025-03-22 12:35:19 | 2025-04-06 17:50:55 |
            | rpl.rpl_mysqldump_gtid_slave_pos                             |  27 | 2025-03-28 14:15:00 | 2025-03-28 14:33:40 |
            | main.mysql-interactive                                       |  26 | 2025-03-03 01:17:22 | 2025-04-14 14:45:43 |
            | mariabackup.full_backup                                      |  24 | 2025-03-11 02:07:47 | 2025-04-16 06:04:52 |
            | mariabackup.partial                                          |  24 | 2025-03-07 15:06:24 | 2025-04-15 10:05:51 |
            | encryption.doublewrite_debug                                 |  23 | 2025-03-07 15:07:04 | 2025-04-16 06:04:52 |
            | binlog_encryption.rpl_sync                                   |  22 | 2025-03-18 09:17:10 | 2025-04-17 21:26:07 |
            | rpl.parallel_backup_xa_debug                                 |  22 | 2025-03-01 01:24:54 | 2025-04-15 10:04:29 |
            | binlog_encryption.rpl_binlog_errors                          |  21 | 2025-03-18 09:17:10 | 2025-04-17 21:26:07 |
            | binlog_encryption.rpl_relayrotate                            |  21 | 2025-03-18 09:17:10 | 2025-04-17 21:26:07 |
            | innodb.temp_truncate_freed                                   |  21 | 2025-03-28 10:29:32 | 2025-04-19 18:08:41 |
            | main.mysqldump                                               |  21 | 2025-03-05 11:49:53 | 2025-03-31 10:03:15 |
            | parts.partition_alter4_innodb                                |  21 | 2025-03-03 11:11:02 | 2025-04-19 13:19:17 |
            | rpl.rpl_do_grant                                             |  21 | 2025-03-02 13:25:18 | 2025-04-08 10:09:14 |
            | rpl.rpl_gtid_crash                                           |  21 | 2025-03-11 02:07:47 | 2025-04-15 11:47:24 |
            | binlog_encryption.rpl_init_slave_errors                      |  20 | 2025-03-18 09:17:10 | 2025-04-03 13:51:36 |
            | binlog_encryption.rpl_parallel_analyze_table_hang            |  20 | 2025-03-18 09:17:10 | 2025-04-17 21:26:07 |
            | main.cte_nonrecursive                                        |  20 | 2025-03-25 02:53:12 | 2025-04-08 21:57:41 |
            | main.opt_hints_join_order                                    |  20 | 2025-03-04 11:57:06 | 2025-04-07 18:23:08 |
            | plugins.thread_pool_server_audit                             |  19 | 2025-03-02 14:18:31 | 2025-03-31 07:28:47 |
            | rpl.rpl_parallel_optimistic_xa_lsu_off                       |  19 | 2025-03-11 02:07:47 | 2025-03-26 11:34:05 |
            | s3.mysqldump                                                 |  19 | 2025-03-03 04:47:39 | 2025-03-26 11:45:10 |
            | compat/oracle.exception                                      |  18 | 2025-04-06 14:54:11 | 2025-04-07 16:00:35 |
            | main.alter_table                                             |  18 | 2025-03-11 02:07:47 | 2025-04-06 17:44:02 |
            | mariabackup.incremental_backup                               |  18 | 2025-03-21 13:57:47 | 2025-04-16 06:04:52 |
            | rpl.rpl_read_only                                            |  18 | 2025-03-11 02:07:47 | 2025-03-31 17:23:39 |
            | sys_vars.character_set_results_basic                         |  18 | 2025-04-06 14:54:11 | 2025-04-07 16:00:35 |
            | type_cursor.type_sys_refcursor-type_row-debug                |  18 | 2025-03-17 14:42:41 | 2025-03-18 10:48:12 |
            | binlog_encryption.rpl_parallel_deadlock_corrupt_binlog       |  17 | 2025-03-18 09:17:10 | 2025-04-17 21:26:07 |
            | binlog_encryption.rpl_skip_replication                       |  17 | 2025-03-18 09:17:10 | 2025-04-17 21:26:07 |
            | main.view                                                    |  17 | 2025-03-05 15:16:52 | 2025-04-18 10:47:23 |
            | mariabackup.encrypted_page_corruption                        |  17 | 2025-04-16 05:56:06 | 2025-04-16 06:04:52 |
            | binlog_encryption.rpl_incident                               |  16 | 2025-03-18 09:17:10 | 2025-04-17 21:26:07 |
            | binlog_encryption.rpl_special_charset                        |  16 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 |
            | binlog_encryption.rpl_sporadic_master                        |  16 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 |
            | main.order_by                                                |  16 | 2025-04-14 14:44:23 | 2025-04-17 17:51:52 |
            | main.subselect_sj_jcl6                                       |  16 | 2025-03-25 02:53:12 | 2025-04-10 22:39:57 |
            | mariabackup.incremental_encrypted                            |  16 | 2025-03-27 13:59:37 | 2025-04-16 06:04:52 |
            | vcol.wrong_arena                                             |  16 | 2025-03-13 15:00:54 | 2025-04-08 10:09:14 |
            | binlog_encryption.rpl_corruption                             |  15 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 |
            | innodb.innodb-64k-crash                                      |  15 | 2025-03-25 13:04:20 | 2025-03-28 14:54:46 |
            | innodb.undo_upgrade                                          |  15 | 2025-03-11 02:07:47 | 2025-03-12 02:51:27 |
            | binlog_encryption.binlog_xa_recover                          |  14 | 2025-03-11 02:07:47 | 2025-04-17 21:26:07 |
            | binlog_encryption.rpl_loaddata_local                         |  14 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 |
            | compat/oracle.ora_outer_join                                 |  14 | 2025-03-19 14:37:16 | 2025-04-13 13:20:24 |
            | encryption.innodb-encryption-alter                           |  14 | 2025-03-03 05:31:55 | 2025-04-19 18:08:41 |
            | main.mysql_client_test                                       |  14 | 2025-03-03 17:44:57 | 2025-04-18 10:47:23 |
            | main.mysql_upgrade-28915                                     |  14 | 2025-03-02 14:20:59 | 2025-04-19 17:57:26 |
            | binlog_encryption.rpl_parallel_domain                        |  13 | 2025-03-18 09:17:10 | 2025-04-17 21:26:07 |
            | binlog_encryption.rpl_parallel_show_binlog_events_purge_logs |  13 | 2025-03-18 09:17:10 | 2025-04-17 21:26:07 |
            | main.derived_view                                            |  13 | 2025-04-02 11:13:44 | 2025-04-15 19:01:07 |
            | rpl.parallel_backup_lsu_off                                  |  13 | 2025-03-11 02:07:47 | 2025-04-11 09:56:43 |
            | rpl.parallel_backup_slave_binlog_off                         |  13 | 2025-03-11 02:07:47 | 2025-04-11 09:56:43 |
            | binlog_encryption.rpl_parallel_innodb_lock_conflict          |  12 | 2025-03-01 07:12:19 | 2025-04-02 12:54:57 |
            | connect.alter                                                |  12 | 2025-03-04 12:01:27 | 2025-03-12 07:59:25 |
            | federated.federated_server                                   |  12 | 2025-03-04 12:01:23 | 2025-04-19 09:21:20 |
            | innodb.innodb-system-table-view                              |  12 | 2025-04-16 11:04:16 | 2025-04-16 11:09:07 |
            | innodb_fts.innodb_fts_misc_1                                 |  12 | 2025-03-02 15:55:01 | 2025-04-06 11:05:00 |
            | innodb_gis.rtree_purge                                       |  12 | 2025-03-05 11:49:37 | 2025-04-11 15:45:57 |
            | main.analyze_format_json                                     |  12 | 2025-04-14 14:44:23 | 2025-04-17 07:22:24 |
            | main.explain_json                                            |  12 | 2025-04-14 14:44:23 | 2025-04-17 07:22:24 |
            | main.show_explain_json                                       |  12 | 2025-04-14 14:44:23 | 2025-04-17 07:29:29 |
            | main.subselect4                                              |  12 | 2025-04-14 14:44:23 | 2025-04-17 07:22:24 |
            | main.subselect_cache                                         |  12 | 2025-04-14 14:44:23 | 2025-04-17 07:22:24 |
            | main.subselect_sj                                            |  12 | 2025-03-25 02:53:12 | 2025-04-08 21:57:41 |
            | mariabackup.unsupported_redo                                 |  12 | 2025-03-07 15:06:24 | 2025-03-26 08:59:09 |
            | roles.show_grants_replicated                                 |  12 | 2025-03-13 15:02:07 | 2025-04-08 10:09:14 |
            | rpl.rpl_create_drop_role                                     |  12 | 2025-03-13 15:02:07 | 2025-04-08 10:09:14 |
            | type_cursor.type_sys_refcursor-do-debug                      |  12 | 2025-03-17 14:42:41 | 2025-03-17 19:27:40 |
            | type_cursor.type_sys_refcursor-sf-debug                      |  12 | 2025-03-17 14:42:41 | 2025-03-17 19:27:40 |
            | binlog_encryption.binlog_incident                            |  11 | 2025-03-18 09:17:10 | 2025-03-19 06:32:29 |
            | binlog_encryption.binlog_row_annotate                        |  11 | 2025-03-18 09:17:10 | 2025-03-19 09:17:45 |
            | binlog_encryption.rpl_parallel_free_deferred_event           |  11 | 2025-03-06 09:17:06 | 2025-04-17 21:26:07 |
            | main.derived                                                 |  11 | 2025-03-05 11:53:23 | 2025-04-04 04:15:44 |
            | main.sp-row                                                  |  11 | 2025-03-02 12:23:21 | 2025-04-17 14:31:14 |
            | main.xa                                                      |  11 | 2025-03-11 02:07:47 | 2025-04-11 07:12:05 |
            | rpl.rpl_parallel_multi_domain_xa                             |  11 | 2025-03-11 02:07:47 | 2025-04-14 07:45:53 |
            | binlog.binlog_commit_by_rotate_atomic                        |  10 | 2025-03-11 02:07:47 | 2025-04-17 21:26:07 |
            | binlog.binlog_dmls_on_tmp_tables_readonly                    |  10 | 2025-03-31 17:22:28 | 2025-04-17 21:26:07 |
            | binlog.binlog_empty_xa_prepared                              |  10 | 2025-03-11 02:07:47 | 2025-04-17 21:26:07 |
            +--------------------------------------------------------------+-----+---------------------+---------------------+
            100 rows
            

            I also added first and last occurrence time, it used to help me to see if the failure is likely to be fixed already (if it was happening plenty but stopped some time ago, then it's likely fixed or was a false positive; not a guarantee of course).

            elenst Elena Stepanova added a comment - I'm just concerned that main trees may not have enough runs for sampling. You may be right about it, I didn't realize there would be so few builders, I used to do it for the old buildbot where all its builders counted. It was quite enough for sampling. I've done both, main-only and all trees. In fact, I've done six, the other dimension is time: March only, from the beginning of April, from the beginning of March. Main branches, March +---------------------------------------------------------------+---+---------------------+---------------------+ | test_name | c | min(dt) | max(dt) | +---------------------------------------------------------------+---+---------------------+---------------------+ | main.lock_sync | 4 | 2025-03-11 23:50:24 | 2025-03-26 13:05:56 | | mariabackup.incremental_compressed | 4 | 2025-03-15 11:17:19 | 2025-03-26 14:52:35 | | mariabackup.partial | 3 | 2025-03-19 00:23:51 | 2025-03-24 11:43:31 | | atomic.alter_table | 1 | 2025-03-05 19:27:56 | 2025-03-05 19:27:56 | | atomic.create_table | 1 | 2025-03-05 15:01:41 | 2025-03-05 15:01:41 | | binlog_encryption.encrypted_master_switch_to_unencrypted_gtid | 1 | 2025-03-19 20:14:43 | 2025-03-19 20:14:43 | | encryption.innodb-encryption-alter | 1 | 2025-03-21 17:02:31 | 2025-03-21 17:02:31 | | innodb.doublewrite_debug | 1 | 2025-03-31 09:45:13 | 2025-03-31 09:45:13 | | innodb.temp_truncate_freed | 1 | 2025-03-29 11:34:15 | 2025-03-29 11:34:15 | | main.mysql-interactive | 1 | 2025-03-22 03:17:54 | 2025-03-22 03:17:54 | | main.mysql_upgrade-28915 | 1 | 2025-03-18 12:13:49 | 2025-03-18 12:13:49 | | main.query_cache_debug | 1 | 2025-03-14 15:09:47 | 2025-03-14 15:09:47 | | main.vector2 | 1 | 2025-03-06 02:12:59 | 2025-03-06 02:12:59 | +---------------------------------------------------------------+---+---------------------+---------------------+ 13 rows All branches, March +--------------------------------------------------------------+----+---------------------+---------------------+ | test_name | c | min(dt) | max(dt) | +--------------------------------------------------------------+----+---------------------+---------------------+ | versioning.foreign | 85 | 2025-03-02 12:32:33 | 2025-03-30 15:44:30 | | main.timezone | 75 | 2025-03-30 03:53:14 | 2025-03-31 21:59:00 | | innodb.doublewrite | 51 | 2025-03-06 04:44:18 | 2025-03-29 11:27:28 | | innodb.insert_into_empty_notembedded | 45 | 2025-03-28 14:15:00 | 2025-03-28 14:33:40 | | versioning.delete | 44 | 2025-03-02 12:32:33 | 2025-03-30 15:35:02 | | binlog_encryption.encrypted_slave | 40 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 | | main.mysqldump-system | 39 | 2025-03-11 10:29:17 | 2025-03-26 13:59:44 | | versioning.rpl | 39 | 2025-03-02 15:55:01 | 2025-03-30 15:44:30 | | main.mysql_upgrade | 31 | 2025-03-02 13:22:14 | 2025-03-31 18:42:35 | | binlog_encryption.rpl_gtid_basic | 30 | 2025-03-06 09:17:06 | 2025-03-26 14:47:54 | | federated.federated | 30 | 2025-03-04 12:01:23 | 2025-03-12 07:57:33 | | mariabackup.incremental_compressed | 29 | 2025-03-07 15:25:09 | 2025-03-30 11:58:34 | | innodb_fts.misc_1 | 28 | 2025-03-02 15:55:01 | 2025-03-30 15:44:30 | | binlog_encryption.encrypted_master | 27 | 2025-03-06 09:17:06 | 2025-03-26 14:47:54 | | rpl.rpl_mysqldump_gtid_slave_pos | 27 | 2025-03-28 14:15:00 | 2025-03-28 14:33:40 | | main.lock_sync | 24 | 2025-03-05 14:36:26 | 2025-03-31 12:29:04 | | main.mysql-interactive | 24 | 2025-03-03 01:17:22 | 2025-03-30 15:35:02 | | atomic.alter_table | 22 | 2025-03-03 04:04:53 | 2025-03-28 10:33:50 | | main.mysqldump | 21 | 2025-03-05 11:49:53 | 2025-03-31 10:03:15 | | mariabackup.partial | 20 | 2025-03-07 15:06:24 | 2025-03-31 16:43:59 | | binlog_encryption.rpl_sync | 19 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 | | plugins.thread_pool_server_audit | 19 | 2025-03-02 14:18:31 | 2025-03-31 07:28:47 | | rpl.rpl_gtid_crash | 19 | 2025-03-11 02:07:47 | 2025-03-31 11:48:34 | | rpl.rpl_parallel_optimistic_xa_lsu_off | 19 | 2025-03-11 02:07:47 | 2025-03-26 11:34:05 | | s3.mysqldump | 19 | 2025-03-03 04:47:39 | 2025-03-26 11:45:10 | | binlog_encryption.rpl_binlog_errors | 18 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 | | binlog_encryption.rpl_relayrotate | 18 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 | | rpl.parallel_backup_xa_debug | 18 | 2025-03-01 01:24:54 | 2025-03-30 09:25:59 | | rpl.rpl_read_only | 18 | 2025-03-11 02:07:47 | 2025-03-31 17:23:39 | | rpl.rpl_xa | 18 | 2025-03-11 02:07:47 | 2025-03-26 11:34:05 | | type_cursor.type_sys_refcursor-type_row-debug | 18 | 2025-03-17 14:42:41 | 2025-03-18 10:48:12 | | binlog_encryption.rpl_parallel_analyze_table_hang | 17 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 | | mariabackup.full_backup | 17 | 2025-03-11 02:07:47 | 2025-03-26 12:38:55 | | binlog_encryption.rpl_special_charset | 16 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 | | binlog_encryption.rpl_sporadic_master | 16 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 | | main.opt_hints_join_order | 16 | 2025-03-04 11:57:06 | 2025-03-13 14:52:01 | | binlog_encryption.rpl_corruption | 15 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 | | encryption.doublewrite_debug | 15 | 2025-03-07 15:07:04 | 2025-03-28 10:38:05 | | innodb.innodb-64k-crash | 15 | 2025-03-25 13:04:20 | 2025-03-28 14:54:46 | | innodb.undo_upgrade | 15 | 2025-03-11 02:07:47 | 2025-03-12 02:51:27 | | rpl.rpl_xa_gtid_pos_auto_engine | 15 | 2025-03-11 02:07:47 | 2025-03-26 11:34:05 | | binlog_encryption.rpl_loaddata_local | 14 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 | | binlog_encryption.rpl_parallel_deadlock_corrupt_binlog | 14 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 | | binlog_encryption.binlog_xa_recover | 13 | 2025-03-11 02:07:47 | 2025-03-19 06:32:29 | | binlog_encryption.rpl_incident | 13 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 | | parts.partition_alter4_innodb | 13 | 2025-03-03 11:11:02 | 2025-03-31 12:43:12 | | binlog_encryption.rpl_parallel_show_binlog_events_purge_logs | 12 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 | | binlog_encryption.rpl_skip_replication | 12 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 | | connect.alter | 12 | 2025-03-04 12:01:27 | 2025-03-12 07:59:25 | | mariabackup.unsupported_redo | 12 | 2025-03-07 15:06:24 | 2025-03-26 08:59:09 | | rpl.rpl_do_grant | 12 | 2025-03-02 13:25:18 | 2025-03-13 15:02:07 | | type_cursor.type_sys_refcursor-do-debug | 12 | 2025-03-17 14:42:41 | 2025-03-17 19:27:40 | | type_cursor.type_sys_refcursor-sf-debug | 12 | 2025-03-17 14:42:41 | 2025-03-17 19:27:40 | | binlog_encryption.binlog_incident | 11 | 2025-03-18 09:17:10 | 2025-03-19 06:32:29 | | binlog_encryption.binlog_row_annotate | 11 | 2025-03-18 09:17:10 | 2025-03-19 09:17:45 | | binlog_encryption.rpl_init_slave_errors | 11 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 | | binlog_encryption.rpl_parallel_innodb_lock_conflict | 11 | 2025-03-01 07:12:19 | 2025-03-26 14:47:54 | | innodb_gis.rtree_purge | 11 | 2025-03-05 11:49:37 | 2025-03-25 17:27:36 | | binlog_encryption.rpl_parallel_domain | 10 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 | | compat/oracle.ora_outer_join | 10 | 2025-03-19 14:37:16 | 2025-03-31 12:40:31 | | compat/oracle.sp-sys_refcursor | 10 | 2025-03-15 16:22:04 | 2025-03-17 16:42:05 | | federated.federatedx | 10 | 2025-03-26 11:34:05 | 2025-03-28 14:33:40 | | federated.federated_server | 10 | 2025-03-04 12:01:23 | 2025-03-12 07:57:17 | | rpl.parallel_backup_lsu_off | 10 | 2025-03-11 02:07:47 | 2025-03-26 11:34:05 | | rpl.parallel_backup_slave_binlog_off | 10 | 2025-03-11 02:07:47 | 2025-03-26 11:34:05 | | rpl.rpl_parallel_innodb_lock_conflict | 10 | 2025-03-03 03:59:55 | 2025-03-21 20:22:56 | | rpl.rpl_parallel_multi_domain_xa | 10 | 2025-03-11 02:07:47 | 2025-03-28 15:03:11 | | rpl.rpl_xa_survive_disconnect_lsu_off | 10 | 2025-03-11 02:07:47 | 2025-03-26 11:34:05 | | spider/bugfix.mdev_27902 | 10 | 2025-03-04 12:01:27 | 2025-03-12 07:57:33 | | spider/bugfix.mdev_28683 | 10 | 2025-03-04 12:01:27 | 2025-03-12 07:57:33 | | spider/bugfix.mdev_29421 | 10 | 2025-03-04 12:01:27 | 2025-03-12 07:57:33 | | spider/bugfix.mdev_31475 | 10 | 2025-03-04 12:01:27 | 2025-03-12 07:57:33 | | spider/bugfix.mdev_32492 | 10 | 2025-03-04 12:01:27 | 2025-03-12 07:57:33 | | spider/bugfix.mdev_34541 | 10 | 2025-03-04 12:01:27 | 2025-03-12 07:57:33 | | spider/bugfix.mdev_34555 | 10 | 2025-03-04 12:01:27 | 2025-03-12 07:57:33 | | spider/bugfix.mdev_34636 | 10 | 2025-03-04 12:01:27 | 2025-03-12 07:57:33 | | spider/bugfix.self_reference | 10 | 2025-03-04 12:01:27 | 2025-03-12 07:57:33 | | spider/bugfix.self_reference_multi | 10 | 2025-03-04 12:01:27 | 2025-03-12 07:57:33 | | binlog.binlog_checkpoint_flush_logs | 9 | 2025-03-11 02:07:47 | 2025-03-12 02:52:39 | | binlog.binlog_commit_by_rotate_atomic | 9 | 2025-03-11 02:07:47 | 2025-03-12 02:52:39 | | binlog.binlog_dmls_on_tmp_tables_readonly | 9 | 2025-03-31 17:22:28 | 2025-03-31 17:23:39 | | binlog.binlog_empty_xa_prepared | 9 | 2025-03-11 02:07:47 | 2025-03-12 02:52:39 | | binlog.binlog_mdev342 | 9 | 2025-03-11 02:07:47 | 2025-03-12 02:52:39 | | binlog.binlog_recover_async_rollback_trx | 9 | 2025-03-11 02:07:47 | 2025-03-12 02:52:39 | | binlog.binlog_truncate_active_log | 9 | 2025-03-11 02:07:47 | 2025-03-12 02:52:39 | | binlog.binlog_xa_prepared_disconnect | 9 | 2025-03-11 02:07:47 | 2025-03-12 02:52:39 | | binlog.binlog_xa_recover | 9 | 2025-03-11 02:07:47 | 2025-03-12 02:52:39 | | binlog_encryption.rpl_parallel_free_deferred_event | 9 | 2025-03-06 09:17:06 | 2025-03-26 14:47:54 | | binlog_encryption.rpl_parallel_incorrect_relay_pos | 9 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 | | binlog_encryption.rpl_temporal_format_default_to_default | 9 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 | | innodb_fts.innodb_fts_misc_1 | 9 | 2025-03-02 15:55:01 | 2025-03-30 15:44:30 | | innodb_fts.misc_debug | 9 | 2025-03-05 11:57:13 | 2025-03-05 13:29:15 | | main.loadxml | 9 | 2025-03-28 14:15:00 | 2025-03-28 14:33:40 | | main.mariadb-dump-debug | 9 | 2025-03-28 14:15:00 | 2025-03-28 14:33:40 | | main.mysqldump-compat | 9 | 2025-03-28 14:15:00 | 2025-03-28 14:33:40 | | plugins.sql_error_log_withdbinfo | 9 | 2025-03-02 14:18:31 | 2025-03-31 07:28:47 | | rpl.parallel_backup | 9 | 2025-03-11 02:07:47 | 2025-03-12 02:52:39 | | rpl.rpl_parallel_optimistic_xa | 9 | 2025-03-11 02:07:47 | 2025-03-12 02:51:27 | | rpl.rpl_perfschema_connect_config | 9 | 2025-03-28 14:40:32 | 2025-03-31 17:23:39 | | rpl.rpl_xa_empty_transaction | 9 | 2025-03-11 02:07:47 | 2025-03-12 02:51:27 | +--------------------------------------------------------------+----+---------------------+---------------------+ 100 rows Main branches, April +--------------------------------------------+---+---------------------+---------------------+ | test_name | c | min(dt) | max(dt) | +--------------------------------------------+---+---------------------+---------------------+ | innodb.doublewrite | 4 | 2025-04-02 12:11:57 | 2025-04-19 17:53:54 | | innodb.temp_truncate_freed | 3 | 2025-04-14 16:08:23 | 2025-04-19 17:53:54 | | atomic.create_table | 2 | 2025-04-18 10:09:22 | 2025-04-19 13:37:54 | | main.lock_sync | 2 | 2025-04-18 23:04:57 | 2025-04-19 17:55:51 | | main.mysql_upgrade-28915 | 2 | 2025-04-16 20:25:47 | 2025-04-19 17:57:26 | | main.timezone | 2 | 2025-04-15 11:50:34 | 2025-04-15 11:53:41 | | atomic.alter_table | 1 | 2025-04-02 06:49:06 | 2025-04-02 06:49:06 | | atomic.drop_table | 1 | 2025-04-19 13:37:54 | 2025-04-19 13:37:54 | | encryption.innochecksum | 1 | 2025-04-19 09:23:59 | 2025-04-19 09:23:59 | | innodb.doublewrite_debug | 1 | 2025-04-03 13:39:25 | 2025-04-03 13:39:25 | | innodb.innodb-64k | 1 | 2025-04-10 16:20:37 | 2025-04-10 16:20:37 | | innodb.innodb-lock-inherit-read_commited | 1 | 2025-04-15 11:47:24 | 2025-04-15 11:47:24 | | innodb.innodb_stats_table_flag_auto_recalc | 1 | 2025-04-17 07:05:30 | 2025-04-17 07:05:30 | | innodb.instant_alter_crash | 1 | 2025-04-16 20:21:32 | 2025-04-16 20:21:32 | | main.mysql_upgrade | 1 | 2025-04-18 10:09:22 | 2025-04-18 10:09:22 | | main.rowid_filter_innodb | 1 | 2025-04-03 13:39:25 | 2025-04-03 13:39:25 | | mariabackup.partial | 1 | 2025-04-01 23:59:40 | 2025-04-01 23:59:40 | | rpl.rpl_gtid_crash | 1 | 2025-04-15 11:47:24 | 2025-04-15 11:47:24 | | rpl.rpl_parallel_multi_domain_xa | 1 | 2025-04-14 07:45:53 | 2025-04-14 07:45:53 | | rpl.rpl_row_create_table | 1 | 2025-04-19 09:21:21 | 2025-04-19 09:21:21 | +--------------------------------------------+---+---------------------+---------------------+ 20 rows All branches, April +-----------------------------------------------------+-----+---------------------+---------------------+ | test_name | c | min(dt) | max(dt) | +-----------------------------------------------------+-----+---------------------+---------------------+ | main.timezone | 209 | 2025-04-01 03:20:45 | 2025-04-19 13:20:06 | | versioning.update | 40 | 2025-04-17 17:47:33 | 2025-04-17 21:04:39 | | main.mysql_upgrade | 38 | 2025-04-03 10:45:27 | 2025-04-18 19:28:10 | | innodb.doublewrite | 34 | 2025-04-01 03:22:55 | 2025-04-20 10:58:51 | | main.lock_sync | 34 | 2025-04-02 07:23:05 | 2025-04-19 17:55:51 | | main.mysqld--help | 29 | 2025-04-06 14:53:55 | 2025-04-16 06:04:05 | | versioning.foreign | 24 | 2025-04-06 11:00:42 | 2025-04-06 11:05:00 | | perfschema.table_schema | 20 | 2025-04-06 14:53:55 | 2025-04-06 17:50:55 | | versioning.rpl | 19 | 2025-04-06 11:00:42 | 2025-04-06 11:05:00 | | compat/oracle.exception | 18 | 2025-04-06 14:54:11 | 2025-04-07 16:00:35 | | rpl.rpl_xa | 18 | 2025-04-17 17:49:40 | 2025-04-17 21:04:39 | | rpl.rpl_xa_gtid_pos_auto_engine | 18 | 2025-04-17 17:49:40 | 2025-04-17 21:04:39 | | sys_vars.character_set_results_basic | 18 | 2025-04-06 14:54:11 | 2025-04-07 16:00:35 | | mariabackup.encrypted_page_corruption | 17 | 2025-04-16 05:56:06 | 2025-04-16 06:04:52 | | innodb.temp_truncate_freed | 16 | 2025-04-08 07:32:28 | 2025-04-19 18:08:41 | | main.alter_table | 16 | 2025-04-06 14:53:55 | 2025-04-06 17:44:02 | | main.cte_nonrecursive | 16 | 2025-04-02 11:13:44 | 2025-04-08 21:57:41 | | main.order_by | 16 | 2025-04-14 14:44:23 | 2025-04-17 17:51:52 | | main.view | 15 | 2025-04-02 11:13:44 | 2025-04-18 10:47:23 | | mariabackup.incremental_backup | 15 | 2025-04-16 05:56:06 | 2025-04-16 06:04:52 | | mariabackup.incremental_encrypted | 15 | 2025-04-16 05:56:06 | 2025-04-16 06:04:52 | | main.derived_view | 13 | 2025-04-02 11:13:44 | 2025-04-15 19:01:07 | | innodb.innodb-system-table-view | 12 | 2025-04-16 11:04:16 | 2025-04-16 11:09:07 | | main.analyze_format_json | 12 | 2025-04-14 14:44:23 | 2025-04-17 07:22:24 | | main.explain_json | 12 | 2025-04-14 14:44:23 | 2025-04-17 07:22:24 | | main.show_explain_json | 12 | 2025-04-14 14:44:23 | 2025-04-17 07:29:29 | | main.subselect4 | 12 | 2025-04-14 14:44:23 | 2025-04-17 07:22:24 | | main.subselect_cache | 12 | 2025-04-14 14:44:23 | 2025-04-17 07:22:24 | | main.subselect_sj_jcl6 | 12 | 2025-04-08 01:17:51 | 2025-04-10 22:39:57 | | vcol.wrong_arena | 12 | 2025-04-03 10:42:52 | 2025-04-08 10:09:14 | | binlog_encryption.encrypted_master | 11 | 2025-04-14 23:07:12 | 2025-04-17 21:26:07 | | atomic.alter_table | 9 | 2025-04-02 06:49:06 | 2025-04-18 18:18:02 | | binlog_encryption.rpl_init_slave_errors | 9 | 2025-04-03 13:48:18 | 2025-04-03 13:51:36 | | innodb_fts.misc_1 | 9 | 2025-04-06 11:01:59 | 2025-04-06 11:05:00 | | main.mysql_client_test | 9 | 2025-04-03 10:45:27 | 2025-04-18 10:47:23 | | perfschema.ortho_iter | 9 | 2025-04-02 11:13:44 | 2025-04-14 20:05:10 | | roles.show_grants_replicated | 9 | 2025-04-03 10:45:27 | 2025-04-08 10:09:14 | | rpl.rpl_binlog_compress | 9 | 2025-04-14 23:07:12 | 2025-04-14 23:09:40 | | rpl.rpl_create_drop_role | 9 | 2025-04-03 10:45:27 | 2025-04-08 10:09:14 | | rpl.rpl_do_grant | 9 | 2025-04-03 10:45:27 | 2025-04-08 10:09:14 | | sequence.group_by | 9 | 2025-04-01 04:56:12 | 2025-04-01 06:34:25 | | encryption.doublewrite_debug | 8 | 2025-04-16 05:56:06 | 2025-04-16 06:04:52 | | main.group_by | 8 | 2025-04-15 03:04:07 | 2025-04-17 17:51:52 | | main.opt_hint_timeout | 8 | 2025-04-08 10:08:54 | 2025-04-08 10:30:30 | | main.partition_explicit_prune | 8 | 2025-04-06 14:53:55 | 2025-04-06 16:41:33 | | main.subselect_sj | 8 | 2025-04-08 01:17:51 | 2025-04-08 21:57:41 | | main.union | 8 | 2025-04-02 11:13:44 | 2025-04-02 11:21:06 | | main.xa | 8 | 2025-04-11 06:53:10 | 2025-04-11 07:12:05 | | parts.partition_alter4_innodb | 8 | 2025-04-03 10:34:28 | 2025-04-19 13:19:17 | | vcol.partition | 8 | 2025-04-17 17:47:33 | 2025-04-17 21:04:39 | | encryption.innodb-encryption-alter | 7 | 2025-04-02 07:22:40 | 2025-04-19 18:08:41 | | innodb.fk_col_alter | 7 | 2025-04-09 13:34:36 | 2025-04-10 06:39:55 | | innodb.foreign_null | 7 | 2025-04-09 13:34:36 | 2025-04-10 06:39:55 | | main.connect | 7 | 2025-04-02 12:45:14 | 2025-04-08 10:09:14 | | main.func_json | 7 | 2025-04-03 10:45:27 | 2025-04-10 02:42:04 | | main.grant4 | 7 | 2025-04-03 10:45:27 | 2025-04-09 08:17:36 | | main.mysql_upgrade-28915 | 7 | 2025-04-01 19:40:45 | 2025-04-19 17:57:26 | | main.ps_error | 7 | 2025-04-04 00:17:32 | 2025-04-19 18:19:18 | | mariabackup.full_backup | 7 | 2025-04-16 05:56:06 | 2025-04-16 06:04:52 | | sql_sequence.grant | 7 | 2025-04-03 15:45:28 | 2025-04-17 10:36:42 | | sys_vars.sysvars_innodb | 7 | 2025-04-16 07:17:19 | 2025-04-19 05:59:08 | | binlog.binlog_invalid_read_in_rotate | 6 | 2025-04-17 21:26:07 | 2025-04-17 21:26:07 | | binlog_encryption.rpl_gtid_basic | 6 | 2025-04-17 21:26:07 | 2025-04-17 21:26:07 | | federated.federatedx_create_handlers | 6 | 2025-04-02 11:13:53 | 2025-04-02 11:18:40 | | innodb_zip.prefix_index_liftedlimit | 6 | 2025-04-17 17:49:40 | 2025-04-17 21:04:39 | | main.derived | 6 | 2025-04-04 02:29:45 | 2025-04-04 04:15:44 | | main.grant5 | 6 | 2025-04-03 10:42:52 | 2025-04-08 10:09:14 | | main.ssl_7937 | 6 | 2025-04-02 12:45:14 | 2025-04-02 12:47:27 | | mariabackup.small_ibd | 6 | 2025-04-16 05:56:06 | 2025-04-16 06:04:52 | | roles.acl_statistics | 6 | 2025-04-03 10:42:52 | 2025-04-08 10:09:14 | | spider/bugfix.mdev_20502 | 6 | 2025-04-01 03:54:27 | 2025-04-01 06:25:01 | | sql_sequence.check | 6 | 2025-04-07 06:42:01 | 2025-04-07 07:59:20 | | sql_sequence.other | 6 | 2025-04-18 10:23:18 | 2025-04-18 10:47:23 | | sys_vars.innodb_purge_rseg_truncate_frequency_basic | 6 | 2025-04-17 17:49:40 | 2025-04-17 21:04:39 | | sys_vars.max_sort_length_func | 6 | 2025-04-17 17:49:40 | 2025-04-17 21:04:39 | | atomic.create_table | 5 | 2025-04-08 17:13:20 | 2025-04-19 13:37:54 | | binlog_encryption.rpl_skip_replication | 5 | 2025-04-01 19:35:31 | 2025-04-17 21:26:07 | | main.set_password | 5 | 2025-04-02 12:45:14 | 2025-04-08 10:09:14 | | main.show_analyze_json | 5 | 2025-04-02 12:45:41 | 2025-04-17 07:29:29 | | compat/oracle.ora_outer_join | 4 | 2025-04-13 10:53:13 | 2025-04-13 13:20:24 | | innodb.alter_varchar_change | 4 | 2025-04-16 11:04:16 | 2025-04-16 11:09:07 | | innodb.innodb-lock-inherit-read_commited | 4 | 2025-04-07 19:45:25 | 2025-04-19 09:11:55 | | innodb.innodb_buffer_pool_resize_temporary | 4 | 2025-04-11 04:42:54 | 2025-04-11 04:46:15 | | innodb.innodb_ctype_ldml | 4 | 2025-04-09 07:56:52 | 2025-04-09 08:01:55 | | innodb.instant_alter_debug | 4 | 2025-04-03 06:56:29 | 2025-04-03 07:00:53 | | innodb.instant_alter_index_rename | 4 | 2025-04-16 11:04:16 | 2025-04-16 11:09:07 | | main.commit_1innodb | 4 | 2025-04-17 05:07:03 | 2025-04-17 05:12:20 | | main.ctype_gbk | 4 | 2025-04-17 17:47:33 | 2025-04-17 17:51:52 | | main.ctype_ldml | 4 | 2025-04-10 13:10:13 | 2025-04-10 13:31:13 | | main.func_extract | 4 | 2025-04-15 15:44:22 | 2025-04-15 15:48:08 | | main.mariadb-import | 4 | 2025-04-10 12:39:10 | 2025-04-10 12:44:29 | | main.mysqltest | 4 | 2025-04-10 13:10:13 | 2025-04-10 13:31:13 | | main.opt_hints_join_order | 4 | 2025-04-07 18:19:50 | 2025-04-07 18:23:08 | | main.ps_ddl | 4 | 2025-04-09 07:56:52 | 2025-04-09 08:01:55 | | main.ps_show_log | 4 | 2025-04-06 14:53:55 | 2025-04-06 14:58:19 | | main.show_check | 4 | 2025-04-09 07:56:52 | 2025-04-09 08:01:55 | | main.sp-anchor-row-type-table | 4 | 2025-04-17 14:26:24 | 2025-04-17 14:31:14 | | main.sp-row | 4 | 2025-04-17 14:26:24 | 2025-04-17 14:31:14 | | mariabackup.partial | 4 | 2025-04-01 08:19:39 | 2025-04-15 10:05:51 | | mariabackup.partial_exclude | 4 | 2025-04-16 05:56:06 | 2025-04-16 06:04:52 | +-----------------------------------------------------+-----+---------------------+---------------------+ 100 rows Main branches, from the beginning of March +---------------------------------------------------------------+---+---------------------+---------------------+ | test_name | c | min(dt) | max(dt) | +---------------------------------------------------------------+---+---------------------+---------------------+ | main.lock_sync | 6 | 2025-03-11 23:50:24 | 2025-04-19 17:55:51 | | innodb.doublewrite | 4 | 2025-04-02 12:11:57 | 2025-04-19 17:53:54 | | innodb.temp_truncate_freed | 4 | 2025-03-29 11:34:15 | 2025-04-19 17:53:54 | | mariabackup.incremental_compressed | 4 | 2025-03-15 11:17:19 | 2025-03-26 14:52:35 | | mariabackup.partial | 4 | 2025-03-19 00:23:51 | 2025-04-01 23:59:40 | | atomic.create_table | 3 | 2025-03-05 15:01:41 | 2025-04-19 13:37:54 | | main.mysql_upgrade-28915 | 3 | 2025-03-18 12:13:49 | 2025-04-19 17:57:26 | | atomic.alter_table | 2 | 2025-03-05 19:27:56 | 2025-04-02 06:49:06 | | innodb.doublewrite_debug | 2 | 2025-03-31 09:45:13 | 2025-04-03 13:39:25 | | main.timezone | 2 | 2025-04-15 11:50:34 | 2025-04-15 11:53:41 | | atomic.drop_table | 1 | 2025-04-19 13:37:54 | 2025-04-19 13:37:54 | | binlog_encryption.encrypted_master_switch_to_unencrypted_gtid | 1 | 2025-03-19 20:14:43 | 2025-03-19 20:14:43 | | encryption.innochecksum | 1 | 2025-04-19 09:23:59 | 2025-04-19 09:23:59 | | encryption.innodb-encryption-alter | 1 | 2025-03-21 17:02:31 | 2025-03-21 17:02:31 | | innodb.innodb-64k | 1 | 2025-04-10 16:20:37 | 2025-04-10 16:20:37 | | innodb.innodb-lock-inherit-read_commited | 1 | 2025-04-15 11:47:24 | 2025-04-15 11:47:24 | | innodb.innodb_stats_table_flag_auto_recalc | 1 | 2025-04-17 07:05:30 | 2025-04-17 07:05:30 | | innodb.instant_alter_crash | 1 | 2025-04-16 20:21:32 | 2025-04-16 20:21:32 | | main.mysql-interactive | 1 | 2025-03-22 03:17:54 | 2025-03-22 03:17:54 | | main.mysql_upgrade | 1 | 2025-04-18 10:09:22 | 2025-04-18 10:09:22 | | main.query_cache_debug | 1 | 2025-03-14 15:09:47 | 2025-03-14 15:09:47 | | main.rowid_filter_innodb | 1 | 2025-04-03 13:39:25 | 2025-04-03 13:39:25 | | main.vector2 | 1 | 2025-03-06 02:12:59 | 2025-03-06 02:12:59 | | rpl.rpl_gtid_crash | 1 | 2025-04-15 11:47:24 | 2025-04-15 11:47:24 | | rpl.rpl_parallel_multi_domain_xa | 1 | 2025-04-14 07:45:53 | 2025-04-14 07:45:53 | | rpl.rpl_row_create_table | 1 | 2025-04-19 09:21:21 | 2025-04-19 09:21:21 | +---------------------------------------------------------------+---+---------------------+---------------------+ 26 rows All branches, from the beginning of March +--------------------------------------------------------------+-----+---------------------+---------------------+ | test_name | c | min(dt) | max(dt) | +--------------------------------------------------------------+-----+---------------------+---------------------+ | main.timezone | 284 | 2025-03-30 03:53:14 | 2025-04-19 13:20:06 | | versioning.foreign | 109 | 2025-03-02 12:32:33 | 2025-04-06 11:05:00 | | innodb.doublewrite | 85 | 2025-03-06 04:44:18 | 2025-04-20 10:58:51 | | main.mysql_upgrade | 69 | 2025-03-02 13:22:14 | 2025-04-18 19:28:10 | | main.lock_sync | 58 | 2025-03-05 14:36:26 | 2025-04-19 17:55:51 | | versioning.rpl | 58 | 2025-03-02 15:55:01 | 2025-04-06 11:05:00 | | innodb.insert_into_empty_notembedded | 45 | 2025-03-28 14:15:00 | 2025-03-28 14:33:40 | | versioning.delete | 44 | 2025-03-02 12:32:33 | 2025-03-30 15:35:02 | | main.mysqldump-system | 42 | 2025-03-11 10:29:17 | 2025-04-08 10:09:14 | | binlog_encryption.encrypted_slave | 40 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 | | versioning.update | 40 | 2025-04-17 17:47:33 | 2025-04-17 21:04:39 | | binlog_encryption.encrypted_master | 38 | 2025-03-06 09:17:06 | 2025-04-17 21:26:07 | | innodb_fts.misc_1 | 37 | 2025-03-02 15:55:01 | 2025-04-06 11:05:00 | | main.mysqld--help | 37 | 2025-03-19 11:19:21 | 2025-04-16 06:04:05 | | binlog_encryption.rpl_gtid_basic | 36 | 2025-03-06 09:17:06 | 2025-04-17 21:26:07 | | rpl.rpl_xa | 36 | 2025-03-11 02:07:47 | 2025-04-17 21:04:39 | | rpl.rpl_xa_gtid_pos_auto_engine | 33 | 2025-03-11 02:07:47 | 2025-04-17 21:04:39 | | atomic.alter_table | 31 | 2025-03-03 04:04:53 | 2025-04-18 18:18:02 | | federated.federated | 30 | 2025-03-04 12:01:23 | 2025-03-12 07:57:33 | | mariabackup.incremental_compressed | 29 | 2025-03-07 15:25:09 | 2025-03-30 11:58:34 | | perfschema.table_schema | 28 | 2025-03-22 12:35:19 | 2025-04-06 17:50:55 | | rpl.rpl_mysqldump_gtid_slave_pos | 27 | 2025-03-28 14:15:00 | 2025-03-28 14:33:40 | | main.mysql-interactive | 26 | 2025-03-03 01:17:22 | 2025-04-14 14:45:43 | | mariabackup.full_backup | 24 | 2025-03-11 02:07:47 | 2025-04-16 06:04:52 | | mariabackup.partial | 24 | 2025-03-07 15:06:24 | 2025-04-15 10:05:51 | | encryption.doublewrite_debug | 23 | 2025-03-07 15:07:04 | 2025-04-16 06:04:52 | | binlog_encryption.rpl_sync | 22 | 2025-03-18 09:17:10 | 2025-04-17 21:26:07 | | rpl.parallel_backup_xa_debug | 22 | 2025-03-01 01:24:54 | 2025-04-15 10:04:29 | | binlog_encryption.rpl_binlog_errors | 21 | 2025-03-18 09:17:10 | 2025-04-17 21:26:07 | | binlog_encryption.rpl_relayrotate | 21 | 2025-03-18 09:17:10 | 2025-04-17 21:26:07 | | innodb.temp_truncate_freed | 21 | 2025-03-28 10:29:32 | 2025-04-19 18:08:41 | | main.mysqldump | 21 | 2025-03-05 11:49:53 | 2025-03-31 10:03:15 | | parts.partition_alter4_innodb | 21 | 2025-03-03 11:11:02 | 2025-04-19 13:19:17 | | rpl.rpl_do_grant | 21 | 2025-03-02 13:25:18 | 2025-04-08 10:09:14 | | rpl.rpl_gtid_crash | 21 | 2025-03-11 02:07:47 | 2025-04-15 11:47:24 | | binlog_encryption.rpl_init_slave_errors | 20 | 2025-03-18 09:17:10 | 2025-04-03 13:51:36 | | binlog_encryption.rpl_parallel_analyze_table_hang | 20 | 2025-03-18 09:17:10 | 2025-04-17 21:26:07 | | main.cte_nonrecursive | 20 | 2025-03-25 02:53:12 | 2025-04-08 21:57:41 | | main.opt_hints_join_order | 20 | 2025-03-04 11:57:06 | 2025-04-07 18:23:08 | | plugins.thread_pool_server_audit | 19 | 2025-03-02 14:18:31 | 2025-03-31 07:28:47 | | rpl.rpl_parallel_optimistic_xa_lsu_off | 19 | 2025-03-11 02:07:47 | 2025-03-26 11:34:05 | | s3.mysqldump | 19 | 2025-03-03 04:47:39 | 2025-03-26 11:45:10 | | compat/oracle.exception | 18 | 2025-04-06 14:54:11 | 2025-04-07 16:00:35 | | main.alter_table | 18 | 2025-03-11 02:07:47 | 2025-04-06 17:44:02 | | mariabackup.incremental_backup | 18 | 2025-03-21 13:57:47 | 2025-04-16 06:04:52 | | rpl.rpl_read_only | 18 | 2025-03-11 02:07:47 | 2025-03-31 17:23:39 | | sys_vars.character_set_results_basic | 18 | 2025-04-06 14:54:11 | 2025-04-07 16:00:35 | | type_cursor.type_sys_refcursor-type_row-debug | 18 | 2025-03-17 14:42:41 | 2025-03-18 10:48:12 | | binlog_encryption.rpl_parallel_deadlock_corrupt_binlog | 17 | 2025-03-18 09:17:10 | 2025-04-17 21:26:07 | | binlog_encryption.rpl_skip_replication | 17 | 2025-03-18 09:17:10 | 2025-04-17 21:26:07 | | main.view | 17 | 2025-03-05 15:16:52 | 2025-04-18 10:47:23 | | mariabackup.encrypted_page_corruption | 17 | 2025-04-16 05:56:06 | 2025-04-16 06:04:52 | | binlog_encryption.rpl_incident | 16 | 2025-03-18 09:17:10 | 2025-04-17 21:26:07 | | binlog_encryption.rpl_special_charset | 16 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 | | binlog_encryption.rpl_sporadic_master | 16 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 | | main.order_by | 16 | 2025-04-14 14:44:23 | 2025-04-17 17:51:52 | | main.subselect_sj_jcl6 | 16 | 2025-03-25 02:53:12 | 2025-04-10 22:39:57 | | mariabackup.incremental_encrypted | 16 | 2025-03-27 13:59:37 | 2025-04-16 06:04:52 | | vcol.wrong_arena | 16 | 2025-03-13 15:00:54 | 2025-04-08 10:09:14 | | binlog_encryption.rpl_corruption | 15 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 | | innodb.innodb-64k-crash | 15 | 2025-03-25 13:04:20 | 2025-03-28 14:54:46 | | innodb.undo_upgrade | 15 | 2025-03-11 02:07:47 | 2025-03-12 02:51:27 | | binlog_encryption.binlog_xa_recover | 14 | 2025-03-11 02:07:47 | 2025-04-17 21:26:07 | | binlog_encryption.rpl_loaddata_local | 14 | 2025-03-18 09:17:10 | 2025-03-26 14:47:54 | | compat/oracle.ora_outer_join | 14 | 2025-03-19 14:37:16 | 2025-04-13 13:20:24 | | encryption.innodb-encryption-alter | 14 | 2025-03-03 05:31:55 | 2025-04-19 18:08:41 | | main.mysql_client_test | 14 | 2025-03-03 17:44:57 | 2025-04-18 10:47:23 | | main.mysql_upgrade-28915 | 14 | 2025-03-02 14:20:59 | 2025-04-19 17:57:26 | | binlog_encryption.rpl_parallel_domain | 13 | 2025-03-18 09:17:10 | 2025-04-17 21:26:07 | | binlog_encryption.rpl_parallel_show_binlog_events_purge_logs | 13 | 2025-03-18 09:17:10 | 2025-04-17 21:26:07 | | main.derived_view | 13 | 2025-04-02 11:13:44 | 2025-04-15 19:01:07 | | rpl.parallel_backup_lsu_off | 13 | 2025-03-11 02:07:47 | 2025-04-11 09:56:43 | | rpl.parallel_backup_slave_binlog_off | 13 | 2025-03-11 02:07:47 | 2025-04-11 09:56:43 | | binlog_encryption.rpl_parallel_innodb_lock_conflict | 12 | 2025-03-01 07:12:19 | 2025-04-02 12:54:57 | | connect.alter | 12 | 2025-03-04 12:01:27 | 2025-03-12 07:59:25 | | federated.federated_server | 12 | 2025-03-04 12:01:23 | 2025-04-19 09:21:20 | | innodb.innodb-system-table-view | 12 | 2025-04-16 11:04:16 | 2025-04-16 11:09:07 | | innodb_fts.innodb_fts_misc_1 | 12 | 2025-03-02 15:55:01 | 2025-04-06 11:05:00 | | innodb_gis.rtree_purge | 12 | 2025-03-05 11:49:37 | 2025-04-11 15:45:57 | | main.analyze_format_json | 12 | 2025-04-14 14:44:23 | 2025-04-17 07:22:24 | | main.explain_json | 12 | 2025-04-14 14:44:23 | 2025-04-17 07:22:24 | | main.show_explain_json | 12 | 2025-04-14 14:44:23 | 2025-04-17 07:29:29 | | main.subselect4 | 12 | 2025-04-14 14:44:23 | 2025-04-17 07:22:24 | | main.subselect_cache | 12 | 2025-04-14 14:44:23 | 2025-04-17 07:22:24 | | main.subselect_sj | 12 | 2025-03-25 02:53:12 | 2025-04-08 21:57:41 | | mariabackup.unsupported_redo | 12 | 2025-03-07 15:06:24 | 2025-03-26 08:59:09 | | roles.show_grants_replicated | 12 | 2025-03-13 15:02:07 | 2025-04-08 10:09:14 | | rpl.rpl_create_drop_role | 12 | 2025-03-13 15:02:07 | 2025-04-08 10:09:14 | | type_cursor.type_sys_refcursor-do-debug | 12 | 2025-03-17 14:42:41 | 2025-03-17 19:27:40 | | type_cursor.type_sys_refcursor-sf-debug | 12 | 2025-03-17 14:42:41 | 2025-03-17 19:27:40 | | binlog_encryption.binlog_incident | 11 | 2025-03-18 09:17:10 | 2025-03-19 06:32:29 | | binlog_encryption.binlog_row_annotate | 11 | 2025-03-18 09:17:10 | 2025-03-19 09:17:45 | | binlog_encryption.rpl_parallel_free_deferred_event | 11 | 2025-03-06 09:17:06 | 2025-04-17 21:26:07 | | main.derived | 11 | 2025-03-05 11:53:23 | 2025-04-04 04:15:44 | | main.sp-row | 11 | 2025-03-02 12:23:21 | 2025-04-17 14:31:14 | | main.xa | 11 | 2025-03-11 02:07:47 | 2025-04-11 07:12:05 | | rpl.rpl_parallel_multi_domain_xa | 11 | 2025-03-11 02:07:47 | 2025-04-14 07:45:53 | | binlog.binlog_commit_by_rotate_atomic | 10 | 2025-03-11 02:07:47 | 2025-04-17 21:26:07 | | binlog.binlog_dmls_on_tmp_tables_readonly | 10 | 2025-03-31 17:22:28 | 2025-04-17 21:26:07 | | binlog.binlog_empty_xa_prepared | 10 | 2025-03-11 02:07:47 | 2025-04-17 21:26:07 | +--------------------------------------------------------------+-----+---------------------+---------------------+ 100 rows I also added first and last occurrence time, it used to help me to see if the failure is likely to be fixed already (if it was happening plenty but stopped some time ago, then it's likely fixed or was a false positive; not a guarantee of course).

            Amazing! That's what I need! I'll check those out and report/link issues accordingly.

            svoj Sergey Vojtovich added a comment - Amazing! That's what I need! I'll check those out and report/link issues accordingly.

            People

              svoj Sergey Vojtovich
              svoj Sergey Vojtovich
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.