Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-20398

Slave sql thread blocked by Update_rows_log_event::find_row(-1)

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Duplicate
    • 10.1.22
    • N/A
    • Replication
    • 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1+deb8u3 (2015-08-04) x86_64 GNU/Linux

    Description

      There are more than 32 thread running on the instance,lead to many slow log,but user's connections are still running,but slow than normal.I check engine innodb status found these messages:

      Trx #rec lock waits 320708 #table lock waits 0
      Trx total rec lock wait time 792 SEC
      Trx total table lock wait time 0 SEC
      ---TRANSACTION 86665517604, ACTIVE (PREPARED) 0 sec
      3 lock struct(s), heap size 1184, 1 row lock(s), undo log entries 2
      MySQL thread id 878349428, OS thread handle 0x7fb5255fab00, query id 189853061018 Unlocking tables
      Trx #rec lock waits 318751 #table lock waits 0
      Trx total rec lock wait time 774 SEC
      Trx total table lock wait time 0 SEC
      ---TRANSACTION 86665517602, ACTIVE (PREPARED) 0 sec
      3 lock struct(s), heap size 1184, 1 row lock(s), undo log entries 2

      878349423 system user NULL Connect 0 Waiting for prior transaction to start commit before starting next transaction NULL 0.000
      878349424 system user NULL Connect 0 Waiting for prior transaction to start commit before starting next transaction NULL 0.000
      878349425 system user NULL Connect 0 Waiting for prior transaction to start commit before starting next transaction NULL 0.000
      878349426 system user NULL Connect 0 Waiting for prior transaction to start commit before starting next transaction NULL 0.000
      878349427 system user NULL Connect 0 Waiting for prior transaction to start commit before starting next transaction NULL 0.000
      878349428 system user NULL Connect 0 Update_rows_log_event::find_row(-1) NULL 0.000
      878349429 system user NULL Connect 0 Waiting for prior transaction to start commit before starting next transaction NULL 0.000
      878349430 system user NULL Connect 0 Waiting for prior transaction to start commit before starting next transaction NULL 0.000
      878349431 system user NULL Connect 799 Slave has read all relay log; waiting for the slave I/O thread to update it NULL 0.000
      889402627 repl IP1:13232 NULL Binlog Dump 2062590 Master has sent all binlog to slave; waiting for binlog to be updated NULL 0.000
      889832413 repl IP2:59589 NULL Binlog Dump 1782552 Master has sent all binlog to slave; waiting for binlog to be updated NULL 0.000

      ---TRANSACTION 2819, not started
      MySQL thread id 1, OS thread handle 0x7fb554d58b00, query id 0 Waiting for background binlog tasks
      ---TRANSACTION 86665661686, ACTIVE 0 sec starting index read
      mysql tables in use 1, locked 1
      1 lock struct(s), heap size 360, 0 row lock(s)
      MySQL thread id 878349428, OS thread handle 0x7fb5255fab00, query id 189853425690 Update_rows_log_event::find_row(-1)
      Trx #rec lock waits 318754 #table lock waits 0
      Trx total rec lock wait time 774 SEC
      Trx total table lock wait time 0 SEC
      --------
      ROW OPERATIONS
      --------------
      32 queries inside InnoDB, 134 queries in queue
      33 read views open inside InnoDB
      2 RW transactions active inside InnoDB
      0 RO transactions active inside InnoDB
      2 out of 1000 descriptors used

      I killed the thread 878349428,and user's queries become normal.
      and then I found slave io thread is not running,so I started it.then everything was ok.

      Attachments

        Issue Links

          Activity

            mperez Miguel P added a comment -

            Hi Andrei Elkin, thanks!

            The hang happens right before an event for a Replicate_Ignore_Table in this example, but it also happened in another table before I had this table ignored (I've reconfigured replication from a fresh dump a few times, always hanged at either update or delete with find_rows(-1)). I'm ignoring the `md_session` table because I don't need it and because it's MyISAM, but it didn't help.

            We have nothing doing FLUSH TABLES WITH READ LOCK, nor any other threads or clients running when it hangs. (Com_flush is 4 because I did a few FLUSH TABLES during some tests prior to setting up replication.)

            Doesn't seem to be an instance of MDEV-20247 as there the SQL thread seems to be done ("Slave has read all relay log; waiting for the slave I/O thread to update it") with the IO thread stuck.

            Here's the SHOW CREATE TABLE:

            CREATE TABLE `ContactCredential` (
              `Contact` int(10) unsigned NOT NULL,
              `Branding` varchar(22) NOT NULL,
              `Password` binary(20) DEFAULT NULL COMMENT '...',
              `OldPassword` binary(20) DEFAULT NULL COMMENT '...',
              `FailedLoginCount` tinyint(4) NOT NULL DEFAULT '0' COMMENT '...',
              PRIMARY KEY (`Contact`,`Branding`),
              CONSTRAINT `ContactCredential_ibfk_1` FOREIGN KEY (`Contact`) REFERENCES `Contact` (`ID`) ON DELETE CASCADE ON UPDATE CASCADE
            ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
            

            SHOW GLOBAL VARIABLES:

            Variable_name	Value
            aria_block_size	8192
            aria_checkpoint_interval	30
            aria_checkpoint_log_activity	1048576
            aria_encrypt_tables	OFF
            aria_force_start_after_recovery_failures	0
            aria_group_commit	none
            aria_group_commit_interval	0
            aria_log_file_size	1073741824
            aria_log_purge_type	immediate
            aria_max_sort_file_size	9223372036853727232
            aria_page_checksum	ON
            aria_pagecache_age_threshold	300
            aria_pagecache_buffer_size	134217728
            aria_pagecache_division_limit	100
            aria_pagecache_file_hash_size	512
            aria_recover	NORMAL
            aria_repair_threads	1
            aria_sort_buffer_size	268434432
            aria_stats_method	nulls_unequal
            aria_sync_log_dir	NEWFILE
            aria_used_for_temp_tables	ON
            auto_increment_increment	1
            auto_increment_offset	1
            autocommit	ON
            automatic_sp_privileges	ON
            back_log	52
            basedir	/usr
            big_tables	OFF
            binlog_annotate_row_events	OFF
            binlog_cache_size	65536
            binlog_checksum	NONE
            binlog_commit_wait_count	0
            binlog_commit_wait_usec	100000
            binlog_direct_non_transactional_updates	OFF
            binlog_format	MIXED
            binlog_optimize_thread_scheduling	ON
            binlog_row_image	FULL
            binlog_stmt_cache_size	32768
            bulk_insert_buffer_size	8388608
            character_set_client	utf8mb4
            character_set_connection	utf8mb4
            character_set_database	utf8mb4
            character_set_filesystem	binary
            character_set_results	utf8mb4
            character_set_server	utf8mb4
            character_set_system	utf8
            character_sets_dir	/usr/share/mysql/charsets/
            collation_connection	utf8mb4_general_ci
            collation_database	utf8mb4_general_ci
            collation_server	utf8mb4_general_ci
            completion_type	NO_CHAIN
            concurrent_insert	AUTO
            connect_timeout	10
            core_file	OFF
            datadir	/data/mysql/
            date_format	%Y-%m-%d
            datetime_format	%Y-%m-%d %H:%i:%s
            deadlock_search_depth_long	15
            deadlock_search_depth_short	4
            deadlock_timeout_long	50000000
            deadlock_timeout_short	10000
            debug_no_thread_alarm	OFF
            default_regex_flags	
            default_storage_engine	InnoDB
            default_tmp_storage_engine	
            default_week_format	0
            delay_key_write	ON
            delayed_insert_limit	100
            delayed_insert_timeout	300
            delayed_queue_size	1000
            div_precision_increment	4
            encrypt_binlog	OFF
            encrypt_tmp_disk_tables	OFF
            encrypt_tmp_files	OFF
            enforce_storage_engine	
            event_scheduler	DISABLED
            expensive_subquery_limit	100
            expire_logs_days	15
            explicit_defaults_for_timestamp	OFF
            extra_max_connections	1
            extra_port	0
            flush	OFF
            flush_time	0
            foreign_key_checks	ON
            ft_boolean_syntax	+ -><()~*:""&|
            ft_max_word_len	20
            ft_min_word_len	3
            ft_query_expansion_limit	20
            ft_stopword_file	(built-in)
            general_log	OFF
            general_log_file	/data/mysql-log/mysql.log
            group_concat_max_len	16384
            gtid_binlog_pos	
            gtid_binlog_state	
            gtid_current_pos	0-1-139164901
            gtid_domain_id	0
            gtid_ignore_duplicates	OFF
            gtid_slave_pos	0-1-139164901
            gtid_strict_mode	OFF
            have_compress	YES
            have_crypt	YES
            have_dynamic_loading	YES
            have_geometry	YES
            have_openssl	NO
            have_profiling	YES
            have_query_cache	YES
            have_rtree_keys	YES
            have_ssl	DISABLED
            have_symlink	YES
            histogram_size	0
            histogram_type	SINGLE_PREC_HB
            host_cache_size	138
            hostname	SssssSlave
            ignore_builtin_innodb	OFF
            ignore_db_dirs	
            init_connect	
            init_file	
            init_slave	
            innodb_adaptive_flushing	ON
            innodb_adaptive_flushing_lwm	10.000000
            innodb_adaptive_hash_index	ON
            innodb_adaptive_hash_index_partitions	1
            innodb_adaptive_max_sleep_delay	150000
            innodb_additional_mem_pool_size	2097152
            innodb_api_bk_commit_interval	5
            innodb_api_disable_rowlock	OFF
            innodb_api_enable_binlog	OFF
            innodb_api_enable_mdl	OFF
            innodb_api_trx_level	0
            innodb_autoextend_increment	256
            innodb_autoinc_lock_mode	1
            innodb_background_scrub_data_check_interval	3600
            innodb_background_scrub_data_compressed	OFF
            innodb_background_scrub_data_interval	604800
            innodb_background_scrub_data_uncompressed	OFF
            innodb_buf_dump_status_frequency	0
            innodb_buffer_pool_dump_at_shutdown	OFF
            innodb_buffer_pool_dump_now	OFF
            innodb_buffer_pool_dump_pct	100
            innodb_buffer_pool_filename	ib_buffer_pool
            innodb_buffer_pool_instances	8
            innodb_buffer_pool_load_abort	OFF
            innodb_buffer_pool_load_at_startup	OFF
            innodb_buffer_pool_load_now	OFF
            innodb_buffer_pool_populate	OFF
            innodb_buffer_pool_size	134217728
            innodb_change_buffer_max_size	25
            innodb_change_buffering	all
            innodb_checksum_algorithm	INNODB
            innodb_checksums	ON
            innodb_cleaner_lsn_age_factor	HIGH_CHECKPOINT
            innodb_cmp_per_index_enabled	OFF
            innodb_commit_concurrency	0
            innodb_compression_algorithm	zlib
            innodb_compression_failure_threshold_pct	5
            innodb_compression_level	6
            innodb_compression_pad_pct_max	50
            innodb_concurrency_tickets	5000
            innodb_corrupt_table_action	assert
            innodb_data_file_path	ibdata1:4096M:autoextend
            innodb_data_home_dir	
            innodb_default_encryption_key_id	1
            innodb_default_row_format	compact
            innodb_defragment	OFF
            innodb_defragment_fill_factor	0.900000
            innodb_defragment_fill_factor_n_recs	20
            innodb_defragment_frequency	40
            innodb_defragment_n_pages	7
            innodb_defragment_stats_accuracy	0
            innodb_disable_sort_file_cache	OFF
            innodb_disallow_writes	OFF
            innodb_doublewrite	ON
            innodb_empty_free_list_algorithm	LEGACY
            innodb_encrypt_log	OFF
            innodb_encrypt_tables	OFF
            innodb_encryption_rotate_key_age	1
            innodb_encryption_rotation_iops	100
            innodb_encryption_threads	0
            innodb_fake_changes	OFF
            innodb_fast_shutdown	1
            innodb_fatal_semaphore_wait_threshold	600
            innodb_file_format	Antelope
            innodb_file_format_check	ON
            innodb_file_format_max	Antelope
            innodb_file_per_table	ON
            innodb_flush_log_at_timeout	1
            innodb_flush_log_at_trx_commit	0
            innodb_flush_method	
            innodb_flush_neighbors	1
            innodb_flushing_avg_loops	30
            innodb_force_load_corrupted	OFF
            innodb_force_primary_key	OFF
            innodb_force_recovery	0
            innodb_foreground_preflush	EXPONENTIAL_BACKOFF
            innodb_ft_aux_table	
            innodb_ft_cache_size	8000000
            innodb_ft_enable_diag_print	OFF
            innodb_ft_enable_stopword	ON
            innodb_ft_max_token_size	84
            innodb_ft_min_token_size	3
            innodb_ft_num_word_optimize	2000
            innodb_ft_result_cache_limit	2000000000
            innodb_ft_server_stopword_table	
            innodb_ft_sort_pll_degree	2
            innodb_ft_total_cache_size	640000000
            innodb_ft_user_stopword_table	
            innodb_idle_flush_pct	100
            innodb_immediate_scrub_data_uncompressed	OFF
            innodb_instrument_semaphores	OFF
            innodb_io_capacity	200
            innodb_io_capacity_max	2000
            innodb_kill_idle_transaction	0
            innodb_large_prefix	OFF
            innodb_lock_schedule_algorithm	fcfs
            innodb_lock_wait_timeout	50
            innodb_locking_fake_changes	ON
            innodb_locks_unsafe_for_binlog	OFF
            innodb_log_arch_dir	./
            innodb_log_arch_expire_sec	0
            innodb_log_archive	OFF
            innodb_log_block_size	512
            innodb_log_buffer_size	1048576
            innodb_log_checksum_algorithm	INNODB
            innodb_log_compressed_pages	ON
            innodb_log_file_size	20971520
            innodb_log_files_in_group	2
            innodb_log_group_home_dir	./
            innodb_lru_scan_depth	1024
            innodb_max_bitmap_file_size	104857600
            innodb_max_changed_pages	1000000
            innodb_max_dirty_pages_pct	75.000000
            innodb_max_dirty_pages_pct_lwm	0.001000
            innodb_max_purge_lag	0
            innodb_max_purge_lag_delay	0
            innodb_mirrored_log_groups	1
            innodb_monitor_disable	
            innodb_monitor_enable	
            innodb_monitor_reset	
            innodb_monitor_reset_all	
            innodb_mtflush_threads	8
            innodb_old_blocks_pct	37
            innodb_old_blocks_time	1000
            innodb_online_alter_log_max_size	134217728
            innodb_open_files	300
            innodb_optimize_fulltext_only	OFF
            innodb_page_size	16384
            innodb_prefix_index_cluster_optimization	OFF
            innodb_print_all_deadlocks	OFF
            innodb_print_lock_wait_timeout_info	OFF
            innodb_purge_batch_size	300
            innodb_purge_threads	1
            innodb_random_read_ahead	OFF
            innodb_read_ahead_threshold	56
            innodb_read_io_threads	4
            innodb_read_only	OFF
            innodb_replication_delay	0
            innodb_rollback_on_timeout	OFF
            innodb_rollback_segments	128
            innodb_sched_priority_cleaner	19
            innodb_scrub_log	OFF
            innodb_scrub_log_speed	256
            innodb_show_locks_held	10
            innodb_show_verbose_locks	0
            innodb_simulate_comp_failures	0
            innodb_sort_buffer_size	1048576
            innodb_spin_wait_delay	6
            innodb_stats_auto_recalc	ON
            innodb_stats_include_delete_marked	OFF
            innodb_stats_method	nulls_equal
            innodb_stats_modified_counter	0
            innodb_stats_on_metadata	OFF
            innodb_stats_persistent	ON
            innodb_stats_persistent_sample_pages	20
            innodb_stats_sample_pages	8
            innodb_stats_traditional	ON
            innodb_stats_transient_sample_pages	8
            innodb_status_output	OFF
            innodb_status_output_locks	OFF
            innodb_strict_mode	OFF
            innodb_support_xa	ON
            innodb_sync_array_size	1
            innodb_sync_spin_loops	30
            innodb_table_locks	ON
            innodb_thread_concurrency	7
            innodb_thread_sleep_delay	0
            innodb_tmpdir	
            innodb_track_changed_pages	OFF
            innodb_undo_directory	.
            innodb_undo_logs	128
            innodb_undo_tablespaces	0
            innodb_use_atomic_writes	OFF
            innodb_use_fallocate	OFF
            innodb_use_global_flush_log_at_trx_commit	ON
            innodb_use_mtflush	OFF
            innodb_use_native_aio	ON
            innodb_use_stacktrace	OFF
            innodb_use_sys_malloc	ON
            innodb_use_trim	OFF
            innodb_version	5.6.44-86.0
            innodb_write_io_threads	4
            interactive_timeout	28800
            join_buffer_size	524288
            join_buffer_space_limit	2097152
            join_cache_level	2
            keep_files_on_create	OFF
            key_buffer_size	8388608
            key_cache_age_threshold	300
            key_cache_block_size	1024
            key_cache_division_limit	100
            key_cache_file_hash_size	512
            key_cache_segments	0
            large_files_support	ON
            large_page_size	0
            large_pages	OFF
            lc_messages	en_US
            lc_messages_dir	/usr/share/mysql
            lc_time_names	en_US
            license	GPL
            local_infile	ON
            lock_wait_timeout	31536000
            locked_in_memory	OFF
            log_bin	OFF
            log_bin_basename	
            log_bin_index	
            log_bin_trust_function_creators	OFF
            log_error	/data/mysql-log/mysql.err
            log_output	FILE
            log_queries_not_using_indexes	OFF
            log_slave_updates	OFF
            log_slow_admin_statements	OFF
            log_slow_filter	admin,filesort,filesort_on_disk,full_join,full_scan,query_cache,query_cache_miss,tmp_table,tmp_table_on_disk
            log_slow_rate_limit	1
            log_slow_slave_statements	OFF
            log_slow_verbosity	
            log_tc_size	24576
            log_warnings	2
            long_query_time	10.000000
            low_priority_updates	OFF
            lower_case_file_system	OFF
            lower_case_table_names	0
            master_verify_checksum	OFF
            max_allowed_packet	4194304
            max_binlog_cache_size	18446744073709547520
            max_binlog_size	8388608
            max_binlog_stmt_cache_size	18446744073709547520
            max_connect_errors	8
            max_connections	10
            max_delayed_threads	20
            max_digest_length	1024
            max_error_count	64
            max_heap_table_size	33554432
            max_insert_delayed_threads	20
            max_join_size	18446744073709551615
            max_length_for_sort_data	1024
            max_long_data_size	4194304
            max_prepared_stmt_count	16382
            max_relay_log_size	8388608
            max_seeks_for_key	4294967295
            max_session_mem_used	9223372036854775807
            max_sort_length	256
            max_sp_recursion_depth	8
            max_statement_time	0.000000
            max_tmp_tables	32
            max_user_connections	19
            max_write_lock_count	4294967295
            metadata_locks_cache_size	1024
            metadata_locks_hash_instances	8
            min_examined_row_limit	0
            mrr_buffer_size	262144
            multi_range_count	256
            myisam_block_size	1024
            myisam_data_pointer_size	6
            myisam_max_sort_file_size	2147483648
            myisam_mmap_size	18446744073709551615
            myisam_recover_options	BACKUP
            myisam_repair_threads	1
            myisam_sort_buffer_size	16777216
            myisam_stats_method	NULLS_UNEQUAL
            myisam_use_mmap	OFF
            mysql56_temporal_format	ON
            net_buffer_length	16384
            net_read_timeout	30
            net_retry_count	10
            net_write_timeout	60
            old	OFF
            old_alter_table	OFF
            old_mode	
            old_passwords	OFF
            open_files_limit	743
            optimizer_prune_level	1
            optimizer_search_depth	62
            optimizer_selectivity_sampling_limit	100
            optimizer_switch	index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=off
            optimizer_use_condition_selectivity	1
            performance_schema	OFF
            performance_schema_accounts_size	-1
            performance_schema_digests_size	-1
            performance_schema_events_stages_history_long_size	-1
            performance_schema_events_stages_history_size	-1
            performance_schema_events_statements_history_long_size	-1
            performance_schema_events_statements_history_size	-1
            performance_schema_events_waits_history_long_size	-1
            performance_schema_events_waits_history_size	-1
            performance_schema_hosts_size	-1
            performance_schema_max_cond_classes	80
            performance_schema_max_cond_instances	-1
            performance_schema_max_digest_length	1024
            performance_schema_max_file_classes	50
            performance_schema_max_file_handles	32768
            performance_schema_max_file_instances	-1
            performance_schema_max_mutex_classes	200
            performance_schema_max_mutex_instances	-1
            performance_schema_max_rwlock_classes	40
            performance_schema_max_rwlock_instances	-1
            performance_schema_max_socket_classes	10
            performance_schema_max_socket_instances	-1
            performance_schema_max_stage_classes	150
            performance_schema_max_statement_classes	178
            performance_schema_max_table_handles	-1
            performance_schema_max_table_instances	-1
            performance_schema_max_thread_classes	50
            performance_schema_max_thread_instances	-1
            performance_schema_session_connect_attrs_size	-1
            performance_schema_setup_actors_size	100
            performance_schema_setup_objects_size	100
            performance_schema_users_size	-1
            pid_file	/var/run/mysqld/mysqld.pid
            plugin_dir	/usr/lib/x86_64-linux-gnu/mariadb18/plugin/
            plugin_maturity	unknown
            port	3306
            preload_buffer_size	32768
            profiling	OFF
            profiling_history_size	15
            progress_report_time	5
            protocol_version	10
            query_alloc_block_size	16384
            query_cache_limit	262144
            query_cache_min_res_unit	4096
            query_cache_size	33554432
            query_cache_strip_comments	OFF
            query_cache_type	ON
            query_cache_wlock_invalidate	ON
            query_prealloc_size	24576
            range_alloc_block_size	4096
            read_buffer_size	786432
            read_only	OFF
            read_rnd_buffer_size	2097152
            relay_log	/data/mysql-log/relay
            relay_log_basename	/data/mysql-log/relay
            relay_log_index	/data/mysql-log/relay.index
            relay_log_info_file	/data/mysql-log/relay.info
            relay_log_purge	ON
            relay_log_recovery	OFF
            relay_log_space_limit	0
            replicate_annotate_row_events	OFF
            replicate_do_db	
            replicate_do_table	
            replicate_events_marked_for_skip	REPLICATE
            replicate_ignore_db	
            replicate_ignore_table	sssss.md_session
            replicate_wild_do_table	
            replicate_wild_ignore_table	
            report_host	
            report_password	
            report_port	3306
            report_user	
            rowid_merge_buff_size	8388608
            secure_auth	ON
            secure_file_priv	
            server_id	2
            skip_external_locking	ON
            skip_name_resolve	ON
            skip_networking	OFF
            skip_show_database	OFF
            slave_compressed_protocol	OFF
            slave_ddl_exec_mode	IDEMPOTENT
            slave_domain_parallel_threads	0
            slave_exec_mode	STRICT
            slave_load_tmpdir	/tmp
            slave_max_allowed_packet	1073741824
            slave_net_timeout	3600
            slave_parallel_max_queued	131072
            slave_parallel_mode	conservative
            slave_parallel_threads	0
            slave_run_triggers_for_rbr	NO
            slave_skip_errors	OFF
            slave_sql_verify_checksum	ON
            slave_transaction_retries	10
            slave_type_conversions	
            slow_launch_time	2
            slow_query_log	ON
            slow_query_log_file	/data/mysql-log/mysql-slow.log
            socket	/var/run/mysqld/mysqld.sock
            sort_buffer_size	786432
            sql_auto_is_null	OFF
            sql_big_selects	ON
            sql_buffer_result	OFF
            sql_log_bin	ON
            sql_log_off	OFF
            sql_mode	PIPES_AS_CONCAT,STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
            sql_notes	ON
            sql_quote_show_create	ON
            sql_safe_updates	OFF
            sql_select_limit	18446744073709551615
            sql_slave_skip_counter	0
            sql_warnings	OFF
            ssl_ca	
            ssl_capath	
            ssl_cert	
            ssl_cipher	
            ssl_crl	
            ssl_crlpath	
            ssl_key	
            storage_engine	InnoDB
            stored_program_cache	256
            strict_password_validation	ON
            sync_binlog	0
            sync_frm	ON
            sync_master_info	10000
            sync_relay_log	10000
            sync_relay_log_info	10000
            system_time_zone	CET
            table_definition_cache	400
            table_open_cache	350
            thread_cache_size	4
            thread_concurrency	3
            thread_handling	one-thread-per-connection
            thread_pool_idle_timeout	60
            thread_pool_max_threads	1000
            thread_pool_oversubscribe	3
            thread_pool_size	2
            thread_pool_stall_limit	500
            thread_stack	196608
            time_format	%H:%i:%s
            time_zone	SYSTEM
            timed_mutexes	OFF
            tmp_table_size	12582912
            tmpdir	/tmp
            transaction_alloc_block_size	8192
            transaction_prealloc_size	4096
            tx_isolation	READ-COMMITTED
            tx_read_only	OFF
            unique_checks	ON
            updatable_views_with_limit	YES
            use_stat_tables	NEVER
            userstat	OFF
            version	10.1.41-MariaDB-0+deb9u1
            version_comment	Debian 9.9
            version_compile_machine	x86_64
            version_compile_os	debian-linux-gnu
            version_malloc_library	system jemalloc
            version_ssl_library	YaSSL 2.4.4
            wait_timeout	28800
            wsrep_osu_method	TOI
            wsrep_auto_increment_control	ON
            wsrep_causal_reads	OFF
            wsrep_certification_rules	strict
            wsrep_certify_nonpk	ON
            wsrep_cluster_address	
            wsrep_cluster_name	my_wsrep_cluster
            wsrep_convert_lock_to_trx	OFF
            wsrep_data_home_dir	/data/mysql/
            wsrep_dbug_option	
            wsrep_debug	OFF
            wsrep_desync	OFF
            wsrep_dirty_reads	OFF
            wsrep_drupal_282555_workaround	OFF
            wsrep_forced_binlog_format	NONE
            wsrep_gtid_domain_id	0
            wsrep_gtid_mode	OFF
            wsrep_load_data_splitting	ON
            wsrep_log_conflicts	OFF
            wsrep_max_ws_rows	0
            wsrep_max_ws_size	2147483647
            wsrep_mysql_replication_bundle	0
            wsrep_node_address	
            wsrep_node_incoming_address	AUTO
            wsrep_node_name	SssssSlave
            wsrep_notify_cmd	
            wsrep_on	OFF
            wsrep_patch_version	wsrep_25.24
            wsrep_provider	none
            wsrep_provider_options	
            wsrep_recover	OFF
            wsrep_reject_queries	NONE
            wsrep_replicate_myisam	OFF
            wsrep_restart_slave	OFF
            wsrep_retry_autocommit	1
            wsrep_slave_fk_checks	ON
            wsrep_slave_uk_checks	OFF
            wsrep_slave_threads	1
            wsrep_sst_auth	
            wsrep_sst_donor	
            wsrep_sst_donor_rejects_queries	OFF
            wsrep_sst_method	rsync
            wsrep_sst_receive_address	AUTO
            wsrep_start_position	00000000-0000-0000-0000-000000000000:-1
            wsrep_sync_wait	0
            

            SHOW GLOBAL STATUS

            Variable_name	Value
            Aborted_clients	0
            Aborted_connects	0
            Access_denied_errors	0
            Acl_column_grants	0
            Acl_database_grants	10
            Acl_function_grants	0
            Acl_procedure_grants	0
            Acl_proxy_users	1
            Acl_role_grants	0
            Acl_roles	0
            Acl_table_grants	6
            Acl_users	11
            Aria_pagecache_blocks_not_flushed	0
            Aria_pagecache_blocks_unused	15706
            Aria_pagecache_blocks_used	0
            Aria_pagecache_read_requests	0
            Aria_pagecache_reads	0
            Aria_pagecache_write_requests	0
            Aria_pagecache_writes	0
            Aria_transaction_log_syncs	0
            Binlog_commits	0
            Binlog_group_commits	0
            Binlog_group_commit_trigger_count	0
            Binlog_group_commit_trigger_lock_wait	0
            Binlog_group_commit_trigger_timeout	0
            Binlog_snapshot_file	
            Binlog_snapshot_position	0
            Binlog_bytes_written	0
            Binlog_cache_disk_use	0
            Binlog_cache_use	0
            Binlog_stmt_cache_disk_use	0
            Binlog_stmt_cache_use	0
            Busy_time	0.000000
            Bytes_received	10712143359
            Bytes_sent	3290678354
            Com_admin_commands	2
            Com_alter_db	70
            Com_alter_db_upgrade	0
            Com_alter_event	0
            Com_alter_function	0
            Com_alter_procedure	0
            Com_alter_server	0
            Com_alter_table	674
            Com_alter_tablespace	0
            Com_analyze	0
            Com_assign_to_keycache	0
            Com_begin	5
            Com_binlog	0
            Com_call_procedure	0
            Com_change_db	2
            Com_change_master	2
            Com_check	0
            Com_checksum	0
            Com_commit	341
            Com_compound_sql	0
            Com_create_db	1
            Com_create_event	0
            Com_create_function	4
            Com_create_index	0
            Com_create_procedure	3
            Com_create_role	0
            Com_create_server	0
            Com_create_table	337
            Com_create_temporary_table	0
            Com_create_trigger	86
            Com_create_udf	0
            Com_create_user	0
            Com_create_view	0
            Com_dealloc_sql	0
            Com_delete	0
            Com_delete_multi	0
            Com_do	0
            Com_drop_db	1
            Com_drop_event	0
            Com_drop_function	4
            Com_drop_index	0
            Com_drop_procedure	3
            Com_drop_role	0
            Com_drop_server	0
            Com_drop_table	337
            Com_drop_temporary_table	0
            Com_drop_trigger	0
            Com_drop_user	0
            Com_drop_view	0
            Com_empty_query	0
            Com_execute_sql	0
            Com_flush	4
            Com_get_diagnostics	0
            Com_grant	0
            Com_grant_role	0
            Com_ha_close	0
            Com_ha_open	0
            Com_ha_read	0
            Com_help	0
            Com_insert	9480
            Com_insert_select	0
            Com_install_plugin	0
            Com_kill	0
            Com_load	0
            Com_lock_tables	337
            Com_optimize	0
            Com_preload_keys	0
            Com_prepare_sql	0
            Com_purge	0
            Com_purge_before_date	0
            Com_release_savepoint	0
            Com_rename_table	0
            Com_rename_user	0
            Com_repair	0
            Com_replace	0
            Com_replace_select	0
            Com_reset	0
            Com_resignal	0
            Com_revoke	0
            Com_revoke_all	0
            Com_revoke_role	0
            Com_rollback	0
            Com_rollback_to_savepoint	0
            Com_savepoint	0
            Com_select	274
            Com_set_option	2587
            Com_show_authors	0
            Com_show_binlog_events	0
            Com_show_binlogs	0
            Com_show_charsets	0
            Com_show_collations	0
            Com_show_contributors	0
            Com_show_create_db	0
            Com_show_create_event	0
            Com_show_create_func	0
            Com_show_create_proc	0
            Com_show_create_table	2
            Com_show_create_trigger	0
            Com_show_databases	12
            Com_show_engine_logs	0
            Com_show_engine_mutex	0
            Com_show_engine_status	2
            Com_show_errors	0
            Com_show_events	0
            Com_show_explain	0
            Com_show_fields	3735
            Com_show_function_status	0
            Com_show_generic	0
            Com_show_grants	0
            Com_show_keys	0
            Com_show_master_status	0
            Com_show_open_tables	0
            Com_show_plugins	0
            Com_show_privileges	0
            Com_show_procedure_status	0
            Com_show_processlist	66
            Com_show_profile	0
            Com_show_profiles	0
            Com_show_relaylog_events	0
            Com_show_slave_hosts	0
            Com_show_slave_status	22
            Com_show_status	1
            Com_show_storage_engines	0
            Com_show_table_status	0
            Com_show_tables	12
            Com_show_triggers	0
            Com_show_variables	4
            Com_show_warnings	0
            Com_shutdown	0
            Com_signal	0
            Com_start_all_slaves	0
            Com_start_slave	1
            Com_stmt_close	0
            Com_stmt_execute	0
            Com_stmt_fetch	0
            Com_stmt_prepare	0
            Com_stmt_reprepare	0
            Com_stmt_reset	0
            Com_stmt_send_long_data	0
            Com_stop_all_slaves	0
            Com_stop_slave	0
            Com_truncate	0
            Com_uninstall_plugin	0
            Com_unlock_tables	337
            Com_update	1
            Com_update_multi	0
            Com_xa_commit	0
            Com_xa_end	0
            Com_xa_prepare	0
            Com_xa_recover	0
            Com_xa_rollback	0
            Com_xa_start	0
            Compression	OFF
            Connection_errors_accept	0
            Connection_errors_internal	0
            Connection_errors_max_connections	0
            Connection_errors_peer_address	0
            Connection_errors_select	0
            Connection_errors_tcpwrap	0
            Connections	285
            Cpu_time	0.000000
            Created_tmp_disk_tables	2
            Created_tmp_files	6
            Created_tmp_tables	42
            Delayed_errors	0
            Delayed_insert_threads	0
            Delayed_writes	0
            Delete_scan	0
            Empty_queries	238
            Executed_events	0
            Executed_triggers	0
            Feature_delay_key_write	0
            Feature_dynamic_columns	0
            Feature_fulltext	0
            Feature_gis	0
            Feature_locale	0
            Feature_subquery	12
            Feature_timezone	5
            Feature_trigger	210
            Feature_xml	0
            Flush_commands	1
            Handler_commit	10461
            Handler_delete	10
            Handler_discover	0
            Handler_external_lock	0
            Handler_icp_attempts	0
            Handler_icp_match	0
            Handler_mrr_init	0
            Handler_mrr_key_refills	0
            Handler_mrr_rowid_refills	0
            Handler_prepare	0
            Handler_read_first	5
            Handler_read_key	6579291
            Handler_read_last	0
            Handler_read_next	7970057
            Handler_read_prev	0
            Handler_read_retry	0
            Handler_read_rnd	0
            Handler_read_rnd_deleted	0
            Handler_read_rnd_next	807268
            Handler_rollback	0
            Handler_savepoint	0
            Handler_savepoint_rollback	0
            Handler_tmp_update	0
            Handler_tmp_write	18025
            Handler_update	0
            Handler_write	24673912
            Innodb_available_undo_logs	128
            Innodb_background_log_sync	92972
            Innodb_buffer_pool_bytes_data	103809024
            Innodb_buffer_pool_bytes_dirty	0
            Innodb_buffer_pool_dump_status	Dumping buffer pool(s) not yet started
            Innodb_buffer_pool_load_status	Loading buffer pool(s) not yet started
            Innodb_buffer_pool_pages_data	6336
            Innodb_buffer_pool_pages_dirty	0
            Innodb_buffer_pool_pages_flushed	811467
            Innodb_buffer_pool_pages_free	1024
            Innodb_buffer_pool_pages_lru_flushed	0
            Innodb_buffer_pool_pages_made_not_young	11840343
            Innodb_buffer_pool_pages_made_young	15865
            Innodb_buffer_pool_pages_misc	831
            Innodb_buffer_pool_pages_old	2318
            Innodb_buffer_pool_pages_total	8191
            Innodb_buffer_pool_read_ahead	198337
            Innodb_buffer_pool_read_ahead_evicted	395
            Innodb_buffer_pool_read_ahead_rnd	0
            Innodb_buffer_pool_read_requests	228086694
            Innodb_buffer_pool_reads	169377
            Innodb_buffer_pool_wait_free	0
            Innodb_buffer_pool_write_requests	88089440
            Innodb_checkpoint_age	0
            Innodb_checkpoint_max_age	32849880
            Innodb_data_fsyncs	41692
            Innodb_data_pending_fsyncs	0
            Innodb_data_pending_reads	0
            Innodb_data_pending_writes	0
            Innodb_data_read	6056645120
            Innodb_data_reads	369806
            Innodb_data_writes	873397
            Innodb_data_written	37874769408
            Innodb_dblwr_pages_written	811467
            Innodb_dblwr_writes	8730
            Innodb_deadlocks	0
            Innodb_have_atomic_builtins	ON
            Innodb_history_list_length	56
            Innodb_ibuf_discarded_delete_marks	0
            Innodb_ibuf_discarded_deletes	0
            Innodb_ibuf_discarded_inserts	0
            Innodb_ibuf_free_list	717
            Innodb_ibuf_merged_delete_marks	0
            Innodb_ibuf_merged_deletes	0
            Innodb_ibuf_merged_inserts	28530
            Innodb_ibuf_merges	2075
            Innodb_ibuf_segment_size	719
            Innodb_ibuf_size	1
            Innodb_log_waits	73
            Innodb_log_write_requests	25277620
            Innodb_log_writes	29807
            Innodb_lsn_current	306283005919
            Innodb_lsn_flushed	306283005919
            Innodb_lsn_last_checkpoint	306283005919
            Innodb_master_thread_active_loops	816
            Innodb_master_thread_idle_loops	92174
            Innodb_max_trx_id	13477470
            Innodb_mem_adaptive_hash	15832672
            Innodb_mem_dictionary	972528
            Innodb_mem_total	140574720
            Innodb_mutex_os_waits	12372
            Innodb_mutex_spin_rounds	408545
            Innodb_mutex_spin_waits	41928
            Innodb_oldest_view_low_limit_trx_id	0
            Innodb_os_log_fsyncs	7712
            Innodb_os_log_pending_fsyncs	0
            Innodb_os_log_pending_writes	0
            Innodb_os_log_written	11284195840
            Innodb_page_size	16384
            Innodb_pages_created	692551
            Innodb_pages_read	369663
            Innodb_pages0_read	132
            Innodb_pages_written	811467
            Innodb_purge_trx_id	13477395
            Innodb_purge_undo_no	0
            Innodb_read_views_memory	104
            Innodb_row_lock_current_waits	0
            Innodb_row_lock_time	0
            Innodb_row_lock_time_avg	0
            Innodb_row_lock_time_max	0
            Innodb_row_lock_waits	0
            Innodb_rows_deleted	0
            Innodb_rows_inserted	24597562
            Innodb_rows_read	9704240
            Innodb_rows_updated	0
            Innodb_system_rows_deleted	0
            Innodb_system_rows_inserted	0
            Innodb_system_rows_read	0
            Innodb_system_rows_updated	0
            Innodb_s_lock_os_waits	658
            Innodb_s_lock_spin_rounds	31759
            Innodb_s_lock_spin_waits	6659
            Innodb_truncated_status_writes	0
            Innodb_x_lock_os_waits	3221
            Innodb_x_lock_spin_rounds	128098
            Innodb_x_lock_spin_waits	10750
            Innodb_page_compression_saved	0
            Innodb_page_compression_trim_sect512	0
            Innodb_page_compression_trim_sect1024	0
            Innodb_page_compression_trim_sect2048	0
            Innodb_page_compression_trim_sect4096	0
            Innodb_page_compression_trim_sect8192	0
            Innodb_page_compression_trim_sect16384	0
            Innodb_page_compression_trim_sect32768	0
            Innodb_num_index_pages_written	622214
            Innodb_num_non_index_pages_written	236593
            Innodb_num_pages_page_compressed	0
            Innodb_num_page_compressed_trim_op	0
            Innodb_num_page_compressed_trim_op_saved	0
            Innodb_num_pages_page_decompressed	0
            Innodb_num_pages_page_compression_error	0
            Innodb_num_pages_encrypted	0
            Innodb_num_pages_decrypted	0
            Innodb_have_lz4	OFF
            Innodb_have_lzo	OFF
            Innodb_have_lzma	OFF
            Innodb_have_bzip2	OFF
            Innodb_have_snappy	OFF
            Innodb_defragment_compression_failures	0
            Innodb_defragment_failures	0
            Innodb_defragment_count	0
            Innodb_onlineddl_rowlog_rows	0
            Innodb_onlineddl_rowlog_pct_used	0
            Innodb_onlineddl_pct_progress	0
            Innodb_secondary_index_triggered_cluster_reads	2548316
            Innodb_secondary_index_triggered_cluster_reads_avoided	0
            Innodb_buffered_aio_submitted	198337
            Innodb_encryption_rotation_pages_read_from_cache	0
            Innodb_encryption_rotation_pages_read_from_disk	0
            Innodb_encryption_rotation_pages_modified	0
            Innodb_encryption_rotation_pages_flushed	0
            Innodb_encryption_rotation_estimated_iops	0
            Innodb_encryption_key_rotation_list_length	0
            Innodb_encryption_n_merge_blocks_encrypted	0
            Innodb_encryption_n_merge_blocks_decrypted	0
            Innodb_encryption_n_rowlog_blocks_encrypted	0
            Innodb_encryption_n_rowlog_blocks_decrypted	0
            Innodb_scrub_background_page_reorganizations	0
            Innodb_scrub_background_page_splits	0
            Innodb_scrub_background_page_split_failures_underflow	0
            Innodb_scrub_background_page_split_failures_out_of_filespace	0
            Innodb_scrub_background_page_split_failures_missing_index	0
            Innodb_scrub_background_page_split_failures_unknown	0
            Innodb_encryption_num_key_requests	0
            Key_blocks_not_flushed	0
            Key_blocks_unused	6330
            Key_blocks_used	1231
            Key_blocks_warm	81
            Key_read_requests	19180717
            Key_reads	366
            Key_write_requests	81645
            Key_writes	1965
            Last_query_cost	0.000000
            Master_gtid_wait_count	0
            Master_gtid_wait_time	0
            Master_gtid_wait_timeouts	0
            Max_statement_time_exceeded	0
            Max_used_connections	2
            Memory_used	243120432
            Not_flushed_delayed_rows	0
            Open_files	443
            Open_streams	0
            Open_table_definitions	370
            Open_tables	350
            Opened_files	5982
            Opened_plugin_libraries	0
            Opened_table_definitions	1465
            Opened_tables	1535
            Opened_views	0
            Performance_schema_accounts_lost	0
            Performance_schema_cond_classes_lost	0
            Performance_schema_cond_instances_lost	0
            Performance_schema_digest_lost	0
            Performance_schema_file_classes_lost	0
            Performance_schema_file_handles_lost	0
            Performance_schema_file_instances_lost	0
            Performance_schema_hosts_lost	0
            Performance_schema_locker_lost	0
            Performance_schema_mutex_classes_lost	0
            Performance_schema_mutex_instances_lost	0
            Performance_schema_rwlock_classes_lost	0
            Performance_schema_rwlock_instances_lost	0
            Performance_schema_session_connect_attrs_lost	0
            Performance_schema_socket_classes_lost	0
            Performance_schema_socket_instances_lost	0
            Performance_schema_stage_classes_lost	0
            Performance_schema_statement_classes_lost	0
            Performance_schema_table_handles_lost	0
            Performance_schema_table_instances_lost	0
            Performance_schema_thread_classes_lost	0
            Performance_schema_thread_instances_lost	0
            Performance_schema_users_lost	0
            Prepared_stmt_count	0
            Qcache_free_blocks	1
            Qcache_free_memory	33536824
            Qcache_hits	0
            Qcache_inserts	0
            Qcache_lowmem_prunes	0
            Qcache_not_cached	274
            Qcache_queries_in_cache	0
            Qcache_total_blocks	1
            Queries	19036
            Questions	19022
            Rows_read	11721162
            Rows_sent	3402308
            Rows_tmp_read	15272
            Rpl_status	AUTH_MASTER
            Select_full_join	0
            Select_full_range_join	0
            Select_range	0
            Select_range_check	0
            Select_scan	52
            Slave_connections	0
            Slave_heartbeat_period	1800.000
            Slave_open_temp_tables	0
            Slave_received_heartbeats	0
            Slave_retried_transactions	0
            Slave_running	ON
            Slave_skipped_errors	0
            Slaves_connected	0
            Slaves_running	1
            Slow_launch_threads	0
            Slow_queries	4
            Sort_merge_passes	0
            Sort_priority_queue_sorts	0
            Sort_range	0
            Sort_rows	0
            Sort_scan	0
            Ssl_accept_renegotiates	0
            Ssl_accepts	0
            Ssl_callback_cache_hits	0
            Ssl_cipher	
            Ssl_cipher_list	
            Ssl_client_connects	0
            Ssl_connect_renegotiates	0
            Ssl_ctx_verify_depth	0
            Ssl_ctx_verify_mode	0
            Ssl_default_timeout	0
            Ssl_finished_accepts	0
            Ssl_finished_connects	0
            Ssl_server_not_after	
            Ssl_server_not_before	
            Ssl_session_cache_hits	0
            Ssl_session_cache_misses	0
            Ssl_session_cache_mode	NONE
            Ssl_session_cache_overflows	0
            Ssl_session_cache_size	0
            Ssl_session_cache_timeouts	0
            Ssl_sessions_reused	0
            Ssl_used_session_cache_entries	0
            Ssl_verify_depth	0
            Ssl_verify_mode	0
            Ssl_version	
            Subquery_cache_hit	15158
            Subquery_cache_miss	12054
            Syncs	3058
            Table_locks_immediate	3249
            Table_locks_waited	0
            Tc_log_max_pages_used	0
            Tc_log_page_size	4096
            Tc_log_page_waits	0
            Threadpool_idle_threads	0
            Threadpool_threads	0
            Threads_cached	1
            Threads_connected	1
            Threads_created	2
            Threads_running	1
            Update_scan	0
            Uptime	93089
            Uptime_since_flush_status	93089
            wsrep_cluster_conf_id	18446744073709551615
            wsrep_cluster_size	0
            wsrep_cluster_state_uuid	
            wsrep_cluster_status	Disconnected
            wsrep_connected	OFF
            wsrep_local_bf_aborts	0
            wsrep_local_index	18446744073709551615
            wsrep_provider_name	
            wsrep_provider_vendor	
            wsrep_provider_version	
            wsrep_ready	OFF
            wsrep_thread_count	0
            

            mperez Miguel P added a comment - Hi Andrei Elkin , thanks! The hang happens right before an event for a Replicate_Ignore_Table in this example, but it also happened in another table before I had this table ignored (I've reconfigured replication from a fresh dump a few times, always hanged at either update or delete with find_rows(-1)). I'm ignoring the `md_session` table because I don't need it and because it's MyISAM, but it didn't help. We have nothing doing FLUSH TABLES WITH READ LOCK, nor any other threads or clients running when it hangs. (Com_flush is 4 because I did a few FLUSH TABLES during some tests prior to setting up replication.) Doesn't seem to be an instance of MDEV-20247 as there the SQL thread seems to be done ("Slave has read all relay log; waiting for the slave I/O thread to update it") with the IO thread stuck. Here's the SHOW CREATE TABLE: CREATE TABLE `ContactCredential` ( `Contact` int(10) unsigned NOT NULL, `Branding` varchar(22) NOT NULL, `Password` binary(20) DEFAULT NULL COMMENT '...', `OldPassword` binary(20) DEFAULT NULL COMMENT '...', `FailedLoginCount` tinyint(4) NOT NULL DEFAULT '0' COMMENT '...', PRIMARY KEY (`Contact`,`Branding`), CONSTRAINT `ContactCredential_ibfk_1` FOREIGN KEY (`Contact`) REFERENCES `Contact` (`ID`) ON DELETE CASCADE ON UPDATE CASCADE ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 SHOW GLOBAL VARIABLES: Variable_name Value aria_block_size 8192 aria_checkpoint_interval 30 aria_checkpoint_log_activity 1048576 aria_encrypt_tables OFF aria_force_start_after_recovery_failures 0 aria_group_commit none aria_group_commit_interval 0 aria_log_file_size 1073741824 aria_log_purge_type immediate aria_max_sort_file_size 9223372036853727232 aria_page_checksum ON aria_pagecache_age_threshold 300 aria_pagecache_buffer_size 134217728 aria_pagecache_division_limit 100 aria_pagecache_file_hash_size 512 aria_recover NORMAL aria_repair_threads 1 aria_sort_buffer_size 268434432 aria_stats_method nulls_unequal aria_sync_log_dir NEWFILE aria_used_for_temp_tables ON auto_increment_increment 1 auto_increment_offset 1 autocommit ON automatic_sp_privileges ON back_log 52 basedir /usr big_tables OFF binlog_annotate_row_events OFF binlog_cache_size 65536 binlog_checksum NONE binlog_commit_wait_count 0 binlog_commit_wait_usec 100000 binlog_direct_non_transactional_updates OFF binlog_format MIXED binlog_optimize_thread_scheduling ON binlog_row_image FULL binlog_stmt_cache_size 32768 bulk_insert_buffer_size 8388608 character_set_client utf8mb4 character_set_connection utf8mb4 character_set_database utf8mb4 character_set_filesystem binary character_set_results utf8mb4 character_set_server utf8mb4 character_set_system utf8 character_sets_dir /usr/share/mysql/charsets/ collation_connection utf8mb4_general_ci collation_database utf8mb4_general_ci collation_server utf8mb4_general_ci completion_type NO_CHAIN concurrent_insert AUTO connect_timeout 10 core_file OFF datadir /data/mysql/ date_format %Y-%m-%d datetime_format %Y-%m-%d %H:%i:%s deadlock_search_depth_long 15 deadlock_search_depth_short 4 deadlock_timeout_long 50000000 deadlock_timeout_short 10000 debug_no_thread_alarm OFF default_regex_flags default_storage_engine InnoDB default_tmp_storage_engine default_week_format 0 delay_key_write ON delayed_insert_limit 100 delayed_insert_timeout 300 delayed_queue_size 1000 div_precision_increment 4 encrypt_binlog OFF encrypt_tmp_disk_tables OFF encrypt_tmp_files OFF enforce_storage_engine event_scheduler DISABLED expensive_subquery_limit 100 expire_logs_days 15 explicit_defaults_for_timestamp OFF extra_max_connections 1 extra_port 0 flush OFF flush_time 0 foreign_key_checks ON ft_boolean_syntax + -><()~*:""&| ft_max_word_len 20 ft_min_word_len 3 ft_query_expansion_limit 20 ft_stopword_file (built-in) general_log OFF general_log_file /data/mysql-log/mysql.log group_concat_max_len 16384 gtid_binlog_pos gtid_binlog_state gtid_current_pos 0-1-139164901 gtid_domain_id 0 gtid_ignore_duplicates OFF gtid_slave_pos 0-1-139164901 gtid_strict_mode OFF have_compress YES have_crypt YES have_dynamic_loading YES have_geometry YES have_openssl NO have_profiling YES have_query_cache YES have_rtree_keys YES have_ssl DISABLED have_symlink YES histogram_size 0 histogram_type SINGLE_PREC_HB host_cache_size 138 hostname SssssSlave ignore_builtin_innodb OFF ignore_db_dirs init_connect init_file init_slave innodb_adaptive_flushing ON innodb_adaptive_flushing_lwm 10.000000 innodb_adaptive_hash_index ON innodb_adaptive_hash_index_partitions 1 innodb_adaptive_max_sleep_delay 150000 innodb_additional_mem_pool_size 2097152 innodb_api_bk_commit_interval 5 innodb_api_disable_rowlock OFF innodb_api_enable_binlog OFF innodb_api_enable_mdl OFF innodb_api_trx_level 0 innodb_autoextend_increment 256 innodb_autoinc_lock_mode 1 innodb_background_scrub_data_check_interval 3600 innodb_background_scrub_data_compressed OFF innodb_background_scrub_data_interval 604800 innodb_background_scrub_data_uncompressed OFF innodb_buf_dump_status_frequency 0 innodb_buffer_pool_dump_at_shutdown OFF innodb_buffer_pool_dump_now OFF innodb_buffer_pool_dump_pct 100 innodb_buffer_pool_filename ib_buffer_pool innodb_buffer_pool_instances 8 innodb_buffer_pool_load_abort OFF innodb_buffer_pool_load_at_startup OFF innodb_buffer_pool_load_now OFF innodb_buffer_pool_populate OFF innodb_buffer_pool_size 134217728 innodb_change_buffer_max_size 25 innodb_change_buffering all innodb_checksum_algorithm INNODB innodb_checksums ON innodb_cleaner_lsn_age_factor HIGH_CHECKPOINT innodb_cmp_per_index_enabled OFF innodb_commit_concurrency 0 innodb_compression_algorithm zlib innodb_compression_failure_threshold_pct 5 innodb_compression_level 6 innodb_compression_pad_pct_max 50 innodb_concurrency_tickets 5000 innodb_corrupt_table_action assert innodb_data_file_path ibdata1:4096M:autoextend innodb_data_home_dir innodb_default_encryption_key_id 1 innodb_default_row_format compact innodb_defragment OFF innodb_defragment_fill_factor 0.900000 innodb_defragment_fill_factor_n_recs 20 innodb_defragment_frequency 40 innodb_defragment_n_pages 7 innodb_defragment_stats_accuracy 0 innodb_disable_sort_file_cache OFF innodb_disallow_writes OFF innodb_doublewrite ON innodb_empty_free_list_algorithm LEGACY innodb_encrypt_log OFF innodb_encrypt_tables OFF innodb_encryption_rotate_key_age 1 innodb_encryption_rotation_iops 100 innodb_encryption_threads 0 innodb_fake_changes OFF innodb_fast_shutdown 1 innodb_fatal_semaphore_wait_threshold 600 innodb_file_format Antelope innodb_file_format_check ON innodb_file_format_max Antelope innodb_file_per_table ON innodb_flush_log_at_timeout 1 innodb_flush_log_at_trx_commit 0 innodb_flush_method innodb_flush_neighbors 1 innodb_flushing_avg_loops 30 innodb_force_load_corrupted OFF innodb_force_primary_key OFF innodb_force_recovery 0 innodb_foreground_preflush EXPONENTIAL_BACKOFF innodb_ft_aux_table innodb_ft_cache_size 8000000 innodb_ft_enable_diag_print OFF innodb_ft_enable_stopword ON innodb_ft_max_token_size 84 innodb_ft_min_token_size 3 innodb_ft_num_word_optimize 2000 innodb_ft_result_cache_limit 2000000000 innodb_ft_server_stopword_table innodb_ft_sort_pll_degree 2 innodb_ft_total_cache_size 640000000 innodb_ft_user_stopword_table innodb_idle_flush_pct 100 innodb_immediate_scrub_data_uncompressed OFF innodb_instrument_semaphores OFF innodb_io_capacity 200 innodb_io_capacity_max 2000 innodb_kill_idle_transaction 0 innodb_large_prefix OFF innodb_lock_schedule_algorithm fcfs innodb_lock_wait_timeout 50 innodb_locking_fake_changes ON innodb_locks_unsafe_for_binlog OFF innodb_log_arch_dir ./ innodb_log_arch_expire_sec 0 innodb_log_archive OFF innodb_log_block_size 512 innodb_log_buffer_size 1048576 innodb_log_checksum_algorithm INNODB innodb_log_compressed_pages ON innodb_log_file_size 20971520 innodb_log_files_in_group 2 innodb_log_group_home_dir ./ innodb_lru_scan_depth 1024 innodb_max_bitmap_file_size 104857600 innodb_max_changed_pages 1000000 innodb_max_dirty_pages_pct 75.000000 innodb_max_dirty_pages_pct_lwm 0.001000 innodb_max_purge_lag 0 innodb_max_purge_lag_delay 0 innodb_mirrored_log_groups 1 innodb_monitor_disable innodb_monitor_enable innodb_monitor_reset innodb_monitor_reset_all innodb_mtflush_threads 8 innodb_old_blocks_pct 37 innodb_old_blocks_time 1000 innodb_online_alter_log_max_size 134217728 innodb_open_files 300 innodb_optimize_fulltext_only OFF innodb_page_size 16384 innodb_prefix_index_cluster_optimization OFF innodb_print_all_deadlocks OFF innodb_print_lock_wait_timeout_info OFF innodb_purge_batch_size 300 innodb_purge_threads 1 innodb_random_read_ahead OFF innodb_read_ahead_threshold 56 innodb_read_io_threads 4 innodb_read_only OFF innodb_replication_delay 0 innodb_rollback_on_timeout OFF innodb_rollback_segments 128 innodb_sched_priority_cleaner 19 innodb_scrub_log OFF innodb_scrub_log_speed 256 innodb_show_locks_held 10 innodb_show_verbose_locks 0 innodb_simulate_comp_failures 0 innodb_sort_buffer_size 1048576 innodb_spin_wait_delay 6 innodb_stats_auto_recalc ON innodb_stats_include_delete_marked OFF innodb_stats_method nulls_equal innodb_stats_modified_counter 0 innodb_stats_on_metadata OFF innodb_stats_persistent ON innodb_stats_persistent_sample_pages 20 innodb_stats_sample_pages 8 innodb_stats_traditional ON innodb_stats_transient_sample_pages 8 innodb_status_output OFF innodb_status_output_locks OFF innodb_strict_mode OFF innodb_support_xa ON innodb_sync_array_size 1 innodb_sync_spin_loops 30 innodb_table_locks ON innodb_thread_concurrency 7 innodb_thread_sleep_delay 0 innodb_tmpdir innodb_track_changed_pages OFF innodb_undo_directory . innodb_undo_logs 128 innodb_undo_tablespaces 0 innodb_use_atomic_writes OFF innodb_use_fallocate OFF innodb_use_global_flush_log_at_trx_commit ON innodb_use_mtflush OFF innodb_use_native_aio ON innodb_use_stacktrace OFF innodb_use_sys_malloc ON innodb_use_trim OFF innodb_version 5.6.44-86.0 innodb_write_io_threads 4 interactive_timeout 28800 join_buffer_size 524288 join_buffer_space_limit 2097152 join_cache_level 2 keep_files_on_create OFF key_buffer_size 8388608 key_cache_age_threshold 300 key_cache_block_size 1024 key_cache_division_limit 100 key_cache_file_hash_size 512 key_cache_segments 0 large_files_support ON large_page_size 0 large_pages OFF lc_messages en_US lc_messages_dir /usr/share/mysql lc_time_names en_US license GPL local_infile ON lock_wait_timeout 31536000 locked_in_memory OFF log_bin OFF log_bin_basename log_bin_index log_bin_trust_function_creators OFF log_error /data/mysql-log/mysql.err log_output FILE log_queries_not_using_indexes OFF log_slave_updates OFF log_slow_admin_statements OFF log_slow_filter admin,filesort,filesort_on_disk,full_join,full_scan,query_cache,query_cache_miss,tmp_table,tmp_table_on_disk log_slow_rate_limit 1 log_slow_slave_statements OFF log_slow_verbosity log_tc_size 24576 log_warnings 2 long_query_time 10.000000 low_priority_updates OFF lower_case_file_system OFF lower_case_table_names 0 master_verify_checksum OFF max_allowed_packet 4194304 max_binlog_cache_size 18446744073709547520 max_binlog_size 8388608 max_binlog_stmt_cache_size 18446744073709547520 max_connect_errors 8 max_connections 10 max_delayed_threads 20 max_digest_length 1024 max_error_count 64 max_heap_table_size 33554432 max_insert_delayed_threads 20 max_join_size 18446744073709551615 max_length_for_sort_data 1024 max_long_data_size 4194304 max_prepared_stmt_count 16382 max_relay_log_size 8388608 max_seeks_for_key 4294967295 max_session_mem_used 9223372036854775807 max_sort_length 256 max_sp_recursion_depth 8 max_statement_time 0.000000 max_tmp_tables 32 max_user_connections 19 max_write_lock_count 4294967295 metadata_locks_cache_size 1024 metadata_locks_hash_instances 8 min_examined_row_limit 0 mrr_buffer_size 262144 multi_range_count 256 myisam_block_size 1024 myisam_data_pointer_size 6 myisam_max_sort_file_size 2147483648 myisam_mmap_size 18446744073709551615 myisam_recover_options BACKUP myisam_repair_threads 1 myisam_sort_buffer_size 16777216 myisam_stats_method NULLS_UNEQUAL myisam_use_mmap OFF mysql56_temporal_format ON net_buffer_length 16384 net_read_timeout 30 net_retry_count 10 net_write_timeout 60 old OFF old_alter_table OFF old_mode old_passwords OFF open_files_limit 743 optimizer_prune_level 1 optimizer_search_depth 62 optimizer_selectivity_sampling_limit 100 optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=off optimizer_use_condition_selectivity 1 performance_schema OFF performance_schema_accounts_size -1 performance_schema_digests_size -1 performance_schema_events_stages_history_long_size -1 performance_schema_events_stages_history_size -1 performance_schema_events_statements_history_long_size -1 performance_schema_events_statements_history_size -1 performance_schema_events_waits_history_long_size -1 performance_schema_events_waits_history_size -1 performance_schema_hosts_size -1 performance_schema_max_cond_classes 80 performance_schema_max_cond_instances -1 performance_schema_max_digest_length 1024 performance_schema_max_file_classes 50 performance_schema_max_file_handles 32768 performance_schema_max_file_instances -1 performance_schema_max_mutex_classes 200 performance_schema_max_mutex_instances -1 performance_schema_max_rwlock_classes 40 performance_schema_max_rwlock_instances -1 performance_schema_max_socket_classes 10 performance_schema_max_socket_instances -1 performance_schema_max_stage_classes 150 performance_schema_max_statement_classes 178 performance_schema_max_table_handles -1 performance_schema_max_table_instances -1 performance_schema_max_thread_classes 50 performance_schema_max_thread_instances -1 performance_schema_session_connect_attrs_size -1 performance_schema_setup_actors_size 100 performance_schema_setup_objects_size 100 performance_schema_users_size -1 pid_file /var/run/mysqld/mysqld.pid plugin_dir /usr/lib/x86_64-linux-gnu/mariadb18/plugin/ plugin_maturity unknown port 3306 preload_buffer_size 32768 profiling OFF profiling_history_size 15 progress_report_time 5 protocol_version 10 query_alloc_block_size 16384 query_cache_limit 262144 query_cache_min_res_unit 4096 query_cache_size 33554432 query_cache_strip_comments OFF query_cache_type ON query_cache_wlock_invalidate ON query_prealloc_size 24576 range_alloc_block_size 4096 read_buffer_size 786432 read_only OFF read_rnd_buffer_size 2097152 relay_log /data/mysql-log/relay relay_log_basename /data/mysql-log/relay relay_log_index /data/mysql-log/relay.index relay_log_info_file /data/mysql-log/relay.info relay_log_purge ON relay_log_recovery OFF relay_log_space_limit 0 replicate_annotate_row_events OFF replicate_do_db replicate_do_table replicate_events_marked_for_skip REPLICATE replicate_ignore_db replicate_ignore_table sssss.md_session replicate_wild_do_table replicate_wild_ignore_table report_host report_password report_port 3306 report_user rowid_merge_buff_size 8388608 secure_auth ON secure_file_priv server_id 2 skip_external_locking ON skip_name_resolve ON skip_networking OFF skip_show_database OFF slave_compressed_protocol OFF slave_ddl_exec_mode IDEMPOTENT slave_domain_parallel_threads 0 slave_exec_mode STRICT slave_load_tmpdir /tmp slave_max_allowed_packet 1073741824 slave_net_timeout 3600 slave_parallel_max_queued 131072 slave_parallel_mode conservative slave_parallel_threads 0 slave_run_triggers_for_rbr NO slave_skip_errors OFF slave_sql_verify_checksum ON slave_transaction_retries 10 slave_type_conversions slow_launch_time 2 slow_query_log ON slow_query_log_file /data/mysql-log/mysql-slow.log socket /var/run/mysqld/mysqld.sock sort_buffer_size 786432 sql_auto_is_null OFF sql_big_selects ON sql_buffer_result OFF sql_log_bin ON sql_log_off OFF sql_mode PIPES_AS_CONCAT,STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION sql_notes ON sql_quote_show_create ON sql_safe_updates OFF sql_select_limit 18446744073709551615 sql_slave_skip_counter 0 sql_warnings OFF ssl_ca ssl_capath ssl_cert ssl_cipher ssl_crl ssl_crlpath ssl_key storage_engine InnoDB stored_program_cache 256 strict_password_validation ON sync_binlog 0 sync_frm ON sync_master_info 10000 sync_relay_log 10000 sync_relay_log_info 10000 system_time_zone CET table_definition_cache 400 table_open_cache 350 thread_cache_size 4 thread_concurrency 3 thread_handling one-thread-per-connection thread_pool_idle_timeout 60 thread_pool_max_threads 1000 thread_pool_oversubscribe 3 thread_pool_size 2 thread_pool_stall_limit 500 thread_stack 196608 time_format %H:%i:%s time_zone SYSTEM timed_mutexes OFF tmp_table_size 12582912 tmpdir /tmp transaction_alloc_block_size 8192 transaction_prealloc_size 4096 tx_isolation READ-COMMITTED tx_read_only OFF unique_checks ON updatable_views_with_limit YES use_stat_tables NEVER userstat OFF version 10.1.41-MariaDB-0+deb9u1 version_comment Debian 9.9 version_compile_machine x86_64 version_compile_os debian-linux-gnu version_malloc_library system jemalloc version_ssl_library YaSSL 2.4.4 wait_timeout 28800 wsrep_osu_method TOI wsrep_auto_increment_control ON wsrep_causal_reads OFF wsrep_certification_rules strict wsrep_certify_nonpk ON wsrep_cluster_address wsrep_cluster_name my_wsrep_cluster wsrep_convert_lock_to_trx OFF wsrep_data_home_dir /data/mysql/ wsrep_dbug_option wsrep_debug OFF wsrep_desync OFF wsrep_dirty_reads OFF wsrep_drupal_282555_workaround OFF wsrep_forced_binlog_format NONE wsrep_gtid_domain_id 0 wsrep_gtid_mode OFF wsrep_load_data_splitting ON wsrep_log_conflicts OFF wsrep_max_ws_rows 0 wsrep_max_ws_size 2147483647 wsrep_mysql_replication_bundle 0 wsrep_node_address wsrep_node_incoming_address AUTO wsrep_node_name SssssSlave wsrep_notify_cmd wsrep_on OFF wsrep_patch_version wsrep_25.24 wsrep_provider none wsrep_provider_options wsrep_recover OFF wsrep_reject_queries NONE wsrep_replicate_myisam OFF wsrep_restart_slave OFF wsrep_retry_autocommit 1 wsrep_slave_fk_checks ON wsrep_slave_uk_checks OFF wsrep_slave_threads 1 wsrep_sst_auth wsrep_sst_donor wsrep_sst_donor_rejects_queries OFF wsrep_sst_method rsync wsrep_sst_receive_address AUTO wsrep_start_position 00000000-0000-0000-0000-000000000000:-1 wsrep_sync_wait 0 SHOW GLOBAL STATUS Variable_name Value Aborted_clients 0 Aborted_connects 0 Access_denied_errors 0 Acl_column_grants 0 Acl_database_grants 10 Acl_function_grants 0 Acl_procedure_grants 0 Acl_proxy_users 1 Acl_role_grants 0 Acl_roles 0 Acl_table_grants 6 Acl_users 11 Aria_pagecache_blocks_not_flushed 0 Aria_pagecache_blocks_unused 15706 Aria_pagecache_blocks_used 0 Aria_pagecache_read_requests 0 Aria_pagecache_reads 0 Aria_pagecache_write_requests 0 Aria_pagecache_writes 0 Aria_transaction_log_syncs 0 Binlog_commits 0 Binlog_group_commits 0 Binlog_group_commit_trigger_count 0 Binlog_group_commit_trigger_lock_wait 0 Binlog_group_commit_trigger_timeout 0 Binlog_snapshot_file Binlog_snapshot_position 0 Binlog_bytes_written 0 Binlog_cache_disk_use 0 Binlog_cache_use 0 Binlog_stmt_cache_disk_use 0 Binlog_stmt_cache_use 0 Busy_time 0.000000 Bytes_received 10712143359 Bytes_sent 3290678354 Com_admin_commands 2 Com_alter_db 70 Com_alter_db_upgrade 0 Com_alter_event 0 Com_alter_function 0 Com_alter_procedure 0 Com_alter_server 0 Com_alter_table 674 Com_alter_tablespace 0 Com_analyze 0 Com_assign_to_keycache 0 Com_begin 5 Com_binlog 0 Com_call_procedure 0 Com_change_db 2 Com_change_master 2 Com_check 0 Com_checksum 0 Com_commit 341 Com_compound_sql 0 Com_create_db 1 Com_create_event 0 Com_create_function 4 Com_create_index 0 Com_create_procedure 3 Com_create_role 0 Com_create_server 0 Com_create_table 337 Com_create_temporary_table 0 Com_create_trigger 86 Com_create_udf 0 Com_create_user 0 Com_create_view 0 Com_dealloc_sql 0 Com_delete 0 Com_delete_multi 0 Com_do 0 Com_drop_db 1 Com_drop_event 0 Com_drop_function 4 Com_drop_index 0 Com_drop_procedure 3 Com_drop_role 0 Com_drop_server 0 Com_drop_table 337 Com_drop_temporary_table 0 Com_drop_trigger 0 Com_drop_user 0 Com_drop_view 0 Com_empty_query 0 Com_execute_sql 0 Com_flush 4 Com_get_diagnostics 0 Com_grant 0 Com_grant_role 0 Com_ha_close 0 Com_ha_open 0 Com_ha_read 0 Com_help 0 Com_insert 9480 Com_insert_select 0 Com_install_plugin 0 Com_kill 0 Com_load 0 Com_lock_tables 337 Com_optimize 0 Com_preload_keys 0 Com_prepare_sql 0 Com_purge 0 Com_purge_before_date 0 Com_release_savepoint 0 Com_rename_table 0 Com_rename_user 0 Com_repair 0 Com_replace 0 Com_replace_select 0 Com_reset 0 Com_resignal 0 Com_revoke 0 Com_revoke_all 0 Com_revoke_role 0 Com_rollback 0 Com_rollback_to_savepoint 0 Com_savepoint 0 Com_select 274 Com_set_option 2587 Com_show_authors 0 Com_show_binlog_events 0 Com_show_binlogs 0 Com_show_charsets 0 Com_show_collations 0 Com_show_contributors 0 Com_show_create_db 0 Com_show_create_event 0 Com_show_create_func 0 Com_show_create_proc 0 Com_show_create_table 2 Com_show_create_trigger 0 Com_show_databases 12 Com_show_engine_logs 0 Com_show_engine_mutex 0 Com_show_engine_status 2 Com_show_errors 0 Com_show_events 0 Com_show_explain 0 Com_show_fields 3735 Com_show_function_status 0 Com_show_generic 0 Com_show_grants 0 Com_show_keys 0 Com_show_master_status 0 Com_show_open_tables 0 Com_show_plugins 0 Com_show_privileges 0 Com_show_procedure_status 0 Com_show_processlist 66 Com_show_profile 0 Com_show_profiles 0 Com_show_relaylog_events 0 Com_show_slave_hosts 0 Com_show_slave_status 22 Com_show_status 1 Com_show_storage_engines 0 Com_show_table_status 0 Com_show_tables 12 Com_show_triggers 0 Com_show_variables 4 Com_show_warnings 0 Com_shutdown 0 Com_signal 0 Com_start_all_slaves 0 Com_start_slave 1 Com_stmt_close 0 Com_stmt_execute 0 Com_stmt_fetch 0 Com_stmt_prepare 0 Com_stmt_reprepare 0 Com_stmt_reset 0 Com_stmt_send_long_data 0 Com_stop_all_slaves 0 Com_stop_slave 0 Com_truncate 0 Com_uninstall_plugin 0 Com_unlock_tables 337 Com_update 1 Com_update_multi 0 Com_xa_commit 0 Com_xa_end 0 Com_xa_prepare 0 Com_xa_recover 0 Com_xa_rollback 0 Com_xa_start 0 Compression OFF Connection_errors_accept 0 Connection_errors_internal 0 Connection_errors_max_connections 0 Connection_errors_peer_address 0 Connection_errors_select 0 Connection_errors_tcpwrap 0 Connections 285 Cpu_time 0.000000 Created_tmp_disk_tables 2 Created_tmp_files 6 Created_tmp_tables 42 Delayed_errors 0 Delayed_insert_threads 0 Delayed_writes 0 Delete_scan 0 Empty_queries 238 Executed_events 0 Executed_triggers 0 Feature_delay_key_write 0 Feature_dynamic_columns 0 Feature_fulltext 0 Feature_gis 0 Feature_locale 0 Feature_subquery 12 Feature_timezone 5 Feature_trigger 210 Feature_xml 0 Flush_commands 1 Handler_commit 10461 Handler_delete 10 Handler_discover 0 Handler_external_lock 0 Handler_icp_attempts 0 Handler_icp_match 0 Handler_mrr_init 0 Handler_mrr_key_refills 0 Handler_mrr_rowid_refills 0 Handler_prepare 0 Handler_read_first 5 Handler_read_key 6579291 Handler_read_last 0 Handler_read_next 7970057 Handler_read_prev 0 Handler_read_retry 0 Handler_read_rnd 0 Handler_read_rnd_deleted 0 Handler_read_rnd_next 807268 Handler_rollback 0 Handler_savepoint 0 Handler_savepoint_rollback 0 Handler_tmp_update 0 Handler_tmp_write 18025 Handler_update 0 Handler_write 24673912 Innodb_available_undo_logs 128 Innodb_background_log_sync 92972 Innodb_buffer_pool_bytes_data 103809024 Innodb_buffer_pool_bytes_dirty 0 Innodb_buffer_pool_dump_status Dumping buffer pool(s) not yet started Innodb_buffer_pool_load_status Loading buffer pool(s) not yet started Innodb_buffer_pool_pages_data 6336 Innodb_buffer_pool_pages_dirty 0 Innodb_buffer_pool_pages_flushed 811467 Innodb_buffer_pool_pages_free 1024 Innodb_buffer_pool_pages_lru_flushed 0 Innodb_buffer_pool_pages_made_not_young 11840343 Innodb_buffer_pool_pages_made_young 15865 Innodb_buffer_pool_pages_misc 831 Innodb_buffer_pool_pages_old 2318 Innodb_buffer_pool_pages_total 8191 Innodb_buffer_pool_read_ahead 198337 Innodb_buffer_pool_read_ahead_evicted 395 Innodb_buffer_pool_read_ahead_rnd 0 Innodb_buffer_pool_read_requests 228086694 Innodb_buffer_pool_reads 169377 Innodb_buffer_pool_wait_free 0 Innodb_buffer_pool_write_requests 88089440 Innodb_checkpoint_age 0 Innodb_checkpoint_max_age 32849880 Innodb_data_fsyncs 41692 Innodb_data_pending_fsyncs 0 Innodb_data_pending_reads 0 Innodb_data_pending_writes 0 Innodb_data_read 6056645120 Innodb_data_reads 369806 Innodb_data_writes 873397 Innodb_data_written 37874769408 Innodb_dblwr_pages_written 811467 Innodb_dblwr_writes 8730 Innodb_deadlocks 0 Innodb_have_atomic_builtins ON Innodb_history_list_length 56 Innodb_ibuf_discarded_delete_marks 0 Innodb_ibuf_discarded_deletes 0 Innodb_ibuf_discarded_inserts 0 Innodb_ibuf_free_list 717 Innodb_ibuf_merged_delete_marks 0 Innodb_ibuf_merged_deletes 0 Innodb_ibuf_merged_inserts 28530 Innodb_ibuf_merges 2075 Innodb_ibuf_segment_size 719 Innodb_ibuf_size 1 Innodb_log_waits 73 Innodb_log_write_requests 25277620 Innodb_log_writes 29807 Innodb_lsn_current 306283005919 Innodb_lsn_flushed 306283005919 Innodb_lsn_last_checkpoint 306283005919 Innodb_master_thread_active_loops 816 Innodb_master_thread_idle_loops 92174 Innodb_max_trx_id 13477470 Innodb_mem_adaptive_hash 15832672 Innodb_mem_dictionary 972528 Innodb_mem_total 140574720 Innodb_mutex_os_waits 12372 Innodb_mutex_spin_rounds 408545 Innodb_mutex_spin_waits 41928 Innodb_oldest_view_low_limit_trx_id 0 Innodb_os_log_fsyncs 7712 Innodb_os_log_pending_fsyncs 0 Innodb_os_log_pending_writes 0 Innodb_os_log_written 11284195840 Innodb_page_size 16384 Innodb_pages_created 692551 Innodb_pages_read 369663 Innodb_pages0_read 132 Innodb_pages_written 811467 Innodb_purge_trx_id 13477395 Innodb_purge_undo_no 0 Innodb_read_views_memory 104 Innodb_row_lock_current_waits 0 Innodb_row_lock_time 0 Innodb_row_lock_time_avg 0 Innodb_row_lock_time_max 0 Innodb_row_lock_waits 0 Innodb_rows_deleted 0 Innodb_rows_inserted 24597562 Innodb_rows_read 9704240 Innodb_rows_updated 0 Innodb_system_rows_deleted 0 Innodb_system_rows_inserted 0 Innodb_system_rows_read 0 Innodb_system_rows_updated 0 Innodb_s_lock_os_waits 658 Innodb_s_lock_spin_rounds 31759 Innodb_s_lock_spin_waits 6659 Innodb_truncated_status_writes 0 Innodb_x_lock_os_waits 3221 Innodb_x_lock_spin_rounds 128098 Innodb_x_lock_spin_waits 10750 Innodb_page_compression_saved 0 Innodb_page_compression_trim_sect512 0 Innodb_page_compression_trim_sect1024 0 Innodb_page_compression_trim_sect2048 0 Innodb_page_compression_trim_sect4096 0 Innodb_page_compression_trim_sect8192 0 Innodb_page_compression_trim_sect16384 0 Innodb_page_compression_trim_sect32768 0 Innodb_num_index_pages_written 622214 Innodb_num_non_index_pages_written 236593 Innodb_num_pages_page_compressed 0 Innodb_num_page_compressed_trim_op 0 Innodb_num_page_compressed_trim_op_saved 0 Innodb_num_pages_page_decompressed 0 Innodb_num_pages_page_compression_error 0 Innodb_num_pages_encrypted 0 Innodb_num_pages_decrypted 0 Innodb_have_lz4 OFF Innodb_have_lzo OFF Innodb_have_lzma OFF Innodb_have_bzip2 OFF Innodb_have_snappy OFF Innodb_defragment_compression_failures 0 Innodb_defragment_failures 0 Innodb_defragment_count 0 Innodb_onlineddl_rowlog_rows 0 Innodb_onlineddl_rowlog_pct_used 0 Innodb_onlineddl_pct_progress 0 Innodb_secondary_index_triggered_cluster_reads 2548316 Innodb_secondary_index_triggered_cluster_reads_avoided 0 Innodb_buffered_aio_submitted 198337 Innodb_encryption_rotation_pages_read_from_cache 0 Innodb_encryption_rotation_pages_read_from_disk 0 Innodb_encryption_rotation_pages_modified 0 Innodb_encryption_rotation_pages_flushed 0 Innodb_encryption_rotation_estimated_iops 0 Innodb_encryption_key_rotation_list_length 0 Innodb_encryption_n_merge_blocks_encrypted 0 Innodb_encryption_n_merge_blocks_decrypted 0 Innodb_encryption_n_rowlog_blocks_encrypted 0 Innodb_encryption_n_rowlog_blocks_decrypted 0 Innodb_scrub_background_page_reorganizations 0 Innodb_scrub_background_page_splits 0 Innodb_scrub_background_page_split_failures_underflow 0 Innodb_scrub_background_page_split_failures_out_of_filespace 0 Innodb_scrub_background_page_split_failures_missing_index 0 Innodb_scrub_background_page_split_failures_unknown 0 Innodb_encryption_num_key_requests 0 Key_blocks_not_flushed 0 Key_blocks_unused 6330 Key_blocks_used 1231 Key_blocks_warm 81 Key_read_requests 19180717 Key_reads 366 Key_write_requests 81645 Key_writes 1965 Last_query_cost 0.000000 Master_gtid_wait_count 0 Master_gtid_wait_time 0 Master_gtid_wait_timeouts 0 Max_statement_time_exceeded 0 Max_used_connections 2 Memory_used 243120432 Not_flushed_delayed_rows 0 Open_files 443 Open_streams 0 Open_table_definitions 370 Open_tables 350 Opened_files 5982 Opened_plugin_libraries 0 Opened_table_definitions 1465 Opened_tables 1535 Opened_views 0 Performance_schema_accounts_lost 0 Performance_schema_cond_classes_lost 0 Performance_schema_cond_instances_lost 0 Performance_schema_digest_lost 0 Performance_schema_file_classes_lost 0 Performance_schema_file_handles_lost 0 Performance_schema_file_instances_lost 0 Performance_schema_hosts_lost 0 Performance_schema_locker_lost 0 Performance_schema_mutex_classes_lost 0 Performance_schema_mutex_instances_lost 0 Performance_schema_rwlock_classes_lost 0 Performance_schema_rwlock_instances_lost 0 Performance_schema_session_connect_attrs_lost 0 Performance_schema_socket_classes_lost 0 Performance_schema_socket_instances_lost 0 Performance_schema_stage_classes_lost 0 Performance_schema_statement_classes_lost 0 Performance_schema_table_handles_lost 0 Performance_schema_table_instances_lost 0 Performance_schema_thread_classes_lost 0 Performance_schema_thread_instances_lost 0 Performance_schema_users_lost 0 Prepared_stmt_count 0 Qcache_free_blocks 1 Qcache_free_memory 33536824 Qcache_hits 0 Qcache_inserts 0 Qcache_lowmem_prunes 0 Qcache_not_cached 274 Qcache_queries_in_cache 0 Qcache_total_blocks 1 Queries 19036 Questions 19022 Rows_read 11721162 Rows_sent 3402308 Rows_tmp_read 15272 Rpl_status AUTH_MASTER Select_full_join 0 Select_full_range_join 0 Select_range 0 Select_range_check 0 Select_scan 52 Slave_connections 0 Slave_heartbeat_period 1800.000 Slave_open_temp_tables 0 Slave_received_heartbeats 0 Slave_retried_transactions 0 Slave_running ON Slave_skipped_errors 0 Slaves_connected 0 Slaves_running 1 Slow_launch_threads 0 Slow_queries 4 Sort_merge_passes 0 Sort_priority_queue_sorts 0 Sort_range 0 Sort_rows 0 Sort_scan 0 Ssl_accept_renegotiates 0 Ssl_accepts 0 Ssl_callback_cache_hits 0 Ssl_cipher Ssl_cipher_list Ssl_client_connects 0 Ssl_connect_renegotiates 0 Ssl_ctx_verify_depth 0 Ssl_ctx_verify_mode 0 Ssl_default_timeout 0 Ssl_finished_accepts 0 Ssl_finished_connects 0 Ssl_server_not_after Ssl_server_not_before Ssl_session_cache_hits 0 Ssl_session_cache_misses 0 Ssl_session_cache_mode NONE Ssl_session_cache_overflows 0 Ssl_session_cache_size 0 Ssl_session_cache_timeouts 0 Ssl_sessions_reused 0 Ssl_used_session_cache_entries 0 Ssl_verify_depth 0 Ssl_verify_mode 0 Ssl_version Subquery_cache_hit 15158 Subquery_cache_miss 12054 Syncs 3058 Table_locks_immediate 3249 Table_locks_waited 0 Tc_log_max_pages_used 0 Tc_log_page_size 4096 Tc_log_page_waits 0 Threadpool_idle_threads 0 Threadpool_threads 0 Threads_cached 1 Threads_connected 1 Threads_created 2 Threads_running 1 Update_scan 0 Uptime 93089 Uptime_since_flush_status 93089 wsrep_cluster_conf_id 18446744073709551615 wsrep_cluster_size 0 wsrep_cluster_state_uuid wsrep_cluster_status Disconnected wsrep_connected OFF wsrep_local_bf_aborts 0 wsrep_local_index 18446744073709551615 wsrep_provider_name wsrep_provider_vendor wsrep_provider_version wsrep_ready OFF wsrep_thread_count 0
            Elkin Andrei Elkin added a comment - - edited

            mperez: I still think we can't exclude MDEV-20247 because there's a path from Rows_log_event::find_row() into MDEV-20247's hanging innobase_srv_conc_enter_innodb:

            Thread 29 (Thread 0x7f9318e1f700 (LWP 17465)):
            #0 0x00007f93177a7603 in select () from /lib/x86_64-linux-gnu/libc.so.6
            #1 0x000055ec5da3f4ff in ?? ()
            #2 0x000055ec5d9bf542 in ?? ()
            #3 0x000055ec5d9d6ff8 in ?? ()
            #4 0x000055ec5d74d082 in handler::ha_rnd_pos(unsigned char*, unsigned char*) ()
            #5 0x000055ec5d81bf8c in Rows_log_event::find_row(rpl_group_info*) ()
            #6 0x000055ec5d81c604 in Update_rows_log_event::do_exec_row(rpl_group_info*) ()
            #7 0x000055ec5d80ef3b in Rows_log_event::do_apply_event(rpl_group_info*) ()
            #8 0x000055ec5d52c4ab in ?? ()
            #9 0x000055ec5d537553 in handle_slave_sql ()
            #10 0x00007f9318b814a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
            #11 0x00007f93177aed0f in clone () from /lib/x86_64-linux-gnu/libc.so.6

            Elkin Andrei Elkin added a comment - - edited mperez : I still think we can't exclude MDEV-20247 because there's a path from Rows_log_event::find_row() into MDEV-20247 's hanging innobase_srv_conc_enter_innodb : Thread 29 (Thread 0x7f9318e1f700 (LWP 17465)): #0 0x00007f93177a7603 in select () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x000055ec5da3f4ff in ?? () #2 0x000055ec5d9bf542 in ?? () #3 0x000055ec5d9d6ff8 in ?? () #4 0x000055ec5d74d082 in handler::ha_rnd_pos(unsigned char*, unsigned char*) () #5 0x000055ec5d81bf8c in Rows_log_event::find_row(rpl_group_info*) () #6 0x000055ec5d81c604 in Update_rows_log_event::do_exec_row(rpl_group_info*) () #7 0x000055ec5d80ef3b in Rows_log_event::do_apply_event(rpl_group_info*) () #8 0x000055ec5d52c4ab in ?? () #9 0x000055ec5d537553 in handle_slave_sql () #10 0x00007f9318b814a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #11 0x00007f93177aed0f in clone () from /lib/x86_64-linux-gnu/libc.so.6

            Hello Miguel P,

            Thank you for providing the above details.
            Is there a difference in table structure between 'master' and 'slave'.
            Can you please share 'SHOW CREATE TABLE ContactCredential;' output on master.
            Since you are able to reproduce the issue consistently at your end with few
            hundred rows will it be possible to make a simple test case with dummy tables.

            Thank you!

            sujatha.sivakumar Sujatha Sivakumar (Inactive) added a comment - Hello Miguel P, Thank you for providing the above details. Is there a difference in table structure between 'master' and 'slave'. Can you please share 'SHOW CREATE TABLE ContactCredential;' output on master. Since you are able to reproduce the issue consistently at your end with few hundred rows will it be possible to make a simple test case with dummy tables. Thank you!
            mperez Miguel P added a comment -

            Hi Elkin,

            MDEV-20247 says the IO thread hangs on "preparing", yet this is about the SQL thread hanging on find_rows(-1)... Yet I did notice MDEV-20247 says it affected 10.1.41 and a comment said a downgrade (for 10.2) fixed it.

            When I was setting this up I had an older package with an older 10.1, and I believe replication worked fine for a day, but I didn't watch it. On the next day I upgraded Debian packages and I noticed this started happening. It could be a bug specific to 10.1.41, I'll try to get 10.1.42+, 10.2.27+, 10.3.18+ or 10.4.8+ as indicated by MDEV-20247 . If the upgrade solves it I'll report back here.

            Hi sujatha.sivakumar,

            The tables have identical SHOW CREATE TABLE, and they have been freshly created with a new mysqldump. I've double checkde it's the same above verbatim including all options.

            I'll try updating to the versions I mentioned above, and if this still happens, I'll try to isolate it and share the DB dumps and setups that will reproduce it.

            mperez Miguel P added a comment - Hi Elkin , MDEV-20247 says the IO thread hangs on "preparing", yet this is about the SQL thread hanging on find_rows(-1)... Yet I did notice MDEV-20247 says it affected 10.1.41 and a comment said a downgrade (for 10.2) fixed it. When I was setting this up I had an older package with an older 10.1, and I believe replication worked fine for a day, but I didn't watch it. On the next day I upgraded Debian packages and I noticed this started happening. It could be a bug specific to 10.1.41, I'll try to get 10.1.42+, 10.2.27+, 10.3.18+ or 10.4.8+ as indicated by MDEV-20247 . If the upgrade solves it I'll report back here. Hi sujatha.sivakumar , The tables have identical SHOW CREATE TABLE, and they have been freshly created with a new mysqldump. I've double checkde it's the same above verbatim including all options. I'll try updating to the versions I mentioned above, and if this still happens, I'll try to isolate it and share the DB dumps and setups that will reproduce it.
            mperez Miguel P added a comment -

            I can confirm replication works after upgrading to 10.3.18, so it had to be that bug. You can close this, thank you very much for your help.

            Not sure if it's worth notifying Debian that replication will be broken for Stretch (Debian 9, oldstable) as it's using 10.1.41 and they should try to upgrade the MariaDB packages to a newer 10.1.

            Cheers

            mperez Miguel P added a comment - I can confirm replication works after upgrading to 10.3.18, so it had to be that bug. You can close this, thank you very much for your help. Not sure if it's worth notifying Debian that replication will be broken for Stretch (Debian 9, oldstable) as it's using 10.1.41 and they should try to upgrade the MariaDB packages to a newer 10.1. Cheers

            People

              sujatha.sivakumar Sujatha Sivakumar (Inactive)
              Sandy Sandy
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.