[MDEV-32039] Some records are not updated on storage engine (innodb) but exist on binary log Created: 2023-08-30  Updated: 2023-10-30  Resolved: 2023-10-30

Status: Closed
Project: MariaDB Server
Component/s: None
Affects Version/s: 10.0.38
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Michael Velayo Assignee: Unassigned
Resolution: Incomplete Votes: 0
Labels: innodb
Environment:

OS: CentOS release 6.7 (Final)
Application: Node v0.12.12



 Description   

I have a nodejs application that build a query to update records on my table.

In the binary logs there is a transaction that occurred but in the storage engine (innodb) it did not update.

for example in the binary log, the transaction is like this. but the id with 2 and 3 are not updated in the storage engine(innodb).

BEGIN
UPDATE my_table SET status = 1, updated_at = NOW() WHERE id = 1
UPDATE my_table SET status = 1, updated_at = NOW() WHERE id = 2
UPDATE my_table SET status = 1, updated_at = NOW() WHERE id = 3
UPDATE my_table SET status = 1, updated_at = NOW() WHERE id = 4
UPDATE my_table SET status = 1, updated_at = NOW() WHERE id = 5
COMMIT



 Comments   
Comment by Daniel Black [ 2023-08-30 ]

Can you test against a maintained 10.4 version?

What is the table structure of my_table (from SHOW CREATE TABLE) (change/omit field names if needed, keep indexes relevant to query and types.

If its repeatable, what non-default replication, binary log and innodb settings are in use?

Comment by Michael Velayo [ 2023-08-30 ]

Can you test against a maintained 10.4 version?

I can't test because I can't reproduce the problem. but it always happen.

Table Structure

 CREATE TABLE `autocalls` (
  `id` int(11) NOT NULL,
  `status` tinyint(3) NOT NULL DEFAULT '0',
  `priority` int(11) NOT NULL DEFAULT '0',
  `updated_at` datetime NOT NULL,
  PRIMARY KEY (`id`),
  KEY `index_priority` (`status`,`priority`),
  KEY `index_campaign` (`campaign_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 |

Configuaration

log-bin=mysql-bin
binlog_format=mixed
 
replicate-wild-ignore-table=mysql.%
 
innodb_data_home_dir = /var/lib/mysql
innodb_data_file_path = ibdata1:10M:autoextend
innodb_log_group_home_dir = /var/lib/mysql
innodb_buffer_pool_size = 1G
innodb_additional_mem_pool_size = 20M
innodb_log_file_size = 1G
innodb_log_buffer_size = 8M
innodb_flush_log_at_trx_commit = 2
innodb_lock_wait_timeout = 50

Innodb settings

MariaDB [(none)]> show variables like 'innodb%';
+-------------------------------------------+------------------------+
| Variable_name                             | Value                  |
+-------------------------------------------+------------------------+
| innodb_adaptive_flushing                  | ON                     |
| innodb_adaptive_flushing_lwm              | 10.000000              |
| innodb_adaptive_hash_index                | ON                     |
| innodb_adaptive_hash_index_partitions     | 1                      |
| innodb_adaptive_max_sleep_delay           | 150000                 |
| innodb_additional_mem_pool_size           | 20971520               |
| innodb_api_bk_commit_interval             | 5                      |
| innodb_api_disable_rowlock                | OFF                    |
| innodb_api_enable_binlog                  | OFF                    |
| innodb_api_enable_mdl                     | OFF                    |
| innodb_api_trx_level                      | 0                      |
| innodb_autoextend_increment               | 64                     |
| innodb_autoinc_lock_mode                  | 1                      |
| innodb_buffer_pool_dump_at_shutdown       | OFF                    |
| innodb_buffer_pool_dump_now               | OFF                    |
| innodb_buffer_pool_dump_pct               | 100                    |
| innodb_buffer_pool_filename               | ib_buffer_pool         |
| innodb_buffer_pool_instances              | 8                      |
| innodb_buffer_pool_load_abort             | OFF                    |
| innodb_buffer_pool_load_at_startup        | OFF                    |
| innodb_buffer_pool_load_now               | OFF                    |
| innodb_buffer_pool_populate               | OFF                    |
| innodb_buffer_pool_size                   | 1073741824             |
| innodb_change_buffer_max_size             | 25                     |
| innodb_change_buffering                   | all                    |
| innodb_checksum_algorithm                 | innodb                 |
| innodb_checksums                          | ON                     |
| innodb_cleaner_lsn_age_factor             | high_checkpoint        |
| innodb_cmp_per_index_enabled              | OFF                    |
| innodb_commit_concurrency                 | 0                      |
| innodb_compression_failure_threshold_pct  | 5                      |
| innodb_compression_level                  | 6                      |
| innodb_compression_pad_pct_max            | 50                     |
| innodb_concurrency_tickets                | 5000                   |
| innodb_corrupt_table_action               | assert                 |
| innodb_data_file_path                     | ibdata1:10M:autoextend |
| innodb_data_home_dir                      | /var/lib/mysql         |
| innodb_disable_sort_file_cache            | OFF                    |
| innodb_doublewrite                        | ON                     |
| innodb_empty_free_list_algorithm          | backoff                |
| innodb_fake_changes                       | OFF                    |
| innodb_fast_shutdown                      | 1                      |
| innodb_file_format                        | Antelope               |
| innodb_file_format_check                  | ON                     |
| innodb_file_format_max                    | Antelope               |
| innodb_file_per_table                     | ON                     |
| innodb_flush_log_at_timeout               | 1                      |
| innodb_flush_log_at_trx_commit            | 2                      |
| innodb_flush_method                       |                        |
| innodb_flush_neighbors                    | 1                      |
| innodb_flushing_avg_loops                 | 30                     |
| innodb_force_load_corrupted               | OFF                    |
| innodb_force_recovery                     | 0                      |
| innodb_foreground_preflush                | exponential_backoff    |
| innodb_ft_aux_table                       |                        |
| innodb_ft_cache_size                      | 8000000                |
| innodb_ft_enable_diag_print               | OFF                    |
| innodb_ft_enable_stopword                 | ON                     |
| innodb_ft_max_token_size                  | 84                     |
| innodb_ft_min_token_size                  | 3                      |
| innodb_ft_num_word_optimize               | 2000                   |
| innodb_ft_result_cache_limit              | 2000000000             |
| innodb_ft_server_stopword_table           |                        |
| innodb_ft_sort_pll_degree                 | 2                      |
| innodb_ft_total_cache_size                | 640000000              |
| innodb_ft_user_stopword_table             |                        |
| innodb_io_capacity                        | 200                    |
| innodb_io_capacity_max                    | 2000                   |
| innodb_kill_idle_transaction              | 0                      |
| innodb_large_prefix                       | OFF                    |
| innodb_lock_wait_timeout                  | 50                     |
| innodb_locking_fake_changes               | ON                     |
| innodb_locks_unsafe_for_binlog            | OFF                    |
| innodb_log_arch_dir                       | /var/lib/mysql         |
| innodb_log_arch_expire_sec                | 0                      |
| innodb_log_archive                        | OFF                    |
| innodb_log_block_size                     | 512                    |
| innodb_log_buffer_size                    | 8388608                |
| innodb_log_checksum_algorithm             | innodb                 |
| innodb_log_compressed_pages               | ON                     |
| innodb_log_file_size                      | 1073741824             |
| innodb_log_files_in_group                 | 2                      |
| innodb_log_group_home_dir                 | /var/lib/mysql         |
| innodb_lru_scan_depth                     | 1024                   |
| innodb_max_bitmap_file_size               | 104857600              |
| innodb_max_changed_pages                  | 1000000                |
| innodb_max_dirty_pages_pct                | 75.000000              |
| innodb_max_dirty_pages_pct_lwm            | 0.001000               |
| innodb_max_purge_lag                      | 0                      |
| innodb_max_purge_lag_delay                | 0                      |
| innodb_mirrored_log_groups                | 1                      |
| innodb_monitor_disable                    |                        |
| innodb_monitor_enable                     |                        |
| innodb_monitor_reset                      |                        |
| innodb_monitor_reset_all                  |                        |
| innodb_old_blocks_pct                     | 37                     |
| innodb_old_blocks_time                    | 1000                   |
| innodb_online_alter_log_max_size          | 134217728              |
| innodb_open_files                         | 300                    |
| innodb_optimize_fulltext_only             | OFF                    |
| innodb_page_size                          | 16384                  |
| innodb_print_all_deadlocks                | OFF                    |
| innodb_print_lock_wait_timeout_info       | OFF                    |
| innodb_purge_batch_size                   | 300                    |
| innodb_purge_threads                      | 1                      |
| innodb_random_read_ahead                  | OFF                    |
| innodb_read_ahead_threshold               | 56                     |
| innodb_read_io_threads                    | 4                      |
| innodb_read_only                          | OFF                    |
| innodb_replication_delay                  | 0                      |
| innodb_rollback_on_timeout                | OFF                    |
| innodb_rollback_segments                  | 128                    |
| innodb_sched_priority_cleaner             | 19                     |
| innodb_show_locks_held                    | 10                     |
| innodb_show_verbose_locks                 | 0                      |
| innodb_simulate_comp_failures             | 0                      |
| innodb_sort_buffer_size                   | 1048576                |
| innodb_spin_wait_delay                    | 6                      |
| innodb_stats_auto_recalc                  | ON                     |
| innodb_stats_include_delete_marked        | OFF                    |
| innodb_stats_method                       | nulls_equal            |
| innodb_stats_modified_counter             | 0                      |
| innodb_stats_on_metadata                  | OFF                    |
| innodb_stats_persistent                   | ON                     |
| innodb_stats_persistent_sample_pages      | 20                     |
| innodb_stats_sample_pages                 | 8                      |
| innodb_stats_traditional                  | ON                     |
| innodb_stats_transient_sample_pages       | 8                      |
| innodb_status_output                      | OFF                    |
| innodb_status_output_locks                | OFF                    |
| innodb_strict_mode                        | OFF                    |
| innodb_support_xa                         | ON                     |
| innodb_sync_array_size                    | 1                      |
| innodb_sync_spin_loops                    | 30                     |
| innodb_table_locks                        | ON                     |
| innodb_thread_concurrency                 | 0                      |
| innodb_thread_sleep_delay                 | 10000                  |
| innodb_tmpdir                             |                        |
| innodb_track_changed_pages                | OFF                    |
| innodb_undo_directory                     | .                      |
| innodb_undo_logs                          | 128                    |
| innodb_undo_tablespaces                   | 0                      |
| innodb_use_atomic_writes                  | OFF                    |
| innodb_use_fallocate                      | OFF                    |
| innodb_use_global_flush_log_at_trx_commit | ON                     |
| innodb_use_native_aio                     | ON                     |
| innodb_use_stacktrace                     | OFF                    |
| innodb_use_sys_malloc                     | ON                     |
| innodb_version                            | 5.6.42-84.2            |
| innodb_write_io_threads                   | 4                      |
+-------------------------------------------+------------------------+

Binlog settings

MariaDB [(none)]> show variables like '%binlog%';
+-----------------------------------------+----------------------+
| Variable_name                           | Value                |
+-----------------------------------------+----------------------+
| binlog_annotate_row_events              | OFF                  |
| binlog_cache_size                       | 32768                |
| binlog_checksum                         | NONE                 |
| binlog_commit_wait_count                | 0                    |
| binlog_commit_wait_usec                 | 100000               |
| binlog_direct_non_transactional_updates | OFF                  |
| binlog_format                           | MIXED                |
| binlog_optimize_thread_scheduling       | ON                   |
| binlog_stmt_cache_size                  | 32768                |
| gtid_binlog_pos                         | 0-101-165906         |
| gtid_binlog_state                       | 0-101-165906         |
| innodb_api_enable_binlog                | OFF                  |
| innodb_locks_unsafe_for_binlog          | OFF                  |
| max_binlog_cache_size                   | 18446744073709547520 |
| max_binlog_size                         | 1073741824           |
| max_binlog_stmt_cache_size              | 18446744073709547520 |
| sync_binlog                             | 0                    |
+-----------------------------------------+----------------------+

Comment by Sergei Golubchik [ 2023-10-02 ]

There will be no new releases of MariaDB 10.0, so this bug will never be fixed there.
Please try to upgrade to one of the still maintained versions and if the bug will still be there we'll try to figure out how to reproduce and fix it.

Generated at Thu Feb 08 10:28:21 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.