Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-28975

AWS RDS mariadb 10.5.12 crashes upon OS update

Details

    Description

      My RDS instance was showing `incompatible parameter` error on the console; users getting connection refused. I followed this to reset the parameters to default and rebooted, but still getting same error. Then I saw there's an OS update available, so I hit the `upgrade now` button. Then it starts crashing, never booted again.

      here's the logs

      2022-06-24 2:59:54 0 [Warning] You need to use --log-bin to make --log-slave-updates work.
      2022-06-24 2:59:54 0 [Note] InnoDB: Uses event mutexes
      2022-06-24 2:59:54 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
      2022-06-24 2:59:54 0 [Note] InnoDB: Number of pools: 1
      2022-06-24 2:59:54 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
      2022-06-24 2:59:54 0 [Note] InnoDB: Using Linux native AIO
      2022-06-24 2:59:54 0 [Note] InnoDB: Initializing buffer pool, total size = 2147483648, chunk size = 134217728
      2022-06-24 2:59:54 0 [Note] InnoDB: Completed initialization of buffer pool
      2022-06-24 2:59:54 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=1343302636904,1343302636904
      2022-06-24 3:00:00 0 [Note] InnoDB: 2 transaction(s) which must be rolled back or cleaned up in total 2 row operations to undo
      2022-06-24 3:00:00 0 [Note] InnoDB: Trx id counter is 8615938331
      2022-06-24 3:00:00 0 [Note] InnoDB: Starting final batch to recover 25079 pages from redo log.
      2022-06-24 3:00:16 0 [Note] InnoDB: Last binlog file '/rdsdbdata/log/binlog/mysql-bin-changelog.667099', position 4312
      2022-06-24 3:00:16 0 [Note] InnoDB: 128 rollback segments are active.
      2022-06-24 3:00:16 0 [Note] InnoDB: Starting in background the rollback of recovered transactions
      2022-06-24 3:00:16 0 [Note] InnoDB: To roll back: 2 transactions, 2 rows
      2022-06-24 3:00:16 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
      2022-06-24 3:00:16 0 [Note] InnoDB: Creating shared tablespace for temporary tables
      2022-06-24 3:00:16 0 [Note] InnoDB: Setting file '/rdsdbdata/db/innodb/ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
      2022-06-24 3:00:16 0 [Note] InnoDB: File '/rdsdbdata/db/innodb/ibtmp1' size is now 12 MB.
      2022-06-24 3:00:16 0 [ERROR] [FATAL] InnoDB: Page old data size 8606 new data size 5454, page old max ins size 7544 new max ins size 10696
      220624 3:00:16 [ERROR] mysqld got signal 6 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.
       
      To report this bug, see https://mariadb.com/kb/en/reporting-bugs
       
      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed,
      something is definitely wrong and this may fail.
       
      Server version: 10.5.12-MariaDB
      key_buffer_size=16777216
      read_buffer_size=262144
      max_used_connections=0
      max_threads=5002
      thread_count=0
      It is possible that mysqld could use up to
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 11660859 K bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
       
      Thread pointer: 0x0
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x0 thread_stack 0x40000
      2022-06-24 3:00:16 0 [Note] InnoDB: 10.5.12 started; log sequence number 1343572670349; transaction id 8615938332
      2022-06-24 3:00:16 0 [Note] InnoDB: Loading buffer pool(s) from /rdsdbdata/db/innodb/ib_buffer_pool
      2022-06-24 3:00:16 0 [Note] Server socket created on IP: '::'.
      2022-06-24 3:00:16 0 [Note] Reading of all Master_info entries succeeded
      2022-06-24 3:00:16 0 [Note] Added new Master_info '' to hash table
      2022-06-24 3:00:16 0 [Note] /rdsdbbin/mysql/bin/mysqld: ready for connections.
      Version: '10.5.12-MariaDB' socket: '/tmp/mysql.sock' port: 3306 managed by https://aws.amazon.com/rds/
      mysys/stacktrace.c:213(my_print_stacktrace)[0x55b7e23a1c5e]
      sql/signal_handler.cc:221(handle_fatal_signal)[0x55b7e1b5376a]
      sigaction.c:0(__restore_rt)[0x2b656a1e6100]
      :0(__GI_raise)[0x2b656a4285f7]
      :0(__GI_abort)[0x2b656a429ce8]
      :0(ib::fatal::~fatal())[0x55b7e21f57d1]
      btr/btr0btr.cc:1484(btr_page_reorganize_low)[0x55b7e21fb246]
      btr/btr0btr.cc:1646(btr_page_reorganize(page_cur_t*, dict_index_t*, mtr_t*))[0x55b7e21fb617]
      include/dict0mem.h:1840(dict_table_t::not_redundant() const)[0x55b7e20e0f93]
      ibuf/ibuf0ibuf.cc:3881(ibuf_insert_to_index_page)[0x55b7e20e6a11]
      buf/buf0buf.cc:3339(buf_page_get_low(page_id_t, unsigned long, unsigned long, buf_block_t*, unsigned long, char const*, unsigned int, mtr_t*, dberr_t*, bool))[0x55b7e223052b]
      buf/buf0buf.cc:3421(buf_page_get_gen(page_id_t, unsigned long, unsigned long, buf_block_t*, unsigned long, char const*, unsigned int, mtr_t*, dberr_t*, bool))[0x55b7e2230576]
      btr/btr0cur.cc:1622(btr_cur_search_to_nth_level_func(dict_index_t*, unsigned long, dtuple_t const*, page_cur_mode_t, unsigned long, btr_cur_t*, rw_lock_t*, char const*, unsigned int, mtr_t*, unsigned long))[0x55b7e2210941]
      include/btr0pcur.ic:448(btr_pcur_open_low)[0x55b7e2189ca2]
      row/row0row.cc:1306(row_search_index_entry(dict_index_t*, dtuple_t const*, unsigned long, btr_pcur_t*, mtr_t*))[0x55b7e2189e0f]
      row/row0uins.cc:260(row_undo_ins_remove_sec_low)[0x55b7e22faee3]
      row/row0uins.cc:320(row_undo_ins_remove_sec)[0x55b7e22fb5b2]
      row/row0undo.cc:420(row_undo)[0x55b7e2194d0a]
      que/que0que.cc:944(que_thr_step)[0x55b7e213ba30]
      trx/trx0roll.cc:631(trx_rollback_active)[0x55b7e21d1e90]
      trx/trx0roll.cc:784(trx_rollback_recovered(bool))[0x55b7e21d255f]
      trx/trx0roll.cc:838(trx_rollback_all_recovered)[0x55b7e21d2c6a]
      pthread_create.c:0(start_thread)[0x2b656a1dedc5]
      ??:0(__clone)[0x2b656a4e9c9d]
      The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
      information that should help you find out what is causing the crash.
      Writing a core file...
      Working directory at /rdsdbdata/db
      Resource Limits:
      Limit Soft Limit Hard Limit Units
      Max cpu time unlimited unlimited seconds
      Max file size unlimited unlimited bytes
      Max data size unlimited unlimited bytes
      Max stack size unlimited unlimited bytes
      Max core file size 0 unlimited bytes
      Max resident set unlimited unlimited bytes
      Max processes unlimited unlimited processes
      Max open files 65535 65535 files
      Max locked memory 65536 65536 bytes
      Max address space unlimited unlimited bytes
      Max file locks unlimited unlimited locks
      Max pending signals 15735 15735 signals
      Max msgqueue size 819200 819200 bytes
      Max nice priority 0 0
      Max realtime priority 0 0
      Max realtime timeout unlimited unlimited us
      Core pattern: /rdsdbdata/tmp/core-%e-%p
       
      2022-06-24 3:01:11 0 [Note] /rdsdbbin/mysql/bin/mysqld (mysqld 10.5.12-MariaDB) starting as process 17497 ...
      2022-06-24 3:01:11 0 [Warning] You need to use --log-bin to make --log-slave-updates work.
      2022-06-24 3:01:11 0 [Note] InnoDB: Uses event mutexes
      2022-06-24 3:01:11 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
      2022-06-24 3:01:11 0 [Note] InnoDB: Number of pools: 1
      2022-06-24 3:01:11 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
      2022-06-24 3:01:11 0 [Note] InnoDB: Using Linux native AIO
      2022-06-24 3:01:11 0 [Note] InnoDB: Initializing buffer pool, total size = 2147483648, chunk size = 134217728
      2022-06-24 3:01:11 0 [Note] InnoDB: Completed initialization of buffer pool
      2022-06-24 3:01:11 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=1343302636904,1343302636904
      2022-06-24 3:01:17 0 [Note] InnoDB: 2 transaction(s) which must be rolled back or cleaned up in total 2 row operations to undo
      2022-06-24 3:01:17 0 [Note] InnoDB: Trx id counter is 8615938331
      2022-06-24 3:01:17 0 [Note] InnoDB: Starting final batch to recover 25079 pages from redo log.
      2022-06-24 3:01:33 0 [Note] InnoDB: Last binlog file '/rdsdbdata/log/binlog/mysql-bin-changelog.667099', position 4312
      2022-06-24 3:01:33 0 [Note] InnoDB: 128 rollback segments are active.
      2022-06-24 3:01:33 0 [Note] InnoDB: Starting in background the rollback of recovered transactions
      2022-06-24 3:01:33 0 [Note] InnoDB: To roll back: 2 transactions, 2 rows
      2022-06-24 3:01:33 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
      2022-06-24 3:01:33 0 [Note] InnoDB: Creating shared tablespace for temporary tables
      2022-06-24 3:01:33 0 [Note] InnoDB: Setting file '/rdsdbdata/db/innodb/ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
      2022-06-24 3:01:33 0 [Note] InnoDB: File '/rdsdbdata/db/innodb/ibtmp1' size is now 12 MB.
      2022-06-24 3:01:33 0 [ERROR] [FATAL] InnoDB: Page old data size 8606 new data size 5454, page old max ins size 7544 new max ins size 10696
      220624 3:01:33 [ERROR] mysqld got signal 6 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.
       
      To report this bug, see https://mariadb.com/kb/en/reporting-bugs
       
      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed,
      something is definitely wrong and this may fail.
       
      Server version: 10.5.12-MariaDB
      key_buffer_size=16777216
      read_buffer_size=262144
      max_used_connections=0
      max_threads=5002
      thread_count=0
      It is possible that mysqld could use up to
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 11660859 K bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
       
      Thread pointer: 0x0
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x0 thread_stack 0x40000
      2022-06-24 3:01:33 0 [Note] InnoDB: 10.5.12 started; log sequence number 1343572974007; transaction id 8615938332
      2022-06-24 3:01:33 0 [Note] InnoDB: Loading buffer pool(s) from /rdsdbdata/db/innodb/ib_buffer_pool
      2022-06-24 3:01:33 0 [Note] Server socket created on IP: '::'.
      2022-06-24 3:01:33 0 [Note] Reading of all Master_info entries succeeded
      2022-06-24 3:01:33 0 [Note] Added new Master_info '' to hash table
      2022-06-24 3:01:33 0 [Note] /rdsdbbin/mysql/bin/mysqld: ready for connections.
      Version: '10.5.12-MariaDB' socket: '/tmp/mysql.sock' port: 3306 managed by https://aws.amazon.com/rds/
      mysys/stacktrace.c:213(my_print_stacktrace)[0x560cd2b17c5e]
      sql/signal_handler.cc:221(handle_fatal_signal)[0x560cd22c976a]
      sigaction.c:0(__restore_rt)[0x2abefd95a100]
      :0(__GI_raise)[0x2abefdb9c5f7]
      :0(__GI_abort)[0x2abefdb9dce8]
      :0(ib::fatal::~fatal())[0x560cd296b7d1]
      btr/btr0btr.cc:1484(btr_page_reorganize_low)[0x560cd2971246]
      btr/btr0btr.cc:1646(btr_page_reorganize(page_cur_t*, dict_index_t*, mtr_t*))[0x560cd2971617]
      include/dict0mem.h:1840(dict_table_t::not_redundant() const)[0x560cd2856f93]
      ibuf/ibuf0ibuf.cc:3881(ibuf_insert_to_index_page)[0x560cd285ca11]
      buf/buf0buf.cc:3339(buf_page_get_low(page_id_t, unsigned long, unsigned long, buf_block_t*, unsigned long, char const*, unsigned int, mtr_t*, dberr_t*, bool))[0x560cd29a652b]
      buf/buf0buf.cc:3421(buf_page_get_gen(page_id_t, unsigned long, unsigned long, buf_block_t*, unsigned long, char const*, unsigned int, mtr_t*, dberr_t*, bool))[0x560cd29a6576]
      btr/btr0cur.cc:1622(btr_cur_search_to_nth_level_func(dict_index_t*, unsigned long, dtuple_t const*, page_cur_mode_t, unsigned long, btr_cur_t*, rw_lock_t*, char const*, unsigned int, mtr_t*, unsigned long))[0x560cd2986941]
      include/btr0pcur.ic:448(btr_pcur_open_low)[0x560cd28ffca2]
      row/row0row.cc:1306(row_search_index_entry(dict_index_t*, dtuple_t const*, unsigned long, btr_pcur_t*, mtr_t*))[0x560cd28ffe0f]
      row/row0uins.cc:260(row_undo_ins_remove_sec_low)[0x560cd2a70ee3]
      row/row0uins.cc:320(row_undo_ins_remove_sec)[0x560cd2a715b2]
      row/row0undo.cc:420(row_undo)[0x560cd290ad0a]
      que/que0que.cc:944(que_thr_step)[0x560cd28b1a30]
      trx/trx0roll.cc:631(trx_rollback_active)[0x560cd2947e90]
      trx/trx0roll.cc:784(trx_rollback_recovered(bool))[0x560cd294855f]
      trx/trx0roll.cc:838(trx_rollback_all_recovered)[0x560cd2948c6a]
      pthread_create.c:0(start_thread)[0x2abefd952dc5]
      ??:0(__clone)[0x2abefdc5dc9d]
      The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
      information that should help you find out what is causing the crash.
      Writing a core file...
      Working directory at /rdsdbdata/db
      Resource Limits:
      Limit Soft Limit Hard Limit Units
      Max cpu time unlimited unlimited seconds
      Max file size unlimited unlimited bytes
      Max data size unlimited unlimited bytes
      Max stack size unlimited unlimited bytes
      Max core file size 0 unlimited bytes
      Max resident set unlimited unlimited bytes
      Max processes unlimited unlimited processes
      Max open files 65535 65535 files
      Max locked memory 65536 65536 bytes
      Max address space unlimited unlimited bytes
      Max file locks unlimited unlimited locks
      Max pending signals 15735 15735 signals
      Max msgqueue size 819200 819200 bytes
      Max nice priority 0 0
      Max realtime priority 0 0
      Max realtime timeout unlimited unlimited us
      Core pattern: /rdsdbdata/tmp/core-%e-%p
       
      2022-06-24 3:02:28 0 [Warning] You need to use --log-bin to make --log-slave-updates work.
      2022-06-24 3:02:28 0 [Note] InnoDB: Uses event mutexes
      2022-06-24 3:02:28 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
      2022-06-24 3:02:28 0 [Note] InnoDB: Number of pools: 1
      2022-06-24 3:02:28 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
      2022-06-24 3:02:28 0 [Note] InnoDB: Using Linux native AIO
      2022-06-24 3:02:28 0 [Note] InnoDB: Initializing buffer pool, total size = 2147483648, chunk size = 134217728
      2022-06-24 3:02:29 0 [Note] InnoDB: Completed initialization of buffer pool
      2022-06-24 3:02:29 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=1343302636904,1343302636904
      2022-06-24 3:02:34 0 [Note] InnoDB: 2 transaction(s) which must be rolled back or cleaned up in total 2 row operations to undo
      2022-06-24 3:02:34 0 [Note] InnoDB: Trx id counter is 8615938331
      2022-06-24 3:02:34 0 [Note] InnoDB: Starting final batch to recover 25079 pages from redo log.
      2022-06-24 3:02:50 0 [Note] InnoDB: Last binlog file '/rdsdbdata/log/binlog/mysql-bin-changelog.667099', position 4312
      2022-06-24 3:02:50 0 [Note] InnoDB: 128 rollback segments are active.
      2022-06-24 3:02:50 0 [Note] InnoDB: Starting in background the rollback of recovered transactions
      2022-06-24 3:02:50 0 [Note] InnoDB: To roll back: 2 transactions, 2 rows
      2022-06-24 3:02:50 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
      2022-06-24 3:02:50 0 [Note] InnoDB: Creating shared tablespace for temporary tables
      2022-06-24 3:02:50 0 [Note] InnoDB: Setting file '/rdsdbdata/db/innodb/ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
      2022-06-24 3:02:50 0 [Note] InnoDB: File '/rdsdbdata/db/innodb/ibtmp1' size is now 12 MB.
      2022-06-24 3:02:50 0 [ERROR] [FATAL] InnoDB: Page old data size 8606 new data size 5454, page old max ins size 7544 new max ins size 10696
      220624 3:02:50 [ERROR] mysqld got signal 6 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.
       
      To report this bug, see https://mariadb.com/kb/en/reporting-bugs
       
      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed,
      something is definitely wrong and this may fail.
       
      Server version: 10.5.12-MariaDB
      key_buffer_size=16777216
      read_buffer_size=262144
      max_used_connections=0
      max_threads=5002
      thread_count=0
      It is possible that mysqld could use up to
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 11660859 K bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
       
      Thread pointer: 0x0
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x0 thread_stack 0x40000
      2022-06-24 3:02:50 0 [Note] InnoDB: 10.5.12 started; log sequence number 1343573277665; transaction id 8615938332
      2022-06-24 3:02:50 0 [Note] InnoDB: Loading buffer pool(s) from /rdsdbdata/db/innodb/ib_buffer_pool
      2022-06-24 3:02:50 0 [Note] Server socket created on IP: '::'.
      2022-06-24 3:02:50 0 [Note] Reading of all Master_info entries succeeded
      2022-06-24 3:02:50 0 [Note] Added new Master_info '' to hash table
      2022-06-24 3:02:50 0 [Note] /rdsdbbin/mysql/bin/mysqld: ready for connections.
      Version: '10.5.12-MariaDB' socket: '/tmp/mysql.sock' port: 3306 managed by https://aws.amazon.com/rds/
      mysys/stacktrace.c:213(my_print_stacktrace)[0x563f02949c5e]
      sql/signal_handler.cc:221(handle_fatal_signal)[0x563f020fb76a]
      sigaction.c:0(__restore_rt)[0x2b828c588100]
      :0(__GI_raise)[0x2b828c7ca5f7]
      :0(__GI_abort)[0x2b828c7cbce8]
      :0(ib::fatal::~fatal())[0x563f0279d7d1]
      btr/btr0btr.cc:1484(btr_page_reorganize_low)[0x563f027a3246]
      btr/btr0btr.cc:1646(btr_page_reorganize(page_cur_t*, dict_index_t*, mtr_t*))[0x563f027a3617]
      include/dict0mem.h:1840(dict_table_t::not_redundant() const)[0x563f02688f93]
      ibuf/ibuf0ibuf.cc:3881(ibuf_insert_to_index_page)[0x563f0268ea11]
      buf/buf0buf.cc:3339(buf_page_get_low(page_id_t, unsigned long, unsigned long, buf_block_t*, unsigned long, char const*, unsigned int, mtr_t*, dberr_t*, bool))[0x563f027d852b]
      buf/buf0buf.cc:3421(buf_page_get_gen(page_id_t, unsigned long, unsigned long, buf_block_t*, unsigned long, char const*, unsigned int, mtr_t*, dberr_t*, bool))[0x563f027d8576]
      btr/btr0cur.cc:1622(btr_cur_search_to_nth_level_func(dict_index_t*, unsigned long, dtuple_t const*, page_cur_mode_t, unsigned long, btr_cur_t*, rw_lock_t*, char const*, unsigned int, mtr_t*, unsigned long))[0x563f027b8941]
      include/btr0pcur.ic:448(btr_pcur_open_low)[0x563f02731ca2]
      row/row0row.cc:1306(row_search_index_entry(dict_index_t*, dtuple_t const*, unsigned long, btr_pcur_t*, mtr_t*))[0x563f02731e0f]
      row/row0uins.cc:260(row_undo_ins_remove_sec_low)[0x563f028a2ee3]
      row/row0uins.cc:320(row_undo_ins_remove_sec)[0x563f028a35b2]
      row/row0undo.cc:420(row_undo)[0x563f0273cd0a]
      que/que0que.cc:944(que_thr_step)[0x563f026e3a30]
      trx/trx0roll.cc:631(trx_rollback_active)[0x563f02779e90]
      trx/trx0roll.cc:784(trx_rollback_recovered(bool))[0x563f0277a55f]
      trx/trx0roll.cc:838(trx_rollback_all_recovered)[0x563f0277ac6a]
      pthread_create.c:0(start_thread)[0x2b828c580dc5]
      ??:0(__clone)[0x2b828c88bc9d]
      The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
      information that should help you find out what is causing the crash.
      Writing a core file...
      Working directory at /rdsdbdata/db
      Resource Limits:
      Limit Soft Limit Hard Limit Units
      Max cpu time unlimited unlimited seconds
      Max file size unlimited unlimited bytes
      Max data size unlimited unlimited bytes
      Max stack size unlimited unlimited bytes
      Max core file size 0 unlimited bytes
      Max resident set unlimited unlimited bytes
      Max processes unlimited unlimited processes
      Max open files 65535 65535 files
      Max locked memory 65536 65536 bytes
      Max address space unlimited unlimited bytes
      Max file locks unlimited unlimited locks
      Max pending signals 15735 15735 signals
      Max msgqueue size 819200 819200 bytes
      Max nice priority 0 0
      Max realtime priority 0 0
      Max realtime timeout unlimited unlimited us
      Core pattern: /rdsdbdata/tmp/core-%e-%p
      

      Attachments

        Issue Links

          Activity

            The stack trace shows that corruption was detected during a change buffer merge, while rolling back a transaction (maybe a recovered incomplete transaction).

            Unfortunately, we have been unable to reproduce this corruption in our internal testing. The change buffer was disabled by default in MDEV-27734. Disabling it will not cure any previous corruption; it will only prevent the use of the change buffer in future operations. But, you could try if the database would start if you specify innodb_change_buffering=none. If that does not help, then with innodb_force_recovery=3 the database should start up, without rolling back that incomplete transaction. That should allow you to create a logical backup of the data (with mariadb-dump a.k.a. mysqldump or similar tools).

            This crash (but not the corruption that triggers it) should have been fixed in MDEV-13542. Because we have not been able to reproduce this crash internally, I can’t guarantee what exactly should happen in this case. The intention is that the affected secondary index will be marked as corrupted and operations that would use the index would return an error to the SQL statement.

            A support customer of ours hit a similar crash some time ago, but they had overwritten the corrupted database with a backup, so there was no chance to test the fix.

            Also the reports MDEV-22739 and MDEV-26917 are due to change buffer induced corruption. If anyone can provide a set of SQL statements that reproduce this type of errors, please post them there. The test must start from initializing the database from the scratch, with no physical backup of any files.

            marko Marko Mäkelä added a comment - The stack trace shows that corruption was detected during a change buffer merge, while rolling back a transaction (maybe a recovered incomplete transaction). Unfortunately, we have been unable to reproduce this corruption in our internal testing. The change buffer was disabled by default in MDEV-27734 . Disabling it will not cure any previous corruption; it will only prevent the use of the change buffer in future operations. But, you could try if the database would start if you specify innodb_change_buffering=none . If that does not help, then with innodb_force_recovery=3 the database should start up, without rolling back that incomplete transaction. That should allow you to create a logical backup of the data (with mariadb-dump a.k.a. mysqldump or similar tools). This crash (but not the corruption that triggers it) should have been fixed in MDEV-13542 . Because we have not been able to reproduce this crash internally, I can’t guarantee what exactly should happen in this case. The intention is that the affected secondary index will be marked as corrupted and operations that would use the index would return an error to the SQL statement. A support customer of ours hit a similar crash some time ago, but they had overwritten the corrupted database with a backup, so there was no chance to test the fix. Also the reports MDEV-22739 and MDEV-26917 are due to change buffer induced corruption. If anyone can provide a set of SQL statements that reproduce this type of errors, please post them there. The test must start from initializing the database from the scratch, with no physical backup of any files.

            I am closing this as a duplicate of MDEV-13542 because this crash should not occur with that version. For reproducing the corruption itself, MDEV-22739 or MDEV-26917 can be updated.

            marko Marko Mäkelä added a comment - I am closing this as a duplicate of MDEV-13542 because this crash should not occur with that version. For reproducing the corruption itself, MDEV-22739 or MDEV-26917 can be updated.

            People

              marko Marko Mäkelä
              laocius TAO ZHOU
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.