Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-35795

mysqld Signal 11 Shutdown Due to Page Corruption in MariaDB 10.11.7

    XMLWordPrintable

Details

    • Bug
    • Status: Needs Feedback (View Workflow)
    • Major
    • Resolution: Unresolved
    • 10.11.7
    • 10.11
    • MariaDB 10.11.7

      Rocky Linux 8.6 / kernel : 4.18.0-372.9.1.el8.x86_64
      Rocky Linux 8.10 kernel : 4.18.0-553.el8_10.x86_64

    Description

      Hello,

      I am currently using MariaDB version 10.11.7.

      Since upgrading to this version, I have observed that certain UPDATE operations occasionally lead to significant issues, including deadlocks or excessive resource consumption. These situations eventually result in corrupted pages in MariaDB, followed by a mysqld got signal 11 shutdown.

      Before the upgrade, I was using version 10.5.17, and such issues never occurred. These problems have only started appearing after upgrading to version 10.11.7.

      Could you please help me identify the root cause of this issue?
      Is this a known bug in version 10.11.7?
      If so, is there a fix or a specific version where this issue has been resolved?

      Below is a relevant call stack for reference, and I have attached multiple related files for your review.

      Thank you in advance for your assistance.

      Best regards,

      241016 14:37:53 [ERROR] mysqld got signal 11 ;
      Sorry, we probably made a mistake, and this is a bug.
       
      Your assistance in bug reporting will enable us to fix this for the next release.
      To report this bug, see https://mariadb.com/kb/en/reporting-bugs
       
      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed, 
      something is definitely wrong and this may fail.
       
      Server version: 10.11.7-MariaDB-log source revision: 87e13722a95af5d9378d990caf48cb6874439347
      key_buffer_size=25165824
      read_buffer_size=2097152
      max_used_connections=39
      max_threads=65537
      thread_count=22
      It is possible that mysqld could use up to 
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 268477510 K  bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
       
      Thread pointer: 0x7fdba8000d38
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x0 thread_stack 0x49000
      2024-10-16 14:37:54 0x7fdbbea4f700  InnoDB: Assertion failure in file /home/buildbot/buildbot/build/storage/innobase/buf/buf0lru.cc line 281
      InnoDB: Failing assertion: !block->page.in_file()
      InnoDB: We intentionally generate a memory trap.
      InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
      InnoDB: If you get repeated assertion failures or crashes, even
      InnoDB: immediately after the mariadbd startup, there may be
      InnoDB: corruption in the InnoDB tablespace. Please refer to
      InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
      InnoDB: about forcing recovery.
      Fatal signal 6 while backtracing
       
      ...
       
      Thread pointer: 0x7f92a0001118
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x0 thread_stack 0x49000
      mysys/stacktrace.c:216(my_print_stacktrace)[0x556ffb8adfde]
      sql/signal_handler.cc:238(handle_fatal_signal)[0x556ffb296857]
      /lib64/libpthread.so.0(+0x12ce0)[0x7f95ad234ce0]
      fil/fil0fil.cc:2776(fil_space_t::io(IORequest const&, unsigned long, unsigned long, void*, buf_page_t*))[0x556ffb76bd04]
      buf/buf0rea.cc:324(buf_read_page_low)[0x556ffb72d5b5]
      buf/buf0buf.cc:2827(buf_page_get_low(page_id_t, unsigned long, unsigned long, buf_block_t*, unsigned long, mtr_t*, dberr_t*, bool))[0x556ffb7137a4]
      trx/trx0undo.cc:1385(buf_block_t* trx_undo_assign_low<false>(trx_t*, trx_rseg_t*, trx_undo_t**, mtr_t*, dberr_t*))[0x556ffb6d77bd]
      trx/trx0rec.cc:1898(trx_undo_report_row_operation(que_thr_t*, dict_index_t*, dtuple_t const*, upd_t const*, unsigned long, unsigned char const*, unsigned short const*, unsigned long*))[0x556ffb6afb7f]
      btr/btr0cur.cc:2834(btr_cur_upd_lock_and_undo)[0x556ffb6fc53c]
      btr/btr0cur.cc:3616(btr_cur_optimistic_update(unsigned long, btr_cur_t*, unsigned short**, mem_block_info_t**, upd_t const*, unsigned long, que_thr_t*, unsigned long, mtr_t*))[0x556ffb6fc75c]
      row/row0ins.cc:328(row_ins_clust_index_entry_by_modify)[0x556ffb64a2e6]
      row/row0ins.cc:3281(row_ins_clust_index_entry(dict_index_t*, dtuple_t*, que_thr_t*, unsigned long))[0x556ffb64e866]
      row/row0ins.cc:3409(row_ins_index_entry)[0x556ffb64f18a]
      que/que0que.cc:567(que_thr_step)[0x556ffb62ff06]
      que/que0que.cc:408(que_graph_free)[0x556ffb630230]
      dict/dict0stats.cc:3349(dict_stats_save(dict_table_t*, unsigned long const*))[0x556ffb7619ba]
      dict/dict0stats.cc:4136(dict_stats_update(dict_table_t*, dict_stats_upd_option_t))[0x556ffb76263f]
      dict/dict0stats_bg.cc:349(dict_stats_process_entry_from_recalc_pool)[0x556ffb764d63]
      tpool/tpool_generic.cc:342(tpool::thread_pool_generic::timer_generic::execute(void*))[0x556ffb7dbc00]
      tpool/task.cc:38(tpool::task::execute())[0x556ffb7dcbfb]
      tpool/tpool_generic.cc:583(tpool::thread_pool_generic::worker_main(tpool::worker_data*))[0x556ffb7d9f79]
      /lib64/libstdc++.so.6(+0xc2f54)[0x7f95acd4bf54]
      /lib64/libpthread.so.0(+0x81cf)[0x7f95ad22a1cf]
      /lib64/libc.so.6(clone+0x43)[0x7f95ac57bd83]
       
      ....
       
      Thread pointer: 0x7f6b9001b7b8
       
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x7f6c2152e440 thread_stack 0x49000
      mysys/stacktrace.c:216(my_print_stacktrace)[0x55acb9c62fde]
      sql/signal_handler.cc:238(handle_fatal_signal)[0x55acb964b857]
      /lib64/libpthread.so.0(+0x12ce0)[0x7f72715fcce0]
      fil/fil0fil.cc:2776(fil_space_t::io(IORequest const&, unsigned long, unsigned long, void*, buf_page_t*))[0x55acb9b20d04]
      buf/buf0rea.cc:324(buf_read_page_low)[0x55acb9ae25b5]
      buf/buf0buf.cc:2827(buf_page_get_low(page_id_t, unsigned long, unsigned long, buf_block_t*, unsigned long, mtr_t*, dberr_t*, bool))[0x55acb9ac87a4]
      trx/trx0undo.cc:1385(buf_block_t* trx_undo_assign_low<false>(trx_t*, trx_rseg_t*, trx_undo_t**, mtr_t*, dberr_t*))[0x55acb9a8c7bd]
      trx/trx0rec.cc:1898(trx_undo_report_row_operation(que_thr_t*, dict_index_t*, dtuple_t const*, upd_t const*, unsigned long, unsigned char const*, unsigned short const*, unsigned long*))[0x55acb9a64b7f]
      btr/btr0cur.cc:2254(btr_cur_ins_lock_and_undo)[0x55acb9aae21a]
      row/row0ins.cc:2892(row_ins_clust_index_entry_low(unsigned long, btr_latch_mode, dict_index_t*, unsigned long, dtuple_t*, unsigned long, que_thr_t*))[0x55acb99fe4af]
      row/row0ins.cc:3281(row_ins_clust_index_entry(dict_index_t*, dtuple_t*, que_thr_t*, unsigned long))[0x55acb9a03866]
      row/row0ins.cc:3409(row_ins_index_entry)[0x55acb9a0418a]
      row/row0mysql.cc:1319(row_insert_for_mysql(unsigned char const*, row_prebuilt_t*, ins_mode_t))[0x55acb9a15bca]
      handler/ha_innodb.cc:7844(ha_innobase::write_row(unsigned char const*))[0x55acb994dd6e]
      sql/handler.cc:7662(handler::ha_write_row(unsigned char const*))[0x55acb965ad0f]
      sql/rpl_gtid.cc:751(rpl_slave_state::record_gtid(THD*, rpl_gtid const*, unsigned long long, bool, bool, void**))[0x55acb95785a8]
      sql/log_event_server.cc:4321(Xid_apply_log_event::do_record_gtid(THD*, rpl_group_info*, bool, void**, bool))[0x55acb977d566]
      sql/log_event_server.cc:4399(Xid_apply_log_event::do_apply_event(rpl_group_info*))[0x55acb977d99f]
      sql/log_event.cc:4220(Log_event::apply_event(rpl_group_info*))[0x55acb976bc07]
      sql/slave.cc:3918(apply_event_and_update_pos_apply(Log_event*, THD*, rpl_group_info*, int))[0x55acb934ace8]
      sql/slave.cc:4513(exec_relay_log_event)[0x55acb93547bb]
      perfschema/pfs.cc:2204(pfs_spawn_thread)[0x55acb989cc3c]
      /lib64/libpthread.so.0(+0x81cf)[0x7f72715f21cf]
      /lib64/libc.so.6(clone+0x43)[0x7f7270943d83]
      

      Attachments

        1. !block.page.in_file().err
          7 kB
        2. bpage.can_relocate.err
          4 kB
        3. buf_read_page_low.err
          6 kB
        4. fil_space_t.err
          8 kB
        5. rpl_slave_state.err
          6 kB

        Activity

          People

            marko Marko Mäkelä
            SeongGuk Kim SeongGuk Kim
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.