Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-29215

MariaDB server crashes upon serving SST to another node!?

    XMLWordPrintable

Details

    • Bug
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • 10.6.7, 10.6
    • 10.6
    • Galera
    • None

    Description

      Here are the logs from the crash, if anyone can tell me what's going wrong here? Is it a parameter problem or a bug?

      I'm running MariaDB 10.6.7, upgraded a couple of weeks ago from 10.5.15, and they all crash regularly!?

      2022-07-30 19:28:36 2 [Note] WSREP: Server status change synced -> donor
      2022-07-30 19:28:36 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
      2022-07-30 19:28:36 0 [Note] WSREP: async IST sender starting to serve tcp://192.168.0.2:4568 sending 218663-218766, preload starts from 218663
      2022-07-30 19:28:36 0 [Note] WSREP: Donor monitor thread started to monitor
      2022-07-30 19:28:36 0 [Note] WSREP: Running: 'wsrep_sst_mariabackup --role 'donor' --address '192.168.0.2:4444/xtrabackup_sst//1' --local-port '3306' --socket '/run/mysqld/mysqld.sock' --datadir '/var/lib/mysql/' --gtid 'b3d8aa1b-1026-11ed-be37-1610a57a02f8:218903' --gtid-domain-id '300' --binlog '/var/lib/mysql/mysql-bin' --binlog-index '/var/lib/mysql/mysql-bin.index' --mysqld-args --wsrep_start_position=00000000-0000-0000-0000-000000000000:-1,0-0-0'
      2022-07-30 19:28:36 0 [Note] WSREP: IST sender 218663 -> 218766
      2022-07-30 19:28:36 2 [Note] WSREP: sst_donor_thread signaled with 0
      WSREP_SST: [INFO] SSL configuration: CA='', CAPATH='', CERT='', KEY='', MODE='DISABLED', encrypt='0' (20220730 19:28:36.821)
      WSREP_SST: [INFO] Logging all stderr of SST/mariabackup to syslog (20220730 19:28:36.875)
      2022-07-30 19:28:38 0 [Note] WSREP: (50c987a0-908b, 'tcp://0.0.0.0:4567') turning message relay requesting off
      2022-07-30 19:29:21 0 [Note] WSREP: Created page /var/lib/mysql/galera.page.000054 of size 134217696 bytes
      mariadbd: ./wsrep-lib/include/wsrep/client_state.hpp:668: int wsrep::client_state::bf_abort(wsrep::seqno): Assertion `mode_ == m_local || transaction_.is_streaming()' failed.
      220730 19:30:02 [ERROR] mysqld got signal 6 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.
       
      To report this bug, see https://mariadb.com/kb/en/reporting-bugs
       
      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed,
      something is definitely wrong and this may fail.
       
      Server version: 10.6.7-MariaDB-2ubuntu1.1-log
      key_buffer_size=268435456
      read_buffer_size=131072
      max_used_connections=25
      max_threads=5002
      thread_count=41
      It is possible that mysqld could use up to
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 11276717 K  bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
       
      Thread pointer: 0x7f0b7c000c68
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x7f24a8223ca8 thread_stack 0x49000
      ??:0(my_print_stacktrace)[0x560640f31702]
      ??:0(handle_fatal_signal)[0x560640a6c4d8]
      ??:0(__sigaction)[0x7f26b4760520]
      ??:0(pthread_kill)[0x7f26b47b4a7c]
      ??:0(raise)[0x7f26b4760476]
      ??:0(abort)[0x7f26b47467f3]
      /lib/x86_64-linux-gnu/libc.so.6(+0x2871b)[0x7f26b474671b]
      ??:0(__assert_fail)[0x7f26b4757e96]
      ??:0(wsrep_bf_abort(THD*, THD*))[0x560640d164c7]
      ??:0(wsrep_thd_bf_abort)[0x560640d1c5ad]
      ??:0(wsrep_notify_status(wsrep::server_state::state, wsrep::view const*))[0x560640d398a2]
      ??:0(Wsrep_server_service::log_dummy_write_set(wsrep::client_state&, wsrep::ws_meta const&))[0x5606406f7827]
      ??:0(Wsrep_server_service::log_dummy_write_set(wsrep::client_state&, wsrep::ws_meta const&))[0x5606406f7979]
      ??:0(void std::this_thread::sleep_for<long, std::ratio<1l, 1l> >(std::chrono::duration<long, std::ratio<1l, 1l> > const&))[0x560640ddaff7]
      ??:0(void std::this_thread::sleep_for<long, std::ratio<1l, 1l> >(std::chrono::duration<long, std::ratio<1l, 1l> > const&))[0x560640ddb8f6]
      ??:0(wsrep_notify_status(wsrep::server_state::state, wsrep::view const*))[0x560640d3fc12]
      ??:0(handler::ha_write_row(unsigned char const*))[0x560640a79950]
      ??:0(Rows_log_event::write_row(rpl_group_info*, bool))[0x560640b8d924]
      ??:0(Write_rows_log_event::do_exec_row(rpl_group_info*))[0x560640b8df94]
      ??:0(Rows_log_event::do_apply_event(rpl_group_info*))[0x560640b82b87]
      ??:0(wsrep_apply_events(THD*, Relay_log_info*, void const*, unsigned long))[0x560640d14470]
      ??:0(Wsrep_high_priority_service::remove_fragments(wsrep::ws_meta const&))[0x560640cfbfe0]
      ??:0(Wsrep_applier_service::apply_write_set(wsrep::ws_meta const&, wsrep::const_buffer const&, wsrep::mutable_buffer&))[0x560640cfce26]
      ??:0(wsrep::server_state::start_streaming_applier(wsrep::id const&, wsrep::transaction_id const&, wsrep::high_priority_service*))[0x560640fa6efb]
      ??:0(wsrep::wsrep_provider_v26::options[abi:cxx11]() const)[0x560640fb972e]
      /usr/lib/galera/libgalera_smm.so(+0x53b14)[0x7f26a3284b14]
      2022-07-30 19:30:04 202 [Warning] 'proxies_priv' entry '@% root@ovh2.1check.com' ignored in --skip-name-resolve mode.
      /usr/lib/galera/libgalera_smm.so(+0x5bb55)[0x7f26a328cb55]
      src/trx_handle.cpp:396(galera::TrxHandleSlave::apply(void*, wsrep_cb_status (*)(void*, wsrep_ws_handle const*, unsigned int, wsrep_buf const*, wsrep_trx_meta const*, bool*), wsrep_trx_meta const&, bool&))[0x7f26a3298ba8]
      /usr/lib/galera/libgalera_smm.so(+0x87595)[0x7f26a32b8595]
      /usr/lib/galera/libgalera_smm.so(+0x5fcd0)[0x7f26a3290cd0]
      /usr/lib/galera/libgalera_smm.so(+0x463c1)[0x7f26a32773c1]
      ??:0(wsrep::wsrep_provider_v26::run_applier(wsrep::high_priority_service*))[0x560640fb9dd2]
      ??:0(wsrep_bf_abort(THD*, THD*))[0x560640d16577]
      ??:0(start_wsrep_THD(void*))[0x560640d075a3]
      ??:0(MyCTX_nopad::finish(unsigned char*, unsigned int*))[0x560640c96386]
      ??:0(pthread_condattr_setpshared)[0x7f26b47b2b43]
      ??:0(__xmknodat)[0x7f26b4844a00]
       
      Trying to get some variables.
      Some pointers may be invalid and cause the dump to abort.
      

      Attachments

        Activity

          People

            seppo Seppo Jaakola
            ccounotte COUNOTTE CEDRIC
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.