Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-23101

SIGSEGV in lock_rec_unlock() when Galera is enabled

    XMLWordPrintable

Details

    Description

      Server is crashing few times a day with the following stacktrace:

      /lib64/libc.so.6(+0x115b62)[0x7fcadad58b62]
      /lib64/libc.so.6(+0x117947)[0x7fcadad5a947]
      /usr/sbin/mysqld(my_addr_resolve+0xda)[0x55f2dc88214a]
      /usr/sbin/mysqld(my_print_stacktrace+0x1c2)[0x55f2dc86a862]
      /usr/sbin/mysqld(handle_fatal_signal+0x30f)[0x55f2dc300f7f]
      /lib64/libpthread.so.0(+0xf5d0)[0x7fcadc9a65d0]
      /usr/sbin/mysqld(+0xa9a908)[0x55f2dc4eb908]
      /usr/sbin/mysqld(+0xa9d47e)[0x55f2dc4ee47e]
      /usr/sbin/mysqld(+0xb22626)[0x55f2dc573626]
      /usr/sbin/mysqld(_Z12mysql_updateP3THDP10TABLE_LISTR4ListI4ItemES6_PS4_jP8st_orderybPySA_+0x2110)[0x55f2dc1a5c60]
      /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x2ff5)[0x55f2dc0fcd15]
      /usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_statebb+0x1fb)[0x55f2dc1028ab]
      /usr/sbin/mysqld(+0x5a0090)[0x55f2dbff1090]
      /usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjbb+0x28f8)[0x55f2dc1059d8]
      /usr/sbin/mysqld(_Z10do_commandP3THD+0x109)[0x55f2dc106049]
      /usr/sbin/mysqld(_Z24do_handle_one_connectionP7CONNECT+0x1fa)[0x55f2dc1e3bfa]
      /usr/sbin/mysqld(handle_one_connection+0x3d)[0x55f2dc1e3cdd]
      /usr/sbin/mysqld(+0xdcb78d)[0x55f2dc81c78d]
      /lib64/libpthread.so.0(+0x7dd5)[0x7fcadc99edd5]
      /lib64/libc.so.6(clone+0x6d)[0x7fcadad40ead]
      

      3 Top frames, resolved

      /usr/sbin/mysqld(+0xa9a908)[0x55f2dc4eb908] lock_rec_has_to_wait_in_queue(ib_lock_t const*) (/usr/src/debug/MariaDB-10.4.13/src_0/storage/innobase/include/lock0priv.ic:79).
       /usr/sbin/mysqld(0xa9d47e)[0x55f2dc4ee47e] lock_rec_unlock(trx_t*, buf_block_t const*, unsigned char const*, lock_mode) (/usr/src/debug/MariaDB-10.4.13/src_0/storage/innobase/lock/lock0lock.cc:4214).
      /usr/sbin/mysqld(+0xb22626)[0x55f2dc573626] row_unlock_for_mysql(row_prebuilt_t*, unsigned long) (/usr/src/debug/MariaDB-10.4.13/src_0/storage/innobase/row/row0mysql.cc:2093).
      

      We tried to set up for core dump generation, but it appears it crashes trying to access memory and therefore no core dump is generated:

      200703 13:10:16 [ERROR] mysqld got signal 11 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.
       
      To report this bug, see https://mariadb.com/kb/en/reporting-bugs
       
      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed, 
      something is definitely wrong and this may fail.
       
      Server version: 10.4.13-MariaDB-log
      key_buffer_size=134217728
      read_buffer_size=2097152
      max_used_connections=10223
      max_threads=20002
      thread_count=8406
      It is possible that mysqld could use up to 
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 123517134 K  bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
       
      Thread pointer: 0x7faf48086b98
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x7facc4190cc0 thread_stack 0x49000
      *** buffer overflow detected ***: /usr/sbin/mysqld terminated
      ======= Backtrace: =========
      /lib64/libc.so.6(__fortify_fail+0x37)[0x7fcadad5a9e7]
      /lib64/libc.so.6(+0x115b62)[0x7fcadad58b62]
      /lib64/libc.so.6(+0x117947)[0x7fcadad5a947]
      /usr/sbin/mysqld(my_addr_resolve+0xda)[0x55f2dc88214a]
      /usr/sbin/mysqld(my_print_stacktrace+0x1c2)[0x55f2dc86a862]
      /usr/sbin/mysqld(handle_fatal_signal+0x30f)[0x55f2dc300f7f]
      /lib64/libpthread.so.0(+0xf5d0)[0x7fcadc9a65d0]
      /usr/sbin/mysqld(+0xa9a908)[0x55f2dc4eb908]
      /usr/sbin/mysqld(+0xa9d47e)[0x55f2dc4ee47e]
      /usr/sbin/mysqld(+0xb22626)[0x55f2dc573626]
      /usr/sbin/mysqld(_Z12mysql_updateP3THDP10TABLE_LISTR4ListI4ItemES6_PS4_jP8st_orderybPySA_+0x2110)[0x55f2dc1a5c60]
      /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x2ff5)[0x55f2dc0fcd15]
      /usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_statebb+0x1fb)[0x55f2dc1028ab]
      /usr/sbin/mysqld(+0x5a0090)[0x55f2dbff1090]
      /usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjbb+0x28f8)[0x55f2dc1059d8]
      /usr/sbin/mysqld(_Z10do_commandP3THD+0x109)[0x55f2dc106049]
      /usr/sbin/mysqld(_Z24do_handle_one_connectionP7CONNECT+0x1fa)[0x55f2dc1e3bfa]
      /usr/sbin/mysqld(handle_one_connection+0x3d)[0x55f2dc1e3cdd]
      /usr/sbin/mysqld(+0xdcb78d)[0x55f2dc81c78d]
      /lib64/libpthread.so.0(+0x7dd5)[0x7fcadc99edd5]
      /lib64/libc.so.6(clone+0x6d)[0x7fcadad40ead]
      ======= Memory map: ========
      55f2dba51000-55f2dceca000 r-xp 00000000 fd:00 436210356                  /usr/sbin/mysqld
      55f2dd0ca000-55f2dd1fd000 r--p 01479000 fd:00 436210356                  /usr/sbin/mysqld
      55f2dd1fd000-55f2dd2b7000 rw-p 015ac000 fd:00 436210356                  /usr/sbin/mysqld
      55f2dd2b7000-55f2ddb66000 rw-p 00000000 00:00 0 
      55f2de4cb000-55f348284000 rw-p 00000000 00:00 0                          [heap]
      7fabcc000000-7fabcdb8a000 rw-p 00000000 00:00 0 
       
      [ ... ]
       
      7fcadcdd6000-7fcadcdd7000 rw-p 00000000 00:00 0 
      7fff76cfc000-7fff76d1d000 rw-p 00000000 00:00 0                          [stack]
      7fff76d84000-7fff76d86000 r-xp 00000000 00:00 0                          [vdso]
      ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
      Fatal signal 6 while backtracing
      
      

      Attachments

        1. thd.txt
          71 kB
          Jan Lindström
        2. trx.txt
          8 kB
          Jan Lindström

        Issue Links

          Activity

            People

              jplindst Jan Lindström (Inactive)
              rpizzi Rick Pizzi (Inactive)
              Votes:
              5 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.