Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-25579

MariaDB 10.4.18 crashes with InnoDB: Conflicting lock on table

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 10.4.18
    • Fix Version/s: 10.4.19
    • Component/s: Galera
    • Labels:
      None
    • Environment:
      Ubuntu 20.04, hosted on GCP.
      MariaDB running in Galera cluster (nodes b, c and d), node d is slave to a MariaDB 10.3 Galera cluster (Ubuntu 18.04)

      Description

      MariaDB on node C crashed yesterday, did a SST afterwards to perform recovery. The MariaDB logs stated to file a bug report, doing so accordingly.
      The log contained this:

      {{2021-05-02 0:01:50 8 [ERROR] InnoDB: Conflicting lock on table: `my_db`.`property_alias` index: PRIMARY that has lock
      RECORD LOCKS space id 981076 page no 6 n bits 336 index PRIMARY of table `my_db`.`property_alias` trx id 42949686941 lock_mode X locks rec but not gap
      Record lock, heap no 56 PHYSICAL RECORD: n_fields 10; compact format; info bits 32
      0: len 4; hex 800012d1; asc ;;
      1: len 6; hex 000a0000369d; asc 6 ;;
      2: len 7; hex 240000402c0890; asc $ @, ;;
      3: len 4; hex 80000030; asc 0;;
      4: len 7; hex 4472656e746865; asc Drenthe;;
      5: len 7; hex 4472656e746865; asc Drenthe;;
      6: len 4; hex 8000003c; asc <;;
      7: SQL NULL;
      8: len 1; hex 80; asc ;;
      9: SQL NULL;

      2021-05-02 0:01:50 8 [ERROR] InnoDB: WSREP state:
      2021-05-02 0:01:50 8 [ERROR] WSREP: Thread BF trx_id: 42949686944 thread: 8 seqno: 52272178 client_state: exec client_mode: high priority transaction_mode: executing applier: 1 toi: 0 local: 0 query: DELETE FROM property WHERE id=48 LIMIT 500�ύ`
      2021-05-02 0:01:50 8 [ERROR] WSREP: Thread BF trx_id: 42949686941 thread: 12 seqno: 52272175 client_state: exec client_mode: high priority transaction_mode: executing applier: 1 toi: 0 local: 0 query: DELETE FROM property_alias WHERE property_id=48 LIMIT 500�ύ`
      2021-05-02 00:01:50 0x7ff79c076700 InnoDB: Assertion failure in file /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/lock/lock0lock.cc line 706
      InnoDB: We intentionally generate a memory trap.
      InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
      InnoDB: If you get repeated assertion failures or crashes, even
      InnoDB: immediately after the mysqld startup, there may be
      InnoDB: corruption in the InnoDB tablespace. Please refer to
      InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
      InnoDB: about forcing recovery.
      210502 0:01:50 [ERROR] mysqld got signal 6 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.

      To report this bug, see https://mariadb.com/kb/en/reporting-bugs

      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed,
      something is definitely wrong and this may fail.

      Server version: 10.4.18-MariaDB-1:10.4.18+maria~focal-log
      key_buffer_size=134217728
      read_buffer_size=131072
      max_used_connections=3
      max_threads=386
      thread_count=11
      It is possible that mysqld could use up to
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 980533 K bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.

      Thread pointer: 0x7ff21c000c48
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x7ff79c075dd8 thread_stack 0x49000
      /usr/sbin/mysqld(my_print_stacktrace+0x32)[0x556d00484192]
      /usr/sbin/mysqld(handle_fatal_signal+0x55d)[0x556cfff3b54d]
      /lib/x86_64-linux-gnu/libpthread.so.0(+0x153c0)[0x7ffba43473c0]
      /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcb)[0x7ffba3a3418b]
      /lib/x86_64-linux-gnu/libc.so.6(abort+0x12b)[0x7ffba3a13859]
      /usr/sbin/mysqld(+0x5b0d47)[0x556cffc08d47]
      /usr/sbin/mysqld(+0x58d3b4)[0x556cffbe53b4]
      /usr/sbin/mysqld(+0xafc1d1)[0x556d001541d1]
      /usr/sbin/mysqld(+0xb01a52)[0x556d00159a52]
      /usr/sbin/mysqld(+0xb02e26)[0x556d0015ae26]
      /usr/sbin/mysqld(+0xb5aa40)[0x556d001b2a40]
      /usr/sbin/mysqld(+0xb8fdc7)[0x556d001e7dc7]
      /usr/sbin/mysqld(+0xb94a98)[0x556d001eca98]
      /usr/sbin/mysqld(+0xb963f7)[0x556d001ee3f7]
      /usr/sbin/mysqld(+0xb7057d)[0x556d001c857d]
      /usr/sbin/mysqld(+0xaaf062)[0x556d00107062]
      /usr/sbin/mysqld(_ZN7handler13ha_delete_rowEPKh+0x1b1)[0x556cfff49a71]
      /usr/sbin/mysqld(_ZN21Delete_rows_log_event11do_exec_rowEP14rpl_group_info+0x131)[0x556d00056ea1]
      /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEP14rpl_group_info+0x23f)[0x556d0004a31f]
      /usr/sbin/mysqld(_Z18wsrep_apply_eventsP3THDP14Relay_log_infoPKvm+0x1b9)[0x556cffeb5779]
      /usr/sbin/mysqld(_ZN21Wsrep_applier_service15apply_write_setERKN5wsrep7ws_metaERKNS0_12const_bufferERNS0_14mutable_bufferE+0xa3)[0x556cffe9da03]
      /usr/sbin/mysqld(+0xe99ca0)[0x556d004f1ca0]
      /usr/sbin/mysqld(_ZN5wsrep12server_state8on_applyERNS_21high_priority_serviceERKNS_9ws_handleERKNS_7ws_metaERKNS_12const_bufferE+0xc1)[0x556d004f2cf1]
      /usr/sbin/mysqld(+0xeac04c)[0x556d0050404c]
      /usr/lib/galera/libgalera_smm.so(+0x6241b)[0x7ffb8544f41b]
      /usr/lib/galera/libgalera_smm.so(+0x7426e)[0x7ffb8546126e]
      /usr/lib/galera/libgalera_smm.so(+0x76dd8)[0x7ffb85463dd8]
      /usr/lib/galera/libgalera_smm.so(+0xaaa82)[0x7ffb85497a82]
      /usr/lib/galera/libgalera_smm.so(+0xaac0b)[0x7ffb85497c0b]
      /usr/lib/galera/libgalera_smm.so(+0xaae02)[0x7ffb85497e02]
      /usr/lib/galera/libgalera_smm.so(+0x777d0)[0x7ffb854647d0]
      /usr/lib/galera/libgalera_smm.so(+0x4cdf1)[0x7ffb85439df1]
      /usr/sbin/mysqld(_ZN5wsrep18wsrep_provider_v2611run_applierEPNS_21high_priority_serviceE+0x12)[0x556d00504592]
      /usr/sbin/mysqld(+0x85f5f3)[0x556cffeb75f3]
      /usr/sbin/mysqld(_Z15start_wsrep_THDPv+0x2ec)[0x556cffea889c]
      /usr/sbin/mysqld(+0xdda376)[0x556d00432376]
      /lib/x86_64-linux-gnu/libpthread.so.0(+0x9609)[0x7ffba433b609]
      /lib/x86_64-linux-gnu/libc.so.6(clone+0x43)[0x7ffba3b10293]

      Trying to get some variables.
      Some pointers may be invalid and cause the dump to abort.
      Query (0x7ff9b362f33b): DELETE FROM property WHERE id=48 LIMIT 500

      Connection ID (thread ID): 8
      Status: NOT_KILLED

      Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on

      The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
      information that should help you find out what is causing the crash.
      Writing a core file...
      Working directory at /data/mysql/current
      Resource Limits:
      Limit Soft Limit Hard Limit Units
      Max cpu time unlimited unlimited seconds
      Max file size unlimited unlimited bytes
      Max data size unlimited unlimited bytes
      Max stack size 8388608 unlimited bytes
      Max core file size 0 unlimited bytes
      Max resident set unlimited unlimited bytes
      Max processes 128433 128433 processes
      Max open files 32767 32767 files
      Max locked memory 65536 65536 bytes
      Max address space unlimited unlimited bytes
      Max file locks unlimited unlimited locks
      Max pending signals 128433 128433 signals
      Max msgqueue size 819200 819200 bytes
      Max nice priority 0 0
      Max realtime priority 0 0
      Max realtime timeout unlimited unlimited us
      Core pattern: |/usr/share/apport/apport %p %s %c %d %P %E

      2021-05-02 0:03:50 0 [Note] WSREP: Loading provider /usr/lib/galera/libgalera_smm.so initial position: 20982d02-a280-11eb-8f51-be4af2f78621:52272167
      2021-05-02 0:03:50 0 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so'
      2021-05-02 0:03:50 0 [Note] WSREP: wsrep_load(): Galera 4.7(ree4f10fc) by Codership Oy <info@codership.com> loaded successfully.}}

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              jplindst Jan Lindström
              Reporter:
              lieuwe Lieuwe Hummel
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: