Details

    Description

      140810 19:08:23 [Note] WSREP: Receiving IST: 801 writesets, seqnos 9164663-9165464
      BF-BF X lock conflict
      RECORD LOCKS space id 7259 page no 3 n bits 72 index `GEN_CLUST_INDEX` of table `hpssicccsornlgov_hpssic_prod`.`test_update_f` trx id B64402 lock_mode X locks rec but not gap
      140810 19:08:23 [ERROR] mysqld got signal 6 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.

      To report this bug, see http://kb.askmonty.org/en/reporting-bugs

      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed,
      something is definitely wrong and this may fail.

      Server version: 5.5.38-MariaDB-wsrep-log
      key_buffer_size=33554432
      read_buffer_size=131072
      max_used_connections=0
      max_threads=2050
      thread_count=22
      It is possible that mysqld could use up to
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 4531026 K bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
      Thread pointer: 0x0x7fea89412000
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      140810 19:08:23 [Note] /usr/sbin/mysqld: ready for connections.
      Version: '5.5.38-MariaDB-wsrep-log' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server, wsrep_25.10.r3997
      stack_bottom = 0x7fea8a5f6940 thread_stack 0x48000
      /usr/sbin/mysqld(my_print_stacktrace+0x2b)[0xa95bab]
      /usr/sbin/mysqld(handle_fatal_signal+0x398)[0x6ebc58]
      /lib64/libpthread.so.0[0x3575a0f710]
      /lib64/libc.so.6(gsignal+0x35)[0x3575232925]
      /lib64/libc.so.6(abort+0x175)[0x3575234105]
      /usr/sbin/mysqld[0x50e940]
      /usr/sbin/mysqld[0x92c0ff]
      /usr/sbin/mysqld[0x931799]
      /usr/sbin/mysqld[0x931b0a]
      /usr/sbin/mysqld[0x857d72]
      /usr/sbin/mysqld[0x85be51]
      /usr/sbin/mysqld[0x830233]
      /usr/sbin/mysqld[0x824327]
      /usr/sbin/mysqld[0x83276f]
      /usr/sbin/mysqld(_ZN7handler11ha_rnd_nextEPh+0x15)[0x55edd5]
      /usr/sbin/mysqld(_ZN14Rows_log_event8find_rowEPK14Relay_log_info+0x5d6)[0x7a3d26]
      /usr/sbin/mysqld(_ZN21Update_rows_log_event11do_exec_rowEPK14Relay_log_info+0x67)[0x7a3e07]
      /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0x26a)[0x798e2a]
      /usr/sbin/mysqld(_Z14wsrep_apply_cbPvPKvmjPK14wsrep_trx_meta+0x598)[0x69f988]
      /usr/lib64/galera/libgalera_smm.so(ZNK6galera9TrxHandle5applyEPvPF15wsrep_cb_statusS1_PKvmjPK14wsrep_trx_metaERS6+0xb1)[0x7fed0e6c02c1]
      /usr/lib64/galera/libgalera_smm.so(+0x1aaf95)[0x7fed0e6f7f95]
      /usr/lib64/galera/libgalera_smm.so(_ZN6galera13ReplicatorSMM9apply_trxEPvPNS_9TrxHandleE+0x283)[0x7fed0e6f8e03]
      /usr/lib64/galera/libgalera_smm.so(_ZN6galera13ReplicatorSMM8recv_ISTEPv+0x332)[0x7fed0e705652]
      /usr/lib64/galera/libgalera_smm.so(_ZN6galera13ReplicatorSMM22request_state_transferEPvRK10wsrep_uuidlPKvl+0x2e7)[0x7fed0e706a87]
      /usr/lib64/galera/libgalera_smm.so(_ZN6galera13ReplicatorSMM19process_conf_changeEPvRK15wsrep_view_infoiNS_10Replicator5StateEl+0x923)[0x7fed0e6fc403]
      /usr/lib64/galera/libgalera_smm.so(_ZN6galera15GcsActionSource8dispatchEPvRK10gcs_actionRb+0x587)[0x7fed0e6d6607]
      /usr/lib64/galera/libgalera_smm.so(_ZN6galera15GcsActionSource7processEPvRb+0x63)[0x7fed0e6d6823]
      /usr/lib64/galera/libgalera_smm.so(_ZN6galera13ReplicatorSMM10async_recvEPv+0x93)[0x7fed0e6f53f3]
      /usr/lib64/galera/libgalera_smm.so(galera_recv+0x23)[0x7fed0e70a743]
      /usr/sbin/mysqld[0x6a037f]
      /usr/sbin/mysqld(start_wsrep_THD+0x365)[0x527415]
      /lib64/libpthread.so.0[0x3575a079d1]
      /lib64/libc.so.6(clone+0x6d)[0x35752e8b6d]

      Trying to get some variables.
      Some pointers may be invalid and cause the dump to abort.
      Query (0x0): is an invalid pointer
      Connection ID (thread ID): 2
      Status: NOT_KILLED

      Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=off

      The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
      information that should help you find out what is causing the crash.

      Attachments

        Issue Links

          Activity

            nirbhay_c Nirbhay Choubey (Inactive) added a comment - https://bugs.launchpad.net/percona-xtradb-cluster/+bug/1321767

            I'm experiencing the same issue:

            140827 7:11:29 [Note] WSREP: Member 0.0 (ip-172-31-8-102) requested state transfer from 'any'. Selected 1.0 (ip-172-31-8-213)(SYNCED) as donor.
            140827 7:11:29 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 4497)
            140827 7:11:29 [Note] WSREP: IST request: aa917245-2d1e-11e4-ab4a-fba5d2333991:4491-4497|tcp://172.31.8.102:4568
            140827 7:11:29 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
            140827 7:11:29 [Note] WSREP: Running: 'wsrep_sst_rsync --role 'donor' --address '172.31.8.102:4444/rsync_sst' --auth '(null)' --socket '/var/lib/mysql/mysql.sock' --datadir '/var/lib/mysql/' --defaults-file '/etc/my.cnf' '' --gtid 'aa917245-2d1e-11e4-ab4a-fba5d2333991:4491' --bypass'
            140827 7:11:29 [Note] WSREP: sst_donor_thread signaled with 0
            140827 7:11:29 [Note] WSREP: async IST sender starting to serve tcp://172.31.8.102:4568 sending 4492-4497
            WSREP_SST: [INFO] Bypassing state dump. (20140827 07:11:29.686)
            140827 7:11:31 [Note] WSREP: async IST sender served
            140827 7:11:31 [Note] WSREP: 1.0 (ip-172-31-8-213): State transfer to 0.0 (ip-172-31-8-102) complete.
            140827 7:11:31 [Note] WSREP: Shifting DONOR/DESYNCED -> JOINED (TO: 4497)
            140827 7:11:31 [Note] WSREP: 0.0 (ip-172-31-8-102): State transfer from 1.0 (ip-172-31-8-213) complete.
            140827 7:11:31 [Note] WSREP: Member 1.0 (ip-172-31-8-213) synced with group.
            140827 7:11:31 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 4497)
            140827 7:11:31 [Note] WSREP: Synchronized with group, ready for connections
            140827 7:11:31 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
            140827 7:11:31 [Note] WSREP: Member 0.0 (ip-172-31-8-102) synced with group.
            BF-BF X lock conflict
            RECORD LOCKS space id 180 page no 3 n bits 72 index `GEN_CLUST_INDEX` of table `jackrabbit`.`LOCAL_REVISIONS` trx id 32999 lock_mode X locks rec but not gap
            140827 8:12:33 [ERROR] mysqld got signal 6 ;
            This could be because you hit a bug. It is also possible that this binary
            or one of the libraries it was linked against is corrupt, improperly built,
            or misconfigured. This error can also be caused by malfunctioning hardware.

            To report this bug, see http://kb.askmonty.org/en/reporting-bugs

            We will try our best to scrape up some info that will hopefully help
            diagnose the problem, but since we have already crashed,
            something is definitely wrong and this may fail.

            Server version: 10.0.11-MariaDB-wsrep
            key_buffer_size=134217728
            read_buffer_size=131072
            max_used_connections=14
            max_threads=153
            thread_count=16
            It is possible that mysqld could use up to
            key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 467215 K bytes of memory
            Hope that's ok; if not, decrease some variables in the equation.

            Thread pointer: 0x0x7fbc4db3d008
            Attempting backtrace. You can use the following information to find out
            where mysqld died. If you see no messages after this, something went
            terribly wrong...
            stack_bottom = 0x7fbc7389ccb0 thread_stack 0x48000
            /usr/sbin/mysqld(my_print_stacktrace+0x2b)[0xb8d2ab]
            /usr/sbin/mysqld(handle_fatal_signal+0x398)[0x740b88]
            /lib64/libpthread.so.0(+0xf710)[0x7fbc7360b710]
            /lib64/libc.so.6(gsignal+0x35)[0x7fbc71c6c925]
            /lib64/libc.so.6(abort+0x175)[0x7fbc71c6e105]
            /usr/sbin/mysqld[0x5302a6]
            /usr/sbin/mysqld[0x8c0f11]
            /usr/sbin/mysqld[0x8c41b1]
            /usr/sbin/mysqld[0x8c81f1]
            /usr/sbin/mysqld[0x93ff6f]
            /usr/sbin/mysqld[0x945312]
            /usr/sbin/mysqld[0x89b3a5]
            /usr/sbin/mysqld[0x884927]
            /usr/sbin/mysqld[0x89b01f]
            /usr/sbin/mysqld(_ZN7handler11ha_rnd_nextEPh+0x164)[0x745314]
            /usr/sbin/mysqld(_ZN14Rows_log_event8find_rowEP14rpl_group_info+0x4ce)[0x80541e]
            /usr/sbin/mysqld(_ZN21Update_rows_log_event11do_exec_rowEP14rpl_group_info+0x67)[0x805507]
            /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEP14rpl_group_info+0x251)[0x7f91e1]
            /usr/sbin/mysqld(_Z14wsrep_apply_cbPvPKvmjPK14wsrep_trx_meta+0x4d5)[0x6ee5b5]
            /usr/lib64/galera/libgalera_smm.so(ZNK6galera9TrxHandle5applyEPvPF15wsrep_cb_statusS1_PKvmjPK14wsrep_trx_metaERS6+0xb1)[0x7fbc6c3462c1]
            /usr/lib64/galera/libgalera_smm.so(+0x1aaf95)[0x7fbc6c37df95]
            /usr/lib64/galera/libgalera_smm.so(_ZN6galera13ReplicatorSMM10replay_trxEPNS_9TrxHandleEPv+0x12e)[0x7fbc6c37e85e]
            /usr/lib64/galera/libgalera_smm.so(galera_replay_trx+0x5c)[0x7fbc6c39245c]
            /usr/sbin/mysqld(_Z24wsrep_replay_transactionP3THD+0x2de)[0x6f015e]
            /usr/sbin/mysqld[0x5df310]
            /usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x16d0)[0x5e0a90]
            /usr/sbin/mysqld(_Z10do_commandP3THD+0x132)[0x5e1262]
            /usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x54b)[0x69fb6b]
            /usr/sbin/mysqld(handle_one_connection+0x42)[0x69fc62]
            /usr/sbin/mysqld[0xa831cd]
            /lib64/libpthread.so.0(+0x79d1)[0x7fbc736039d1]
            /lib64/libc.so.6(clone+0x6d)[0x7fbc71d22b6d]

            Trying to get some variables.
            Some pointers may be invalid and cause the dump to abort.
            Query (0x7fbc5f990020): is an invalid pointer
            Connection ID (thread ID): 18
            Status: NOT_KILLED

            Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=off

            spektom Michael Spector added a comment - I'm experiencing the same issue: 140827 7:11:29 [Note] WSREP: Member 0.0 (ip-172-31-8-102) requested state transfer from ' any '. Selected 1.0 (ip-172-31-8-213)(SYNCED) as donor. 140827 7:11:29 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 4497) 140827 7:11:29 [Note] WSREP: IST request: aa917245-2d1e-11e4-ab4a-fba5d2333991:4491-4497|tcp://172.31.8.102:4568 140827 7:11:29 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 140827 7:11:29 [Note] WSREP: Running: 'wsrep_sst_rsync --role 'donor' --address '172.31.8.102:4444/rsync_sst' --auth '(null)' --socket '/var/lib/mysql/mysql.sock' --datadir '/var/lib/mysql/' --defaults-file '/etc/my.cnf' '' --gtid 'aa917245-2d1e-11e4-ab4a-fba5d2333991:4491' --bypass' 140827 7:11:29 [Note] WSREP: sst_donor_thread signaled with 0 140827 7:11:29 [Note] WSREP: async IST sender starting to serve tcp://172.31.8.102:4568 sending 4492-4497 WSREP_SST: [INFO] Bypassing state dump. (20140827 07:11:29.686) 140827 7:11:31 [Note] WSREP: async IST sender served 140827 7:11:31 [Note] WSREP: 1.0 (ip-172-31-8-213): State transfer to 0.0 (ip-172-31-8-102) complete. 140827 7:11:31 [Note] WSREP: Shifting DONOR/DESYNCED -> JOINED (TO: 4497) 140827 7:11:31 [Note] WSREP: 0.0 (ip-172-31-8-102): State transfer from 1.0 (ip-172-31-8-213) complete. 140827 7:11:31 [Note] WSREP: Member 1.0 (ip-172-31-8-213) synced with group. 140827 7:11:31 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 4497) 140827 7:11:31 [Note] WSREP: Synchronized with group, ready for connections 140827 7:11:31 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 140827 7:11:31 [Note] WSREP: Member 0.0 (ip-172-31-8-102) synced with group. BF-BF X lock conflict RECORD LOCKS space id 180 page no 3 n bits 72 index `GEN_CLUST_INDEX` of table `jackrabbit`.`LOCAL_REVISIONS` trx id 32999 lock_mode X locks rec but not gap 140827 8:12:33 [ERROR] mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. To report this bug, see http://kb.askmonty.org/en/reporting-bugs We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Server version: 10.0.11-MariaDB-wsrep key_buffer_size=134217728 read_buffer_size=131072 max_used_connections=14 max_threads=153 thread_count=16 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 467215 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x0x7fbc4db3d008 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x7fbc7389ccb0 thread_stack 0x48000 /usr/sbin/mysqld(my_print_stacktrace+0x2b) [0xb8d2ab] /usr/sbin/mysqld(handle_fatal_signal+0x398) [0x740b88] /lib64/libpthread.so.0(+0xf710) [0x7fbc7360b710] /lib64/libc.so.6(gsignal+0x35) [0x7fbc71c6c925] /lib64/libc.so.6(abort+0x175) [0x7fbc71c6e105] /usr/sbin/mysqld [0x5302a6] /usr/sbin/mysqld [0x8c0f11] /usr/sbin/mysqld [0x8c41b1] /usr/sbin/mysqld [0x8c81f1] /usr/sbin/mysqld [0x93ff6f] /usr/sbin/mysqld [0x945312] /usr/sbin/mysqld [0x89b3a5] /usr/sbin/mysqld [0x884927] /usr/sbin/mysqld [0x89b01f] /usr/sbin/mysqld(_ZN7handler11ha_rnd_nextEPh+0x164) [0x745314] /usr/sbin/mysqld(_ZN14Rows_log_event8find_rowEP14rpl_group_info+0x4ce) [0x80541e] /usr/sbin/mysqld(_ZN21Update_rows_log_event11do_exec_rowEP14rpl_group_info+0x67) [0x805507] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEP14rpl_group_info+0x251) [0x7f91e1] /usr/sbin/mysqld(_Z14wsrep_apply_cbPvPKvmjPK14wsrep_trx_meta+0x4d5) [0x6ee5b5] /usr/lib64/galera/libgalera_smm.so( ZNK6galera9TrxHandle5applyEPvPF15wsrep_cb_statusS1_PKvmjPK14wsrep_trx_metaERS6 +0xb1) [0x7fbc6c3462c1] /usr/lib64/galera/libgalera_smm.so(+0x1aaf95) [0x7fbc6c37df95] /usr/lib64/galera/libgalera_smm.so(_ZN6galera13ReplicatorSMM10replay_trxEPNS_9TrxHandleEPv+0x12e) [0x7fbc6c37e85e] /usr/lib64/galera/libgalera_smm.so(galera_replay_trx+0x5c) [0x7fbc6c39245c] /usr/sbin/mysqld(_Z24wsrep_replay_transactionP3THD+0x2de) [0x6f015e] /usr/sbin/mysqld [0x5df310] /usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x16d0) [0x5e0a90] /usr/sbin/mysqld(_Z10do_commandP3THD+0x132) [0x5e1262] /usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x54b) [0x69fb6b] /usr/sbin/mysqld(handle_one_connection+0x42) [0x69fc62] /usr/sbin/mysqld [0xa831cd] /lib64/libpthread.so.0(+0x79d1) [0x7fbc736039d1] /lib64/libc.so.6(clone+0x6d) [0x7fbc71d22b6d] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0x7fbc5f990020): is an invalid pointer Connection ID (thread ID): 18 Status: NOT_KILLED Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=off
            noc@cloudlynx.ch cloudlynx added a comment -

            we got the same issue

            141021 11:06:28 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
            141021 11:06:28 [Note] WSREP: REPL Protocols: 5 (3, 1)
            141021 11:06:28 [Note] WSREP: Service thread queue flushed.
            141021 11:06:28 [Note] WSREP: Assign initial position for certification: 22496157, protocol version: 3
            141021 11:06:28 [Note] WSREP: Service thread queue flushed.
            141021 11:06:28 [Warning] WSREP: Releasing seqno 22496157 before 22496158 was assigned.
            141021 11:06:30 [Note] WSREP: Member 1.0 (gtbcs-dbdev-vm-8e719eac) requested state transfer from 'any'. Selected 0.0 (lupin-gen-r72012014-4bs1202)(SYNCED) as donor.
            141021 11:06:30 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 22496166)
            141021 11:06:30 [Note] WSREP: IST request: ae79d681-40e2-11e4-ad0f-8afe6a8541b3:13990739-22496157|tcp://10.1.1.90:4568
            141021 11:06:30 [Note] WSREP: IST first seqno 13990740 not found from cache, falling back to SST
            141021 11:06:30 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
            141021 11:06:30 [Note] WSREP: Running: 'wsrep_sst_rsync --role 'donor' --address '10.1.1.90:4444/rsync_sst' --auth 'root:vlPaVrbpx1ZNwf6KcUAI' --socket '/var/lib/mysql/mysql.sock' --datadir '/var/lib/mysql/' --defaults-file '/etc/my.cnf' --gtid 'ae79d681-40e2-11e4-ad0f-8afe6a8541b3:22496166''
            141021 11:06:30 [Note] WSREP: sst_donor_thread signaled with 0
            141021 11:06:30 [Note] WSREP: Flushing tables for SST...
            141021 11:06:30 [Note] WSREP: Provider paused at ae79d681-40e2-11e4-ad0f-8afe6a8541b3:22496167 (12143365)
            141021 11:06:30 [Note] WSREP: Tables flushed.
            141021 11:09:04 [Note] WSREP: resuming provider at 12143365
            141021 11:09:04 [Note] WSREP: Provider resumed.
            141021 11:09:04 [Note] WSREP: 0.0 (lupin-gen-r72012014-4bs1202): State transfer to 1.0 (gtbcs-dbdev-vm-8e719eac) complete.
            141021 11:09:04 [Note] WSREP: Shifting DONOR/DESYNCED -> JOINED (TO: 22497379)
            141021 11:09:05 [Note] WSREP: Member 0.0 (lupin-gen-r72012014-4bs1202) synced with group.
            141021 11:09:05 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 22497393)
            141021 11:09:05 [Note] WSREP: Synchronized with group, ready for connections
            141021 11:09:05 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
            141021 11:09:11 [Note] WSREP: 1.0 (gtbcs-dbdev-vm-8e719eac): State transfer from 0.0 (lupin-gen-r72012014-4bs1202) complete.
            141021 11:09:18 [Note] WSREP: Member 1.0 (gtbcs-dbdev-vm-8e719eac) synced with group.
            BF-BF X lock conflict
            RECORD LOCKS space id 1719 page no 3 n bits 144 index `GEN_CLUST_INDEX` of table `otrs`.`process_id` trx id 2DBE14A lock_mode X locks rec but not gap
            141027 4:10:02 [ERROR] mysqld got signal 6 ;
            This could be because you hit a bug. It is also possible that this binary
            or one of the libraries it was linked against is corrupt, improperly built,
            or misconfigured. This error can also be caused by malfunctioning hardware.

            To report this bug, see http://kb.askmonty.org/en/reporting-bugs

            We will try our best to scrape up some info that will hopefully help
            diagnose the problem, but since we have already crashed,
            something is definitely wrong and this may fail.

            Server version: 5.5.39-MariaDB-wsrep
            key_buffer_size=25165824
            read_buffer_size=131072
            max_used_connections=3
            max_threads=502
            thread_count=50
            It is possible that mysqld could use up to
            key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1126086 K bytes of memory
            Hope that's ok; if not, decrease some variables in the equation.

            Thread pointer: 0x0x7f626d812000
            Attempting backtrace. You can use the following information to find out
            where mysqld died. If you see no messages after this, something went
            terribly wrong...
            stack_bottom = 0x7f625cdb4940 thread_stack 0x48000
            /usr/sbin/mysqld(my_print_stacktrace+0x2b)[0xa95ecb]
            /usr/sbin/mysqld(handle_fatal_signal+0x398)[0x6ebbc8]
            /lib64/libpthread.so.0[0x333ba0f710]
            /lib64/libc.so.6(gsignal+0x35)[0x333b632635]
            /lib64/libc.so.6(abort+0x175)[0x333b633e15]
            /usr/sbin/mysqld[0x50eb40]
            /usr/sbin/mysqld[0x92c23f]
            /usr/sbin/mysqld[0x9318d9]
            /usr/sbin/mysqld[0x931c4a]
            /usr/sbin/mysqld[0x858192]
            /usr/sbin/mysqld[0x85c271]
            /usr/sbin/mysqld[0x830453]
            /usr/sbin/mysqld[0x824467]
            /usr/sbin/mysqld[0x83298f]
            /usr/sbin/mysqld(_ZN7handler11ha_rnd_nextEPh+0x15)[0x55f445]
            /usr/sbin/mysqld(_ZN14Rows_log_event8find_rowEPK14Relay_log_info+0x5d6)[0x7a3a26]
            /usr/sbin/mysqld(_ZN21Delete_rows_log_event11do_exec_rowEPK14Relay_log_info+0x65)[0x7a3e35]
            /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0x26a)[0x798b3a]
            /usr/sbin/mysqld(_Z14wsrep_apply_cbPvPKvmjPK14wsrep_trx_meta+0x5e8)[0x69f848]
            /usr/lib64/galera/libgalera_smm.so(ZNK6galera9TrxHandle5applyEPvPF15wsrep_cb_statusS1_PKvmjPK14wsrep_trx_metaERS6+0xb1)[0x7f641b5542c1]
            /usr/lib64/galera/libgalera_smm.so(+0x1aaf95)[0x7f641b58bf95]
            /usr/lib64/galera/libgalera_smm.so(_ZN6galera13ReplicatorSMM9apply_trxEPvPNS_9TrxHandleE+0x283)[0x7f641b58ce03]
            /usr/lib64/galera/libgalera_smm.so(_ZN6galera13ReplicatorSMM11process_trxEPvPNS_9TrxHandleE+0x45)[0x7f641b58d6f5]
            /usr/lib64/galera/libgalera_smm.so(_ZN6galera15GcsActionSource8dispatchEPvRK10gcs_actionRb+0x2c9)[0x7f641b56a349]
            /usr/lib64/galera/libgalera_smm.so(_ZN6galera15GcsActionSource7processEPvRb+0x63)[0x7f641b56a823]
            /usr/lib64/galera/libgalera_smm.so(_ZN6galera13ReplicatorSMM10async_recvEPv+0x93)[0x7f641b5893f3]
            /usr/lib64/galera/libgalera_smm.so(galera_recv+0x23)[0x7f641b59e743]
            /usr/sbin/mysqld[0x6a023f]
            /usr/sbin/mysqld(start_wsrep_THD+0x365)[0x5278c5]
            /lib64/libpthread.so.0[0x333ba079d1]
            /lib64/libc.so.6(clone+0x6d)[0x333b6e886d]

            Trying to get some variables.
            Some pointers may be invalid and cause the dump to abort.
            Query (0x0): is an invalid pointer
            Connection ID (thread ID): 3
            Status: NOT_KILLED

            Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=on,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=off

            The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
            information that should help you find out what is causing the crash.
            141027 04:10:03 mysqld_safe Number of processes running now: 0
            141027 04:10:03 mysqld_safe WSREP: not restarting wsrep node automatically
            141027 04:10:03 mysqld_safe mysqld from pid file /var/lib/mysql/mysqld.pid ended
            [root@lupin-gen-r72012014-4bs1202 mysql]#
            [root@lupin-gen-r72012014-4bs1202 mysql]# ps -ef | egrep mysql
            root 16296 16093 0 09:08 pts/0 00:00:00 egrep mysql
            [root@lupin-gen-r72012014-4bs1202 mysql]#
            [root@lupin-gen-r72012014-4bs1202 mysql]# rpm -qa | egrep -i "galera|maria"
            MariaDB-client-5.5.39-1.el6.x86_64
            galera-25.3.5-1.rhel6.x86_64
            MariaDB-common-5.5.39-1.el6.x86_64
            MariaDB-compat-5.5.39-1.el6.x86_64
            MariaDB-Galera-server-5.5.39-1.el6.x86_64

            noc@cloudlynx.ch cloudlynx added a comment - we got the same issue 141021 11:06:28 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 141021 11:06:28 [Note] WSREP: REPL Protocols: 5 (3, 1) 141021 11:06:28 [Note] WSREP: Service thread queue flushed. 141021 11:06:28 [Note] WSREP: Assign initial position for certification: 22496157, protocol version: 3 141021 11:06:28 [Note] WSREP: Service thread queue flushed. 141021 11:06:28 [Warning] WSREP: Releasing seqno 22496157 before 22496158 was assigned. 141021 11:06:30 [Note] WSREP: Member 1.0 (gtbcs-dbdev-vm-8e719eac) requested state transfer from ' any '. Selected 0.0 (lupin-gen-r72012014-4bs1202)(SYNCED) as donor. 141021 11:06:30 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 22496166) 141021 11:06:30 [Note] WSREP: IST request: ae79d681-40e2-11e4-ad0f-8afe6a8541b3:13990739-22496157|tcp://10.1.1.90:4568 141021 11:06:30 [Note] WSREP: IST first seqno 13990740 not found from cache, falling back to SST 141021 11:06:30 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 141021 11:06:30 [Note] WSREP: Running: 'wsrep_sst_rsync --role 'donor' --address '10.1.1.90:4444/rsync_sst' --auth 'root:vlPaVrbpx1ZNwf6KcUAI' --socket '/var/lib/mysql/mysql.sock' --datadir '/var/lib/mysql/' --defaults-file '/etc/my.cnf' --gtid 'ae79d681-40e2-11e4-ad0f-8afe6a8541b3:22496166'' 141021 11:06:30 [Note] WSREP: sst_donor_thread signaled with 0 141021 11:06:30 [Note] WSREP: Flushing tables for SST... 141021 11:06:30 [Note] WSREP: Provider paused at ae79d681-40e2-11e4-ad0f-8afe6a8541b3:22496167 (12143365) 141021 11:06:30 [Note] WSREP: Tables flushed. 141021 11:09:04 [Note] WSREP: resuming provider at 12143365 141021 11:09:04 [Note] WSREP: Provider resumed. 141021 11:09:04 [Note] WSREP: 0.0 (lupin-gen-r72012014-4bs1202): State transfer to 1.0 (gtbcs-dbdev-vm-8e719eac) complete. 141021 11:09:04 [Note] WSREP: Shifting DONOR/DESYNCED -> JOINED (TO: 22497379) 141021 11:09:05 [Note] WSREP: Member 0.0 (lupin-gen-r72012014-4bs1202) synced with group. 141021 11:09:05 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 22497393) 141021 11:09:05 [Note] WSREP: Synchronized with group, ready for connections 141021 11:09:05 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 141021 11:09:11 [Note] WSREP: 1.0 (gtbcs-dbdev-vm-8e719eac): State transfer from 0.0 (lupin-gen-r72012014-4bs1202) complete. 141021 11:09:18 [Note] WSREP: Member 1.0 (gtbcs-dbdev-vm-8e719eac) synced with group. BF-BF X lock conflict RECORD LOCKS space id 1719 page no 3 n bits 144 index `GEN_CLUST_INDEX` of table `otrs`.`process_id` trx id 2DBE14A lock_mode X locks rec but not gap 141027 4:10:02 [ERROR] mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. To report this bug, see http://kb.askmonty.org/en/reporting-bugs We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Server version: 5.5.39-MariaDB-wsrep key_buffer_size=25165824 read_buffer_size=131072 max_used_connections=3 max_threads=502 thread_count=50 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1126086 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x0x7f626d812000 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x7f625cdb4940 thread_stack 0x48000 /usr/sbin/mysqld(my_print_stacktrace+0x2b) [0xa95ecb] /usr/sbin/mysqld(handle_fatal_signal+0x398) [0x6ebbc8] /lib64/libpthread.so.0 [0x333ba0f710] /lib64/libc.so.6(gsignal+0x35) [0x333b632635] /lib64/libc.so.6(abort+0x175) [0x333b633e15] /usr/sbin/mysqld [0x50eb40] /usr/sbin/mysqld [0x92c23f] /usr/sbin/mysqld [0x9318d9] /usr/sbin/mysqld [0x931c4a] /usr/sbin/mysqld [0x858192] /usr/sbin/mysqld [0x85c271] /usr/sbin/mysqld [0x830453] /usr/sbin/mysqld [0x824467] /usr/sbin/mysqld [0x83298f] /usr/sbin/mysqld(_ZN7handler11ha_rnd_nextEPh+0x15) [0x55f445] /usr/sbin/mysqld(_ZN14Rows_log_event8find_rowEPK14Relay_log_info+0x5d6) [0x7a3a26] /usr/sbin/mysqld(_ZN21Delete_rows_log_event11do_exec_rowEPK14Relay_log_info+0x65) [0x7a3e35] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0x26a) [0x798b3a] /usr/sbin/mysqld(_Z14wsrep_apply_cbPvPKvmjPK14wsrep_trx_meta+0x5e8) [0x69f848] /usr/lib64/galera/libgalera_smm.so( ZNK6galera9TrxHandle5applyEPvPF15wsrep_cb_statusS1_PKvmjPK14wsrep_trx_metaERS6 +0xb1) [0x7f641b5542c1] /usr/lib64/galera/libgalera_smm.so(+0x1aaf95) [0x7f641b58bf95] /usr/lib64/galera/libgalera_smm.so(_ZN6galera13ReplicatorSMM9apply_trxEPvPNS_9TrxHandleE+0x283) [0x7f641b58ce03] /usr/lib64/galera/libgalera_smm.so(_ZN6galera13ReplicatorSMM11process_trxEPvPNS_9TrxHandleE+0x45) [0x7f641b58d6f5] /usr/lib64/galera/libgalera_smm.so(_ZN6galera15GcsActionSource8dispatchEPvRK10gcs_actionRb+0x2c9) [0x7f641b56a349] /usr/lib64/galera/libgalera_smm.so(_ZN6galera15GcsActionSource7processEPvRb+0x63) [0x7f641b56a823] /usr/lib64/galera/libgalera_smm.so(_ZN6galera13ReplicatorSMM10async_recvEPv+0x93) [0x7f641b5893f3] /usr/lib64/galera/libgalera_smm.so(galera_recv+0x23) [0x7f641b59e743] /usr/sbin/mysqld [0x6a023f] /usr/sbin/mysqld(start_wsrep_THD+0x365) [0x5278c5] /lib64/libpthread.so.0 [0x333ba079d1] /lib64/libc.so.6(clone+0x6d) [0x333b6e886d] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0x0): is an invalid pointer Connection ID (thread ID): 3 Status: NOT_KILLED Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=on,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=off The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 141027 04:10:03 mysqld_safe Number of processes running now: 0 141027 04:10:03 mysqld_safe WSREP: not restarting wsrep node automatically 141027 04:10:03 mysqld_safe mysqld from pid file /var/lib/mysql/mysqld.pid ended [root@lupin-gen-r72012014-4bs1202 mysql] # [root@lupin-gen-r72012014-4bs1202 mysql] # ps -ef | egrep mysql root 16296 16093 0 09:08 pts/0 00:00:00 egrep mysql [root@lupin-gen-r72012014-4bs1202 mysql] # [root@lupin-gen-r72012014-4bs1202 mysql] # rpm -qa | egrep -i "galera|maria" MariaDB-client-5.5.39-1.el6.x86_64 galera-25.3.5-1.rhel6.x86_64 MariaDB-common-5.5.39-1.el6.x86_64 MariaDB-compat-5.5.39-1.el6.x86_64 MariaDB-Galera-server-5.5.39-1.el6.x86_64
            nirbhay_c Nirbhay Choubey (Inactive) added a comment - https://bugs.launchpad.net/codership-mysql/+bug/1280896

            People

              nirbhay_c Nirbhay Choubey (Inactive)
              nirbhay_c Nirbhay Choubey (Inactive)
              Votes:
              3 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.