[MDEV-10650] MariaDB does not start with Galera - signal 11 Created: 2016-08-23  Updated: 2016-08-24  Resolved: 2016-08-24

Status: Closed
Project: MariaDB Server
Component/s: wsrep
Affects Version/s: 10.0.16
Fix Version/s: 10.0.27-galera

Type: Bug Priority: Major
Reporter: Florijan Hamzic Assignee: Nirbhay Choubey (Inactive)
Resolution: Duplicate Votes: 0
Labels: None
Environment:

Archlinux
Linux vserver3727 4.6.4-1-ARCH #1 SMP PREEMPT Mon Jul 11 19:12:32 CEST 2016 x86_64 GNU/Linux

extra/mariadb 10.1.16-1
galera-release_25.3.17

Attachments: File core.mysqld.89.b18796af17b5441c92d2dbfbd65219d7.17489.1471955514000000000000.lz4     Text File more_details.txt    
Issue Links:
Duplicate
is duplicated by MDEV-10335 Galera initialization breaks when com... Closed

 Description   

Hello,

i can't get galera to work with MariaDB and Archlinux (x64), iam using:

MariaDB: 10.1.16-1 official package from Archlinux also tried it with own compiled same result
Galera: release_25.3.17 compiled with scons (no errors or anything fine build)

I practices the things like described in MariaDB - Galera Get Started :

I started mysqld with this script (replaced mariadb):
https://github.com/mariadb-corporation/mariadb-columnstore-server/blob/master/scripts/galera_new_cluster.sh

But the mariadb always crashes with signal 11 code, any idea?

i get this:

Aug 23 11:07:25 vserver3727 systemd[1]: Starting MariaDB database server...
 
Aug 23 11:07:31 vserver3727 sh[6978]: WSREP: Recovered position 00000000-0000-0000-0000-000000000000:-1
 
Aug 23 11:07:31 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] /usr/sbin/mysqld (mysqld 10.1.16-MariaDB) starting as process 7104 ...
 
Aug 23 11:07:31 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: Read nil XID from storage engines, skipping position init
 
Aug 23 11:07:31 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so'
 
Aug 23 11:07:31 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: wsrep_load(): Galera 3.17(rXXXX) by Codership Oy <info@codership.com> loaded successfully.
 
Aug 23 11:07:31 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: CRC-32C: using hardware acceleration.
 
Aug 23 11:07:31 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Warning] WSREP: Could not open state file for reading: '/var/lib/mysql//grastate.dat'
 
Aug 23 11:07:31 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1
 
Aug 23 11:07:31 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: Passing config to GCS: base_dir = /var/lib/mysql/; base_host = 109.73.52.248; base_port = 4567; cert.log_conflicts = no; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.size = 128M; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.checksum = false; pc.ignore_quorum = false
 
Aug 23 11:07:31 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851089311488 [Note] WSREP: Service thread queue flushed.
 
Aug 23 11:07:31 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1
 
Aug 23 11:07:31 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: wsrep_sst_grab()
 
Aug 23 11:07:31 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: Start replication
 
Aug 23 11:07:31 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: 'wsrep-new-cluster' option used, bootstrapping the cluster
 
Aug 23 11:07:31 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
 
Aug 23 11:07:31 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: protonet asio version 0
 
Aug 23 11:07:31 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: Using CRC-32C for message checksums.
 
Aug 23 11:07:31 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: backend: asio
 
Aug 23 11:07:31 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: gcomm thread scheduling priority set to other:0
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Warning] WSREP: access file(/var/lib/mysql//gvwstate.dat) failed(No such file or directory)
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: restore pc from disk failed
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: GMCast version 0
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: (05ce3876, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: (05ce3876, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: EVS version 0
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: gcomm: bootstrapping new group 'my_galera_cluster'
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: start_prim is enabled, turn off pc_recovery
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: Node 05ce3876 state prim
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: view(view_id(PRIM,05ce3876,1) memb {
 
Aug 23 11:07:32 vserver3727 mysqld[7104]:       05ce3876,0
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: } joined {
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: } left {
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: } partitioned {
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: })
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: save pc into disk
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: discarding pending addr without UUID: tcp://109.73.52.248:4567
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: discarding pending addr proto entry 0x7f31a7866b40
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: gcomm: connected
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636
 
Aug 23 11:07:32 vserver3727 systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: Opened channel 'my_galera_cluster'
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851304656064 [Note] WSREP: Waiting for SST to complete.
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851038975744 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 1
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851038975744 [Note] WSREP: Starting new group from scratch: 05ced9b2-6911-11e6-bc01-7e6c54885059
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851038975744 [Note] WSREP: STATE_EXCHANGE: sent state UUID: 05ceece4-6911-11e6-8231-7a7f1a4b797e
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851038975744 [Note] WSREP: STATE EXCHANGE: sent state msg: 05ceece4-6911-11e6-8231-7a7f1a4b797e
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851038975744 [Note] WSREP: STATE EXCHANGE: got state msg: 05ceece4-6911-11e6-8231-7a7f1a4b797e from 0 (node1)
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851038975744 [Note] WSREP: Quorum results:
 
Aug 23 11:07:32 vserver3727 mysqld[7104]:       version    = 4,
 
Aug 23 11:07:32 vserver3727 mysqld[7104]:       component  = PRIMARY,
 
Aug 23 11:07:32 vserver3727 mysqld[7104]:       conf_id    = 0,
 
Aug 23 11:07:32 vserver3727 mysqld[7104]:       members    = 1/1 (joined/total),
 
Aug 23 11:07:32 vserver3727 mysqld[7104]:       act_id     = 0,
 
Aug 23 11:07:32 vserver3727 mysqld[7104]:       last_appl. = -1,
 
Aug 23 11:07:32 vserver3727 mysqld[7104]:       protocols  = 0/7/3 (gcs/repl/appl),
 
Aug 23 11:07:32 vserver3727 mysqld[7104]:       group UUID = 05ced9b2-6911-11e6-bc01-7e6c54885059
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851038975744 [Note] WSREP: Flow-control interval: [16, 16]
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851038975744 [Note] WSREP: Restored state OPEN -> JOINED (0)
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: 160823 11:07:31 [ERROR] mysqld got signal 11 ;
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: This could be because you hit a bug. It is also possible that this binary
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: or one of the libraries it was linked against is corrupt, improperly built,
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: or misconfigured. This error can also be caused by malfunctioning hardware.
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: We will try our best to scrape up some info that will hopefully help
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: diagnose the problem, but since we have already crashed,
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: something is definitely wrong and this may fail.
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: Server version: 10.1.16-MariaDB
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: key_buffer_size=0
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: read_buffer_size=262144
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: max_used_connections=0
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: max_threads=153
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: thread_count=2
 
Aug 23 11:07:32 vserver3727 mysqld[7104]: It is possible that mysqld could use up to
 
Aug 23 11:07:32 vserver3727 systemd[1]: Failed to start MariaDB database server.
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 120617 K  bytes of memory
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: Hope that's ok; if not, decrease some variables in the equation.
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: Thread pointer: 0x0x7f319e9a25c8
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: Attempting backtrace. You can use the following information to find out
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: where mysqld died. If you see no messages after this, something went
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: terribly wrong...
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851038975744 [Note] WSREP: Member 0.0 (node1) synced with group.
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: 2016-08-23 11:07:31 139851038975744 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 0)
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: stack_bottom = 0x7f31ab50ed98 thread_stack 0x48400
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x556ab72fd7be]
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: /usr/sbin/mysqld(handle_fatal_signal+0x365)[0x556ab6e3f6b5]
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: /usr/lib/libpthread.so.0(+0x11080)[0x7f31a957c080]
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: /usr/sbin/mysqld(+0x52f656)[0x556ab6dde656]
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: /usr/lib/galera/libgalera_smm.so(_ZN6galera13ReplicatorSMM19process_conf_changeEPvRK15wsrep_view_infoiNS_10Replicator5StateEl+0x457)[0x7f31a6d752a7]
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: /usr/lib/galera/libgalera_smm.so(_ZN6galera15GcsActionSource8dispatchEPvRK10gcs_actionRb+0x4fb)[0x7f31a6d4b9eb]
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: /usr/lib/galera/libgalera_smm.so(_ZN6galera15GcsActionSource7processEPvRb+0x62)[0x7f31a6d4d342]
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: /usr/lib/galera/libgalera_smm.so(_ZN6galera13ReplicatorSMM10async_recvEPv+0x7b)[0x7f31a6d749bb]
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: /usr/lib/galera/libgalera_smm.so(galera_recv+0x18)[0x7f31a6d8e768]
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: /usr/sbin/mysqld(+0x53e837)[0x556ab6ded837]
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: /usr/sbin/mysqld(start_wsrep_THD+0x465)[0x556ab6ddd335]
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: /usr/lib/libpthread.so.0(+0x7454)[0x7f31a9572454]
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: /usr/lib/libc.so.6(clone+0x5f)[0x7f31a8c297df]
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: Trying to get some variables.
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: Some pointers may be invalid and cause the dump to abort.
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: Query (0x0):
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: Connection ID (thread ID): 1
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: Status: NOT_KILLED
 
Aug 23 11:07:33 vserver3727 systemd[1]: mariadb.service: Unit entered failed state.
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=off
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
 
Aug 23 11:07:33 vserver3727 mysqld[7104]: information that should help you find out what is causing the crash.
 
Aug 23 11:07:33 vserver3727 systemd[1]: mariadb.service: Failed with result 'exit-code'.

And That:

Aug 23 11:14:31 vserver3727 systemd[1]: Starting MariaDB database server...
 
Aug 23 11:14:33 vserver3727 sh[7311]: WSREP: Recovered position 00000000-0000-0000-0000-000000000000:-1
 
Aug 23 11:14:33 vserver3727 mysqld[7409]: 2016-08-23 11:14:33 139789470383296 [Note] /usr/sbin/mysqld (mysqld 10.1.16-MariaDB) starting as process 7409 ...
 
Aug 23 11:14:33 vserver3727 mysqld[7409]: 2016-08-23 11:14:33 139789470383296 [Note] WSREP: Read nil XID from storage engines, skipping position init
 
Aug 23 11:14:33 vserver3727 mysqld[7409]: 2016-08-23 11:14:33 139789470383296 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so'
 
Aug 23 11:14:33 vserver3727 mysqld[7409]: 2016-08-23 11:14:33 139789470383296 [Note] WSREP: wsrep_load(): Galera 3.17(rXXXX) by Codership Oy <info@codership.com> loaded successfully.
 
Aug 23 11:14:33 vserver3727 mysqld[7409]: 2016-08-23 11:14:33 139789470383296 [Note] WSREP: CRC-32C: using hardware acceleration.
 
Aug 23 11:14:33 vserver3727 mysqld[7409]: 2016-08-23 11:14:33 139789470383296 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1
 
Aug 23 11:14:33 vserver3727 mysqld[7409]: 2016-08-23 11:14:33 139789470383296 [Note] WSREP: Passing config to GCS: base_dir = /var/lib/mysql/; base_host = 109.73.52.248; base_port = 4567; cert.log_conflicts = no; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.size = 128M; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.checksum = false; pc.ignore_quorum = false
 
Aug 23 11:14:33 vserver3727 mysqld[7409]: 2016-08-23 11:14:33 139789254784768 [Note] WSREP: Service thread queue flushed.
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:33 139789470383296 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789470383296 [Note] WSREP: wsrep_sst_grab()
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789470383296 [Note] WSREP: Start replication
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789470383296 [Note] WSREP: 'wsrep-new-cluster' option used, bootstrapping the cluster
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789470383296 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789470383296 [Note] WSREP: protonet asio version 0
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789470383296 [Note] WSREP: Using CRC-32C for message checksums.
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789470383296 [Note] WSREP: backend: asio
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789470383296 [Note] WSREP: gcomm thread scheduling priority set to other:0
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789470383296 [Note] WSREP: restore pc from disk successfully
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789470383296 [Note] WSREP: GMCast version 0
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789470383296 [Note] WSREP: (05ce3876, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789470383296 [Note] WSREP: (05ce3876, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789470383296 [Note] WSREP: EVS version 0
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789470383296 [Note] WSREP: gcomm: bootstrapping new group 'my_wsrep_cluster'
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789470383296 [Note] WSREP: start_prim is enabled, turn off pc_recovery
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789470383296 [Note] WSREP: Node 05ce3876 state prim
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789470383296 [Note] WSREP: view(view_id(PRIM,05ce3876,3) memb {
 
Aug 23 11:14:34 vserver3727 mysqld[7409]:       05ce3876,0
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: } joined {
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: } left {
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: } partitioned {
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: })
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789470383296 [Note] WSREP: save pc into disk
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789470383296 [Note] WSREP: clear restored view
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789470383296 [Note] WSREP: gcomm: connected
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789470383296 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789470383296 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789470383296 [Note] WSREP: Opened channel 'my_wsrep_cluster'
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789470383296 [Note] WSREP: Waiting for SST to complete.
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789204449024 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 1
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789204449024 [Note] WSREP: Starting new group from scratch: 01b07398-6912-11e6-bf23-8295bbae6dc1
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789204449024 [Note] WSREP: STATE_EXCHANGE: sent state UUID: 01b0833a-6912-11e6-8ebd-a3237237db3c
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789204449024 [Note] WSREP: STATE EXCHANGE: sent state msg: 01b0833a-6912-11e6-8ebd-a3237237db3c
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789204449024 [Note] WSREP: STATE EXCHANGE: got state msg: 01b0833a-6912-11e6-8ebd-a3237237db3c from 0 (vserver3727)
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789204449024 [Note] WSREP: Quorum results:
 
Aug 23 11:14:34 vserver3727 mysqld[7409]:       version    = 4,
 
Aug 23 11:14:34 vserver3727 mysqld[7409]:       component  = PRIMARY,
 
Aug 23 11:14:34 vserver3727 mysqld[7409]:       conf_id    = 0,
 
Aug 23 11:14:34 vserver3727 mysqld[7409]:       members    = 1/1 (joined/total),
 
Aug 23 11:14:34 vserver3727 mysqld[7409]:       act_id     = 0,
 
Aug 23 11:14:34 vserver3727 mysqld[7409]:       last_appl. = -1,
 
Aug 23 11:14:34 vserver3727 mysqld[7409]:       protocols  = 0/7/3 (gcs/repl/appl),
 
Aug 23 11:14:34 vserver3727 mysqld[7409]:       group UUID = 01b07398-6912-11e6-bf23-8295bbae6dc1
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789204449024 [Note] WSREP: Flow-control interval: [16, 16]
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789204449024 [Note] WSREP: Restored state OPEN -> JOINED (0)
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 160823 11:14:34 [ERROR] mysqld got signal 11 ;
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: This could be because you hit a bug. It is also possible that this binary
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: or one of the libraries it was linked against is corrupt, improperly built,
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: or misconfigured. This error can also be caused by malfunctioning hardware.
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: We will try our best to scrape up some info that will hopefully help
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: diagnose the problem, but since we have already crashed,
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: something is definitely wrong and this may fail.
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: Server version: 10.1.16-MariaDB
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: key_buffer_size=0
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: read_buffer_size=262144
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: max_used_connections=0
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: max_threads=153
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: thread_count=2
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: It is possible that mysqld could use up to
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 120617 K  bytes of memory
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: Hope that's ok; if not, decrease some variables in the equation.
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: Thread pointer: 0x0x7f2338f8c5c8
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: Attempting backtrace. You can use the following information to find out
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: where mysqld died. If you see no messages after this, something went
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: terribly wrong...
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: stack_bottom = 0x7f2345b4cd98 thread_stack 0x48400
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789204449024 [Note] WSREP: Member 0.0 (vserver3727) synced with group.
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: 2016-08-23 11:14:34 139789204449024 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 0)
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x5640af9a47be]
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: /usr/sbin/mysqld(handle_fatal_signal+0x365)[0x5640af4e66b5]
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: /usr/lib/libpthread.so.0(+0x11080)[0x7f2343bba080]
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: /usr/sbin/mysqld(+0x52f656)[0x5640af485656]
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: /usr/lib/galera/libgalera_smm.so(_ZN6galera13ReplicatorSMM19process_conf_changeEPvRK15wsrep_view_infoiNS_10Replicator5StateEl+0x457)[0x7f23413752a7]
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: /usr/lib/galera/libgalera_smm.so(_ZN6galera15GcsActionSource8dispatchEPvRK10gcs_actionRb+0x4fb)[0x7f234134b9eb]
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: /usr/lib/galera/libgalera_smm.so(_ZN6galera15GcsActionSource7processEPvRb+0x62)[0x7f234134d342]
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: /usr/lib/galera/libgalera_smm.so(_ZN6galera13ReplicatorSMM10async_recvEPv+0x7b)[0x7f23413749bb]
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: /usr/lib/galera/libgalera_smm.so(galera_recv+0x18)[0x7f234138e768]
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: /usr/sbin/mysqld(+0x53e837)[0x5640af494837]
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: /usr/sbin/mysqld(start_wsrep_THD+0x465)[0x5640af484335]
 
Aug 23 11:14:34 vserver3727 systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: /usr/lib/libpthread.so.0(+0x7454)[0x7f2343bb0454]
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: /usr/lib/libc.so.6(clone+0x5f)[0x7f23432677df]
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: Trying to get some variables.
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: Some pointers may be invalid and cause the dump to abort.
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: Query (0x0):
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: Connection ID (thread ID): 1
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: Status: NOT_KILLED
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=off
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
 
Aug 23 11:14:34 vserver3727 mysqld[7409]: information that should help you find out what is causing the crash.
 
Aug 23 11:14:34 vserver3727 systemd[1]: Failed to start MariaDB database server.
 
Aug 23 11:14:34 vserver3727 systemd[1]: mariadb.service: Unit entered failed state.
 
Aug 23 11:14:34 vserver3727 systemd[1]: mariadb.service: Failed with result 'exit-code'.

i tested it on 3 PCs all with arch linux and same version of mariadb and also tried to compile mariadb with WSREP flags but same result.

 Comments   
Comment by Florijan Hamzic [ 2016-08-23 ] 

(gdb) bt
 
#0  0x00007f4ef7c8333f in pthread_kill () from /usr/lib/libpthread.so.0
 
#1  0x00005581d27ed6ef in handle_fatal_signal ()
 
#2  <signal handler called>
 
#3  0x00005581d278c656 in ?? ()
 
#4  0x00007f4ef54e38ae in galera::ReplicatorSMM::process_conf_change (this=0x7f4ef60ed000, recv_ctx=0x7f4eecf80dc8, view_info=..., repl_proto=7,
 
    next_state=galera::Replicator::S_JOINED, seqno_l=1) at galera/src/replicator_smm.cpp:1393
 
#5  0x00007f4ef54c1385 in galera::GcsActionSource::dispatch (this=0x7f4ef60ed728, recv_ctx=0x7f4eecf80dc8, act=..., exit_loop=@0x7f4ef9c1804f: false)
 
    at galera/src/gcs_action_source.cpp:136
 
#6  0x00007f4ef54c199c in galera::GcsActionSource::process (this=0x7f4ef60ed728, recv_ctx=0x7f4eecf80dc8, exit_loop=@0x7f4ef9c1804f: false)
 
    at galera/src/gcs_action_source.cpp:181
 
#7  0x00007f4ef54de261 in galera::ReplicatorSMM::async_recv (this=0x7f4ef60ed000, recv_ctx=0x7f4eecf80dc8) at galera/src/replicator_smm.cpp:355
 
#8  0x00007f4ef54fa33b in galera_recv (gh=0x7f4ef6066140, recv_ctx=0x7f4eecf80dc8) at galera/src/wsrep_provider.cpp:244
 
#9  0x00005581d279b837 in ?? ()
 
#10 0x00005581d278b335 in start_wsrep_THD ()
 
#11 0x00007f4ef7c7c454 in start_thread () from /usr/lib/libpthread.so.0
 
#12 0x00007f4ef73337df in clone () from /usr/lib/libc.so.6

Comment by Florijan Hamzic [ 2016-08-23 ]

built galera with scon debug=0 and added dump

Comment by Florijan Hamzic [ 2016-08-24 ]

this will fix the issue:
https://github.com/MariaDB/server/commit/fec296cc10f0d1319e032b72e92e3c824b7fc390

Generated at Thu Feb 08 07:43:52 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.