191007 8:30:49 [Note] WSREP: declaring b585b0d7 at tcp://172.31.23.67:4567 stable 191007 8:30:49 [Note] WSREP: forgetting 5ddd2dad (tcp://172.31.21.2:4567) 191007 8:30:49 [Note] WSREP: Node 35b70612 state prim 191007 8:30:49 [Note] WSREP: view(view_id(PRIM,35b70612,45) memb { 35b70612,0 b585b0d7,0 } joined { } left { } partitioned { 5ddd2dad,0 }) 191007 8:30:49 [Note] WSREP: save pc into disk 191007 8:30:49 [Note] WSREP: forgetting 5ddd2dad (tcp://172.31.21.2:4567) 191007 8:30:49 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 2 191007 8:30:49 [Note] WSREP: STATE_EXCHANGE: sent state UUID: c474a19f-e8dc-11e9-a4a0-47460de8fc52 191007 8:30:49 [Note] WSREP: STATE EXCHANGE: sent state msg: c474a19f-e8dc-11e9-a4a0-47460de8fc52 191007 8:30:49 [Note] WSREP: STATE EXCHANGE: got state msg: c474a19f-e8dc-11e9-a4a0-47460de8fc52 from 0 ( node1) 191007 8:30:49 [Note] WSREP: STATE EXCHANGE: got state msg: c474a19f-e8dc-11e9-a4a0-47460de8fc52 from 1 (garb) 191007 8:30:49 [Note] WSREP: 'garb' demoted SYNCED->PRIMARY due to gap in history: 18170877 - 18170887 191007 8:30:49 [Note] WSREP: Quorum results: version = 4, component = PRIMARY, conf_id = 38, members = 1/2 (joined/total), act_id = 18170887, last_appl. = 18170752, protocols = 0/9/3 (gcs/repl/appl), group UUID = d7d97fae-52b8-11e9-86bb-5611caffc126 191007 8:30:49 [Note] WSREP: Flow-control interval: [23, 23] 191007 8:30:49 [Note] WSREP: Trying to continue unpaused monitor 191007 8:30:49 [Note] WSREP: Member 1.0 (garb) requested state transfer from '*any*'. Selected 0.0 ( node1)(SYNCED) as donor. 191007 8:30:49 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 18170887) 191007 8:30:49 [Note] WSREP: 1.0 (garb): State transfer from 0.0 ( node1) complete. 191007 8:30:49 [Note] WSREP: Member 1.0 (garb) synced with group. 191007 8:30:50 [ERROR] Slave SQL: Column 9 of table 'plato_db.ch_newdata' cannot be converted from type 'datetime' to type 'enum('mobile','plato')', Internal MariaDB error code: 1677 191007 8:30:50 [Warning] WSREP: RBR event 69 Update_rows_v1 apply warning: 3, 18170887 191007 8:30:50 [Warning] WSREP: Failed to apply app buffer: seqno: 18170887, status: 1 at galera/src/trx_handle.cpp:apply():353 Retrying 2th time 191007 8:30:50 [ERROR] Slave SQL: Column 9 of table 'plato_db.ch_newdata' cannot be converted from type 'datetime' to type 'enum('mobile','plato')', Internal MariaDB error code: 1677 191007 8:30:50 [Warning] WSREP: RBR event 69 Update_rows_v1 apply warning: 3, 18170887 191007 8:30:50 [Warning] WSREP: Failed to apply app buffer: seqno: 18170887, status: 1 at galera/src/trx_handle.cpp:apply():353 Retrying 3th time 191007 8:30:50 [ERROR] Slave SQL: Column 9 of table 'plato_db.ch_newdata' cannot be converted from type 'datetime' to type 'enum('mobile','plato')', Internal MariaDB error code: 1677 191007 8:30:50 [Warning] WSREP: RBR event 69 Update_rows_v1 apply warning: 3, 18170887 191007 8:30:50 [Warning] WSREP: Failed to apply app buffer: seqno: 18170887, status: 1 at galera/src/trx_handle.cpp:apply():353 Retrying 4th time 191007 8:30:50 [ERROR] Slave SQL: Column 9 of table 'plato_db.ch_newdata' cannot be converted from type 'datetime' to type 'enum('mobile','plato')', Internal MariaDB error code: 1677 191007 8:30:50 [Warning] WSREP: RBR event 69 Update_rows_v1 apply warning: 3, 18170887 191007 8:30:50 [Warning] WSREP: failed to replay trx: source: 35b70612-6a00-11e9-991c-4e3fd08964ff version: 4 local: 1 state: REPLAYING flags: 1 conn_id: 26317908 trx_id: 11124490860 seqnos (l: 18387157, g: 18170887, s: 18170884, d: 18170752, ts: 14086860886139882) 191007 8:30:50 [Warning] WSREP: Failed to apply trx 18170887 4 times 191007 8:30:50 [ERROR] WSREP: trx_replay failed for: 6, schema: plato_db, query:  VD^A 191007 8:30:50 [ERROR] Aborting 191007 8:30:52 [Note] WSREP: killing local connection: 26317931 191007 8:30:52 [Note] WSREP: killing local connection: 26317922 191007 8:30:52 [Note] WSREP: Closing send monitor... 191007 8:30:52 [Note] WSREP: Closed send monitor. 191007 8:30:52 [Note] WSREP: gcomm: terminating thread 191007 8:30:52 [Note] WSREP: gcomm: joining thread 191007 8:30:52 [Note] WSREP: gcomm: closing backend 191007 8:30:55 [Note] WSREP: cleaning up 5ddd2dad (tcp://172.31.21.2:4567) 191007 8:30:55 [Note] WSREP: (35b70612, 'tcp://0.0.0.0:4567') connection to peer b585b0d7 with addr tcp://172.31.23.67:4567 timed out, no messages seen in PT3S 191007 8:30:55 [Note] WSREP: (35b70612, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: tcp://172.31.23.67:4567 191007 8:30:57 [Note] WSREP: (35b70612, 'tcp://0.0.0.0:4567') reconnecting to b585b0d7 (tcp://172.31.23.67:4567), attempt 0 191007 8:30:57 [Note] WSREP: evs::proto(35b70612, LEAVING, view_id(REG,35b70612,45)) suspecting node: b585b0d7 191007 8:30:57 [Note] WSREP: evs::proto(35b70612, LEAVING, view_id(REG,35b70612,45)) suspected node without join message, declaring inactive 191007 8:30:57 [Note] WSREP: view(view_id(NON_PRIM,35b70612,45) memb { 35b70612,0 } joined { } left { } partitioned { b585b0d7,0 }) 191007 8:30:57 [Note] WSREP: view((empty)) 191007 8:30:57 [Note] WSREP: gcomm: closed 191007 8:30:57 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1 191007 8:30:57 [Note] WSREP: Flow-control interval: [16, 16] 191007 8:30:57 [Note] WSREP: Trying to continue unpaused monitor 191007 8:30:57 [Note] WSREP: Received NON-PRIMARY. 191007 8:30:57 [Note] WSREP: Shifting DONOR/DESYNCED -> OPEN (TO: 18170887) 191007 8:30:57 [Note] WSREP: Received self-leave message. 191007 8:30:57 [Note] WSREP: Flow-control interval: [0, 0] 191007 8:30:57 [Note] WSREP: Trying to continue unpaused monitor 191007 8:30:57 [Note] WSREP: Received SELF-LEAVE. Closing connection. 191007 8:30:57 [Note] WSREP: Shifting OPEN -> CLOSED (TO: 18170887) 191007 8:30:57 [Note] WSREP: RECV thread exiting 0: Success 191007 8:30:57 [Note] WSREP: recv_thread() joined. 191007 8:30:57 [Note] WSREP: Closing replication queue. 191007 8:30:57 [Note] WSREP: Closing slave action queue. 191007 8:30:57 [Note] WSREP: Service disconnected. 191007 8:30:57 [Note] WSREP: rollbacker thread exiting 191007 8:30:58 [Note] WSREP: Some threads may fail to exit. 191007 8:30:58 [Note] InnoDB: FTS optimize thread exiting. 191007 8:30:58 [Note] InnoDB: Starting shutdown... 191007 8:30:58 [ERROR] mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. To report this bug, see https://mariadb.com/kb/en/reporting-bugs We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Server version: 10.0.38-MariaDB-wsrep key_buffer_size=134217728 read_buffer_size=131072 max_used_connections=152 max_threads=153 thread_count=3 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 467095 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x7f86b3afb008 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x7f87074c4cf0 thread_stack 0x48000 /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0xbf35fe] /usr/sbin/mysqld(handle_fatal_signal+0x3b6)[0x75f406] sigaction.c:0(__restore_rt)[0x7f87070a91f0] /usr/sbin/mysqld(_Z14ip_to_hostnameP16sockaddr_storagePKcPPcPj+0x1b0)[0x76c800] /usr/sbin/mysqld[0x6b2445] /usr/sbin/mysqld(_Z16login_connectionP3THD+0x42)[0x6b38a2] /usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x15c)[0x6b436c] /usr/sbin/mysqld(handle_one_connection+0x39)[0x6b45e9] pthread_create.c:0(start_thread)[0x7f870709e54b] :0(__GI___clone)[0x7f87057e02ff] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0x0): is an invalid pointer Connection ID (thread ID): 26317960 Status: NOT_KILLED Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. We think the query pointer is invalid, but we will try to print it anyway. Query: 191007 08:30:58 mysqld_safe Number of processes running now: 0 191007 08:30:58 mysqld_safe WSREP: not restarting wsrep node automatically 191007 08:30:58 mysqld_safe mysqld from pid file /var/lib/mysql/ip-172-31-19-249.ap-south-1.compute.internal.pid ended 191007 08:51:00 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql 191007 08:51:00 mysqld_safe WSREP: Running position recovery with --log_error='/var/lib/mysql/wsrep_recovery.R8etVm' --pid-file='/var/lib/mysql/ip-172-31-19-249.ap-south-1.compute.internal-recover.pid' 191007 8:51:00 [Note] /usr/sbin/mysqld (mysqld 10.0.38-MariaDB-wsrep) starting as process 25772 ... 191007 08:51:04 mysqld_safe WSREP: Recovered position d7d97fae-52b8-11e9-86bb-5611caffc126:18170886 191007 8:51:04 [Note] /usr/sbin/mysqld (mysqld 10.0.38-MariaDB-wsrep) starting as process 25814 ... 191007 8:51:04 [Note] WSREP: Read nil XID from storage engines, skipping position init 191007 8:51:04 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib64/galera/libgalera_smm.so' 191007 8:51:04 [Note] WSREP: wsrep_load(): Galera 25.3.25(r3836) by Codership Oy loaded successfully. 191007 8:51:04 [Note] WSREP: CRC-32C: using hardware acceleration. 191007 8:51:04 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1, safe_to_bootstrap: 0 191007 8:51:04 [Note] WSREP: Passing config to GCS: base_dir = /var/lib/mysql/; base_host = 172.31.19.249; base_port = 4567; cert.log_conflicts = no; cert.optimistic_pa = yes; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.recover = no; gcache.size = 128M; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S 191007 8:51:04 [Note] WSREP: GCache history reset: d7d97fae-52b8-11e9-86bb-5611caffc126:0 -> 00000000-0000-0000-0000-000000000000:-1 191007 8:51:04 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1 191007 8:51:04 [Note] WSREP: wsrep_sst_grab() 191007 8:51:04 [Note] WSREP: Start replication 191007 8:51:04 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1 191007 8:51:04 [Note] WSREP: protonet asio version 0 191007 8:51:04 [Note] WSREP: Using CRC-32C for message checksums. 191007 8:51:04 [Note] WSREP: backend: asio 191007 8:51:04 [Note] WSREP: gcomm thread scheduling priority set to other:0 191007 8:51:04 [Warning] WSREP: access file(/var/lib/mysql//gvwstate.dat) failed(No such file or directory) 191007 8:51:04 [Note] WSREP: restore pc from disk failed 191007 8:51:04 [Note] WSREP: GMCast version 0 191007 8:51:04 [Note] WSREP: (985513df, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567 191007 8:51:04 [Note] WSREP: (985513df, 'tcp://0.0.0.0:4567') multicast: , ttl: 1 191007 8:51:04 [Note] WSREP: EVS version 0 191007 8:51:04 [Note] WSREP: gcomm: connecting to group 'galera_cluster', peer '172.31.19.249:,172.31.21.2:,172.31.23.67:'