Aug 23 10:16:14 ramariadb2 mysqld[20501]: 2019-08-23 10:16:14 0 [Note] WSREP: (6462f48d, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: tcp://192.168.9.6:4567 Aug 23 10:16:15 ramariadb2 mysqld[20501]: 2019-08-23 10:16:15 0 [Note] WSREP: (6462f48d, 'tcp://0.0.0.0:4567') reconnecting to 4251276e (tcp://192.168.9.6:4567), attempt 0 Aug 23 10:16:18 ramariadb2 mysqld[20501]: 2019-08-23 10:16:18 0 [Note] WSREP: evs::proto(6462f48d, GATHER, view_id(REG,4251276e,501)) suspecting node: 4251276e Aug 23 10:16:18 ramariadb2 mysqld[20501]: 2019-08-23 10:16:18 0 [Note] WSREP: evs::proto(6462f48d, GATHER, view_id(REG,4251276e,501)) suspected node without join message, declaring inactive Aug 23 10:16:18 ramariadb2 mysqld[20501]: 2019-08-23 10:16:18 0 [Note] WSREP: declaring 7c437ce9 at tcp://192.168.9.8:4567 stable Aug 23 10:16:18 ramariadb2 mysqld[20501]: 2019-08-23 10:16:18 0 [Note] WSREP: Node 6462f48d state prim Aug 23 10:16:18 ramariadb2 mysqld[20501]: 2019-08-23 10:16:18 0 [Note] WSREP: view(view_id(PRIM,6462f48d,502) memb { Aug 23 10:16:18 ramariadb2 mysqld[20501]: #0116462f48d,0 Aug 23 10:16:18 ramariadb2 mysqld[20501]: #0117c437ce9,0 Aug 23 10:16:18 ramariadb2 mysqld[20501]: } joined { Aug 23 10:16:18 ramariadb2 mysqld[20501]: } left { Aug 23 10:16:18 ramariadb2 mysqld[20501]: } partitioned { Aug 23 10:16:18 ramariadb2 mysqld[20501]: #0114251276e,0 Aug 23 10:16:18 ramariadb2 mysqld[20501]: }) Aug 23 10:16:18 ramariadb2 mysqld[20501]: 2019-08-23 10:16:18 0 [Note] WSREP: save pc into disk Aug 23 10:16:18 ramariadb2 mysqld[20501]: 2019-08-23 10:16:18 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 2 Aug 23 10:16:18 ramariadb2 mysqld[20501]: 2019-08-23 10:16:18 0 [Note] WSREP: forgetting 4251276e (tcp://192.168.9.6:4567) Aug 23 10:16:18 ramariadb2 mysqld[20501]: 2019-08-23 10:16:18 0 [Note] WSREP: (6462f48d, 'tcp://0.0.0.0:4567') turning message relay requesting off Aug 23 10:16:18 ramariadb2 mysqld[20501]: 2019-08-23 10:16:18 0 [Note] WSREP: STATE_EXCHANGE: sent state UUID: 488d7b82-c57e-11e9-820b-9f79c4e8b59a Aug 23 10:16:18 ramariadb2 mysqld[20501]: 2019-08-23 10:16:18 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 488d7b82-c57e-11e9-820b-9f79c4e8b59a Aug 23 10:16:18 ramariadb2 mysqld[20501]: 2019-08-23 10:16:18 0 [Note] WSREP: STATE EXCHANGE: got state msg: 488d7b82-c57e-11e9-820b-9f79c4e8b59a from 0 (ramariadb2) Aug 23 10:16:18 ramariadb2 mysqld[20501]: 2019-08-23 10:16:18 0 [Note] WSREP: STATE EXCHANGE: got state msg: 488d7b82-c57e-11e9-820b-9f79c4e8b59a from 1 (ramariadb3) Aug 23 10:16:18 ramariadb2 mysqld[20501]: 2019-08-23 10:16:18 0 [Note] WSREP: Quorum results: Aug 23 10:16:18 ramariadb2 mysqld[20501]: #011version = 4, Aug 23 10:16:18 ramariadb2 mysqld[20501]: #011component = PRIMARY, Aug 23 10:16:18 ramariadb2 mysqld[20501]: #011conf_id = 497, Aug 23 10:16:18 ramariadb2 mysqld[20501]: #011members = 2/2 (joined/total), Aug 23 10:16:18 ramariadb2 mysqld[20501]: #011act_id = 584497204, Aug 23 10:16:18 ramariadb2 mysqld[20501]: #011last_appl. = 584497087, Aug 23 10:16:18 ramariadb2 mysqld[20501]: #011protocols = 0/9/3 (gcs/repl/appl), Aug 23 10:16:18 ramariadb2 mysqld[20501]: #011group UUID = f348bcdc-35b4-11e7-b6da-2610fca8a430 Aug 23 10:16:18 ramariadb2 mysqld[20501]: 2019-08-23 10:16:18 0 [Note] WSREP: Flow-control interval: [23, 23] Aug 23 10:16:18 ramariadb2 mysqld[20501]: 2019-08-23 10:16:18 0 [Note] WSREP: Trying to continue unpaused monitor Aug 23 10:16:21 ramariadb2 mysqld[20501]: 2019-08-23 10:16:21 0 [Note] WSREP: cleaning up 4251276e (tcp://192.168.9.6:4567) Aug 23 10:16:23 ramariadb2 mysqld[20501]: 2019-08-23 10:16:23 22713 [Warning] IP address '192.168.9.11' could not be resolved: Name or service not known Aug 23 10:16:43 ramariadb2 mysqld[20501]: 2019-08-23 10:16:43 22754 [Warning] Aborted connection 22754 to db: 'dev_ca' user: 'dev_ca' host: '192.168.9.11' (Got an error reading communication packets) Aug 23 10:16:43 ramariadb2 mysqld[20501]: 2019-08-23 10:16:43 22737 [Warning] Aborted connection 22737 to db: 'dev_ca' user: 'dev_ca' host: '192.168.9.11' (Got an error reading communication packets) Aug 23 10:16:43 ramariadb2 mysqld[20501]: 2019-08-23 10:16:43 22748 [Warning] Aborted connection 22748 to db: 'dev_ca' user: 'dev_ca' host: '192.168.9.11' (Got an error reading communication packets) Aug 23 10:16:43 ramariadb2 mysqld[20501]: 2019-08-23 10:16:43 22735 [Warning] Aborted connection 22735 to db: 'dev_ca' user: 'dev_ca' host: '192.168.9.11' (Got an error reading communication packets) Aug 23 10:16:44 ramariadb2 mysqld[20501]: 2019-08-23 10:16:44 22756 [Warning] Aborted connection 22756 to db: 'dev_ca' user: 'dev_ca' host: '192.168.9.11' (Got an error reading communication packets) Aug 23 10:16:44 ramariadb2 mysqld[20501]: 2019-08-23 10:16:44 0 [Note] WSREP: (6462f48d, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: tcp://192.168.9.8:4567 Aug 23 10:16:45 ramariadb2 mysqld[20501]: 2019-08-23 10:16:45 22738 [Warning] Aborted connection 22738 to db: 'dev_ca' user: 'dev_ca' host: '192.168.9.11' (Got an error reading communication packets) Aug 23 10:16:45 ramariadb2 mysqld[20501]: 2019-08-23 10:16:45 0 [Note] WSREP: (6462f48d, 'tcp://0.0.0.0:4567') reconnecting to 7c437ce9 (tcp://192.168.9.8:4567), attempt 0 Aug 23 10:16:48 ramariadb2 mysqld[20501]: 2019-08-23 10:16:48 2 [Note] WSREP: New cluster view: global state: f348bcdc-35b4-11e7-b6da-2610fca8a430:584497204, view# 498: Primary, number of nodes: 2, my index: 0, protocol version 3 Aug 23 10:16:49 ramariadb2 mysqld[20501]: 2019-08-23 10:16:49 0 [Note] WSREP: evs::proto(6462f48d, OPERATIONAL, view_id(REG,6462f48d,502)) suspecting node: 7c437ce9 Aug 23 10:16:49 ramariadb2 mysqld[20501]: 2019-08-23 10:16:49 0 [Note] WSREP: evs::proto(6462f48d, OPERATIONAL, view_id(REG,6462f48d,502)) suspected node without join message, declaring inactive Aug 23 10:16:49 ramariadb2 mysqld[20501]: 2019-08-23 10:16:49 2 [Note] WSREP: REPL Protocols: 9 (4, 2) Aug 23 10:16:49 ramariadb2 mysqld[20501]: 2019-08-23 10:16:49 5 [ERROR] InnoDB: Page old data size 8578 new data size 8708, page old max ins size 7608 new max ins size 7478 Aug 23 10:16:49 ramariadb2 mysqld[20501]: 2019-08-23 10:16:49 5 [ERROR] InnoDB: Submit a detailed bug report to https://jira.mariadb.org/ Aug 23 10:16:49 ramariadb2 mysqld[20501]: 2019-08-23 10:16:49 3 [ERROR] [FATAL] InnoDB: Rec offset 99, cur1 offset 10297, cur2 offset 16172 Aug 23 10:16:49 ramariadb2 mysqld[20501]: 190823 10:16:49 [ERROR] mysqld got signal 6 ; Aug 23 10:16:49 ramariadb2 mysqld[20501]: This could be because you hit a bug. It is also possible that this binary Aug 23 10:16:49 ramariadb2 mysqld[20501]: or one of the libraries it was linked against is corrupt, improperly built, Aug 23 10:16:49 ramariadb2 mysqld[20501]: or misconfigured. This error can also be caused by malfunctioning hardware. Aug 23 10:16:49 ramariadb2 mysqld[20501]: To report this bug, see https://mariadb.com/kb/en/reporting-bugs Aug 23 10:16:49 ramariadb2 mysqld[20501]: We will try our best to scrape up some info that will hopefully help Aug 23 10:16:49 ramariadb2 mysqld[20501]: diagnose the problem, but since we have already crashed, Aug 23 10:16:49 ramariadb2 mysqld[20501]: something is definitely wrong and this may fail. Aug 23 10:16:49 ramariadb2 mysqld[20501]: Server version: 10.3.17-MariaDB-1:10.3.17+maria~jessie Aug 23 10:16:49 ramariadb2 mysqld[20501]: key_buffer_size=402653184 Aug 23 10:16:49 ramariadb2 mysqld[20501]: read_buffer_size=2097152 Aug 23 10:16:49 ramariadb2 mysqld[20501]: max_used_connections=96 Aug 23 10:16:49 ramariadb2 mysqld[20501]: max_threads=65537 Aug 23 10:16:49 ramariadb2 mysqld[20501]: thread_count=106 Aug 23 10:16:49 ramariadb2 mysqld[20501]: It is possible that mysqld could use up to Aug 23 10:16:49 ramariadb2 mysqld[20501]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 403277722 K bytes of memory Aug 23 10:16:49 ramariadb2 mysqld[20501]: Hope that's ok; if not, decrease some variables in the equation. Aug 23 10:16:49 ramariadb2 mysqld[20501]: Thread pointer: 0x7f33ec0009a8 Aug 23 10:16:49 ramariadb2 mysqld[20501]: Attempting backtrace. You can use the following information to find out Aug 23 10:16:49 ramariadb2 mysqld[20501]: where mysqld died. If you see no messages after this, something went Aug 23 10:16:49 ramariadb2 mysqld[20501]: terribly wrong... Aug 23 10:16:49 ramariadb2 mysqld[20501]: stack_bottom = 0x7f342bffec48 thread_stack 0x49000 Aug 23 10:16:49 ramariadb2 mysqld[20501]: /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x56040189916e] Aug 23 10:16:49 ramariadb2 mysqld[20501]: 2019-08-23 10:16:49 2 [Note] WSREP: Assign initial position for certification: 584497204, protocol version: 4 Aug 23 10:16:49 ramariadb2 mysqld[20501]: 2019-08-23 10:16:49 0 [Note] WSREP: Service thread queue flushed. Aug 23 10:16:49 ramariadb2 mysqld[20501]: /usr/sbin/mysqld(handle_fatal_signal+0x307)[0x5604013346b7] Aug 23 10:16:50 ramariadb2 mysqld[20501]: 2019-08-23 10:16:50 0 [Note] WSREP: view(view_id(NON_PRIM,6462f48d,502) memb { Aug 23 10:16:50 ramariadb2 mysqld[20501]: #0116462f48d,0 Aug 23 10:16:50 ramariadb2 mysqld[20501]: } joined { Aug 23 10:16:50 ramariadb2 mysqld[20501]: } left { Aug 23 10:16:50 ramariadb2 mysqld[20501]: } partitioned { Aug 23 10:16:50 ramariadb2 mysqld[20501]: #0117c437ce9,0 Aug 23 10:16:50 ramariadb2 mysqld[20501]: }) Aug 23 10:16:50 ramariadb2 mysqld[20501]: 2019-08-23 10:16:50 0 [Note] WSREP: view(view_id(NON_PRIM,6462f48d,503) memb { Aug 23 10:16:50 ramariadb2 mysqld[20501]: #0116462f48d,0 Aug 23 10:16:50 ramariadb2 mysqld[20501]: } joined { Aug 23 10:16:50 ramariadb2 mysqld[20501]: } left { Aug 23 10:16:50 ramariadb2 mysqld[20501]: } partitioned { Aug 23 10:16:50 ramariadb2 mysqld[20501]: #0117c437ce9,0 Aug 23 10:16:50 ramariadb2 mysqld[20501]: }) Aug 23 10:16:50 ramariadb2 mysqld[20501]: 2019-08-23 10:16:50 22728 [Warning] WSREP: Send action {(nil), 720, TORDERED} returned -107 (Transport endpoint is not connected) Aug 23 10:16:50 ramariadb2 mysqld[20501]: 2019-08-23 10:16:50 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1 Aug 23 10:16:50 ramariadb2 mysqld[20501]: 2019-08-23 10:16:50 22752 [Warning] WSREP: Send action {(nil), 512, TORDERED} returned -107 (Transport endpoint is not connected) Aug 23 10:16:50 ramariadb2 mysqld[20501]: 2019-08-23 10:16:50 0 [Note] WSREP: Flow-control interval: [16, 16] Aug 23 10:16:50 ramariadb2 mysqld[20501]: 2019-08-23 10:16:50 0 [Note] WSREP: Trying to continue unpaused monitor Aug 23 10:16:50 ramariadb2 mysqld[20501]: 2019-08-23 10:16:50 0 [Note] WSREP: Received NON-PRIMARY. Aug 23 10:16:50 ramariadb2 mysqld[20501]: 2019-08-23 10:16:50 0 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 584497215) Aug 23 10:16:50 ramariadb2 mysqld[20501]: 2019-08-23 10:16:50 0 [Warning] WSREP: Last Applied Action message in non-primary configuration from member 0 Aug 23 10:16:50 ramariadb2 mysqld[20501]: 2019-08-23 10:16:50 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1 Aug 23 10:16:50 ramariadb2 mysqld[20501]: 2019-08-23 10:16:50 0 [Note] WSREP: Flow-control interval: [16, 16] Aug 23 10:16:50 ramariadb2 mysqld[20501]: 2019-08-23 10:16:50 0 [Note] WSREP: Trying to continue unpaused monitor Aug 23 10:16:50 ramariadb2 mysqld[20501]: 2019-08-23 10:16:50 0 [Note] WSREP: Received NON-PRIMARY. Aug 23 10:16:50 ramariadb2 mysqld[20501]: 2019-08-23 10:16:50 11 [Note] WSREP: New cluster view: global state: f348bcdc-35b4-11e7-b6da-2610fca8a430:584497215, view# -1: non-Primary, number of nodes: 1, my index: 0, protocol version 3 Aug 23 10:16:50 ramariadb2 mysqld[20501]: 2019-08-23 10:16:50 22768 [Warning] WSREP: Send action {(nil), 512, TORDERED} returned -107 (Transport endpoint is not connected) Aug 23 10:16:50 ramariadb2 mysqld[20501]: 2019-08-23 10:16:50 11 [Note] WSREP: New cluster view: global state: f348bcdc-35b4-11e7-b6da-2610fca8a430:584497215, view# -1: non-Primary, number of nodes: 1, my index: 0, protocol version 3 Aug 23 10:16:50 ramariadb2 mysqld[20501]: /lib/x86_64-linux-gnu/libpthread.so.0(+0xf890)[0x7f392bf54890] Aug 23 10:16:50 ramariadb2 mysqld[20501]: /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x37)[0x7f392a69c067] Aug 23 10:16:50 ramariadb2 mysqld[20501]: /lib/x86_64-linux-gnu/libc.so.6(abort+0x148)[0x7f392a69d448] Aug 23 10:16:51 ramariadb2 mysqld[20501]: /usr/sbin/mysqld(+0xa5ae13)[0x5604015fde13] Aug 23 10:16:51 ramariadb2 mysqld[20501]: /usr/sbin/mysqld(+0x98596b)[0x56040152896b] Aug 23 10:16:51 ramariadb2 mysqld[20501]: /usr/sbin/mysqld(+0x988a61)[0x56040152ba61] Aug 23 10:16:51 ramariadb2 mysqld[20501]: /usr/sbin/mysqld(+0xa6a02c)[0x56040160d02c] Aug 23 10:16:51 ramariadb2 mysqld[20501]: /usr/sbin/mysqld(+0xa7920e)[0x56040161c20e] Aug 23 10:16:51 ramariadb2 mysqld[20501]: /usr/sbin/mysqld(+0xa7cef4)[0x56040161fef4] Aug 23 10:16:51 ramariadb2 mysqld[20501]: /usr/sbin/mysqld(+0x9e1db3)[0x560401584db3] Aug 23 10:16:51 ramariadb2 mysqld[20501]: /usr/sbin/mysqld(+0x9e4089)[0x560401587089] Aug 23 10:16:51 ramariadb2 mysqld[20501]: /usr/sbin/mysqld(+0x9e4fa8)[0x560401587fa8] Aug 23 10:16:51 ramariadb2 mysqld[20501]: /usr/sbin/mysqld(+0x99f14e)[0x56040154214e] Aug 23 10:16:51 ramariadb2 mysqld[20501]: /usr/sbin/mysqld(+0xa2baf7)[0x5604015ceaf7] Aug 23 10:16:51 ramariadb2 mysqld[20501]: /usr/sbin/mysqld(+0xa10053)[0x5604015b3053] Aug 23 10:16:51 ramariadb2 mysqld[20501]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8064)[0x7f392bf4d064] Aug 23 10:16:52 ramariadb2 mysqld[20501]: /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f392a74f62d] Aug 23 10:16:52 ramariadb2 mysqld[20501]: Trying to get some variables. Aug 23 10:16:52 ramariadb2 mysqld[20501]: Some pointers may be invalid and cause the dump to abort. Aug 23 10:16:52 ramariadb2 mysqld[20501]: Query (0x0): Aug 23 10:16:52 ramariadb2 mysqld[20501]: Connection ID (thread ID): 3 Aug 23 10:16:52 ramariadb2 mysqld[20501]: Status: NOT_KILLED Aug 23 10:16:52 ramariadb2 mysqld[20501]: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=on,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=on,mrr_cost_based=off,mrr_sort_keys=on,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on Aug 23 10:16:52 ramariadb2 mysqld[20501]: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains Aug 23 10:16:52 ramariadb2 mysqld[20501]: information that should help you find out what is causing the crash. Aug 23 10:16:52 ramariadb2 mysqld[20501]: Writing a core file... Aug 23 10:16:52 ramariadb2 mysqld[20501]: Working directory at /applis/mysql Aug 23 10:16:52 ramariadb2 mysqld[20501]: Resource Limits: Aug 23 10:16:52 ramariadb2 mysqld[20501]: Limit Soft Limit Hard Limit Units Aug 23 10:16:52 ramariadb2 mysqld[20501]: Max cpu time unlimited unlimited seconds Aug 23 10:16:52 ramariadb2 mysqld[20501]: Max file size unlimited unlimited bytes Aug 23 10:16:52 ramariadb2 mysqld[20501]: Max data size unlimited unlimited bytes Aug 23 10:16:52 ramariadb2 mysqld[20501]: Max stack size 8388608 unlimited bytes Aug 23 10:16:52 ramariadb2 mysqld[20501]: Max core file size 0 unlimited bytes Aug 23 10:16:52 ramariadb2 mysqld[20501]: Max resident set unlimited unlimited bytes Aug 23 10:16:52 ramariadb2 mysqld[20501]: Max processes 128555 128555 processes Aug 23 10:16:52 ramariadb2 mysqld[20501]: Max open files 16364 16364 files Aug 23 10:16:52 ramariadb2 mysqld[20501]: Max locked memory 65536 65536 bytes Aug 23 10:16:52 ramariadb2 mysqld[20501]: Max address space unlimited unlimited bytes Aug 23 10:16:52 ramariadb2 mysqld[20501]: Max file locks unlimited unlimited locks Aug 23 10:16:52 ramariadb2 mysqld[20501]: Max pending signals 128555 128555 signals Aug 23 10:16:52 ramariadb2 mysqld[20501]: Max msgqueue size 819200 819200 bytes Aug 23 10:16:52 ramariadb2 mysqld[20501]: Max nice priority 0 0 Aug 23 10:16:52 ramariadb2 mysqld[20501]: Max realtime priority 0 0 Aug 23 10:16:52 ramariadb2 mysqld[20501]: Max realtime timeout unlimited unlimited us Aug 23 10:16:52 ramariadb2 mysqld[20501]: Core pattern: core Aug 23 10:16:53 ramariadb2 systemd[1]: mariadb.service: Main process exited, code=killed, status=6/ABRT Aug 23 10:16:53 ramariadb2 systemd[1]: mariadb.service: Unit entered failed state. Aug 23 10:16:53 ramariadb2 systemd[1]: mariadb.service: Failed with result 'signal'. Aug 23 10:16:58 ramariadb2 systemd[1]: mariadb.service: Service hold-off time over, scheduling restart. Aug 23 10:16:58 ramariadb2 systemd[1]: Stopped MariaDB 10.3.17 database server.