Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Cannot Reproduce
-
10.3.18
-
None
-
Ubuntu 18.04.3 LTS
Linux galera-03 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Description
Oct 22 22:00:53 galera-03 mysqld[1984]: 2019-10-22 22:00:53 14853 [ERROR] WSREP: RecordSet checksum does not match:
Oct 22 22:00:53 galera-03 mysqld[1984]: computed: 1843a2d6 7e37ec48 dadb6da5 f43e61a4
Oct 22 22:00:53 galera-03 mysqld[1984]: found: f97c7ddc 3a0eb101 efb34581 6f9abb70: 22 (Invalid argument)
Oct 22 22:00:53 galera-03 mysqld[1984]: #011 at galerautils/src/gu_rset.cpp:checksum():551
Oct 22 22:00:53 galera-03 mysqld[1984]: 2019-10-22 22:00:53 14853 [ERROR] WSREP: Writeset deserialization failed: Writeset checksum failed: 22 (Invalid argument)
Oct 22 22:00:53 galera-03 mysqld[1984]: #011 at galera/src/write_set_ng.hpp:checksum_fin():815
Oct 22 22:00:53 galera-03 mysqld[1984]: #011 at galera/src/trx_handle.cpp:unserialize():270
Oct 22 22:00:53 galera-03 mysqld[1984]: WS flags: 1
Oct 22 22:00:53 galera-03 mysqld[1984]: Trx proto: 4
Oct 22 22:00:53 galera-03 mysqld[1984]: Trx source: 25d618b4-f4fb-11e9-8880-2b5e714252ba
Oct 22 22:00:53 galera-03 mysqld[1984]: Trx conn_id: 14853
Oct 22 22:00:53 galera-03 mysqld[1984]: Trx trx_id: 1886591184
Oct 22 22:00:53 galera-03 mysqld[1984]: Trx last_seen: 940770072
Oct 22 22:00:53 galera-03 mysqld[1984]: 2019-10-22 22:00:53 14853 [ERROR] WSREP: Writeset checksum failed: 22 (Invalid argument)
Oct 22 22:00:53 galera-03 mysqld[1984]: #011 at galera/src/write_set_ng.hpp:checksum_fin():815
Oct 22 22:00:53 galera-03 mysqld[1984]: #011 at galera/src/trx_handle.cpp:unserialize():270
Oct 22 22:00:53 galera-03 mysqld[1984]: 2019-10-22 22:00:53 14853 [ERROR] WSREP: unknown connection failure
Oct 22 22:00:53 galera-03 mysqld[1984]: 2019-10-22 22:00:53 14853 [ERROR] WSREP: FSM: no such a transition REPLICATING -> ROLLED_BACK
Oct 22 22:00:53 galera-03 mysqld[1984]: 191022 22:00:53 [ERROR] mysqld got signal 6 ;
Oct 22 22:00:53 galera-03 mysqld[1984]: This could be because you hit a bug. It is also possible that this binary
Oct 22 22:00:53 galera-03 mysqld[1984]: or one of the libraries it was linked against is corrupt, improperly built,
Oct 22 22:00:53 galera-03 mysqld[1984]: or misconfigured. This error can also be caused by malfunctioning hardware.
Oct 22 22:00:53 galera-03 mysqld[1984]: To report this bug, see https://mariadb.com/kb/en/reporting-bugs
Oct 22 22:00:53 galera-03 mysqld[1984]: We will try our best to scrape up some info that will hopefully help
Oct 22 22:00:53 galera-03 mysqld[1984]: diagnose the problem, but since we have already crashed,
Oct 22 22:00:53 galera-03 mysqld[1984]: something is definitely wrong and this may fail.
Oct 22 22:00:53 galera-03 mysqld[1984]: Server version: 10.3.18-MariaDB-1:10.3.18+maria~bionic-log
Oct 22 22:00:53 galera-03 mysqld[1984]: key_buffer_size=134217728
Oct 22 22:00:53 galera-03 mysqld[1984]: read_buffer_size=2097152
Oct 22 22:00:53 galera-03 mysqld[1984]: max_used_connections=13
Oct 22 22:00:53 galera-03 mysqld[1984]: max_threads=102
Oct 22 22:00:53 galera-03 mysqld[1984]: thread_count=29
Oct 22 22:00:53 galera-03 mysqld[1984]: It is possible that mysqld could use up to
Oct 22 22:00:53 galera-03 mysqld[1984]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1177826 K bytes of memory
Oct 22 22:00:53 galera-03 mysqld[1984]: Hope that's ok; if not, decrease some variables in the equation.
Oct 22 22:00:53 galera-03 mysqld[1984]: Thread pointer: 0x7fb7c8000c08
Oct 22 22:00:53 galera-03 mysqld[1984]: Attempting backtrace. You can use the following information to find out
Oct 22 22:00:53 galera-03 mysqld[1984]: where mysqld died. If you see no messages after this, something went
Oct 22 22:00:53 galera-03 mysqld[1984]: terribly wrong...
Oct 22 22:00:53 galera-03 mysqld[1984]: stack_bottom = 0x7fc5381ebda8 thread_stack 0x49000
Oct 22 22:00:53 galera-03 mysqld[1984]: 2019-10-22 22:00:53 0 [Note] WSREP: declaring 47272008 at tcp://192.168.3.102:4567 stable
Oct 22 22:00:53 galera-03 mysqld[1984]: 2019-10-22 22:00:53 0 [Note] WSREP: forgetting 7ff63eb5 (tcp://192.168.3.101:4567)
Oct 22 22:00:53 galera-03 mysqld[1984]: 2019-10-22 22:00:53 0 [Note] WSREP: Node 25d618b4 state prim
Oct 22 22:00:53 galera-03 mysqld[1984]: 2019-10-22 22:00:53 0 [Warning] WSREP: 25d618b4 sending install message failed: Resource temporarily unavailable
Oct 22 22:00:53 galera-03 mysqld[1984]: 2019-10-22 22:00:53 0 [Note] WSREP: view(view_id(NON_PRIM,25d618b4,194) memb
joined
{ Oct 22 22:00:53 galera-03 mysqld[1984]: }left
{ Oct 22 22:00:53 galera-03 mysqld[1984]: }partitioned
{ Oct 22 22:00:53 galera-03 mysqld[1984]: #01147272008,0 Oct 22 22:00:53 galera-03 mysqld[1984]: #0117ff63eb5,0 Oct 22 22:00:53 galera-03 mysqld[1984]: })
Oct 22 22:00:53 galera-03 mysqld[1984]: 2019-10-22 22:00:53 0 [Note] WSREP: forgetting 47272008 (tcp://192.168.3.102:4567)
Oct 22 22:00:53 galera-03 mysqld[1984]: 2019-10-22 22:00:53 0 [Note] WSREP: view(view_id(NON_PRIM,25d618b4,195) memb
joined
{ Oct 22 22:00:53 galera-03 mysqld[1984]: }left
{ Oct 22 22:00:53 galera-03 mysqld[1984]: }partitioned
{ Oct 22 22:00:53 galera-03 mysqld[1984]: #01147272008,0 Oct 22 22:00:53 galera-03 mysqld[1984]: #0117ff63eb5,0 Oct 22 22:00:53 galera-03 mysqld[1984]: })
Oct 22 22:00:53 galera-03 mysqld[1984]: 2019-10-22 22:00:53 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
Oct 22 22:00:53 galera-03 mysqld[1984]: 2019-10-22 22:00:53 0 [Note] WSREP: Flow-control interval: [16, 16]
Oct 22 22:00:53 galera-03 mysqld[1984]: 2019-10-22 22:00:53 0 [Note] WSREP: Trying to continue unpaused monitor
Oct 22 22:00:53 galera-03 mysqld[1984]: 2019-10-22 22:00:53 0 [Note] WSREP: Received NON-PRIMARY.
Oct 22 22:00:53 galera-03 mysqld[1984]: 2019-10-22 22:00:53 0 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 940770073)
Oct 22 22:00:53 galera-03 mysqld[1984]: 2019-10-22 22:00:53 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
Oct 22 22:00:53 galera-03 mysqld[1984]: 2019-10-22 22:00:53 0 [Note] WSREP: Flow-control interval: [16, 16]
Oct 22 22:00:53 galera-03 mysqld[1984]: 2019-10-22 22:00:53 0 [Note] WSREP: Trying to continue unpaused monitor
Oct 22 22:00:53 galera-03 mysqld[1984]: 2019-10-22 22:00:53 0 [Note] WSREP: Received NON-PRIMARY.
Oct 22 22:00:53 galera-03 mysqld[1984]: /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x558afbd64a7e]
Oct 22 22:00:53 galera-03 mysqld[1984]: /usr/sbin/mysqld(handle_fatal_signal+0x515)[0x558afb7fb285]
Oct 22 22:00:53 galera-03 mysqld[1984]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x12890)[0x7fc5a6453890]
Oct 22 22:00:54 galera-03 mysqld[1984]: /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xc7)[0x7fc5a5967e97]
Oct 22 22:00:54 galera-03 mysqld[1984]: /lib/x86_64-linux-gnu/libc.so.6(abort+0x141)[0x7fc5a5969801]
Oct 22 22:00:54 galera-03 mysqld[1984]: /usr/lib/galera/libgalera_smm.so(+0x1e6a38)[0x7fc59de39a38]
Oct 22 22:00:54 galera-03 mysqld[1984]: /usr/lib/galera/libgalera_smm.so(+0x1db6d6)[0x7fc59de2e6d6]
Oct 22 22:00:54 galera-03 mysqld[1984]: /usr/lib/galera/libgalera_smm.so(+0x1fba12)[0x7fc59de4ea12]
Oct 22 22:00:54 galera-03 mysqld[1984]: /usr/sbin/mysqld(+0x6c6ba9)[0x558afb767ba9]
Oct 22 22:00:54 galera-03 mysqld[1984]: /usr/sbin/mysqld(_Z17ha_rollback_transP3THDb+0xb7)[0x558afb7fe057]
Oct 22 22:00:54 galera-03 mysqld[1984]: /usr/sbin/mysqld(_Z15ha_commit_transP3THDb+0x385)[0x558afb7fe725]
Oct 22 22:00:54 galera-03 mysqld[1984]: /usr/sbin/mysqld(_Z17trans_commit_stmtP3THD+0x55)[0x558afb704db5]
Oct 22 22:00:54 galera-03 mysqld[1984]: /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x386)[0x558afb61a0c6]
Oct 22 22:00:54 galera-03 mysqld[1984]: /usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_statebb+0x20a)[0x558afb62201a]
Oct 22 22:00:54 galera-03 mysqld[1984]: /usr/sbin/mysqld(+0x5818c6)[0x558afb6228c6]
Oct 22 22:00:54 galera-03 mysqld[1984]: /usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjbb+0x12cf)[0x558afb62400f]
Oct 22 22:00:54 galera-03 mysqld[1984]: /usr/sbin/mysqld(_Z10do_commandP3THD+0x178)[0x558afb625758]
Oct 22 22:00:54 galera-03 mysqld[1984]: /usr/sbin/mysqld(_Z24do_handle_one_connectionP7CONNECT+0x33e)[0x558afb6f7ece]
Oct 22 22:00:54 galera-03 mysqld[1984]: /usr/sbin/mysqld(handle_one_connection+0x3d)[0x558afb6f7f9d]
Oct 22 22:00:54 galera-03 mysqld[1984]: /usr/sbin/mysqld(+0xc6f79a)[0x558afbd1079a]
Oct 22 22:00:54 galera-03 mysqld[1984]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db)[0x7fc5a64486db]
Oct 22 22:00:54 galera-03 mysqld[1984]: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7fc5a5a4a88f]
Oct 22 22:00:54 galera-03 mysqld[1984]: Trying to get some variables.
Oct 22 22:00:54 galera-03 mysqld[1984]: Some pointers may be invalid and cause the dump to abort.
Oct 22 22:00:54 galera-03 mysqld[1984]: Query (0x7fb7c800f6e0): LOAD DATA LOCAL INFILE '/var/www/virtual/web.local/htdocs/var/tmp/sync/avail/avail28' INTO TABLE wuModMapData_0 CHARACTER SET utf8 FIELDS TERMINATED BY ',' ENCLOSED BY '\'' LINES TERMINATED BY '\n' (productID,markerID,avail,productPrice)
Oct 22 22:00:54 galera-03 mysqld[1984]: Connection ID (thread ID): 14853
Oct 22 22:00:54 galera-03 mysqld[1984]: Status: NOT_KILLED
Oct 22 22:00:54 galera-03 mysqld[1984]: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_me
rge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on
Oct 22 22:00:54 galera-03 mysqld[1984]: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
Oct 22 22:00:54 galera-03 mysqld[1984]: information that should help you find out what is causing the crash.
Oct 22 22:00:54 galera-03 mysqld[1984]: Writing a core file...
Oct 22 22:00:54 galera-03 mysqld[1984]: Working directory at /var/lib/mysql
Oct 22 22:00:54 galera-03 mysqld[1984]: Resource Limits:
Oct 22 22:00:54 galera-03 mysqld[1984]: Limit Soft Limit Hard Limit Units
Oct 22 22:00:54 galera-03 mysqld[1984]: Max cpu time unlimited unlimited seconds
Oct 22 22:00:54 galera-03 mysqld[1984]: Max file size unlimited unlimited bytes
Oct 22 22:00:54 galera-03 mysqld[1984]: Max data size unlimited unlimited bytes
Oct 22 22:00:54 galera-03 mysqld[1984]: Max stack size 8388608 unlimited bytes
Oct 22 22:00:54 galera-03 mysqld[1984]: Max core file size 0 unlimited bytes
Oct 22 22:00:54 galera-03 mysqld[1984]: Max resident set unlimited unlimited bytes
Oct 22 22:00:54 galera-03 mysqld[1984]: Max processes 256593 256593 processes
Oct 22 22:00:54 galera-03 mysqld[1984]: Max open files 16364 16364 files
Oct 22 22:00:54 galera-03 mysqld[1984]: Max locked memory 16777216 16777216 bytes
Oct 22 22:00:54 galera-03 mysqld[1984]: Max address space unlimited unlimited bytes
Oct 22 22:00:54 galera-03 mysqld[1984]: Max file locks unlimited unlimited locks
Oct 22 22:00:54 galera-03 mysqld[1984]: Max pending signals 256593 256593 signals
Oct 22 22:00:54 galera-03 mysqld[1984]: Max msgqueue size 819200 819200 bytes
Oct 22 22:00:54 galera-03 mysqld[1984]: Max nice priority 0 0
Oct 22 22:00:54 galera-03 mysqld[1984]: Max realtime priority 0 0
Oct 22 22:00:54 galera-03 mysqld[1984]: Max realtime timeout unlimited unlimited us
Oct 22 22:00:54 galera-03 mysqld[1984]: Core pattern: core
Oct 22 22:00:54 galera-03 systemd[1]: mariadb.service: Main process exited, code=killed, status=6/ABRT
Oct 22 22:00:54 galera-03 systemd[1]: mariadb.service: Failed with result 'signal'.
Oct 22 22:00:59 galera-03 systemd[1]: mariadb.service: Service hold-off time over, scheduling restart.
Oct 22 22:00:59 galera-03 systemd[1]: mariadb.service: Scheduled restart job, restart counter is at 2.
Oct 22 22:00:59 galera-03 systemd[1]: Stopped MariaDB 10.3.18 database server.