Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Incomplete
-
10.6.8
-
None
-
FreeBSD 13, using custom build TODO-3345; datadir on ZFS volumes
https://jira.mariadb.org/browse/TODO-3345
Description
Two nodes of a 5 node Galera cluster on FreeBSD 13 failed at about the same time, catching a SIGABRT signal. Nothing in the error log hits towards the reason, the most recent messages where peer timeout messages, but there were several of these earlier in the log already without causing this.
No assertion check failure messages, no usable backtrace info:
2022-05-15 20:57:50 0 [Note] WSREP: (4c0650f8-9562, 'tcp://10.10.70.231:4567') connection to peer 00000000-0000 with addr tcp://10.10.70.241:4567 timed out, no messages seen in PT3S, socket stats: rtt: 1000 rttvar: 500 rto: 230000 lost: 0 last_data_recv: 3047000 cwnd: 14480 last_queued_since: 3047251488 last_delivered_since: 3047251488 send_queue_length: 0 send_queue_bytes: 0
|
2022-05-15 20:57:54 0 [Note] WSREP: (4c0650f8-9562, 'tcp://10.10.70.231:4567') connection to peer 00000000-0000 with addr tcp://10.10.70.241:4567 timed out, no messages seen in PT3S, socket stats: rtt: 1000 rttvar: 500 rto: 230000 lost: 0 last_data_recv: 3073000 cwnd: 14480 last_queued_since: 3072801757 last_delivered_since: 3072801757 send_queue_length: 0 send_queue_bytes: 0
|
220515 20:57:54 [ERROR] mysqld got signal 6 ;
|
This could be because you hit a bug. It is also possible that this binary
|
or one of the libraries it was linked against is corrupt, improperly built,
|
or misconfigured. This error can also be caused by malfunctioning hardware.
|
|
|
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
|
|
|
We will try our best to scrape up some info that will hopefully help
|
diagnose the problem, but since we have already crashed,
|
something is definitely wrong and this may fail.
|
|
|
Server version: 10.6.8-4-MariaDB-enterprise-log
|
key_buffer_size=5242880
|
read_buffer_size=2097152
|
max_used_connections=4
|
max_threads=1502
|
thread_count=100
|
It is possible that mysqld could use up to
|
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 18500018 K bytes of memory
|
Hope that's ok; if not, decrease some variables in the equation.
|
|
|
Thread pointer: 0x0
|
Attempting backtrace. You can use the following information to find out
|
where mysqld died. If you see no messages after this, something went
|
terribly wrong...
|
stack_bottom = 0x0 thread_stack 0x200000
|
0x13ef71e <my_print_stacktrace+0x2e> at /usr/local/libexec/mariadbd
|
0xce6460 <handle_fatal_signal+0x290> at /usr/local/libexec/mariadbd
|
0x801935580 <pthread_sigmask+0x540> at /lib/libthr.so.3
|
0x801934b3f <pthread_setschedparam+0x82f> at /lib/libthr.so.3
|
0x7ffffffff2d3 <__gxx_personality_v0+0x7ffffeb6ae03> at ???
|
0x801b9791a <thr_kill+0xa> at /lib/libc.so.7
|
0x801b100f4 <_raise+0x34> at /lib/libc.so.7
|
0x801bc16e9 <abort+0x49> at /lib/libc.so.7
|
0x801a32de9 <__cxa_rethrow+0x1d9> at /lib/libcxxrt.so.1
|
0x803a6bc18 <wsrep_ps_free_node_stat+0x159a98> at /rw_part/usr-local/lib/mysql/libgalera_enterprise_smm.so
|
0x803a8777a <wsrep_ps_free_node_stat+0x1755fa> at /rw_part/usr-local/lib/mysql/libgalera_enterprise_smm.so
|
0x803a81bbe <wsrep_ps_free_node_stat+0x16fa3e> at /rw_part/usr-local/lib/mysql/libgalera_enterprise_smm.so
|
0x803a7f444 <wsrep_ps_free_node_stat+0x16d2c4> at /rw_part/usr-local/lib/mysql/libgalera_enterprise_smm.so
|
0x803a8a5b9 <wsrep_ps_free_node_stat+0x178439> at /rw_part/usr-local/lib/mysql/libgalera_enterprise_smm.so
|
0x803a6d9d8 <wsrep_ps_free_node_stat+0x15b858> at /rw_part/usr-local/lib/mysql/libgalera_enterprise_smm.so
|
boost/throw_exception.hpp:0(void boost::throw_exception<std::__1::system_error>(std::__1::system_error const&))[0x803a6e181]
|
0x803a68349 <wsrep_ps_free_node_stat+0x1561c9> at /rw_part/usr-local/lib/mysql/libgalera_enterprise_smm.so
|
impl/throw_error.ipp:0(asio::detail::do_throw_error(std::__1::error_code const&, char const*))[0x80399711c]
|
detail/endpoint.hpp:111(asio::ip::detail::endpoint::is_v4() const)[0x803988ca9]
|
v1/memory:3720(std::__1::shared_ptr<gu::AsioStreamEngine>::operator->() const)[0x803988c19]
|
The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
|
information that should help you find out what is causing the crash.
|
Core pattern: %N.core
|