[MDEV-25068] Node crashes with Transport endpoint is not connected mysqld got signal 6 ; - Jira

Details

Type: Bug
Status: Closed (View Workflow)
Priority: Critical
Resolution: Fixed
Affects Version/s: 10.4.18, 10.4.19, 10.5.9, 10.4.20, 10.4.21, 10.4.22
Fix Version/s: 10.3.36, 10.4.26, 10.5.17, 10.6.9, 10.7.5, 10.8.4
Component/s: Galera
Labels:
None
Environment:
CentOS

Description

3 Node Setup, all 3 nodes crashed. Bootstrap required.

Node 1 crash output

Mar  5 17:48:58  mysqld[116636]: terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<std::system_error> >'

Mar  5 17:48:58  mysqld[116636]: what():  remote_endpoint: Transport endpoint is not connected

Mar  5 17:48:58  mysqld[116636]: 210305 17:48:58 [ERROR] mysqld got signal 6 ;

Mar  5 17:48:58  mysqld[116636]: This could be because you hit a bug. It is also possible that this binary

Mar  5 17:48:58  mysqld[116636]: or one of the libraries it was linked against is corrupt, improperly built,

Mar  5 17:48:58  mysqld[116636]: or misconfigured. This error can also be caused by malfunctioning hardware.

Mar  5 17:48:58  mysqld[116636]: To report this bug, see https://mariadb.com/kb/en/reporting-bugs

Mar  5 17:48:58  mysqld[116636]: We will try our best to scrape up some info that will hopefully help

Mar  5 17:48:58  mysqld[116636]: diagnose the problem, but since we have already crashed,

Mar  5 17:48:58  mysqld[116636]: something is definitely wrong and this may fail.

Mar  5 17:48:58  mysqld[116636]: Server version: 10.4.18-MariaDB-log

Mar  5 17:48:58  mysqld[116636]: key_buffer_size=134217728

Mar  5 17:48:58  mysqld[116636]: read_buffer_size=131072

Mar  5 17:48:58  mysqld[116636]: max_used_connections=76

Mar  5 17:48:58  mysqld[116636]: max_threads=153

Mar  5 17:48:58  mysqld[116636]: thread_count=75

Mar  5 17:48:58  mysqld[116636]: It is possible that mysqld could use up to

Mar  5 17:48:58  mysqld[116636]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 781104 K  bytes of memory

Mar  5 17:48:58  mysqld[116636]: Hope that's ok; if not, decrease some variables in the equation.

Mar  5 17:48:58  mysqld[116636]: Thread pointer: 0x0

Mar  5 17:48:58  mysqld[116636]: Attempting backtrace. You can use the following information to find out

Mar  5 17:48:58  mysqld[116636]: where mysqld died. If you see no messages after this, something went

Mar  5 17:48:58  mysqld[116636]: terribly wrong...

Mar  5 17:48:59  mysqld[116636]: stack_bottom = 0x0 thread_stack 0x49000

Mar  5 17:48:59  mysqld[116636]: /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x55a14151e47e]

Mar  5 17:48:59  mysqld[116636]: /usr/sbin/mysqld(handle_fatal_signal+0x30f)[0x55a140face8f]

Mar  5 17:48:59  mysqld[116636]: /lib64/libpthread.so.0(+0xf630)[0x7f5879976630]

Mar  5 17:48:59  mysqld[116636]: /lib64/libc.so.6(gsignal+0x37)[0x7f5877c463d7]

Mar  5 17:48:59  mysqld[116636]: /lib64/libc.so.6(abort+0x148)[0x7f5877c47ac8]

Mar  5 17:48:59  mysqld[116636]: /lib64/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x165)[0x7f5878340a95]

Mar  5 17:48:59  mysqld[116636]: /lib64/libstdc++.so.6(+0x5ea06)[0x7f587833ea06]

Mar  5 17:48:59  mysqld[116636]: /lib64/libstdc++.so.6(+0x5ea33)[0x7f587833ea33]

Mar  5 17:48:59  mysqld[116636]: /lib64/libstdc++.so.6(+0x5ec53)[0x7f587833ec53]

Mar  5 17:48:59  mysqld[116636]: /usr/lib64/galera-4/libgalera_smm.so(+0x1997a4)[0x7f586ddc07a4]

Mar  5 17:48:59  mysqld[116636]: /usr/lib64/galera-4/libgalera_smm.so(+0x19984c)[0x7f586ddc084c]

Mar  5 17:48:59  mysqld[116636]: /usr/lib64/galera-4/libgalera_smm.so(+0x1a57bf)[0x7f586ddcc7bf]

Mar  5 17:48:59  mysqld[116636]: /usr/lib64/galera-4/libgalera_smm.so(+0x1a78b8)[0x7f586ddce8b8]

Mar  5 17:48:59  mysqld[116636]: /usr/lib64/galera-4/libgalera_smm.so(+0x1aaf13)[0x7f586ddd1f13]

Mar  5 17:48:59  mysqld[116636]: /usr/lib64/galera-4/libgalera_smm.so(+0x19897b)[0x7f586ddbf97b]

Mar  5 17:48:59  mysqld[116636]: /usr/lib64/galera-4/libgalera_smm.so(+0x194b91)[0x7f586ddbbb91]

Mar  5 17:48:59  mysqld[116636]: /usr/lib64/galera-4/libgalera_smm.so(+0xc08c9)[0x7f586dce78c9]

Mar  5 17:48:59  mysqld[116636]: /usr/lib64/galera-4/libgalera_smm.so(+0xa6bcf)[0x7f586dccdbcf]

Mar  5 17:48:59  mysqld[116636]: /usr/lib64/galera-4/libgalera_smm.so(+0xad859)[0x7f586dcd4859]

Mar  5 17:48:59  mysqld[116636]: /lib64/libpthread.so.0(+0x7ea5)[0x7f587996eea5]

Mar  5 17:48:59  mysqld[116636]: /lib64/libc.so.6(clone+0x6d)[0x7f5877d0e9fd]

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending
- Thumbnails
- List
- Download All

_InVuln__PCI-DSS_ORG023150_OneCloudINFRA_day3__jpw1a_2_of_2__09nptb.zip
366 kB
2022-02-21 23:18
gdb.txt
306 kB
2021-10-16 19:37
log-10.4.2
7 kB
2021-10-16 19:21
log-10.5.11
7 kB
2021-10-16 19:21

Issue Links

is duplicated by

MDEV-30336 Galera: Handshake failed: peer did not return a certificate

Closed

relates to

MDEV-27004 McAfee database vulnerability scan caused MariaDB crash with signal 6 (system abort)

Closed

MDEV-26861 Galera Crashing - what(): remote_endpoint: Transport endpoint is not connected

Closed

links to

CVE-2023-5157

Activity

Ascending order - Click to sort in descending order

View 5 older comments

Stuart Green added a comment - 2021-11-19 17:24 - edited

Hi all,

I thought I would verify/update as we have upgraded a cluster to 10.4.22.

@Nicolas I have found slightly different conditions are required to what you suggested

If I scan port 3306 I get denial messages from OpenVAS trying to logon, but the node stays up
If I scan port 4567 I get a lot of internal error messages, but the node stays up.
If I scan port 3306 and port 4567 in the same scan, the node crashes.

It seems that it is required to scan both of these ports at the same time to get the node to crash.

It would make sense that using OpenVAS defaults as a preset would crash the node as sequentially 4567 would follow 3306 as there would be no ports found inbetween.

Output follows:

OpenVAS Scan Port 3306 targeted.

Nov 19 16:55:03 NODE-01 mysqld[78300]: 2021-11-19 16:55:03 72 [Warning] Aborted connection 72 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication)

Nov 19 16:55:03 NODE-01 mysqld[78300]: 2021-11-19 16:55:03 73 [Warning] Aborted connection 73 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication)

Nov 19 16:55:03 NODE-01 mysqld[78300]: 2021-11-19 16:55:03 74 [Warning] Aborted connection 74 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication)

Nov 19 16:55:03 NODE-01 mysqld[78300]: 2021-11-19 16:55:03 75 [Warning] Aborted connection 75 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication)

Nov 19 16:55:03 NODE-01 mysqld[78300]: 2021-11-19 16:55:03 76 [Warning] Aborted connection 76 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication)

Nov 19 16:55:04 NODE-01 mysqld[78300]: 2021-11-19 16:55:04 77 [Warning] Aborted connection 77 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication)

Nov 19 16:55:04 NODE-01 mysqld[78300]: 2021-11-19 16:55:04 78 [Warning] Aborted connection 78 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication)

Nov 19 16:55:04 NODE-01 mysqld[78300]: 2021-11-19 16:55:04 79 [Warning] Aborted connection 79 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication)

Nov 19 16:55:04 NODE-01 mysqld[78300]: 2021-11-19 16:55:04 80 [Warning] Aborted connection 80 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication)

Nov 19 16:55:04 NODE-01 mysqld[78300]: 2021-11-19 16:55:04 81 [Warning] Aborted connection 81 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication)

Nov 19 16:55:04 NODE-01 mysqld[78300]: 2021-11-19 16:55:04 82 [Warning] Aborted connection 82 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication)

Nov 19 16:55:04 NODE-01 mysqld[78300]: 2021-11-19 16:55:04 83 [Warning] Aborted connection 83 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication)

Nov 19 16:55:04 NODE-01 mysqld[78300]: 2021-11-19 16:55:04 84 [Warning] Aborted connection 84 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication)

Nov 19 16:55:45 NODE-01 mysqld[78300]: 2021-11-19 16:55:45 87 [Warning] Access denied for user 'htr'@'the.scanner.ip.address' (using password: YES)

Nov 19 16:55:45 NODE-01 mysqld[78300]: 2021-11-19 16:55:45 88 [Warning] Access denied for user 'root'@'the.scanner.ip.address' (using password: YES)

Nov 19 16:55:45 NODE-01 mysqld[78300]: 2021-11-19 16:55:45 91 [Warning] Access denied for user 'root'@'the.scanner.ip.address' (using password: YES)

Nov 19 16:55:45 NODE-01 mysqld[78300]: 2021-11-19 16:55:45 92 [Warning] Access denied for user 'root'@'the.scanner.ip.address' (using password: YES)

Nov 19 16:55:45 NODE-01 mysqld[78300]: 2021-11-19 16:55:45 93 [Warning] Access denied for user 'root'@'the.scanner.ip.address' (using password: YES)

Nov 19 16:55:45 NODE-01 mysqld[78300]: 2021-11-19 16:55:45 94 [Warning] Access denied for user 'root'@'the.scanner.ip.address' (using password: YES)

Nov 19 16:55:45 NODE-01 mysqld[78300]: 2021-11-19 16:55:45 95 [Warning] Access denied for user 'root'@'the.scanner.ip.address' (using password: YES)

Nov 19 16:55:45 NODE-01 mysqld[78300]: 2021-11-19 16:55:45 96 [Warning] Access denied for user 'root'@'the.scanner.ip.address' (using password: YES)

Nov 19 16:55:45 NODE-01 mysqld[78300]: 2021-11-19 16:55:45 89 [Warning] Aborted connection 89 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication)

Nov 19 16:55:46 NODE-01 mysqld[78300]: 2021-11-19 16:55:46 97 [Warning] Access denied for user 'root'@'the.scanner.ip.address' (using password: YES)

Nov 19 16:55:46 NODE-01 mysqld[78300]: 2021-11-19 16:55:46 98 [Warning] Access denied for user 'root'@'the.scanner.ip.address' (using password: YES)

Nov 19 16:55:46 NODE-01 mysqld[78300]: 2021-11-19 16:55:46 99 [Warning] Access denied for user 'root'@'the.scanner.ip.address' (using password: YES)

Nov 19 16:55:46 NODE-01 mysqld[78300]: 2021-11-19 16:55:46 100 [Warning] Access denied for user 'root'@'the.scanner.ip.address' (using password: YES)

Nov 19 16:55:46 NODE-01 mysqld[78300]: 2021-11-19 16:55:46 101 [Warning] Access denied for user 'root'@'the.scanner.ip.address' (using password: NO)

Nov 19 16:55:47 NODE-01 mysqld[78300]: 2021-11-19 16:55:47 90 [Warning] Aborted connection 90 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication)

Nov 19 16:55:47 NODE-01 mysqld[78300]: 2021-11-19 16:55:47 102 [Warning] Aborted connection 102 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication)

OpenVAS Scan Port 4567 targeted.

Nov 19 17:00:41 NODE-01 mysqld[78300]: 2021-11-19 17:00:41 0 [Warning] WSREP: unserialize error invalid protocol version 2: 71 (Protocol error)

Nov 19 17:00:41 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:00:42 NODE-01 mysqld[78300]: 2021-11-19 17:00:42 0 [Warning] WSREP: unserialize error invalid protocol version 7: 71 (Protocol error)

Nov 19 17:00:42 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:00:45 NODE-01 mysqld[78300]: 2021-11-19 17:00:45 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567') connection to peer 00000000-0000 with addr  timed out, no messages seen in PT3S, socket stats: rtt: 2976 rttvar: 1171 rto: 203000 lost: 0 last_data_recv: 110334802 cwnd: 10 last_queued_since: 3065247393 last_delivered_since: 110634801991969 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0

Nov 19 17:00:45 NODE-01 mysqld[78300]: 2021-11-19 17:00:45 0 [Warning] WSREP: unserialize error invalid protocol version 2: 71 (Protocol error)

Nov 19 17:00:45 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:00:45 NODE-01 mysqld[78300]: 2021-11-19 17:00:45 0 [Warning] WSREP: unserialize error invalid protocol version 2: 71 (Protocol error)

Nov 19 17:00:45 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:00:45 NODE-01 mysqld[78300]: 2021-11-19 17:00:45 0 [Warning] WSREP: unserialize error invalid flags 11: 71 (Protocol error)

Nov 19 17:00:45 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():126

Nov 19 17:00:47 NODE-01 mysqld[78300]: 2021-11-19 17:00:47 0 [Warning] WSREP: unserialize error invalid protocol version 6: 71 (Protocol error)

Nov 19 17:00:47 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:00:47 NODE-01 mysqld[78300]: 2021-11-19 17:00:47 0 [Warning] WSREP: unserialize error invalid protocol version 4: 71 (Protocol error)

Nov 19 17:00:47 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:00:47 NODE-01 mysqld[78300]: 2021-11-19 17:00:47 0 [Warning] WSREP: unserialize error invalid protocol version 2: 71 (Protocol error)

Nov 19 17:00:47 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:00:47 NODE-01 mysqld[78300]: 2021-11-19 17:00:47 0 [Warning] WSREP: unserialize error invalid protocol version 6: 71 (Protocol error)

Nov 19 17:00:47 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:00:47 NODE-01 mysqld[78300]: 2021-11-19 17:00:47 0 [Warning] WSREP: unserialize error invalid protocol version 4: 71 (Protocol error)

Nov 19 17:00:47 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:01:04 NODE-01 mysqld[78300]: 2021-11-19 17:01:04 0 [Warning] WSREP: checksum failed, hdr: len=790 has_crc32=1 has_crc32c=1 crc32=50332065

Nov 19 17:01:04 NODE-01 mysqld[78300]: 2021-11-19 17:01:04 0 [Warning] WSREP: checksum failed, hdr: len=2 has_crc32=0 has_crc32c=0 crc32=16780288

Nov 19 17:01:13 NODE-01 mysqld[78300]: 2021-11-19 17:01:13 0 [Warning] WSREP: unserialize error invalid protocol version 4: 71 (Protocol error)

Nov 19 17:01:13 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:01:13 NODE-01 mysqld[78300]: 2021-11-19 17:01:13 0 [Warning] WSREP: unserialize error invalid protocol version 2: 71 (Protocol error)

Nov 19 17:01:13 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:01:14 NODE-01 mysqld[78300]: 2021-11-19 17:01:14 0 [Warning] WSREP: unserialize error invalid protocol version 5: 71 (Protocol error)

Nov 19 17:01:14 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:01:14 NODE-01 mysqld[78300]: 2021-11-19 17:01:14 0 [Warning] WSREP: unserialize error invalid protocol version 7: 71 (Protocol error)

Nov 19 17:01:14 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:01:14 NODE-01 mysqld[78300]: 2021-11-19 17:01:14 0 [Warning] WSREP: unserialize error invalid protocol version 5: 71 (Protocol error)

Nov 19 17:01:14 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:01:14 NODE-01 mysqld[78300]: 2021-11-19 17:01:14 0 [Warning] WSREP: checksum failed, hdr: len=0 has_crc32=0 has_crc32c=0 crc32=721421568

Nov 19 17:01:14 NODE-01 mysqld[78300]: 2021-11-19 17:01:14 0 [Warning] WSREP: unserialize error invalid protocol version 4: 71 (Protocol error)

Nov 19 17:01:14 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:01:26 NODE-01 mysqld[78300]: 2021-11-19 17:01:26 0 [Warning] WSREP: unserialize error invalid protocol version 5: 71 (Protocol error)

Nov 19 17:01:26 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:01:27 NODE-01 mysqld[78300]: 2021-11-19 17:01:27 0 [Warning] WSREP: unserialize error invalid protocol version 1: 71 (Protocol error)

Nov 19 17:01:27 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:01:32 NODE-01 mysqld[78300]: 2021-11-19 17:01:32 0 [Warning] WSREP: unserialize error invalid protocol version 5: 71 (Protocol error)

Nov 19 17:01:32 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:01:32 NODE-01 mysqld[78300]: 2021-11-19 17:01:32 0 [Warning] WSREP: unserialize error invalid flags 6: 71 (Protocol error)

Nov 19 17:01:32 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():126

Nov 19 17:01:48 NODE-01 mysqld[78300]: 2021-11-19 17:01:48 0 [Warning] WSREP: unserialize error invalid protocol version 6: 71 (Protocol error)

Nov 19 17:01:48 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:01:48 NODE-01 mysqld[78300]: 2021-11-19 17:01:48 0 [Warning] WSREP: unserialize error invalid protocol version 6: 71 (Protocol error)

Nov 19 17:01:48 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:01:48 NODE-01 mysqld[78300]: 2021-11-19 17:01:48 0 [Warning] WSREP: unserialize error invalid protocol version 15: 71 (Protocol error)

Nov 19 17:01:48 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:01:48 NODE-01 mysqld[78300]: 2021-11-19 17:01:48 0 [Warning] WSREP: checksum failed, hdr: len=1 has_crc32=0 has_crc32c=0 crc32=1

Nov 19 17:01:48 NODE-01 mysqld[78300]: 2021-11-19 17:01:48 0 [Warning] WSREP: checksum failed, hdr: len=0 has_crc32=1 has_crc32c=0 crc32=318767104

Nov 19 17:01:48 NODE-01 mysqld[78300]: 2021-11-19 17:01:48 0 [Warning] WSREP: checksum failed, hdr: len=0 has_crc32=1 has_crc32c=0 crc32=318767104

Nov 19 17:01:48 NODE-01 mysqld[78300]: 2021-11-19 17:01:48 0 [Warning] WSREP: unserialize error invalid protocol version 6: 71 (Protocol error)

Nov 19 17:01:48 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:01:48 NODE-01 mysqld[78300]: 2021-11-19 17:01:48 0 [Warning] WSREP: unserialize error invalid protocol version 5: 71 (Protocol error)

Nov 19 17:01:48 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:01:50 NODE-01 mysqld[78300]: 2021-11-19 17:01:50 0 [Warning] WSREP: unserialize error invalid flags 6: 71 (Protocol error)

Nov 19 17:01:50 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():126

Nov 19 17:01:50 NODE-01 mysqld[78300]: 2021-11-19 17:01:50 0 [Warning] WSREP: unserialize error invalid protocol version 5: 71 (Protocol error)

Nov 19 17:01:50 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:01:50 NODE-01 mysqld[78300]: 2021-11-19 17:01:50 0 [Warning] WSREP: unserialize error invalid protocol version 4: 71 (Protocol error)

Nov 19 17:01:50 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:01:51 NODE-01 mysqld[78300]: 2021-11-19 17:01:51 0 [Warning] WSREP: unserialize error invalid flags 4: 71 (Protocol error)

Nov 19 17:01:51 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():126

Nov 19 17:01:51 NODE-01 mysqld[78300]: 2021-11-19 17:01:51 0 [Warning] WSREP: unserialize error invalid protocol version 5: 71 (Protocol error)

Nov 19 17:01:51 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:01:51 NODE-01 mysqld[78300]: 2021-11-19 17:01:51 0 [Warning] WSREP: unserialize error invalid protocol version 12: 71 (Protocol error)

Nov 19 17:01:51 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:01:52 NODE-01 mysqld[78300]: 2021-11-19 17:01:52 0 [Warning] WSREP: unserialize error invalid protocol version 1: 71 (Protocol error)

Nov 19 17:01:52 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:01:52 NODE-01 mysqld[78300]: 2021-11-19 17:01:52 0 [Warning] WSREP: unserialize error invalid protocol version 5: 71 (Protocol error)

Nov 19 17:01:52 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:01:59 NODE-01 mysqld[78300]: 2021-11-19 17:01:59 0 [Warning] WSREP: unserialize error invalid protocol version 5: 71 (Protocol error)

Nov 19 17:01:59 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:01:59 NODE-01 mysqld[78300]: 2021-11-19 17:01:59 0 [Warning] WSREP: unserialize error invalid protocol version 5: 71 (Protocol error)

Nov 19 17:01:59 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:02:00 NODE-01 mysqld[78300]: 2021-11-19 17:02:00 0 [Warning] WSREP: unserialize error invalid protocol version 5: 71 (Protocol error)

Nov 19 17:02:00 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:02:00 NODE-01 mysqld[78300]: 2021-11-19 17:02:00 0 [Warning] WSREP: unserialize error invalid protocol version 5: 71 (Protocol error)

Nov 19 17:02:00 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:02:03 NODE-01 mysqld[78300]: 2021-11-19 17:02:03 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567') connection to peer 00000000-0000 with addr  timed out, no messages seen in PT3S, socket stats: rtt: 2949 rttvar: 1226 rto: 203000 lost: 0 last_data_recv: 110413322 cwnd: 10 last_queued_since: 3024963641 last_delivered_since: 110713321447043 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0

Nov 19 17:02:03 NODE-01 mysqld[78300]: 2021-11-19 17:02:03 0 [Warning] WSREP: unserialize error invalid protocol version 2: 71 (Protocol error)

Nov 19 17:02:03 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:02:07 NODE-01 mysqld[78300]: 2021-11-19 17:02:07 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567') connection to peer 00000000-0000 with addr  timed out, no messages seen in PT3S, socket stats: rtt: 3250 rttvar: 1625 rto: 204000 lost: 0 last_data_recv: 3431 cwnd: 10 last_queued_since: 3433237748 last_delivered_since: 110716822135452 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0

Nov 19 17:02:07 NODE-01 mysqld[78300]: 2021-11-19 17:02:07 0 [Warning] WSREP: unserialize error invalid protocol version 4: 71 (Protocol error)

Nov 19 17:02:07 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:02:07 NODE-01 mysqld[78300]: 2021-11-19 17:02:07 0 [Warning] WSREP: unserialize error invalid protocol version 4: 71 (Protocol error)

Nov 19 17:02:07 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:02:07 NODE-01 mysqld[78300]: 2021-11-19 17:02:07 0 [Warning] WSREP: unserialize error invalid protocol version 2: 71 (Protocol error)

Nov 19 17:02:07 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:02:07 NODE-01 mysqld[78300]: 2021-11-19 17:02:07 0 [Warning] WSREP: unserialize error invalid flags 6: 71 (Protocol error)

Nov 19 17:02:07 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():126

Nov 19 17:02:09 NODE-01 mysqld[78300]: 2021-11-19 17:02:09 0 [Warning] WSREP: unserialize error invalid protocol version 3: 71 (Protocol error)

Nov 19 17:02:09 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:02:10 NODE-01 mysqld[78300]: 2021-11-19 17:02:10 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567') connection to peer 00000000-0000 with addr  timed out, no messages seen in PT3S, socket stats: rtt: 3258 rttvar: 1642 rto: 204000 lost: 0 last_data_recv: 3388 cwnd: 10 last_queued_since: 3391322802 last_delivered_since: 110720322902424 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0

Nov 19 17:02:14 NODE-01 mysqld[78300]: 2021-11-19 17:02:14 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567') connection to peer 00000000-0000 with addr  timed out, no messages seen in PT3S, socket stats: rtt: 2412 rttvar: 1574 rto: 203000 lost: 0 last_data_recv: 3489 cwnd: 10 last_queued_since: 3491386280 last_delivered_since: 110723823637520 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0

Nov 19 17:02:17 NODE-01 mysqld[78300]: 2021-11-19 17:02:17 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567') connection to peer 00000000-0000 with addr  timed out, no messages seen in PT3S, socket stats: rtt: 46870 rttvar: 17884 rto: 247000 lost: 0 last_data_recv: 3447 cwnd: 10 last_queued_since: 3448829822 last_delivered_since: 110727324411113 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0

Nov 19 17:02:17 NODE-01 mysqld[78300]: 2021-11-19 17:02:17 0 [Warning] WSREP: unserialize error invalid protocol version 2: 71 (Protocol error)

Nov 19 17:02:17 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:02:21 NODE-01 mysqld[78300]: 2021-11-19 17:02:21 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567') connection to peer 00000000-0000 with addr  timed out, no messages seen in PT3S, socket stats: rtt: 2380 rttvar: 1510 rto: 203000 lost: 0 last_data_recv: 3387 cwnd: 10 last_queued_since: 3389095114 last_delivered_since: 110730825168063 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0

Nov 19 17:02:21 NODE-01 mysqld[78300]: 2021-11-19 17:02:21 0 [Warning] WSREP: unserialize error invalid protocol version 7: 71 (Protocol error)

Nov 19 17:02:21 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:02:21 NODE-01 mysqld[78300]: 2021-11-19 17:02:21 0 [Warning] WSREP: unserialize error invalid protocol version 10: 71 (Protocol error)

Nov 19 17:02:21 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:02:24 NODE-01 mysqld[78300]: 2021-11-19 17:02:24 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567') connection to peer 00000000-0000 with addr  timed out, no messages seen in PT3S, socket stats: rtt: 2362 rttvar: 1475 rto: 203000 lost: 0 last_data_recv: 3338 cwnd: 10 last_queued_since: 3341175489 last_delivered_since: 110734325955033 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0

Nov 19 17:02:24 NODE-01 mysqld[78300]: 2021-11-19 17:02:24 0 [Warning] WSREP: unserialize error invalid protocol version 2: 71 (Protocol error)

Nov 19 17:02:24 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:02:24 NODE-01 mysqld[78300]: 2021-11-19 17:02:24 0 [Warning] WSREP: unserialize error invalid protocol version 6: 71 (Protocol error)

Nov 19 17:02:24 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:02:28 NODE-01 mysqld[78300]: 2021-11-19 17:02:28 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567') connection to peer 00000000-0000 with addr  timed out, no messages seen in PT3S, socket stats: rtt: 2357 rttvar: 1464 rto: 203000 lost: 0 last_data_recv: 3332 cwnd: 10 last_queued_since: 3334558479 last_delivered_since: 110737826645820 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0

Nov 19 17:02:31 NODE-01 mysqld[78300]: 2021-11-19 17:02:31 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567') connection to peer 00000000-0000 with addr  timed out, no messages seen in PT3S, socket stats: rtt: 46817 rttvar: 17990 rto: 247000 lost: 0 last_data_recv: 3446 cwnd: 10 last_queued_since: 3447757772 last_delivered_since: 110741327415873 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0

Nov 19 17:02:31 NODE-01 mysqld[78300]: 2021-11-19 17:02:31 0 [Warning] WSREP: unserialize error invalid protocol version 4: 71 (Protocol error)

Nov 19 17:02:31 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:02:31 NODE-01 mysqld[78300]: 2021-11-19 17:02:31 0 [Warning] WSREP: unserialize error invalid protocol version 5: 71 (Protocol error)

Nov 19 17:02:31 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:02:31 NODE-01 mysqld[78300]: 2021-11-19 17:02:31 0 [Warning] WSREP: unserialize error invalid flags 11: 71 (Protocol error)

Nov 19 17:02:31 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():126

Nov 19 17:02:31 NODE-01 mysqld[78300]: 2021-11-19 17:02:31 0 [Warning] WSREP: unserialize error invalid protocol version 5: 71 (Protocol error)

Nov 19 17:02:31 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:02:35 NODE-01 mysqld[78300]: 2021-11-19 17:02:35 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567') connection to peer 00000000-0000 with addr  timed out, no messages seen in PT3S, socket stats: rtt: 3234 rttvar: 1594 rto: 204000 lost: 0 last_data_recv: 3229 cwnd: 10 last_queued_since: 3230754498 last_delivered_since: 110744828097513 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0

Nov 19 17:02:38 NODE-01 mysqld[78300]: 2021-11-19 17:02:38 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567') connection to peer 00000000-0000 with addr  timed out, no messages seen in PT3S, socket stats: rtt: 45089 rttvar: 17054 rto: 246000 lost: 0 last_data_recv: 3448 cwnd: 10 last_queued_since: 3449277638 last_delivered_since: 110748328836025 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0

Nov 19 17:02:38 NODE-01 mysqld[78300]: 2021-11-19 17:02:38 0 [Warning] WSREP: unserialize error invalid protocol version 15: 71 (Protocol error)

Nov 19 17:02:38 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:02:42 NODE-01 mysqld[78300]: 2021-11-19 17:02:42 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567') connection to peer 00000000-0000 with addr  timed out, no messages seen in PT3S, socket stats: rtt: 42631 rttvar: 24592 rto: 243000 lost: 0 last_data_recv: 3351 cwnd: 10 last_queued_since: 3353602435 last_delivered_since: 110751829586149 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0

Nov 19 17:02:42 NODE-01 mysqld[78300]: 2021-11-19 17:02:42 0 [Warning] WSREP: unserialize error invalid protocol version 3: 71 (Protocol error)

Nov 19 17:02:42 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:02:45 NODE-01 mysqld[78300]: 2021-11-19 17:02:45 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567') connection to peer 00000000-0000 with addr  timed out, no messages seen in PT3S, socket stats: rtt: 42619 rttvar: 24595 rto: 243000 lost: 0 last_data_recv: 3352 cwnd: 10 last_queued_since: 3354069244 last_delivered_since: 110755330277426 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0

Nov 19 17:02:45 NODE-01 mysqld[78300]: 2021-11-19 17:02:45 0 [Warning] WSREP: unserialize error invalid protocol version 5: 71 (Protocol error)

Nov 19 17:02:45 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:02:45 NODE-01 mysqld[78300]: 2021-11-19 17:02:45 0 [Warning] WSREP: unserialize error invalid protocol version 2: 71 (Protocol error)

Nov 19 17:02:45 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:02:48 NODE-01 mysqld[78300]: 2021-11-19 17:02:48 0 [Warning] WSREP: unserialize error invalid protocol version 5: 71 (Protocol error)

Nov 19 17:02:48 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:03:21 NODE-01 mysqld[78300]: 2021-11-19 17:03:21 0 [Warning] WSREP: checksum failed, hdr: len=0 has_crc32=0 has_crc32c=0 crc32=16777216

Nov 19 17:03:21 NODE-01 mysqld[78300]: 2021-11-19 17:03:21 0 [Warning] WSREP: checksum failed, hdr: len=0 has_crc32=0 has_crc32c=0 crc32=16777216

Nov 19 17:03:21 NODE-01 mysqld[78300]: 2021-11-19 17:03:21 0 [Warning] WSREP: unserialize error invalid protocol version 8: 71 (Protocol error)

Nov 19 17:03:21 NODE-01 mysqld[78300]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize():133

Nov 19 17:03:24 NODE-01 mysqld[78300]: 2021-11-19 17:03:24 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567') connection to peer 00000000-0000 with addr  timed out, no messages seen in PT3S, socket stats: rtt: 4207 rttvar: 1915 rto: 205000 lost: 0 last_data_recv: 1065 cwnd: 10 last_queued_since: 3077417409 last_delivered_since: 110793838298892 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0

OpenVAS Scan Port 3306+4567 Targeted

Nov 19 16:47:42 NODE-01 mysqld[75077]: 2021-11-19 16:47:42 118 [Warning] Aborted connection 118 to db: 'unconnected' user: 'unauthenticated' host: 'connecting host' (This connection closed normally without authentication)

Nov 19 16:47:42 NODE-01 mysqld[75077]: terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<std::system_error> >'

Nov 19 16:47:42 NODE-01 mysqld[75077]: what():  remote_endpoint: Transport endpoint is not connected

Nov 19 16:47:42 NODE-01 mysqld[75077]: 211119 16:47:42 [ERROR] mysqld got signal 6 ;

Nov 19 16:47:42 NODE-01 mysqld[75077]: This could be because you hit a bug. It is also possible that this binary

Nov 19 16:47:42 NODE-01 mysqld[75077]: or one of the libraries it was linked against is corrupt, improperly built,

Nov 19 16:47:42 NODE-01 mysqld[75077]: or misconfigured. This error can also be caused by malfunctioning hardware.

Nov 19 16:47:42 NODE-01 mysqld[75077]: To report this bug, see https://mariadb.com/kb/en/reporting-bugs

Nov 19 16:47:42 NODE-01 mysqld[75077]: We will try our best to scrape up some info that will hopefully help

Nov 19 16:47:42 NODE-01 mysqld[75077]: diagnose the problem, but since we have already crashed,

Nov 19 16:47:42 NODE-01 mysqld[75077]: something is definitely wrong and this may fail.

Nov 19 16:47:42 NODE-01 mysqld[75077]: Server version: 10.4.22-MariaDB-log

Nov 19 16:47:42 NODE-01 mysqld[75077]: key_buffer_size=134217728

Nov 19 16:47:42 NODE-01 mysqld[75077]: read_buffer_size=131072

Nov 19 16:47:42 NODE-01 mysqld[75077]: max_used_connections=32

Nov 19 16:47:42 NODE-01 mysqld[75077]: max_threads=153

Nov 19 16:47:42 NODE-01 mysqld[75077]: thread_count=49

Nov 19 16:47:42 NODE-01 mysqld[75077]: It is possible that mysqld could use up to

Nov 19 16:47:42 NODE-01 mysqld[75077]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 781113 K  bytes of memory

Nov 19 16:47:42 NODE-01 mysqld[75077]: Hope that's ok; if not, decrease some variables in the equation.

Nov 19 16:47:42 NODE-01 mysqld[75077]: Thread pointer: 0x0

Nov 19 16:47:42 NODE-01 mysqld[75077]: Attempting backtrace. You can use the following information to find out

Nov 19 16:47:42 NODE-01 mysqld[75077]: where mysqld died. If you see no messages after this, something went

Nov 19 16:47:42 NODE-01 mysqld[75077]: terribly wrong...

Nov 19 16:47:42 NODE-01 mysqld[75077]: stack_bottom = 0x0 thread_stack 0x49000

Nov 19 16:47:42 NODE-01 mysqld[75077]: /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x55ce5f1dd2be]

Nov 19 16:47:42 NODE-01 mysqld[75077]: /usr/sbin/mysqld(handle_fatal_signal+0x30f)[0x55ce5ec64dbf]

Nov 19 16:47:42 NODE-01 mysqld[75077]: /lib64/libpthread.so.0(+0xf630)[0x7f682458c630]

Nov 19 16:47:42 NODE-01 mysqld[75077]: /lib64/libc.so.6(gsignal+0x37)[0x7f68239d7387]

Nov 19 16:47:42 NODE-01 mysqld[75077]: /lib64/libc.so.6(abort+0x148)[0x7f68239d8a78]

Nov 19 16:47:42 NODE-01 mysqld[75077]: /lib64/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x165)[0x7f68240d1a95]

Nov 19 16:47:42 NODE-01 mysqld[75077]: /lib64/libstdc++.so.6(+0x5ea06)[0x7f68240cfa06]

Nov 19 16:47:42 NODE-01 mysqld[75077]: /lib64/libstdc++.so.6(+0x5ea33)[0x7f68240cfa33]

Nov 19 16:47:42 NODE-01 mysqld[75077]: /lib64/libstdc++.so.6(+0x5ec53)[0x7f68240cfc53]

Nov 19 16:47:42 NODE-01 mysqld[75077]: /usr/lib64/galera-4/libgalera_smm.so(+0x197ab4)[0x7f6819b0bab4]

Nov 19 16:47:42 NODE-01 mysqld[75077]: /usr/lib64/galera-4/libgalera_smm.so(+0x197b5c)[0x7f6819b0bb5c]

Nov 19 16:47:42 NODE-01 mysqld[75077]: /usr/lib64/galera-4/libgalera_smm.so(+0x1a3cbf)[0x7f6819b17cbf]

Nov 19 16:47:42 NODE-01 mysqld[75077]: /usr/lib64/galera-4/libgalera_smm.so(+0x1a62e2)[0x7f6819b1a2e2]

Nov 19 16:47:42 NODE-01 mysqld[75077]: /usr/lib64/galera-4/libgalera_smm.so(+0x1aa533)[0x7f6819b1e533]

Nov 19 16:47:42 NODE-01 mysqld[75077]: /usr/lib64/galera-4/libgalera_smm.so(+0x196eeb)[0x7f6819b0aeeb]

Nov 19 16:47:42 NODE-01 mysqld[75077]: /usr/lib64/galera-4/libgalera_smm.so(+0x192d81)[0x7f6819b06d81]

Nov 19 16:47:42 NODE-01 mysqld[75077]: /usr/lib64/galera-4/libgalera_smm.so(+0xbf599)[0x7f6819a33599]

Nov 19 16:47:42 NODE-01 mysqld[75077]: /usr/lib64/galera-4/libgalera_smm.so(+0xa65d7)[0x7f6819a1a5d7]

Nov 19 16:47:42 NODE-01 mysqld[75077]: /usr/lib64/galera-4/libgalera_smm.so(+0xa6989)[0x7f6819a1a989]

Nov 19 16:47:42 NODE-01 mysqld[75077]: /lib64/libpthread.so.0(+0x7ea5)[0x7f6824584ea5]

Nov 19 16:47:42 NODE-01 mysqld[75077]: /lib64/libc.so.6(clone+0x6d)[0x7f6823a9f9fd]

Nov 19 16:47:42 NODE-01 mysqld[75077]: The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains

Nov 19 16:47:42 NODE-01 mysqld[75077]: information that should help you find out what is causing the crash.

Nov 19 16:47:42 NODE-01 mysqld[75077]: Writing a core file...

Nov 19 16:47:42 NODE-01 mysqld[75077]: Working directory at /var/lib/mysql

Nov 19 16:47:42 NODE-01 mysqld[75077]: Resource Limits:

Nov 19 16:47:42 NODE-01 mysqld[75077]: Fatal signal 11 while backtracing

Nov 19 16:47:43 NODE-01 systemd[1]: mariadb.service: main process exited, code=killed, status=11/SEGV

Nov 19 16:47:43 NODE-01 systemd[1]: Unit mariadb.service entered failed state.

Nov 19 16:47:43 NODE-01 systemd[1]: mariadb.service failed.

Nov 19 16:47:48 NODE-01 systemd[1]: mariadb.service holdoff time over, scheduling restart.

Nov 19 16:47:48 NODE-01 systemd[1]: Stopped MariaDB 10.4.22 database server.

Nov 19 16:47:48 NODE-01 systemd[1]: Starting MariaDB 10.4.22 database server...

Additionally I guess it makes sense that I managed to crash our cluster entirely with our scheduled scan which is my OP here, we can run up to 5 machine scans in parallel so it is highly likely the 3 machines in the cluster being scanned at the same time caused a split-brain/impossible automated recovery.

Stuart Green added a comment - 2021-11-19 17:24 - edited Hi all, I thought I would verify/update as we have upgraded a cluster to 10.4.22. @Nicolas I have found slightly different conditions are required to what you suggested If I scan port 3306 I get denial messages from OpenVAS trying to logon, but the node stays up If I scan port 4567 I get a lot of internal error messages, but the node stays up. If I scan port 3306 and port 4567 in the same scan, the node crashes. It seems that it is required to scan both of these ports at the same time to get the node to crash. It would make sense that using OpenVAS defaults as a preset would crash the node as sequentially 4567 would follow 3306 as there would be no ports found inbetween. Output follows: OpenVAS Scan Port 3306 targeted. Nov 19 16 : 55 : 03 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 03 72 [Warning] Aborted connection 72 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication) Nov 19 16 : 55 : 03 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 03 73 [Warning] Aborted connection 73 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication) Nov 19 16 : 55 : 03 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 03 74 [Warning] Aborted connection 74 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication) Nov 19 16 : 55 : 03 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 03 75 [Warning] Aborted connection 75 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication) Nov 19 16 : 55 : 03 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 03 76 [Warning] Aborted connection 76 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication) Nov 19 16 : 55 : 04 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 04 77 [Warning] Aborted connection 77 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication) Nov 19 16 : 55 : 04 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 04 78 [Warning] Aborted connection 78 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication) Nov 19 16 : 55 : 04 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 04 79 [Warning] Aborted connection 79 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication) Nov 19 16 : 55 : 04 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 04 80 [Warning] Aborted connection 80 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication) Nov 19 16 : 55 : 04 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 04 81 [Warning] Aborted connection 81 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication) Nov 19 16 : 55 : 04 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 04 82 [Warning] Aborted connection 82 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication) Nov 19 16 : 55 : 04 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 04 83 [Warning] Aborted connection 83 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication) Nov 19 16 : 55 : 04 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 04 84 [Warning] Aborted connection 84 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication) Nov 19 16 : 55 : 45 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 45 87 [Warning] Access denied for user 'htr' @ 'the.scanner.ip.address' (using password: YES) Nov 19 16 : 55 : 45 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 45 88 [Warning] Access denied for user 'root' @ 'the.scanner.ip.address' (using password: YES) Nov 19 16 : 55 : 45 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 45 91 [Warning] Access denied for user 'root' @ 'the.scanner.ip.address' (using password: YES) Nov 19 16 : 55 : 45 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 45 92 [Warning] Access denied for user 'root' @ 'the.scanner.ip.address' (using password: YES) Nov 19 16 : 55 : 45 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 45 93 [Warning] Access denied for user 'root' @ 'the.scanner.ip.address' (using password: YES) Nov 19 16 : 55 : 45 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 45 94 [Warning] Access denied for user 'root' @ 'the.scanner.ip.address' (using password: YES) Nov 19 16 : 55 : 45 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 45 95 [Warning] Access denied for user 'root' @ 'the.scanner.ip.address' (using password: YES) Nov 19 16 : 55 : 45 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 45 96 [Warning] Access denied for user 'root' @ 'the.scanner.ip.address' (using password: YES) Nov 19 16 : 55 : 45 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 45 89 [Warning] Aborted connection 89 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication) Nov 19 16 : 55 : 46 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 46 97 [Warning] Access denied for user 'root' @ 'the.scanner.ip.address' (using password: YES) Nov 19 16 : 55 : 46 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 46 98 [Warning] Access denied for user 'root' @ 'the.scanner.ip.address' (using password: YES) Nov 19 16 : 55 : 46 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 46 99 [Warning] Access denied for user 'root' @ 'the.scanner.ip.address' (using password: YES) Nov 19 16 : 55 : 46 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 46 100 [Warning] Access denied for user 'root' @ 'the.scanner.ip.address' (using password: YES) Nov 19 16 : 55 : 46 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 46 101 [Warning] Access denied for user 'root' @ 'the.scanner.ip.address' (using password: NO) Nov 19 16 : 55 : 47 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 47 90 [Warning] Aborted connection 90 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication) Nov 19 16 : 55 : 47 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 16 : 55 : 47 102 [Warning] Aborted connection 102 to db: 'unconnected' user: 'unauthenticated' host: 'the.scanner.ip.address' (This connection closed normally without authentication) OpenVAS Scan Port 4567 targeted. Nov 19 17 : 00 : 41 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 00 : 41 0 [Warning] WSREP: unserialize error invalid protocol version 2 : 71 (Protocol error) Nov 19 17 : 00 : 41 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 00 : 42 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 00 : 42 0 [Warning] WSREP: unserialize error invalid protocol version 7 : 71 (Protocol error) Nov 19 17 : 00 : 42 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 00 : 45 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 00 : 45 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567' ) connection to peer 00000000 - 0000 with addr timed out, no messages seen in PT3S, socket stats: rtt: 2976 rttvar: 1171 rto: 203000 lost: 0 last_data_recv: 110334802 cwnd: 10 last_queued_since: 3065247393 last_delivered_since: 110634801991969 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0 Nov 19 17 : 00 : 45 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 00 : 45 0 [Warning] WSREP: unserialize error invalid protocol version 2 : 71 (Protocol error) Nov 19 17 : 00 : 45 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 00 : 45 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 00 : 45 0 [Warning] WSREP: unserialize error invalid protocol version 2 : 71 (Protocol error) Nov 19 17 : 00 : 45 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 00 : 45 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 00 : 45 0 [Warning] WSREP: unserialize error invalid flags 11 : 71 (Protocol error) Nov 19 17 : 00 : 45 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 126 Nov 19 17 : 00 : 47 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 00 : 47 0 [Warning] WSREP: unserialize error invalid protocol version 6 : 71 (Protocol error) Nov 19 17 : 00 : 47 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 00 : 47 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 00 : 47 0 [Warning] WSREP: unserialize error invalid protocol version 4 : 71 (Protocol error) Nov 19 17 : 00 : 47 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 00 : 47 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 00 : 47 0 [Warning] WSREP: unserialize error invalid protocol version 2 : 71 (Protocol error) Nov 19 17 : 00 : 47 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 00 : 47 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 00 : 47 0 [Warning] WSREP: unserialize error invalid protocol version 6 : 71 (Protocol error) Nov 19 17 : 00 : 47 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 00 : 47 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 00 : 47 0 [Warning] WSREP: unserialize error invalid protocol version 4 : 71 (Protocol error) Nov 19 17 : 00 : 47 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 01 : 04 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 04 0 [Warning] WSREP: checksum failed, hdr: len= 790 has_crc32= 1 has_crc32c= 1 crc32= 50332065 Nov 19 17 : 01 : 04 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 04 0 [Warning] WSREP: checksum failed, hdr: len= 2 has_crc32= 0 has_crc32c= 0 crc32= 16780288 Nov 19 17 : 01 : 13 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 13 0 [Warning] WSREP: unserialize error invalid protocol version 4 : 71 (Protocol error) Nov 19 17 : 01 : 13 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 01 : 13 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 13 0 [Warning] WSREP: unserialize error invalid protocol version 2 : 71 (Protocol error) Nov 19 17 : 01 : 13 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 01 : 14 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 14 0 [Warning] WSREP: unserialize error invalid protocol version 5 : 71 (Protocol error) Nov 19 17 : 01 : 14 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 01 : 14 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 14 0 [Warning] WSREP: unserialize error invalid protocol version 7 : 71 (Protocol error) Nov 19 17 : 01 : 14 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 01 : 14 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 14 0 [Warning] WSREP: unserialize error invalid protocol version 5 : 71 (Protocol error) Nov 19 17 : 01 : 14 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 01 : 14 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 14 0 [Warning] WSREP: checksum failed, hdr: len= 0 has_crc32= 0 has_crc32c= 0 crc32= 721421568 Nov 19 17 : 01 : 14 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 14 0 [Warning] WSREP: unserialize error invalid protocol version 4 : 71 (Protocol error) Nov 19 17 : 01 : 14 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 01 : 26 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 26 0 [Warning] WSREP: unserialize error invalid protocol version 5 : 71 (Protocol error) Nov 19 17 : 01 : 26 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 01 : 27 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 27 0 [Warning] WSREP: unserialize error invalid protocol version 1 : 71 (Protocol error) Nov 19 17 : 01 : 27 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 01 : 32 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 32 0 [Warning] WSREP: unserialize error invalid protocol version 5 : 71 (Protocol error) Nov 19 17 : 01 : 32 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 01 : 32 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 32 0 [Warning] WSREP: unserialize error invalid flags 6 : 71 (Protocol error) Nov 19 17 : 01 : 32 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 126 Nov 19 17 : 01 : 48 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 48 0 [Warning] WSREP: unserialize error invalid protocol version 6 : 71 (Protocol error) Nov 19 17 : 01 : 48 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 01 : 48 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 48 0 [Warning] WSREP: unserialize error invalid protocol version 6 : 71 (Protocol error) Nov 19 17 : 01 : 48 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 01 : 48 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 48 0 [Warning] WSREP: unserialize error invalid protocol version 15 : 71 (Protocol error) Nov 19 17 : 01 : 48 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 01 : 48 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 48 0 [Warning] WSREP: checksum failed, hdr: len= 1 has_crc32= 0 has_crc32c= 0 crc32= 1 Nov 19 17 : 01 : 48 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 48 0 [Warning] WSREP: checksum failed, hdr: len= 0 has_crc32= 1 has_crc32c= 0 crc32= 318767104 Nov 19 17 : 01 : 48 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 48 0 [Warning] WSREP: checksum failed, hdr: len= 0 has_crc32= 1 has_crc32c= 0 crc32= 318767104 Nov 19 17 : 01 : 48 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 48 0 [Warning] WSREP: unserialize error invalid protocol version 6 : 71 (Protocol error) Nov 19 17 : 01 : 48 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 01 : 48 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 48 0 [Warning] WSREP: unserialize error invalid protocol version 5 : 71 (Protocol error) Nov 19 17 : 01 : 48 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 01 : 50 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 50 0 [Warning] WSREP: unserialize error invalid flags 6 : 71 (Protocol error) Nov 19 17 : 01 : 50 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 126 Nov 19 17 : 01 : 50 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 50 0 [Warning] WSREP: unserialize error invalid protocol version 5 : 71 (Protocol error) Nov 19 17 : 01 : 50 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 01 : 50 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 50 0 [Warning] WSREP: unserialize error invalid protocol version 4 : 71 (Protocol error) Nov 19 17 : 01 : 50 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 01 : 51 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 51 0 [Warning] WSREP: unserialize error invalid flags 4 : 71 (Protocol error) Nov 19 17 : 01 : 51 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 126 Nov 19 17 : 01 : 51 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 51 0 [Warning] WSREP: unserialize error invalid protocol version 5 : 71 (Protocol error) Nov 19 17 : 01 : 51 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 01 : 51 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 51 0 [Warning] WSREP: unserialize error invalid protocol version 12 : 71 (Protocol error) Nov 19 17 : 01 : 51 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 01 : 52 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 52 0 [Warning] WSREP: unserialize error invalid protocol version 1 : 71 (Protocol error) Nov 19 17 : 01 : 52 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 01 : 52 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 52 0 [Warning] WSREP: unserialize error invalid protocol version 5 : 71 (Protocol error) Nov 19 17 : 01 : 52 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 01 : 59 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 59 0 [Warning] WSREP: unserialize error invalid protocol version 5 : 71 (Protocol error) Nov 19 17 : 01 : 59 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 01 : 59 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 01 : 59 0 [Warning] WSREP: unserialize error invalid protocol version 5 : 71 (Protocol error) Nov 19 17 : 01 : 59 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 02 : 00 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 00 0 [Warning] WSREP: unserialize error invalid protocol version 5 : 71 (Protocol error) Nov 19 17 : 02 : 00 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 02 : 00 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 00 0 [Warning] WSREP: unserialize error invalid protocol version 5 : 71 (Protocol error) Nov 19 17 : 02 : 00 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 02 : 03 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 03 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567' ) connection to peer 00000000 - 0000 with addr timed out, no messages seen in PT3S, socket stats: rtt: 2949 rttvar: 1226 rto: 203000 lost: 0 last_data_recv: 110413322 cwnd: 10 last_queued_since: 3024963641 last_delivered_since: 110713321447043 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0 Nov 19 17 : 02 : 03 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 03 0 [Warning] WSREP: unserialize error invalid protocol version 2 : 71 (Protocol error) Nov 19 17 : 02 : 03 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 02 : 07 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 07 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567' ) connection to peer 00000000 - 0000 with addr timed out, no messages seen in PT3S, socket stats: rtt: 3250 rttvar: 1625 rto: 204000 lost: 0 last_data_recv: 3431 cwnd: 10 last_queued_since: 3433237748 last_delivered_since: 110716822135452 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0 Nov 19 17 : 02 : 07 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 07 0 [Warning] WSREP: unserialize error invalid protocol version 4 : 71 (Protocol error) Nov 19 17 : 02 : 07 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 02 : 07 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 07 0 [Warning] WSREP: unserialize error invalid protocol version 4 : 71 (Protocol error) Nov 19 17 : 02 : 07 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 02 : 07 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 07 0 [Warning] WSREP: unserialize error invalid protocol version 2 : 71 (Protocol error) Nov 19 17 : 02 : 07 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 02 : 07 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 07 0 [Warning] WSREP: unserialize error invalid flags 6 : 71 (Protocol error) Nov 19 17 : 02 : 07 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 126 Nov 19 17 : 02 : 09 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 09 0 [Warning] WSREP: unserialize error invalid protocol version 3 : 71 (Protocol error) Nov 19 17 : 02 : 09 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 02 : 10 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 10 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567' ) connection to peer 00000000 - 0000 with addr timed out, no messages seen in PT3S, socket stats: rtt: 3258 rttvar: 1642 rto: 204000 lost: 0 last_data_recv: 3388 cwnd: 10 last_queued_since: 3391322802 last_delivered_since: 110720322902424 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0 Nov 19 17 : 02 : 14 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 14 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567' ) connection to peer 00000000 - 0000 with addr timed out, no messages seen in PT3S, socket stats: rtt: 2412 rttvar: 1574 rto: 203000 lost: 0 last_data_recv: 3489 cwnd: 10 last_queued_since: 3491386280 last_delivered_since: 110723823637520 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0 Nov 19 17 : 02 : 17 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 17 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567' ) connection to peer 00000000 - 0000 with addr timed out, no messages seen in PT3S, socket stats: rtt: 46870 rttvar: 17884 rto: 247000 lost: 0 last_data_recv: 3447 cwnd: 10 last_queued_since: 3448829822 last_delivered_since: 110727324411113 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0 Nov 19 17 : 02 : 17 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 17 0 [Warning] WSREP: unserialize error invalid protocol version 2 : 71 (Protocol error) Nov 19 17 : 02 : 17 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 02 : 21 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 21 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567' ) connection to peer 00000000 - 0000 with addr timed out, no messages seen in PT3S, socket stats: rtt: 2380 rttvar: 1510 rto: 203000 lost: 0 last_data_recv: 3387 cwnd: 10 last_queued_since: 3389095114 last_delivered_since: 110730825168063 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0 Nov 19 17 : 02 : 21 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 21 0 [Warning] WSREP: unserialize error invalid protocol version 7 : 71 (Protocol error) Nov 19 17 : 02 : 21 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 02 : 21 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 21 0 [Warning] WSREP: unserialize error invalid protocol version 10 : 71 (Protocol error) Nov 19 17 : 02 : 21 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 02 : 24 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 24 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567' ) connection to peer 00000000 - 0000 with addr timed out, no messages seen in PT3S, socket stats: rtt: 2362 rttvar: 1475 rto: 203000 lost: 0 last_data_recv: 3338 cwnd: 10 last_queued_since: 3341175489 last_delivered_since: 110734325955033 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0 Nov 19 17 : 02 : 24 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 24 0 [Warning] WSREP: unserialize error invalid protocol version 2 : 71 (Protocol error) Nov 19 17 : 02 : 24 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 02 : 24 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 24 0 [Warning] WSREP: unserialize error invalid protocol version 6 : 71 (Protocol error) Nov 19 17 : 02 : 24 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 02 : 28 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 28 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567' ) connection to peer 00000000 - 0000 with addr timed out, no messages seen in PT3S, socket stats: rtt: 2357 rttvar: 1464 rto: 203000 lost: 0 last_data_recv: 3332 cwnd: 10 last_queued_since: 3334558479 last_delivered_since: 110737826645820 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0 Nov 19 17 : 02 : 31 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 31 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567' ) connection to peer 00000000 - 0000 with addr timed out, no messages seen in PT3S, socket stats: rtt: 46817 rttvar: 17990 rto: 247000 lost: 0 last_data_recv: 3446 cwnd: 10 last_queued_since: 3447757772 last_delivered_since: 110741327415873 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0 Nov 19 17 : 02 : 31 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 31 0 [Warning] WSREP: unserialize error invalid protocol version 4 : 71 (Protocol error) Nov 19 17 : 02 : 31 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 02 : 31 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 31 0 [Warning] WSREP: unserialize error invalid protocol version 5 : 71 (Protocol error) Nov 19 17 : 02 : 31 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 02 : 31 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 31 0 [Warning] WSREP: unserialize error invalid flags 11 : 71 (Protocol error) Nov 19 17 : 02 : 31 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 126 Nov 19 17 : 02 : 31 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 31 0 [Warning] WSREP: unserialize error invalid protocol version 5 : 71 (Protocol error) Nov 19 17 : 02 : 31 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 02 : 35 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 35 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567' ) connection to peer 00000000 - 0000 with addr timed out, no messages seen in PT3S, socket stats: rtt: 3234 rttvar: 1594 rto: 204000 lost: 0 last_data_recv: 3229 cwnd: 10 last_queued_since: 3230754498 last_delivered_since: 110744828097513 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0 Nov 19 17 : 02 : 38 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 38 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567' ) connection to peer 00000000 - 0000 with addr timed out, no messages seen in PT3S, socket stats: rtt: 45089 rttvar: 17054 rto: 246000 lost: 0 last_data_recv: 3448 cwnd: 10 last_queued_since: 3449277638 last_delivered_since: 110748328836025 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0 Nov 19 17 : 02 : 38 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 38 0 [Warning] WSREP: unserialize error invalid protocol version 15 : 71 (Protocol error) Nov 19 17 : 02 : 38 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 02 : 42 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 42 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567' ) connection to peer 00000000 - 0000 with addr timed out, no messages seen in PT3S, socket stats: rtt: 42631 rttvar: 24592 rto: 243000 lost: 0 last_data_recv: 3351 cwnd: 10 last_queued_since: 3353602435 last_delivered_since: 110751829586149 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0 Nov 19 17 : 02 : 42 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 42 0 [Warning] WSREP: unserialize error invalid protocol version 3 : 71 (Protocol error) Nov 19 17 : 02 : 42 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 02 : 45 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 45 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567' ) connection to peer 00000000 - 0000 with addr timed out, no messages seen in PT3S, socket stats: rtt: 42619 rttvar: 24595 rto: 243000 lost: 0 last_data_recv: 3352 cwnd: 10 last_queued_since: 3354069244 last_delivered_since: 110755330277426 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0 Nov 19 17 : 02 : 45 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 45 0 [Warning] WSREP: unserialize error invalid protocol version 5 : 71 (Protocol error) Nov 19 17 : 02 : 45 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 02 : 45 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 45 0 [Warning] WSREP: unserialize error invalid protocol version 2 : 71 (Protocol error) Nov 19 17 : 02 : 45 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 02 : 48 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 02 : 48 0 [Warning] WSREP: unserialize error invalid protocol version 5 : 71 (Protocol error) Nov 19 17 : 02 : 48 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 03 : 21 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 03 : 21 0 [Warning] WSREP: checksum failed, hdr: len= 0 has_crc32= 0 has_crc32c= 0 crc32= 16777216 Nov 19 17 : 03 : 21 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 03 : 21 0 [Warning] WSREP: checksum failed, hdr: len= 0 has_crc32= 0 has_crc32c= 0 crc32= 16777216 Nov 19 17 : 03 : 21 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 03 : 21 0 [Warning] WSREP: unserialize error invalid protocol version 8 : 71 (Protocol error) Nov 19 17 : 03 : 21 NODE- 01 mysqld[ 78300 ]: at /home/buildbot/buildbot/build/gcomm/src/gcomm/datagram.hpp:unserialize(): 133 Nov 19 17 : 03 : 24 NODE- 01 mysqld[ 78300 ]: 2021 - 11 - 19 17 : 03 : 24 0 [Note] WSREP: (1132bd81-a3fd, 'tcp://0.0.0.0:4567' ) connection to peer 00000000 - 0000 with addr timed out, no messages seen in PT3S, socket stats: rtt: 4207 rttvar: 1915 rto: 205000 lost: 0 last_data_recv: 1065 cwnd: 10 last_queued_since: 3077417409 last_delivered_since: 110793838298892 send_queue_length: 0 send_queue_bytes: 0 segment: 0 messages: 0 OpenVAS Scan Port 3306+4567 Targeted Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: 2021 - 11 - 19 16 : 47 : 42 118 [Warning] Aborted connection 118 to db: 'unconnected' user: 'unauthenticated' host: 'connecting host' (This connection closed normally without authentication) Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<std::system_error> >' Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: what(): remote_endpoint: Transport endpoint is not connected Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: 211119 16 : 47 : 42 [ERROR] mysqld got signal 6 ; Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: This could be because you hit a bug. It is also possible that this binary Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: or one of the libraries it was linked against is corrupt, improperly built, Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: or misconfigured. This error can also be caused by malfunctioning hardware. Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: To report this bug, see https: //mariadb.com/kb/en/reporting-bugs Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: We will try our best to scrape up some info that will hopefully help Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: diagnose the problem, but since we have already crashed, Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: something is definitely wrong and this may fail. Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: Server version: 10.4 . 22 -MariaDB-log Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: key_buffer_size= 134217728 Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: read_buffer_size= 131072 Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: max_used_connections= 32 Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: max_threads= 153 Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: thread_count= 49 Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: It is possible that mysqld could use up to Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 781113 K bytes of memory Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: Hope that's ok; if not, decrease some variables in the equation. Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: Thread pointer: 0x0 Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: Attempting backtrace. You can use the following information to find out Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: where mysqld died. If you see no messages after this , something went Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: terribly wrong... Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: stack_bottom = 0x0 thread_stack 0x49000 Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: /usr/sbin/mysqld(my_print_stacktrace+ 0x2e )[ 0x55ce5f1dd2be ] Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: /usr/sbin/mysqld(handle_fatal_signal+ 0x30f )[ 0x55ce5ec64dbf ] Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: /lib64/libpthread.so. 0 (+ 0xf630 )[ 0x7f682458c630 ] Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: /lib64/libc.so. 6 (gsignal+ 0x37 )[ 0x7f68239d7387 ] Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: /lib64/libc.so. 6 (abort+ 0x148 )[ 0x7f68239d8a78 ] Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: /lib64/libstdc++.so. 6 (_ZN9__gnu_cxx27__verbose_terminate_handlerEv+ 0x165 )[ 0x7f68240d1a95 ] Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: /lib64/libstdc++.so. 6 (+ 0x5ea06 )[ 0x7f68240cfa06 ] Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: /lib64/libstdc++.so. 6 (+ 0x5ea33 )[ 0x7f68240cfa33 ] Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: /lib64/libstdc++.so. 6 (+ 0x5ec53 )[ 0x7f68240cfc53 ] Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: /usr/lib64/galera- 4 /libgalera_smm.so(+ 0x197ab4 )[ 0x7f6819b0bab4 ] Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: /usr/lib64/galera- 4 /libgalera_smm.so(+ 0x197b5c )[ 0x7f6819b0bb5c ] Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: /usr/lib64/galera- 4 /libgalera_smm.so(+ 0x1a3cbf )[ 0x7f6819b17cbf ] Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: /usr/lib64/galera- 4 /libgalera_smm.so(+ 0x1a62e2 )[ 0x7f6819b1a2e2 ] Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: /usr/lib64/galera- 4 /libgalera_smm.so(+ 0x1aa533 )[ 0x7f6819b1e533 ] Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: /usr/lib64/galera- 4 /libgalera_smm.so(+ 0x196eeb )[ 0x7f6819b0aeeb ] Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: /usr/lib64/galera- 4 /libgalera_smm.so(+ 0x192d81 )[ 0x7f6819b06d81 ] Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: /usr/lib64/galera- 4 /libgalera_smm.so(+ 0xbf599 )[ 0x7f6819a33599 ] Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: /usr/lib64/galera- 4 /libgalera_smm.so(+ 0xa65d7 )[ 0x7f6819a1a5d7 ] Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: /usr/lib64/galera- 4 /libgalera_smm.so(+ 0xa6989 )[ 0x7f6819a1a989 ] Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: /lib64/libpthread.so. 0 (+ 0x7ea5 )[ 0x7f6824584ea5 ] Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: /lib64/libc.so. 6 (clone+ 0x6d )[ 0x7f6823a9f9fd ] Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: The manual page at https: //mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: information that should help you find out what is causing the crash. Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: Writing a core file... Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: Working directory at /var/lib/mysql Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: Resource Limits: Nov 19 16 : 47 : 42 NODE- 01 mysqld[ 75077 ]: Fatal signal 11 while backtracing Nov 19 16 : 47 : 43 NODE- 01 systemd[ 1 ]: mariadb.service: main process exited, code=killed, status= 11 /SEGV Nov 19 16 : 47 : 43 NODE- 01 systemd[ 1 ]: Unit mariadb.service entered failed state. Nov 19 16 : 47 : 43 NODE- 01 systemd[ 1 ]: mariadb.service failed. Nov 19 16 : 47 : 48 NODE- 01 systemd[ 1 ]: mariadb.service holdoff time over, scheduling restart. Nov 19 16 : 47 : 48 NODE- 01 systemd[ 1 ]: Stopped MariaDB 10.4 . 22 database server. Nov 19 16 : 47 : 48 NODE- 01 systemd[ 1 ]: Starting MariaDB 10.4 . 22 database server... Additionally I guess it makes sense that I managed to crash our cluster entirely with our scheduled scan which is my OP here, we can run up to 5 machine scans in parallel so it is highly likely the 3 machines in the cluster being scanned at the same time caused a split-brain/impossible automated recovery.

Brendon Abbott added a comment - 2022-01-05 11:45

Watching.
We have also experienced this exact behaviour on a customer's lab system, and it appears to be due to a scan from Nessus which from wikipedia I understand is the same/similar code base to OpenVAS. Luckily, the DB cluster was being sporadically scanned and the crashed node brought back on line before the scan took out a second node and hence split-brain of the 3 node cluster.

Brendon Abbott added a comment - 2022-01-05 11:45 Watching. We have also experienced this exact behaviour on a customer's lab system, and it appears to be due to a scan from Nessus which from wikipedia I understand is the same/similar code base to OpenVAS. Luckily, the DB cluster was being sporadically scanned and the crashed node brought back on line before the scan took out a second node and hence split-brain of the 3 node cluster.

Henrik Wejdmark added a comment - 2022-06-02 12:03

I have the same issue, but on MariaDB 10.6.7. Maria crashes when OpenVas scans the host.

Henrik Wejdmark added a comment - 2022-06-02 12:03 I have the same issue, but on MariaDB 10.6.7. Maria crashes when OpenVas scans the host.

Jan Lindström (Inactive) added a comment - 2022-06-03 05:31

Fix is in Galera library 26.4.12.

Jan Lindström (Inactive) added a comment - 2022-06-03 05:31 Fix is in Galera library 26.4.12.

Sergei Golubchik added a comment - 2023-12-29 11:01

Also fixed in 25.3.37

Sergei Golubchik added a comment - 2023-12-29 11:01 Also fixed in 25.3.37

People

Assignee:: Jan Lindström (Inactive)

Reporter:: Stuart Green

Votes:: 4 Vote for this issue

Watchers:: 14 Start watching this issue

Dates

Created:: 2021-03-05 18:57

Updated:: 2024-07-07 21:44

Resolved:: 2022-06-03 05:31

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.

MariaDB Server

Details

Description

Attachments

Attachments

Issue Links

Activity

People

Dates

Git Integration