Details
Description
3 Node Setup, all 3 nodes crashed. Bootstrap required.
Node 1 crash output
Mar 5 17:48:58 mysqld[116636]: terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<std::system_error> >' |
Mar 5 17:48:58 mysqld[116636]: what(): remote_endpoint: Transport endpoint is not connected |
Mar 5 17:48:58 mysqld[116636]: 210305 17:48:58 [ERROR] mysqld got signal 6 ; |
Mar 5 17:48:58 mysqld[116636]: This could be because you hit a bug. It is also possible that this binary |
Mar 5 17:48:58 mysqld[116636]: or one of the libraries it was linked against is corrupt, improperly built, |
Mar 5 17:48:58 mysqld[116636]: or misconfigured. This error can also be caused by malfunctioning hardware. |
Mar 5 17:48:58 mysqld[116636]: To report this bug, see https://mariadb.com/kb/en/reporting-bugs |
Mar 5 17:48:58 mysqld[116636]: We will try our best to scrape up some info that will hopefully help |
Mar 5 17:48:58 mysqld[116636]: diagnose the problem, but since we have already crashed, |
Mar 5 17:48:58 mysqld[116636]: something is definitely wrong and this may fail. |
Mar 5 17:48:58 mysqld[116636]: Server version: 10.4.18-MariaDB-log |
Mar 5 17:48:58 mysqld[116636]: key_buffer_size=134217728 |
Mar 5 17:48:58 mysqld[116636]: read_buffer_size=131072 |
Mar 5 17:48:58 mysqld[116636]: max_used_connections=76 |
Mar 5 17:48:58 mysqld[116636]: max_threads=153 |
Mar 5 17:48:58 mysqld[116636]: thread_count=75 |
Mar 5 17:48:58 mysqld[116636]: It is possible that mysqld could use up to |
Mar 5 17:48:58 mysqld[116636]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 781104 K bytes of memory |
Mar 5 17:48:58 mysqld[116636]: Hope that's ok; if not, decrease some variables in the equation. |
Mar 5 17:48:58 mysqld[116636]: Thread pointer: 0x0 |
Mar 5 17:48:58 mysqld[116636]: Attempting backtrace. You can use the following information to find out |
Mar 5 17:48:58 mysqld[116636]: where mysqld died. If you see no messages after this, something went |
Mar 5 17:48:58 mysqld[116636]: terribly wrong... |
Mar 5 17:48:59 mysqld[116636]: stack_bottom = 0x0 thread_stack 0x49000 |
Mar 5 17:48:59 mysqld[116636]: /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x55a14151e47e] |
Mar 5 17:48:59 mysqld[116636]: /usr/sbin/mysqld(handle_fatal_signal+0x30f)[0x55a140face8f] |
Mar 5 17:48:59 mysqld[116636]: /lib64/libpthread.so.0(+0xf630)[0x7f5879976630] |
Mar 5 17:48:59 mysqld[116636]: /lib64/libc.so.6(gsignal+0x37)[0x7f5877c463d7] |
Mar 5 17:48:59 mysqld[116636]: /lib64/libc.so.6(abort+0x148)[0x7f5877c47ac8] |
Mar 5 17:48:59 mysqld[116636]: /lib64/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x165)[0x7f5878340a95] |
Mar 5 17:48:59 mysqld[116636]: /lib64/libstdc++.so.6(+0x5ea06)[0x7f587833ea06] |
Mar 5 17:48:59 mysqld[116636]: /lib64/libstdc++.so.6(+0x5ea33)[0x7f587833ea33] |
Mar 5 17:48:59 mysqld[116636]: /lib64/libstdc++.so.6(+0x5ec53)[0x7f587833ec53] |
Mar 5 17:48:59 mysqld[116636]: /usr/lib64/galera-4/libgalera_smm.so(+0x1997a4)[0x7f586ddc07a4] |
Mar 5 17:48:59 mysqld[116636]: /usr/lib64/galera-4/libgalera_smm.so(+0x19984c)[0x7f586ddc084c] |
Mar 5 17:48:59 mysqld[116636]: /usr/lib64/galera-4/libgalera_smm.so(+0x1a57bf)[0x7f586ddcc7bf] |
Mar 5 17:48:59 mysqld[116636]: /usr/lib64/galera-4/libgalera_smm.so(+0x1a78b8)[0x7f586ddce8b8] |
Mar 5 17:48:59 mysqld[116636]: /usr/lib64/galera-4/libgalera_smm.so(+0x1aaf13)[0x7f586ddd1f13] |
Mar 5 17:48:59 mysqld[116636]: /usr/lib64/galera-4/libgalera_smm.so(+0x19897b)[0x7f586ddbf97b] |
Mar 5 17:48:59 mysqld[116636]: /usr/lib64/galera-4/libgalera_smm.so(+0x194b91)[0x7f586ddbbb91] |
Mar 5 17:48:59 mysqld[116636]: /usr/lib64/galera-4/libgalera_smm.so(+0xc08c9)[0x7f586dce78c9] |
Mar 5 17:48:59 mysqld[116636]: /usr/lib64/galera-4/libgalera_smm.so(+0xa6bcf)[0x7f586dccdbcf] |
Mar 5 17:48:59 mysqld[116636]: /usr/lib64/galera-4/libgalera_smm.so(+0xad859)[0x7f586dcd4859] |
Mar 5 17:48:59 mysqld[116636]: /lib64/libpthread.so.0(+0x7ea5)[0x7f587996eea5] |
Mar 5 17:48:59 mysqld[116636]: /lib64/libc.so.6(clone+0x6d)[0x7f5877d0e9fd] |
|
Attachments
Issue Links
- is duplicated by
-
MDEV-30336 Galera: Handshake failed: peer did not return a certificate
-
- Closed
-
- relates to
-
MDEV-27004 McAfee database vulnerability scan caused MariaDB crash with signal 6 (system abort)
-
- Closed
-
-
MDEV-26861 Galera Crashing - what(): remote_endpoint: Transport endpoint is not connected
-
- Closed
-
- links to
Activity
Field | Original Value | New Value |
---|---|---|
Fix Version/s | 10.4 [ 22408 ] | |
Assignee | Jan Lindström [ jplindst ] |
Attachment | log-10.4.2 [ 59783 ] | |
Attachment | log-10.5.11 [ 59784 ] |
Attachment | gdb.txt [ 59785 ] |
Comment |
[ I've also seen had this crash happen numerous times on 2 separate 3-node clusters.
The 2 clusters are running different versions, one MariaDB 10.4.20 cluster & another 10.5.11 cluster. This has taken one of the clusters down - requiring a manual bootstrap (this happened when 2 nodes in the cluster both suffered from this crash within 20 seconds of each other). Otherwise, the clusters are able to recover when just one node suffers this crash. Included a backtrace for the cluster running 10.5.11 [^log-10.5.11] [^gdb.txt] [^log-10.4.2] ] |
Affects Version/s | 10.4.22 [ 26031 ] |
Summary | 10.4.18 crashes with Transport endpoint is not connected mysqld got signal 6 ; | Node crashes with Transport endpoint is not connected mysqld got signal 6 ; |
Affects Version/s | 10.4.21 [ 26030 ] | |
Affects Version/s | 10.4.20 [ 25733 ] | |
Affects Version/s | 10.4.19 [ 25205 ] |
Affects Version/s | 10.4 [ 22408 ] |
Affects Version/s | 10.4 [ 22408 ] |
Workflow | MariaDB v3 [ 119845 ] | MariaDB v4 [ 142658 ] |
Assignee | Jan Lindström [ jplindst ] | Seppo Jaakola [ seppo ] |
Affects Version/s | 10.5.9 [ 25109 ] |
Assignee | Seppo Jaakola [ seppo ] | Teemu Ollakka [ teemu.ollakka ] |
Assignee | Teemu Ollakka [ teemu.ollakka ] | Mario Karuza [ mkaruza ] |
Assignee | Mario Karuza [ mkaruza ] | Pramod Mahto [ pramod.mahto@mariadb.com ] |
Status | Open [ 1 ] | Needs Feedback [ 10501 ] |
Attachment | _InVuln__PCI-DSS_ORG023150_OneCloudINFRA_day3__jpw1a_2_of_2__09nptb.zip [ 62301 ] |
Assignee | Pramod Mahto [ pramod.mahto@mariadb.com ] | Mario Karuza [ mkaruza ] |
Link | This issue is blocked by MENT-1419 [ MENT-1419 ] |
Status | Needs Feedback [ 10501 ] | Open [ 1 ] |
Assignee | Mario Karuza [ mkaruza ] | Jan Lindström [ jplindst ] |
Status | Open [ 1 ] | Confirmed [ 10101 ] |
Status | Confirmed [ 10101 ] | In Progress [ 3 ] |
Assignee | Jan Lindström [ jplindst ] | Julien Fritsch [ julien.fritsch ] |
Assignee | Julien Fritsch [ julien.fritsch ] | Jan Lindström [ jplindst ] |
Status | In Progress [ 3 ] | In Review [ 10002 ] |
Link |
This issue relates to |
Link |
This issue relates to |
Priority | Major [ 3 ] | Critical [ 2 ] |
issue.field.resolutiondate | 2022-06-03 05:31:13.0 | 2022-06-03 05:31:13.396 |
Component/s | Galera [ 10124 ] | |
Component/s | wsrep [ 11500 ] | |
Fix Version/s | 10.4.26 [ 27511 ] | |
Fix Version/s | 10.5.17 [ 27509 ] | |
Fix Version/s | 10.8.4 [ 27503 ] | |
Fix Version/s | 10.6.9, 10.7.5 [ 28203 ] | |
Fix Version/s | 10.4 [ 22408 ] | |
Resolution | Fixed [ 1 ] | |
Status | In Review [ 10002 ] | Closed [ 6 ] |
Fix Version/s | 10.6.9 [ 27507 ] | |
Fix Version/s | 10.7.5 [ 27505 ] | |
Fix Version/s | 10.6.9, 10.7.5 [ 28203 ] |
Link |
This issue is duplicated by |
Remote Link | This issue links to "CVE-2023-5157 (Web Link)" [ 36165 ] |
Fix Version/s | 10.3.36 [ 27513 ] |
Zendesk Related Tickets | 136631 |