Details
-
Bug
-
Status: Open (View Workflow)
-
Major
-
Resolution: Unresolved
-
10.11.5
Description
Hello Team,
We have again observed DB1 crash with same error as Singal 11 Segmentation Fault.
Please check this issue and provide solution.
Please refer the below logs.
/var/log/messages:
Nov 10 05:33:28 ALEPO-DB1-ILB NetworkManager[1281]: <info> [1731198808.9990] device (enp9s0): Activation: starting connection 'opt-net' (38c00c03-2f91-40f7-a4bb-1a9c28006b7b)
|
Nov 10 05:33:28 ALEPO-DB1-ILB NetworkManager[1281]: <info> [1731198808.9991] device (enp9s0): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'managed')
|
Nov 10 05:33:28 ALEPO-DB1-ILB NetworkManager[1281]: <info> [1731198808.9992] device (enp9s0): state change: prepare -> config (reason 'none', sys-iface-state: 'managed')
|
Nov 10 05:33:29 ALEPO-DB1-ILB NetworkManager[1281]: <info> [1731198809.0047] device (enp9s0): state change: config -> ip-config (reason 'none', sys-iface-state: 'managed')
|
Nov 10 05:33:29 ALEPO-DB1-ILB NetworkManager[1281]: <info> [1731198809.0050] dhcp4 (enp9s0): activation: beginning transaction (timeout in 45 seconds)
|
Nov 10 05:33:34 ALEPO-DB1-ILB systemd[1]: mariadb.service: Main process exited, code=killed, status=11/SEGV
|
Nov 10 05:33:34 ALEPO-DB1-ILB systemd[1]: mariadb.service: Failed with result 'signal'.
|
Nov 10 05:33:39 ALEPO-DB1-ILB systemd[1]: mariadb.service: Service RestartSec=5s expired, scheduling restart.
|
Nov 10 05:33:39 ALEPO-DB1-ILB systemd[1]: mariadb.service: Scheduled restart job, restart counter is at 3.
|
Nov 10 05:33:39 ALEPO-DB1-ILB systemd[1]: Stopped MariaDB 10.11.5 database server.
|
Nov 10 05:33:39 ALEPO-DB1-ILB systemd[1]: Starting MariaDB 10.11.5 database server...
|
Nov 10 05:33:40 ALEPO-DB1-ILB mariadbd[484800]: 2024-11-10 5:33:40 0 [Warning] Could not increase number of max_open_files to more than 32768 (request: 65535)
|
Nov 10 05:34:11 ALEPO-DB1-ILB systemd[1]: Started MariaDB 10.11.5 database server.
|
Nov 10 05:34:13 ALEPO-DB1-ILB NetworkManager[1281]: <info> [1731198853.9165] device (enp9s0): state change: ip-config -> failed (reason 'ip-config-unavailable', sys-iface-state: 'managed')
|
Nov 10 05:34:13 ALEPO-DB1-ILB NetworkManager[1281]: <warn> [1731198853.9169] device (enp9s0): Activation: failed for connection 'opt-net'
|
Nov 10 05:34:13 ALEPO-DB1-ILB NetworkManager[1281]: <info> [1731198853.9170] device (enp9s0): state change: failed -> disconnected (reason 'none', sys-iface-state: 'managed')
|
Nov 10 05:34:13 ALEPO-DB1-ILB NetworkManager[1281]: <info> [1731198853.9260] dhcp4 (enp9s0): canceled DHCP transaction
|
Nov 10 05:34:13 ALEPO-DB1-ILB NetworkManager[1281]: <info> [1731198853.9267] policy: auto-activating connection 'opt-net' (38c00c03-2f91-40f7-a4bb-1a9c28006b7b)
|
Nov 10 05:34:13 ALEPO-DB1-ILB NetworkManager[1281]: <info> [1731198853.9270] device (enp9s0): Activation: starting connection 'opt-net' (38c00c03-2f91-40f7-a4bb-1a9c28006b7b)
|
/var/lib/mysql/mysqld.log
241110 5:29:51 [ERROR] mysqld got signal 11 ;
|
This could be because you hit a bug. It is also possible that this binary
|
or one of the libraries it was linked against is corrupt, improperly built,
|
or misconfigured. This error can also be caused by malfunctioning hardware.
|
|
|
To report this bug, see MariaDB Community Bug Reports and Feature Requests
|
|
|
We will try our best to scrape up some info that will hopefully help
|
diagnose the problem, but since we have already crashed,
|
something is definitely wrong and this may fail.
|
|
|
Server version: 10.11.5-MariaDB-log source revision: 7875294b6b74b53dd3aaa723e6cc103d2bb47b2c
|
key_buffer_size=67108864
|
read_buffer_size=131072
|
max_used_connections=459
|
max_threads=802
|
thread_count=468
|
It is possible that mysqld could use up to
|
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1831964 K bytes of memory
|
Hope that's ok; if not, decrease some variables in the equation.
|
|
|
Thread pointer: 0x7f4db8018fb8
|
Attempting backtrace. You can use the following information to find out
|
where mysqld died. If you see no messages after this, something went
|
terribly wrong...
|
stack_bottom = 0x7f4fd9139b98 thread_stack 0x49000
|
/usr/sbin/mariadbd(my_print_stacktrace+0x2e)[0x558dd648f12e]
|
mysys/stacktrace.c:216(my_print_stacktrace)[0x558dd5f885b5]
|
/lib64/libpthread.so.0(+0x12cf0)[0x7f78b8682cf0]
|
sql/multi_range_read.cc:1383(DsMrr_impl::close_second_handler())[0x558dd5e8e6b0]
|
sql/multi_range_read.cc:1398(DsMrr_impl::dsmrr_close())[0x558dd5e8e7c9]
|
handler/ha_innodb.cc:15770(wsrep_notify_status(wsrep::server_state::state, wsrep::view const*))[0x558dd62323e7]
|
sql/ha_partition.cc:9520(MyCTX_nopad::finish(unsigned char*, unsigned int*))[0x558dd6178481]
|
sql/sql_base.cc:1022(close_thread_table(THD*, TABLE**))[0x558dd5cd67d1]
|
sql/sql_base.cc:965(close_thread_tables(THD*))[0x558dd5cd6aeb]
|
sql/sql_parse.cc:6097(mysql_execute_command(THD*, bool))[0x558dd5d46d7f]
|
sql/sql_parse.cc:8035(mysql_parse(THD*, char*, unsigned int, Parser_state*))[0x558dd5d3ab87]
|
sql/sql_parse.cc:1896(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool))[0x558dd5d446e6]
|
sql/sql_parse.cc:1407(do_command(THD*, bool))[0x558dd5d45dab]
|
sql/sql_connect.cc:1416(do_handle_one_connection(CONNECT*, bool))[0x558dd5e59977]
|
sql/sql_connect.cc:1318(handle_one_connection)[0x558dd5e59cbd]
|
perfschema/pfs.cc:2204(MyCTX_nopad::finish(unsigned char*, unsigned int*))[0x558dd618fc9d]
|
/lib64/libpthread.so.0(+0x81ca)[0x7f78b86781ca]
|
/lib64/libc.so.6(clone+0x43)[0x7f78b79c8e73]
|
|
|
Trying to get some variables.
|
Some pointers may be invalid and cause the dump to abort.
|
Query (0x7f4db8028e30): Insert Into AuthenticationLog (UserIndex,UserName,NasIp,NasPort,RequestId,RadiusCode,RejectReason,PortMessage,TimeStamp,RadiusServerName,AlepoSessionId,PasswordTyped,CallerId,CalledStationId,NasPortType,ErrorCode,CustomField1,CustomField2,CustomField3,CustomField5,customfield4,customnumfield1,customnumfield2,customnumfield3,customnumfield4,customnumfield5) Values(51883661,'03007915007GG',3414627232,339741342,132,2,'USER REJECTED - MAC NOT MATCHED: NAS-ERROR-CODE=0, MAC-ERROR-CODE=22','Access Accept','2024-11-10 05:29:49','AAA1-ILB','HWI-FSD081172718000008ba0e8AAAlnZ03007915007GG','','34:36:54:dd:8c:08','',13,10022,'10.139.4.46 - 0/9/30/0',' - ','34:36:54:dd:8c:07','10.139.4.46-FT-FSD-C11Am-H-M atm 0/9/0/30:0.103','NULL',1,2,0,0,0)
|
|
|
2024-11-10 5:34:11 0 [Note] Crash table recovery finished.
|
2024-11-10 5:34:11 0 [Note] Server socket created on IP: '0.0.0.0'.
|
2024-11-10 5:34:11 0 [Note] Server socket created on IP: '::'.
|
2024-11-10 5:34:11 0 [Warning] 'user' entry '@alepo-db2-ilb' ignored in --skip-name-resolve mode.
|
2024-11-10 5:34:11 0 [Warning] 'user' entry 'root@localhost.localdomain' ignored in --skip-name-resolve mode.
|
2024-11-10 5:34:11 0 [Warning] 'proxies_priv' entry '@% root@alepo-db2-ilb' ignored in --skip-name-resolve mode.
|
2024-11-10 5:34:11 4 [Note] Reading Master_info: 'master-s@002ddb3.info' Relay_info:'relay-log-s@002ddb3.info'
|
2024-11-10 5:34:11 4 [Note] Initialized Master_info from 'master-s@002ddb3.info'
|
2024-11-10 5:34:11 5 [Note] Master 's-db3': Slave I/O thread: Start asynchronous replication to master '@:3306' in log '' at position 4
|
2024-11-10 5:34:11 5 [ERROR] Master 's-db3': Slave I/O: Fatal error: Invalid (empty) username when attempting to connect to the master server. Connection attempt terminated. Internal MariaDB error code: 1593
|
2024-11-10 5:34:11 5 [Note] Master 's-db3': Slave I/O thread killed while connecting to master
|
2024-11-10 5:34:11 5 [Note] Master 's-db3': Slave I/O thread exiting, read up to log 'FIRST', position 4, master :3306
|
2024-11-10 5:34:11 4 [Note] Started replication for 's-db3'
|
2024-11-10 5:34:11 6 [Note] Master 's-db3': Slave SQL thread initialized, starting replication in log 'FIRST' at position 0, relay log './db1-relay-s@002ddb3.000040' position: 4
|
2024-11-10 5:34:11 4 [Note] Reading Master_info: 'master-s@002ddb4.info' Relay_info:'relay-log-s@002ddb4.info'
|
2024-11-10 5:34:11 4 [Note] Initialized Master_info from 'master-s@002ddb4.info'
|
2024-11-10 5:34:11 19 [Note] Master 's-db4': Slave I/O thread: Start asynchronous replication to master 'gtid@10.76.25.11:3306' in log 'db4-binary.236723' at position 157701627
|
2024-11-10 5:34:11 4 [Note] Started replication for 's-db4'
|
2024-11-10 5:34:11 4 [Note] Reading Master_info: 'master-s@002ddb2.info' Relay_info:'relay-log-s@002ddb2.info'
|
|
|
2024-11-10 5:34:11 4 [Note] Reading Master_info: 'master-s@002ddb4.info' Relay_info:'relay-log-s@002ddb4.info'
|
2024-11-10 5:34:11 4 [Note] Initialized Master_info from 'master-s@002ddb4.info'
|
2024-11-10 5:34:11 19 [Note] Master 's-db4': Slave I/O thread: Start asynchronous replication to master 'gtid@10.76.25.11:3306' in log 'db4-binary.236723' at position 157701627
|
2024-11-10 5:34:11 4 [Note] Started replication for 's-db4'
|
2024-11-10 5:34:11 4 [Note] Reading Master_info: 'master-s@002ddb2.info' Relay_info:'relay-log-s@002ddb2.info'
|
2024-11-10 5:34:11 20 [Note] Master 's-db4': Slave SQL thread initialized, starting replication in log 'db4-binary.236723' at position 157701627, relay log './db1-relay-s@002ddb4.171946' position: 783
|
2024-11-10 5:34:11 4 [Note] Initialized Master_info from 'master-s@002ddb2.info'
|
2024-11-10 5:34:11 21 [Note] Master 's-db2': Slave I/O thread: Start asynchronous replication to master '@:3306' in log '' at position 4
|
2024-11-10 5:34:11 21 [ERROR] Master 's-db2': Slave I/O: Fatal error: Invalid (empty) username when attempting to connect to the master server. Connection attempt terminated. Internal MariaDB error code: 1593
|
2024-11-10 5:34:11 21 [Note] Master 's-db2': Slave I/O thread killed while connecting to master
|
2024-11-10 5:34:11 21 [Note] Master 's-db2': Slave I/O thread exiting, read up to log 'FIRST', position 4, master :3306
|
2024-11-10 5:34:11 4 [Note] Started replication for 's-db2'
|
2024-11-10 5:34:11 4 [Note] Reading Master_info: 'master-s@002ddb5.info' Relay_info:'relay-log-s@002ddb5.info'
|
2024-11-10 5:34:11 22 [Note] Master 's-db2': Slave SQL thread initialized, starting replication in log 'FIRST' at position 0, relay log './db1-relay-s@002ddb2.000039' position: 4
|
2024-11-10 5:34:11 4 [Note] Initialized Master_info from 'master-s@002ddb5.info'
|
2024-11-10 5:34:11 23 [Note] Master 's-db5': Slave I/O thread: Start asynchronous replication to master '@:3306' in log '' at position 4
|
2024-11-10 5:34:11 23 [ERROR] Master 's-db5': Slave I/O: Fatal error: Invalid (empty) username when attempting to connect to the master server. Connection attempt terminated. Internal MariaDB error code: 1593
|
2024-11-10 5:34:11 23 [Note] Master 's-db5': Slave I/O thread killed while connecting to master
|
2024-11-10 5:34:11 23 [Note] Master 's-db5': Slave I/O thread exiting, read up to log 'FIRST', position 4, master :3306
|
2024-11-10 5:34:11 4 [Note] Started replication for 's-db5'
|
2024-11-10 5:34:11 24 [Note] Master 's-db5': Slave SQL thread initialized, starting replication in log 'FIRST' at position 0, relay log './db1-relay-s@002ddb5.000031' position: 4
|
MMM_Log
/var/log/mysql-mmm/ mmm_mond.log
2024/11/10 05:30:22 WARN Check 'rep_threads' on 'db1' is in unknown state! Message: UNKNOWN: Connect error (host = 10.76.24.71:3306, user = mmm_monitor)! Lost connection to server at 'handshake: reading initial communication packet', system error: 110
|
2024/11/10 05:30:23 WARN Check 'rep_backlog' on 'db1' is in unknown state! Message: UNKNOWN: Connect error (host = 10.76.24.71:3306, user = mmm_monitor)! Lost connection to server at 'handshake: reading initial communication packet', system error: 110
|
2024/11/10 05:30:39 ERROR Check 'mysql' on 'db1' has failed for 14 seconds! Message: ERROR: Connect error (host = 10.76.24.71:3306, user = mmm_monitor)! Lost connection to server at 'handshake: reading initial communication packet', system error: 110
|
2024/11/10 05:30:42 FATAL State of host 'db1' changed from ONLINE to HARD_OFFLINE (ping: OK, mysql: not OK)
|
2024/11/10 05:30:42 INFO Removing all roles from host 'db1':
|
2024/11/10 05:30:42 INFO Removed role 'writer(10.76.24.100)' from host 'db1'
|
2024/11/10 05:30:45 INFO Orphaned role 'writer(10.76.24.100)' has been assigned to 'db2'
|
2024/11/10 05:34:11 INFO Check 'rep_threads' on 'db1' is ok!
|
2024/11/10 05:34:13 INFO Check 'rep_backlog' on 'db1' is ok!
|
2024/11/10 05:34:14 INFO Check 'mysql' on 'db1' is ok!
|
2024/11/10 05:34:16 FATAL State of host 'db1' changed from HARD_OFFLINE to AWAITING_RECOVERY
|
2024/11/10 05:35:16 FATAL State of host 'db1' changed from AWAITING_RECOVERY to ONLINE because of auto_set_online(60 seconds). It was in state AWAITING_RECOVERY for 60 seconds
|
Coredump:
/var/log/messages:
Nov 10 05:33:28 ALEPO-DB1-ILB NetworkManager[1281]: <info> [1731198808.9990] device (enp9s0): Activation: starting connection 'opt-net' (38c00c03-2f91-40f7-a4bb-1a9c28006b7b)
|
Nov 10 05:33:28 ALEPO-DB1-ILB NetworkManager[1281]: <info> [1731198808.9991] device (enp9s0): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'managed')
|
Nov 10 05:33:28 ALEPO-DB1-ILB NetworkManager[1281]: <info> [1731198808.9992] device (enp9s0): state change: prepare -> config (reason 'none', sys-iface-state: 'managed')
|
Nov 10 05:33:29 ALEPO-DB1-ILB NetworkManager[1281]: <info> [1731198809.0047] device (enp9s0): state change: config -> ip-config (reason 'none', sys-iface-state: 'managed')
|
Nov 10 05:33:29 ALEPO-DB1-ILB NetworkManager[1281]: <info> [1731198809.0050] dhcp4 (enp9s0): activation: beginning transaction (timeout in 45 seconds)
|
Nov 10 05:33:34 ALEPO-DB1-ILB systemd[1]: mariadb.service: Main process exited, code=killed, status=11/SEGV
|
Nov 10 05:33:34 ALEPO-DB1-ILB systemd[1]: mariadb.service: Failed with result 'signal'.
|
Nov 10 05:33:39 ALEPO-DB1-ILB systemd[1]: mariadb.service: Service RestartSec=5s expired, scheduling restart.
|
Nov 10 05:33:39 ALEPO-DB1-ILB systemd[1]: mariadb.service: Scheduled restart job, restart counter is at 3.
|
Nov 10 05:33:39 ALEPO-DB1-ILB systemd[1]: Stopped MariaDB 10.11.5 database server.
|
Nov 10 05:33:39 ALEPO-DB1-ILB systemd[1]: Starting MariaDB 10.11.5 database server...
|
Nov 10 05:33:40 ALEPO-DB1-ILB mariadbd[484800]: 2024-11-10 5:33:40 0 [Warning] Could not increase number of max_open_files to more than 32768 (request: 65535)
|
Nov 10 05:34:11 ALEPO-DB1-ILB systemd[1]: Started MariaDB 10.11.5 database server.
|
Nov 10 05:34:13 ALEPO-DB1-ILB NetworkManager[1281]: <info> [1731198853.9165] device (enp9s0): state change: ip-config -> failed (reason 'ip-config-unavailable', sys-iface-state: 'managed')
|
Nov 10 05:34:13 ALEPO-DB1-ILB NetworkManager[1281]: <warn> [1731198853.9169] device (enp9s0): Activation: failed for connection 'opt-net'
|
Nov 10 05:34:13 ALEPO-DB1-ILB NetworkManager[1281]: <info> [1731198853.9170] device (enp9s0): state change: failed -> disconnected (reason 'none', sys-iface-state: 'managed')
|
Nov 10 05:34:13 ALEPO-DB1-ILB NetworkManager[1281]: <info> [1731198853.9260] dhcp4 (enp9s0): canceled DHCP transaction
|
Nov 10 05:34:13 ALEPO-DB1-ILB NetworkManager[1281]: <info> [1731198853.9267] policy: auto-activating connection 'opt-net' (38c00c03-2f91-40f7-a4bb-1a9c28006b7b)
|
Nov 10 05:34:13 ALEPO-DB1-ILB NetworkManager[1281]: <info> [1731198853.9270] device (enp9s0): Activation: starting connection 'opt-net' (38c00c03-2f91-40f7-a4bb-1a9c28006b7b)
|
/var/lib/mysql/mysqld.log
241110 5:29:51 [ERROR] mysqld got signal 11 ;
|
This could be because you hit a bug. It is also possible that this binary
|
or one of the libraries it was linked against is corrupt, improperly built,
|
or misconfigured. This error can also be caused by malfunctioning hardware.
|
|
|
To report this bug, see MariaDB Community Bug Reports and Feature Requests
|
|
|
We will try our best to scrape up some info that will hopefully help
|
diagnose the problem, but since we have already crashed,
|
something is definitely wrong and this may fail.
|
|
|
Server version: 10.11.5-MariaDB-log source revision: 7875294b6b74b53dd3aaa723e6cc103d2bb47b2c
|
key_buffer_size=67108864
|
read_buffer_size=131072
|
max_used_connections=459
|
max_threads=802
|
thread_count=468
|
It is possible that mysqld could use up to
|
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1831964 K bytes of memory
|
Hope that's ok; if not, decrease some variables in the equation.
|
|
|
Thread pointer: 0x7f4db8018fb8
|
Attempting backtrace. You can use the following information to find out
|
where mysqld died. If you see no messages after this, something went
|
terribly wrong...
|
stack_bottom = 0x7f4fd9139b98 thread_stack 0x49000
|
/usr/sbin/mariadbd(my_print_stacktrace+0x2e)[0x558dd648f12e]
|
mysys/stacktrace.c:216(my_print_stacktrace)[0x558dd5f885b5]
|
/lib64/libpthread.so.0(+0x12cf0)[0x7f78b8682cf0]
|
sql/multi_range_read.cc:1383(DsMrr_impl::close_second_handler())[0x558dd5e8e6b0]
|
sql/multi_range_read.cc:1398(DsMrr_impl::dsmrr_close())[0x558dd5e8e7c9]
|
handler/ha_innodb.cc:15770(wsrep_notify_status(wsrep::server_state::state, wsrep::view const*))[0x558dd62323e7]
|
sql/ha_partition.cc:9520(MyCTX_nopad::finish(unsigned char*, unsigned int*))[0x558dd6178481]
|
sql/sql_base.cc:1022(close_thread_table(THD*, TABLE**))[0x558dd5cd67d1]
|
sql/sql_base.cc:965(close_thread_tables(THD*))[0x558dd5cd6aeb]
|
sql/sql_parse.cc:6097(mysql_execute_command(THD*, bool))[0x558dd5d46d7f]
|
sql/sql_parse.cc:8035(mysql_parse(THD*, char*, unsigned int, Parser_state*))[0x558dd5d3ab87]
|
sql/sql_parse.cc:1896(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool))[0x558dd5d446e6]
|
sql/sql_parse.cc:1407(do_command(THD*, bool))[0x558dd5d45dab]
|
sql/sql_connect.cc:1416(do_handle_one_connection(CONNECT*, bool))[0x558dd5e59977]
|
sql/sql_connect.cc:1318(handle_one_connection)[0x558dd5e59cbd]
|
perfschema/pfs.cc:2204(MyCTX_nopad::finish(unsigned char*, unsigned int*))[0x558dd618fc9d]
|
/lib64/libpthread.so.0(+0x81ca)[0x7f78b86781ca]
|
/lib64/libc.so.6(clone+0x43)[0x7f78b79c8e73]
|
|
|
Trying to get some variables.
|
Some pointers may be invalid and cause the dump to abort.
|
Query (0x7f4db8028e30): Insert Into AuthenticationLog (UserIndex,UserName,NasIp,NasPort,RequestId,RadiusCode,RejectReason,PortMessage,TimeStamp,RadiusServerName,AlepoSessionId,PasswordTyped,CallerId,CalledStationId,NasPortType,ErrorCode,CustomField1,CustomField2,CustomField3,CustomField5,customfield4,customnumfield1,customnumfield2,customnumfield3,customnumfield4,customnumfield5) Values(51883661,'03007915007GG',3414627232,339741342,132,2,'USER REJECTED - MAC NOT MATCHED: NAS-ERROR-CODE=0, MAC-ERROR-CODE=22','Access Accept','2024-11-10 05:29:49','AAA1-ILB','HWI-FSD081172718000008ba0e8AAAlnZ03007915007GG','','34:36:54:dd:8c:08','',13,10022,'10.139.4.46 - 0/9/30/0',' - ','34:36:54:dd:8c:07','10.139.4.46-FT-FSD-C11Am-H-M atm 0/9/0/30:0.103','NULL',1,2,0,0,0)
|
|
|
2024-11-10 5:34:11 0 [Note] Crash table recovery finished.
|
2024-11-10 5:34:11 0 [Note] Server socket created on IP: '0.0.0.0'.
|
2024-11-10 5:34:11 0 [Note] Server socket created on IP: '::'.
|
2024-11-10 5:34:11 0 [Warning] 'user' entry '@alepo-db2-ilb' ignored in --skip-name-resolve mode.
|
2024-11-10 5:34:11 0 [Warning] 'user' entry 'root@localhost.localdomain' ignored in --skip-name-resolve mode.
|
2024-11-10 5:34:11 0 [Warning] 'proxies_priv' entry '@% root@alepo-db2-ilb' ignored in --skip-name-resolve mode.
|
2024-11-10 5:34:11 4 [Note] Reading Master_info: 'master-s@002ddb3.info' Relay_info:'relay-log-s@002ddb3.info'
|
2024-11-10 5:34:11 4 [Note] Initialized Master_info from 'master-s@002ddb3.info'
|
2024-11-10 5:34:11 5 [Note] Master 's-db3': Slave I/O thread: Start asynchronous replication to master '@:3306' in log '' at position 4
|
2024-11-10 5:34:11 5 [ERROR] Master 's-db3': Slave I/O: Fatal error: Invalid (empty) username when attempting to connect to the master server. Connection attempt terminated. Internal MariaDB error code: 1593
|
2024-11-10 5:34:11 5 [Note] Master 's-db3': Slave I/O thread killed while connecting to master
|
2024-11-10 5:34:11 5 [Note] Master 's-db3': Slave I/O thread exiting, read up to log 'FIRST', position 4, master :3306
|
2024-11-10 5:34:11 4 [Note] Started replication for 's-db3'
|
2024-11-10 5:34:11 6 [Note] Master 's-db3': Slave SQL thread initialized, starting replication in log 'FIRST' at position 0, relay log './db1-relay-s@002ddb3.000040' position: 4
|
2024-11-10 5:34:11 4 [Note] Reading Master_info: 'master-s@002ddb4.info' Relay_info:'relay-log-s@002ddb4.info'
|
2024-11-10 5:34:11 4 [Note] Initialized Master_info from 'master-s@002ddb4.info'
|
2024-11-10 5:34:11 19 [Note] Master 's-db4': Slave I/O thread: Start asynchronous replication to master 'gtid@10.76.25.11:3306' in log 'db4-binary.236723' at position 157701627
|
2024-11-10 5:34:11 4 [Note] Started replication for 's-db4'
|
2024-11-10 5:34:11 4 [Note] Reading Master_info: 'master-s@002ddb2.info' Relay_info:'relay-log-s@002ddb2.info'
|
|
|
2024-11-10 5:34:11 4 [Note] Reading Master_info: 'master-s@002ddb4.info' Relay_info:'relay-log-s@002ddb4.info'
|
2024-11-10 5:34:11 4 [Note] Initialized Master_info from 'master-s@002ddb4.info'
|
2024-11-10 5:34:11 19 [Note] Master 's-db4': Slave I/O thread: Start asynchronous replication to master 'gtid@10.76.25.11:3306' in log 'db4-binary.236723' at position 157701627
|
2024-11-10 5:34:11 4 [Note] Started replication for 's-db4'
|
2024-11-10 5:34:11 4 [Note] Reading Master_info: 'master-s@002ddb2.info' Relay_info:'relay-log-s@002ddb2.info'
|
2024-11-10 5:34:11 20 [Note] Master 's-db4': Slave SQL thread initialized, starting replication in log 'db4-binary.236723' at position 157701627, relay log './db1-relay-s@002ddb4.171946' position: 783
|
2024-11-10 5:34:11 4 [Note] Initialized Master_info from 'master-s@002ddb2.info'
|
2024-11-10 5:34:11 21 [Note] Master 's-db2': Slave I/O thread: Start asynchronous replication to master '@:3306' in log '' at position 4
|
2024-11-10 5:34:11 21 [ERROR] Master 's-db2': Slave I/O: Fatal error: Invalid (empty) username when attempting to connect to the master server. Connection attempt terminated. Internal MariaDB error code: 1593
|
2024-11-10 5:34:11 21 [Note] Master 's-db2': Slave I/O thread killed while connecting to master
|
2024-11-10 5:34:11 21 [Note] Master 's-db2': Slave I/O thread exiting, read up to log 'FIRST', position 4, master :3306
|
2024-11-10 5:34:11 4 [Note] Started replication for 's-db2'
|
2024-11-10 5:34:11 4 [Note] Reading Master_info: 'master-s@002ddb5.info' Relay_info:'relay-log-s@002ddb5.info'
|
2024-11-10 5:34:11 22 [Note] Master 's-db2': Slave SQL thread initialized, starting replication in log 'FIRST' at position 0, relay log './db1-relay-s@002ddb2.000039' position: 4
|
2024-11-10 5:34:11 4 [Note] Initialized Master_info from 'master-s@002ddb5.info'
|
2024-11-10 5:34:11 23 [Note] Master 's-db5': Slave I/O thread: Start asynchronous replication to master '@:3306' in log '' at position 4
|
2024-11-10 5:34:11 23 [ERROR] Master 's-db5': Slave I/O: Fatal error: Invalid (empty) username when attempting to connect to the master server. Connection attempt terminated. Internal MariaDB error code: 1593
|
2024-11-10 5:34:11 23 [Note] Master 's-db5': Slave I/O thread killed while connecting to master
|
2024-11-10 5:34:11 23 [Note] Master 's-db5': Slave I/O thread exiting, read up to log 'FIRST', position 4, master :3306
|
2024-11-10 5:34:11 4 [Note] Started replication for 's-db5'
|
2024-11-10 5:34:11 24 [Note] Master 's-db5': Slave SQL thread initialized, starting replication in log 'FIRST' at position 0, relay log './db1-relay-s@002ddb5.000031' position: 4
|
MMM_Log
/var/log/mysql-mmm/ mmm_mond.log
2024/11/10 05:30:22 WARN Check 'rep_threads' on 'db1' is in unknown state! Message: UNKNOWN: Connect error (host = 10.76.24.71:3306, user = mmm_monitor)! Lost connection to server at 'handshake: reading initial communication packet', system error: 110
|
2024/11/10 05:30:23 WARN Check 'rep_backlog' on 'db1' is in unknown state! Message: UNKNOWN: Connect error (host = 10.76.24.71:3306, user = mmm_monitor)! Lost connection to server at 'handshake: reading initial communication packet', system error: 110
|
2024/11/10 05:30:39 ERROR Check 'mysql' on 'db1' has failed for 14 seconds! Message: ERROR: Connect error (host = 10.76.24.71:3306, user = mmm_monitor)! Lost connection to server at 'handshake: reading initial communication packet', system error: 110
|
2024/11/10 05:30:42 FATAL State of host 'db1' changed from ONLINE to HARD_OFFLINE (ping: OK, mysql: not OK)
|
2024/11/10 05:30:42 INFO Removing all roles from host 'db1':
|
2024/11/10 05:30:42 INFO Removed role 'writer(10.76.24.100)' from host 'db1'
|
2024/11/10 05:30:45 INFO Orphaned role 'writer(10.76.24.100)' has been assigned to 'db2'
|
2024/11/10 05:34:11 INFO Check 'rep_threads' on 'db1' is ok!
|
2024/11/10 05:34:13 INFO Check 'rep_backlog' on 'db1' is ok!
|
2024/11/10 05:34:14 INFO Check 'mysql' on 'db1' is ok!
|
2024/11/10 05:34:16 FATAL State of host 'db1' changed from HARD_OFFLINE to AWAITING_RECOVERY
|
2024/11/10 05:35:16 FATAL State of host 'db1' changed from AWAITING_RECOVERY to ONLINE because of auto_set_online(60 seconds). It was in state AWAITING_RECOVERY for 60 seconds
|
gdb analysis:
[Thread debugging using libthread_db enabled]
|
Using host libthread_db library "/lib64/libthread_db.so.1".
|
Core was generated by `/usr/sbin/mariadbd'.
|
Program terminated with signal SIGSEGV, Segmentation fault.
|
#0 0x00007f78b867fa35 in pthread_kill () from /lib64/libpthread.so.0
|
[Current thread is 1 (Thread 0x7f4fd913a700 (LWP 3160532))]
|
Missing separate debuginfos, use: yum debuginfo-install glibc-2.28-211.el8.x86_64 libaio-0.3.112-1.el8.x86_64 libblkid-2.32.1-35.el8.x86_64 libcap-2.48-2.el8.x86_64 libgcc-8.5.0-16.el8_7.x86_64 libgcrypt-1.8.5-7.el8_6.x86_64 libgpg-error-1.31-1.el8.x86_64 libmount-2.32.1-35.el8.x86_64 libpmem-1.6.1-1.el8.x86_64 libselinux-2.9-5.el8.x86_64 libstdc++-8.5.0-16.el8_7.x86_64 libuuid-2.32.1-35.el8.x86_64 libxcrypt-4.1.1-6.el8.x86_64 lz4-libs-1.8.3-3.el8_4.x86_64 openssl-libs-1.1.1k-9.el8_7.x86_64 pcre2-10.32-3.el8_6.x86_64 sssd-client-2.6.2-3.el8.x86_64 systemd-libs-239-74.el8_8.x86_64 xz-libs-5.2.4-4.el8_6.x86_64 zlib-1.2.11-21.el8_7.x86_64
|
|
|
Segmentation Fault: The main process exited due to a segmentation fault (status=11/SEGV).
|
Attachments
Issue Links
- is blocked by
-
-
- Open
-