[MCOL-1605] sendAlarmReport error: InetStreamSocket::connect: connect() error: Connection timed Created: 2018-07-27  Updated: 2023-10-26  Resolved: 2018-08-10

Status: Closed
Project: MariaDB ColumnStore
Component/s: ?
Affects Version/s: 1.1.5
Fix Version/s: 1.1.6

Type: Bug Priority: Minor
Reporter: David Hill (Inactive) Assignee: Daniel Lee (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Sprint: 2018-15, 2018-16

 Description   

Customer reporting this error log, which is relate to trying to issues an alarm. Alarms are being successfully issued on the system, so not sure why the error is occurring.
I can duplicate this when running on amazon using a micro instance with 1 core. Customer has a larger server with 32 cores.

Needs further investigation and maybe determining a fix for the amazon 1 core issue will fix the customer issue.

Jul 10 07:02:14 tarcs01 alarmmanager[23339]: 14.581166 |0|0|0| E 11 CAL0000: sendAlarmReport error: InetStreamSocket::connect: connect() error: Connection timed out to: InetStreamSocket: sd: 3 inet: 127.0.0.1 port: 8606
Jul 10 07:20:31 tarcs01 alarmmanager[27448]: 31.285105 |0|0|0| E 11 CAL0000: sendAlarmReport error: InetStreamSocket::connect: connect() error: Connection timed out to: InetStreamSocket: sd: 3 inet: 127.0.0.1 port: 8606
Jul 10 07:59:12 tarcs01 alarmmanager[32043]: 12.181344 |0|0|0| E 11 CAL0000: sendAlarmReport error: InetStreamSocket::connect: connect() error: Connection timed out to: InetStreamSocket: sd: 3 inet: 127.0.0.1 port: 8606
Jul 10 08:27:20 tarcs01 alarmmanager[6096]: 20.629348 |0|0|0| E 11 CAL0000: sendAlarmReport error: InetStreamSocket::connect: connect() error: Connection timed out to: InetStreamSocket: sd: 3 inet: 127.0.0.1 port: 8606
Jul 10 08:34:17 tarcs01 alarmmanager[7560]: 17.525133 |0|0|0| E 11 CAL0000: sendAlarmReport error: InetStreamSocket::connect: connect() error: Connection timed out to: InetStreamSocket: sd: 3 inet: 127.0.0.1 port: 8606
Jul 10 12:04:17 tarcs01 alarmmanager[5376]: 17.333144 |0|0|0| E 11 CAL0000: sendAlarmReport error: InetStreamSocket::connect: connect() error: Connection timed out to: InetStreamSocket: sd: 3 inet: 127.0.0.1 port: 8606
Jul 10 12:47:37 tarcs01 alarmmanager[14670]: 37.529439 |0|0|0| E 11 CAL0000: sendAlarmReport error: InetStreamSocket::connect: connect() error: Connection timed out to: InetStreamSocket: sd: 3 inet: 127.0.0.1 port: 8606
Jul 10 13:00:04 tarcs01 alarmmanager[20764]: 04.149395 |0|0|0| E 11 CAL0000: sendAlarmReport error: InetStreamSocket::connect: connect() error: Connection timed out to: InetStreamSocket: sd: 3 inet: 127.0.0.1 port: 8606
Jul 10 15:05:40 tarcs01 alarmmanager[6184]: 40.533312 |0|0|0| E 11 CAL0000: sendAlarmReport error: InetStreamSocket::connect: connect() error: Connection timed out to: InetStreamSocket: sd: 3 inet: 127.0.0.1 port: 8606
Jul 10 15:06:32 tarcs01 alarmmanager[6408]: 32.117170 |0|0|0| E 11 CAL0000: sendAlarmReport error: InetStreamSocket::connect: connect() error: Connection timed out to: InetStreamSocket: sd: 3 inet: 127.0.0.1 port: 8606
Jul 10 15:15:36 tarcs01 alarmmanager[7961]: 36.117226 |0|0|0| E 11 CAL0000: sendAlarmReport error: InetStreamSocket::connect: connect() error: Connection timed out to: InetStreamSocket: sd: 3 inet: 127.0.0.1 port: 8606
Jul 10 15:18:03 tarcs01 alarmmanager[8400]: 03.957188 |0|0|0| E 11 CAL0000: sendAlarmReport error: InetStreamSocket::connect: connect() error: Connection timed out to: InetStreamSocket: sd: 3 inet: 127.0.0.1 port: 8606
Jul 10 15:49:30 tarcs01 alarmmanager[13832]: 30.677207 |0|0|0| E 11 CAL0000: sendAlarmReport error: InetStreamSocket::connect: connect() error: Connection timed out to: InetStreamSocket: sd: 3 inet: 127.0.0.1 port: 8606
Jul 10 15:49:40 tarcs01 alarmmanager[13992]: 40.021293 |0|0|0| E 11 CAL0000: sendAlarmReport error: InetStreamSocket::connect: connect() error: Connection timed out to: InetStreamSocket: sd: 3 inet: 127.0.0.1 port: 8606

 Comments   
Comment by David Hill (Inactive) [ 2018-08-03 ]

This is happening because an alarm is trying to be issued before ProcMgr is up

Aug 3 14:01:43 ip-172-31-46-144 ProcessMonitor[2690]: 43.744841 |0|0|0| D 18 CAL0000: STOPPING Process: ProcessManager
Aug 3 14:01:43 ip-172-31-46-144 ProcessMonitor[2690]: 43.745009 |0|0|0| D 18 CAL0000: StatusUpdate of Process ProcessManager State = 1 PID = 0
Aug 3 14:01:43 ip-172-31-46-144 ProcessMonitor[2690]: 43.745972 |0|0|0| D 18 CAL0000: Send SET Alarm ID 13 on device ProcessManager
Aug 3 14:01:43 ip-172-31-46-144 alarmmanager[2690]: 43.747400 |0|0|0| E 11 CAL0000: sendAlarmReport error: InetStreamSocket::connect: connect() error: Connection refused to: InetStreamSocket: sd: 12 inet: 172.31.46.144 port: 8606
Aug 3 14:01:43 ip-172-31-46-144 ProcessMonitor[2690]: 43.747759 |0|0|0| D 18 CAL0000: Send CLEAR Alarm ID 25 on device ProcessManager
Aug 3 14:01:43 ip-172-31-46-144 ProcessMonitor[2690]: 43.749108 |0|0|0| D 18 CAL0000: statusControl: REQUEST RECEIVED: Set Process pm1/ProcessManager State = AUTO_OFFLINE
Aug 3 14:01:43 ip-172-31-46-144 ProcessMonitor[2690]: 43.749144 |0|0|0| D 18 CAL0000: statusControl: Set Process pm1/ProcessManager State = AUTO_OFFLINE PID = 0
Aug 3 14:01:43 ip-172-31-46-144 alarmmanager[2690]: 43.750456 |0|0|0| E 11 CAL0000: sendAlarmReport error: InetStreamSocket::connect: connect() error: Connection refused to: InetStreamSocket: sd: 12 inet: 172.31.46.144 port: 8606
Aug 3 14:01:43 ip-172-31-46-144 ProcessMonitor[2690]: 43.752440 |0|0|0| D 18 CAL0000: Pkill Process just to make sure: ProcMgr*
Aug 3 14:01:43 ip-172-31-46-144 ProcessMonitor[2690]: 43.752628 |0|0|0| D 18 CAL0000: STARTING Process: ProcessManager
Aug 3 14:01:43 ip-172-31-46-144 ProcessMonitor[2690]: 43.752775 |0|0|0| D 18 CAL0000: Process location: /home/guest/mariadb/columnstore/bin/ProcMgr
Aug 3 14:01:43 ip-172-31-46-144 ProcessMonitor[2690]: 43.753751 |0|0|0| D 18 CAL0000: Send CLEAR Alarm ID 27 on device DBRM
Aug 3 14:01:43 ip-172-31-46-144 ProcessMonitor[2690]: 43.756983 |0|0|0| D 18 CAL0000: Pkill Process just to make sure: ProcMgr*
Aug 3 14:01:43 ip-172-31-46-144 ProcessMonitor[2690]: 43.757158 |0|0|0| D 18 CAL0000: StatusUpdate of Process ProcessManager State = 3 PID = 0
Aug 3 14:01:43 ip-172-31-46-144 alarmmanager[2690]: 43.757753 |0|0|0| E 11 CAL0000: sendAlarmReport error: InetStreamSocket::connect: connect() error: Connection refused to: InetStreamSocket: sd: 12 inet: 172.31.46.144 port: 8606

Aug 3 14:01:50 ip-172-31-46-144 ProcessManager[2749]: 50.922351 |0|0|0| D 17 CAL0000: Alarm Message Thread started ..

Comment by David Hill (Inactive) [ 2018-08-07 ]

https://github.com/mariadb-corporation/mariadb-columnstore-engine/pull/535

Comment by Daniel Lee (Inactive) [ 2018-08-10 ]

Build verified: 1.1.6-1

/root/columnstore/mariadb-columnstore-server
commit 513775738f72ec990d055a5d47e2511e3c0e34dd
Merge: 3c37210 9236098
Author: Andrew Hutchings <andrew@linuxjedi.co.uk>
Date: Wed Jul 18 09:37:17 2018 +0100

Merge pull request #123 from drrtuy/MCOL-970

MCOL-970 Slow query log now contains original query even in vtable mode

/root/columnstore/mariadb-columnstore-server/mariadb-columnstore-engine
commit 150171b714c16bd91ef620ea483f6200ad775038
Merge: 1068679 8a42949
Author: benthompson15 <ben.thompson@mariadb.com>
Date: Thu Aug 9 17:49:01 2018 -0500

Merge pull request #535 from mariadb-corporation/MCOL-1605

MCOL-1605 - changed error to debug, alarms trying to get issued befor…

Generated at Thu Feb 08 02:30:00 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.