[CONJ-690] MariaDbPoolDataSource's initiailzation freezes for 2 minutes with aurora Created: 2019-03-05  Updated: 2019-03-13  Resolved: 2019-03-13

Status: Closed
Project: MariaDB Connector/J
Component/s: aurora
Affects Version/s: 2.4.0
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Francesco Vasco Assignee: Diego Dupin
Resolution: Not a Bug Votes: 0
Labels: None
Environment:

AWS Java 8 Lambda

  • and -
    openjdk version "1.8.0_191"
    OpenJDK Runtime Environment (build 1.8.0_191-8u191-b12-2ubuntu0.18.04.1-b12)
    OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)


 Description   

I use a MariaDbPoolDataSource connected to Aurora Mysql.

I use the url `jdbc:mysql:aurora://my.cluster-xxxxxxxx.us-east-1.rds.amazonaws.com/dbuseLegacyDatetimeCode=false&serverTimezone=GMT`, `minPoolSize=1` and `maxPoolSize = 8`.

In AWS Java 8 lambda sometimes the pool initialization freezes.
I reproduce this issue on my PC using Java 11, I get the follow stack trace

"nettyCallPool-4-1 @call-handler#3" #14 prio=10 os_prio=0 tid=0x00007f1bb8088800 nid=0x72c0 runnable [0x00007f1bdcaf8000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at org.mariadb.jdbc.internal.io.input.ReadAheadBufferedStream.fillBuffer(ReadAheadBufferedStream.java:130)
at org.mariadb.jdbc.internal.io.input.ReadAheadBufferedStream.read(ReadAheadBufferedStream.java:103)

  • locked <0x00000000dce8c1b8> (a org.mariadb.jdbc.internal.io.input.ReadAheadBufferedStream)
    at org.mariadb.jdbc.internal.io.input.StandardPacketInputStream.getPacketArray(StandardPacketInputStream.java:244)
    at org.mariadb.jdbc.internal.io.input.StandardPacketInputStream.getPacket(StandardPacketInputStream.java:215)
    at org.mariadb.jdbc.internal.com.read.ReadInitialHandShakePacket.<init>(ReadInitialHandShakePacket.java:89)
    at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.handleConnectionPhases(AbstractConnectProtocol.java:688)
    at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connect(AbstractConnectProtocol.java:412)
    at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connect(AbstractConnectProtocol.java:369)
    at org.mariadb.jdbc.internal.protocol.AuroraProtocol.loop(AuroraProtocol.java:167)
    at org.mariadb.jdbc.internal.failover.impl.AuroraListener.reconnectFailedConnection(AuroraListener.java:210)
    at org.mariadb.jdbc.internal.failover.impl.MastersSlavesListener.initializeConnection(MastersSlavesListener.java:170)
    at org.mariadb.jdbc.internal.failover.FailoverProxy.<init>(FailoverProxy.java:116)
    at org.mariadb.jdbc.internal.util.Utils.retrieveProxy(Utils.java:542)
    at org.mariadb.jdbc.internal.util.pool.Pool.addConnection(Pool.java:218)
    at org.mariadb.jdbc.internal.util.pool.Pool.<init>(Pool.java:127)
    at org.mariadb.jdbc.internal.util.pool.Pools.retrievePool(Pools.java:53)
  • locked <0x00000000dc162190> (a java.util.concurrent.ConcurrentHashMap)
    at org.mariadb.jdbc.MariaDbPoolDataSource.initialize(MariaDbPoolDataSource.java:624)
  • locked <0x00000000dbfdae08> (a org.mariadb.jdbc.MariaDbPoolDataSource)

After two minutes I get the pool working without exception.

This behavior fits really bad in a Lambda.

 Comments   
Comment by Francesco Vasco [ 2019-03-06 ]

This issue looks related to an high active connection count on Aurora master.

Comment by Diego Dupin [ 2019-03-08 ]

This is a strange error, related to aurora : This means that socket to server is created, but server never send any bytes on socket.
Normally when socket is created, server send an initial handshake packet. Connector hangs, waiting for this initial packet from server.

If there was a socket error, that connector would have thrown an exception. Here socket is alright, but server doesn't respect protocol.

I cannot say anything but ask for amazon aurora support about that...

Comment by Francesco Vasco [ 2019-03-08 ]

Thank you for response, this issue is related to MySql max_connection limit, so we have solved raising this value.
Two minutes delay was a coincidence, later tests show that it may vary depending by the connection load.
Finally I do not found any parameter to configure the socket "read timeout", so this kind of issues can be aborted after x seconds.

Comment by Diego Dupin [ 2019-03-08 ]

Ok nice, because 'normal' MySQL or MariaDB servers then throw an error.
The 'socketTimeout' option permit to set socket timeout.

Generated at Thu Feb 08 03:17:36 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.