Uploaded image for project: 'MariaDB Connector/J'
  1. MariaDB Connector/J
  2. CONJ-690

MariaDbPoolDataSource's initiailzation freezes for 2 minutes with aurora

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Not a Bug
    • 2.4.0
    • N/A
    • aurora
    • None
    • AWS Java 8 Lambda
       - and -
      openjdk version "1.8.0_191"
      OpenJDK Runtime Environment (build 1.8.0_191-8u191-b12-2ubuntu0.18.04.1-b12)
      OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)

    Description

      I use a MariaDbPoolDataSource connected to Aurora Mysql.

      I use the url `jdbc:mysql:aurora://my.cluster-xxxxxxxx.us-east-1.rds.amazonaws.com/dbuseLegacyDatetimeCode=false&serverTimezone=GMT`, `minPoolSize=1` and `maxPoolSize = 8`.

      In AWS Java 8 lambda sometimes the pool initialization freezes.
      I reproduce this issue on my PC using Java 11, I get the follow stack trace

      "nettyCallPool-4-1 @call-handler#3" #14 prio=10 os_prio=0 tid=0x00007f1bb8088800 nid=0x72c0 runnable [0x00007f1bdcaf8000]
      java.lang.Thread.State: RUNNABLE
      at java.net.SocketInputStream.socketRead0(Native Method)
      at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
      at java.net.SocketInputStream.read(SocketInputStream.java:171)
      at java.net.SocketInputStream.read(SocketInputStream.java:141)
      at java.io.FilterInputStream.read(FilterInputStream.java:133)
      at org.mariadb.jdbc.internal.io.input.ReadAheadBufferedStream.fillBuffer(ReadAheadBufferedStream.java:130)
      at org.mariadb.jdbc.internal.io.input.ReadAheadBufferedStream.read(ReadAheadBufferedStream.java:103)

      • locked <0x00000000dce8c1b8> (a org.mariadb.jdbc.internal.io.input.ReadAheadBufferedStream)
        at org.mariadb.jdbc.internal.io.input.StandardPacketInputStream.getPacketArray(StandardPacketInputStream.java:244)
        at org.mariadb.jdbc.internal.io.input.StandardPacketInputStream.getPacket(StandardPacketInputStream.java:215)
        at org.mariadb.jdbc.internal.com.read.ReadInitialHandShakePacket.<init>(ReadInitialHandShakePacket.java:89)
        at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.handleConnectionPhases(AbstractConnectProtocol.java:688)
        at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connect(AbstractConnectProtocol.java:412)
        at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connect(AbstractConnectProtocol.java:369)
        at org.mariadb.jdbc.internal.protocol.AuroraProtocol.loop(AuroraProtocol.java:167)
        at org.mariadb.jdbc.internal.failover.impl.AuroraListener.reconnectFailedConnection(AuroraListener.java:210)
        at org.mariadb.jdbc.internal.failover.impl.MastersSlavesListener.initializeConnection(MastersSlavesListener.java:170)
        at org.mariadb.jdbc.internal.failover.FailoverProxy.<init>(FailoverProxy.java:116)
        at org.mariadb.jdbc.internal.util.Utils.retrieveProxy(Utils.java:542)
        at org.mariadb.jdbc.internal.util.pool.Pool.addConnection(Pool.java:218)
        at org.mariadb.jdbc.internal.util.pool.Pool.<init>(Pool.java:127)
        at org.mariadb.jdbc.internal.util.pool.Pools.retrievePool(Pools.java:53)
      • locked <0x00000000dc162190> (a java.util.concurrent.ConcurrentHashMap)
        at org.mariadb.jdbc.MariaDbPoolDataSource.initialize(MariaDbPoolDataSource.java:624)
      • locked <0x00000000dbfdae08> (a org.mariadb.jdbc.MariaDbPoolDataSource)

      After two minutes I get the pool working without exception.

      This behavior fits really bad in a Lambda.

      Attachments

        Activity

          Ascending order - Click to sort in descending order
          vasco.lightstreamer Francesco Vasco added a comment -

          This issue looks related to an high active connection count on Aurora master.

          vasco.lightstreamer Francesco Vasco added a comment - This issue looks related to an high active connection count on Aurora master.
          diego dupin Diego Dupin added a comment - - edited

          This is a strange error, related to aurora : This means that socket to server is created, but server never send any bytes on socket.
          Normally when socket is created, server send an initial handshake packet. Connector hangs, waiting for this initial packet from server.

          If there was a socket error, that connector would have thrown an exception. Here socket is alright, but server doesn't respect protocol.

          I cannot say anything but ask for amazon aurora support about that...

          diego dupin Diego Dupin added a comment - - edited This is a strange error, related to aurora : This means that socket to server is created, but server never send any bytes on socket. Normally when socket is created, server send an initial handshake packet . Connector hangs, waiting for this initial packet from server. If there was a socket error, that connector would have thrown an exception. Here socket is alright, but server doesn't respect protocol. I cannot say anything but ask for amazon aurora support about that...
          vasco.lightstreamer Francesco Vasco added a comment -

          Thank you for response, this issue is related to MySql max_connection limit, so we have solved raising this value.
          Two minutes delay was a coincidence, later tests show that it may vary depending by the connection load.
          Finally I do not found any parameter to configure the socket "read timeout", so this kind of issues can be aborted after x seconds.

          vasco.lightstreamer Francesco Vasco added a comment - Thank you for response, this issue is related to MySql max_connection limit, so we have solved raising this value. Two minutes delay was a coincidence, later tests show that it may vary depending by the connection load. Finally I do not found any parameter to configure the socket "read timeout", so this kind of issues can be aborted after x seconds.
          diego dupin Diego Dupin added a comment -

          Ok nice, because 'normal' MySQL or MariaDB servers then throw an error.
          The 'socketTimeout' option permit to set socket timeout.

          diego dupin Diego Dupin added a comment - Ok nice, because 'normal' MySQL or MariaDB servers then throw an error. The 'socketTimeout' option permit to set socket timeout.

          People

            diego dupin Diego Dupin
            vasco.lightstreamer Francesco Vasco
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.