Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Incomplete
-
1.5.9
-
None
-
Aurora
Description
Hi all,
I'm running Aurora failover testing with the help of MariaDB connector recently and discovered this weird issue.
Preparation:
- Aurora cluster with 2 instances: test-01 and test-02, cluster end point is test-cluster.
- jdbcUrl="jdbc:mysql:aurora://test-cluster.rds.amazonaws.com:3306/Test?log=true"
- Using MariaDB connector version 1.5.9 due to concern on usePipeAuth.
- Java server application have all kinds of queries hitting db server.
Test1:
- test-01 is the writer and test-02 is the reader.
- launch server application.
Result: fine.
(Clean up by terminating server application)
Test2:
- Click failover in AWS console for test cluster.
- Wait until it completes: test-02 is the writer and test-01 is the reader
- launch server application
Result: getting exception.
(Clean up by terminating server application)
Exception:
Caused by: java.lang.ArrayIndexOutOfBoundsException: -10 |
at org.mariadb.jdbc.internal.util.buffer.Buffer.readShort(Buffer.java:98) ~[server.jar:?] |
at org.mariadb.jdbc.internal.packet.dao.ColumnInformation.<init>(ColumnInformation.java:148) ~[server.jar:?] |
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.readResultSet(AbstractQueryProtocol.java:1195) ~[server.jar:?] |
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.readPacket(AbstractQueryProtocol.java:1089) ~[server.jar:?] |
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:1031) ~[server.jar:?] |
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:203) ~[server.jar:?] |
at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) ~[?:?]
|
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121] |
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121] |
at org.mariadb.jdbc.internal.failover.AbstractMastersListener.invoke(AbstractMastersListener.java:355) ~[server.jar:?] |
at org.mariadb.jdbc.internal.failover.FailoverProxy.executeInvocation(FailoverProxy.java:200) ~[server.jar:?] |
at org.mariadb.jdbc.internal.failover.FailoverProxy.invoke(FailoverProxy.java:192) ~[server.jar:?] |
at com.sun.proxy.$Proxy42.executeQuery(Unknown Source) ~[?:?]
|
at org.mariadb.jdbc.MariaDbClientPreparedStatement.executeInternal(MariaDbClientPreparedStatement.java:224) ~[server.jar:?] |
at org.mariadb.jdbc.MariaDbClientPreparedStatement.executeQuery(MariaDbClientPreparedStatement.java:177) ~[server.jar:?] |
Note:
- Tested on the same environment with com.mysql.jdbc driver and removed "aurora" from jdbc url and things are working fine => issue comes from mariadb driver.
- My server application has nothing instance specific: the only information aurora specific is the jdbc url and it was using cluster end point
- For the query that triggered this, I've compared the table description and data between test-01 and test-02 and they are identical.
- Query="select * from table_timestamp"
- Table description(table_timestamp):
+--------------+------------------+------+-----+-------------------+-----------------------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+------------------+------+-----+-------------------+-----------------------------+
| last_updated | timestamp | NO | | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
| update_count | int(10) unsigned | NO | | 0 | |
+--------------+------------------+------+-----+-------------------+-----------------------------+
2 rows in set (0.00 sec)
- Manually tested the same query using mysql command line, getting identical results from test-01 and test-02
- I also got another different exception, but it's from the same query.
java.sql.SQLException: (conn:4027) Packets out of order when reading field packets, expected was EOF stream. Packet contents (hex) = 03 |
Query is: CALL Test.sp_get_timestamp()
|
at org.mariadb.jdbc.internal.util.ExceptionMapper.get(ExceptionMapper.java:150) ~[server.jar:?] |
at org.mariadb.jdbc.internal.util.ExceptionMapper.getException(ExceptionMapper.java:101) ~[server.jar:?] |
at org.mariadb.jdbc.internal.util.ExceptionMapper.throwAndLogException(ExceptionMapper.java:77) ~[server.jar:?] |
at org.mariadb.jdbc.MariaDbStatement.executeQueryEpilog(MariaDbStatement.java:226) ~[server.jar:?] |
at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:413) ~[server.jar:?] |
at org.mariadb.jdbc.MariaDbProcedureStatement.execute(MariaDbProcedureStatement.java:168) ~[server.jar:?] |
at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeQuery(MariaDbServerPreparedStatement.java:343) ~[server.jar:?] |
- The sproc is as simple as one select query as listed above.
BEGINselect * from gift_timestamp;
END
I've tested with other slightly older versions as well and have seen the same result.
Can someone help me out here? Many thanks in advance!
Please don't hesitate to reach out to me if you need more information. I'm subscribed to this thread and should be updated if you leave a comment.
Update
With the same test I'm also seeing this exception periodically:
Caused by: org.mariadb.jdbc.internal.util.dao.QueryException: Access denied for user 'someusername'@'10.%' (using password: YES) |
Query is: CALL Test.sp_get_timestamp()
|
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.readErrorPacket(AbstractQueryProtocol.java:1163) ~[server.jar:?] |
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.readPacket(AbstractQueryProtocol.java:1093) ~[server.jar:?] |
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:1044) ~[server.jar:?] |
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executePreparedQuery(AbstractQueryProtocol.java:628) ~[server.jar:?] |
at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) ~[?:?]
|
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121] |
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121] |
at org.mariadb.jdbc.internal.failover.AbstractMastersListener.invoke(AbstractMastersListener.java:351) ~[server.jar:?] |
at org.mariadb.jdbc.internal.failover.FailoverProxy.invoke(FailoverProxy.java:156) ~[server.jar:?] |
at com.sun.proxy.$Proxy43.executePreparedQuery(Unknown Source) ~[?:?]
|
at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:401) ~[server.jar:?] |
... 36 more |
Note:
This is solvable by restarting the server several times but it will be a blocker for me to deploy mariadb driver to production.
Can someone fix this as well?