[CONJ-481] Buffer overrun reading ResultSet - useServerPrepStmts = true Created: 2017-05-30  Updated: 2017-06-06  Resolved: 2017-06-06

Status: Closed
Project: MariaDB Connector/J
Component/s: Other
Affects Version/s: 1.6.0, 2.0.1
Fix Version/s: 1.6.1, 2.0.2

Type: Bug Priority: Critical
Reporter: Brendon Abbott Assignee: Diego Dupin
Resolution: Fixed Votes: 0
Labels: None
Environment:

Java 8, HikariCP, MariaDB Galera 10.1.23, SpringFramework


Attachments: Java Source File MariaJdbcTest.java     Text File output-useServerPrepStmts_false.txt     Text File output-useServerPrepStmts_true.txt     Text File output_prepStmt_false_02.txt     Text File output_prepStmt_true_02.txt    

 Description   

Since upgrading to 2.0.1, we have numerous unit test cases failing in our project. They all appear to be caused by overruning the data when reading back result sets.

If we repeatedly run the test, the issue does not always occur at the same position in the data stream. i.e. a different column/position.

We will continue to attempt to narrow down a small test case, and get a better understanding of the problem if that would help.

The stack trace below shows a StringIndexOutOfBounds, due to reading a String. Similarly we get, ArrayIndexOutOfBounds of the error occurs whilst reading a column of a different type.

java.lang.StringIndexOutOfBoundsException: String index out of range: 200
 at java.lang.String.checkBounds(Unknown Source)
 at java.lang.String.<init>(Unknown Source)
 at org.mariadb.jdbc.internal.com.read.resultset.SelectResultSet.getInternalString(SelectResultSet.java:1048)
 at org.mariadb.jdbc.internal.com.read.resultset.SelectResultSet.getString(SelectResultSet.java:950)
 at org.mariadb.jdbc.internal.com.read.resultset.SelectResultSet.getString(SelectResultSet.java:957)

Let us know if you need any further information.



 Comments   
Comment by Diego Dupin [ 2017-05-30 ]

Hi, could you please indicate the connection url you are using (to identify if this concern TEXT or BINARY protocol) ?

This must concern a specific data type (blob ?), i'll do some tests to reproduced this issue.

Comment by Brendon Abbott [ 2017-05-30 ]

The URL we are using is

jdbc:mysql://${ut.node}/${ut.database}?useServerPrepStmts=true

It is possible that this could involve blobs, as quite a few of our tables do contain them.

We were running a few versions behind (1.4.6), and scheduled in updating the version to the latest 1.x series - 1.5.9 at the time. However, we think we came across a separate bug, and decided to try 2.0.1 in case it was fixed. We are at the point of testing it now. I will make a separate ticket for our initial problem, in case it is also an issue.

If we use "serverprepstmts=false", then all our TestUnits pass successfully.

Comment by Diego Dupin [ 2017-05-30 ]

ok, so that is concerning binary protocol only.
I've checked code and made differents tests, but i failed to reproduced the issue.

To clearly identify the issue, could you :

  • indicate if you use any proxy (HAproxy / maxscale)
  • Could you please relaunch your tests using 2.0.2-SNAPSHOT version and adding option "&enablePacketDebug=true" to your URL ?

 
<repositories>
    <repository>
        <id>sonatype-nexus-snapshots</id>
        <name>Sonatype Nexus Snapshots</name>
        <url>https://oss.sonatype.org/content/repositories/snapshots</url>
    </repository>
</repositories>
 
<dependencies>
    <dependency>
        <groupId>org.mariadb.jdbc</groupId>
        <artifactId>mariadb-java-client</artifactId>
        <version>2.0.2-SNAPSHOT</version>
    </dependency>
</dependencies>

StrackTrace will list the lasts client-server exchanges.

Comment by Craig bailey [ 2017-05-31 ]

Hi Diego, I am a colleague of Brendon and have some more info on the issue.

I tried adding "&enablePacketDebug=true" but we got no extra output for some reason, however I managed to get the binary logging to show up by using "&log=true". output-useServerPrepStmts_false.txt output-useServerPrepStmts_true.txt

I have attached two logs, both of them make the same query, one of the logs has useServerPrepStmts set to true, the other does not and uses the default behavior (set to false).

We are not behind a proxy.

Comment by Diego Dupin [ 2017-05-31 ]

perfect log ! That just permit me to reproduced the issue.

Comment by Diego Dupin [ 2017-05-31 ]

Correction is done on a new 2.0.2-SNAPSHOT.
Could you please confirm correction ?

Comment by Craig bailey [ 2017-05-31 ]

Thanks for the quick turn around.

It seems to work in the one test case I was using, I will run up our full test suit tomorrow and let you know how that goes.

Comment by sfahren [ 2017-06-01 ]

Good Morning Diego,

i didnt know, if this the same problem but we are testing our product with the Connector/J 2.0.1. On the local system all tests are passed. but on the build server we get the following exception

java.lang.ArrayIndexOutOfBoundsException: 5
at org.mariadb.jdbc.internal.com.read.resultset.SelectResultSet.readNextValue(SelectResultSet.java:481)
at org.mariadb.jdbc.internal.com.read.resultset.SelectResultSet.fetchAllResults(SelectResultSet.java:338)
at org.mariadb.jdbc.internal.com.read.resultset.SelectResultSet.<init>(SelectResultSet.java:209)
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.readResultSet(AbstractQueryProtocol.java:12733)
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.readPacket(AbstractQueryProtocol.java:1033)
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:982)
at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.readRequestSessionVariables(AbstractConnectProtocol.java:498))
at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.readPipelineAdditionalData(AbstractConnectProtocol.java:544)
at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connect(AbstractConnectProtocol.java:410)
at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connectWithoutProxy(AbstractConnectProtocol.java:1013)
at org.mariadb.jdbc.internal.util.Utils.retrieveProxy(Utils.jjava:481)
at org.mariadb.jdbbc.Driver.connect(Driver.java:103)

The JDBC Url is

jdbc:mariadb://localhost:3306/{DB}?user=root&jdbcCompliantTruncation=false

It sees, that the exception comes, on creating another db-shema.

Comment by Diego Dupin [ 2017-06-01 ]

fahrenholz please consider creating a new task for your issue, since it's not related, (probably related to CONJ-479)

Comment by Craig bailey [ 2017-06-01 ]

Unfortunately it does not fix all of our test cases (however it did fix it for the test I provided first), I found another example where it fails with the updated 2.0.2-SNAPSHOT.

Here are the logs: output_prepStmt_false_02.txt output_prepStmt_true_02.txt

Comment by Craig bailey [ 2017-06-01 ]

Further investigation, this might be a different issue (although still related), we are not getting an exception when reading the data.

It seems that when retrieving the results from the result set (with useServerPrepStmts = true), we are failing to read two varchar values: '01:12:00:00' and '01:13:00:00'

All other data seems to be read OK.

Comment by Diego Dupin [ 2017-06-01 ]

hi Craig
That's a strange problem, can you check the exact type of those fields and the engine used (innodb ?) ? server send type like BIGINT, but format is like a TIME field.

Comment by Craig bailey [ 2017-06-01 ]

Here is the description of those columns.
The issue is with START_TC and END_TC.

There are two similarly named columns that are defined as BIGINT, is this causing an issue?

+--------------------+--------------+------+-----+----------------------+-----------------------------+
| Field              | Type         | Null | Key | Default              | Extra                       |
+--------------------+--------------+------+-----+----------------------+-----------------------------+
| START_TC           | varchar(11)  | YES  |     | NULL                 |                             |
| END_TC             | varchar(11)  | YES  |     | NULL                 |                             |
 
| START_TC_NANO      | bigint(20)   | YES  |     | NULL                 |                             |
| END_TC_NANO        | bigint(20)   | YES  |     | NULL                 |                             |
 
+--------------------+--------------+------+-----+----------------------+-----------------------------+

Comment by Diego Dupin [ 2017-06-01 ]

Sorry, i missread the hexadecimals there is no problem about type.

I've recreate the fields /query, results send the exact same hexa, but i
didn't reproduced the issue.
Have you the associated stacktrace ? (there is only "testException = java.lang.NullPointerException" in logs).

Comment by Craig bailey [ 2017-06-01 ]

Th NPE is from my code when trying to access the result of the START_TC column.

When I call the following:

@Override
public void processRow(ResultSet rs) throws SQLException {
...
   String str =  rs.getString("start_tc");
...
}

null is being returned rather than '01:12:00:00'. I will dig deeper and see if I can find any better information for you.

Comment by Craig bailey [ 2017-06-02 ]

Hi Diego,

Sorry this took so long, it took me a while to narrow down an exact test case to make it go wrong. I have attached a junit test file which will highlight the problem. It appears to be something to do with ordering and null vales when it comes to retrieving Strings from the result set. The unit test shows a few working scenario and the one failing scenario.

Note: You will need to modify the unit test to have the correct server information.
MariaJdbcTest.java

Comment by Diego Dupin [ 2017-06-05 ]

Thanks for the detailed test case
And after lot of testing to ensure there is not any remaining issue of this kind, issue is corrected on snapshot.

This will be released in a corrective version 2.0.2 today or tomorrow.

Generated at Thu Feb 08 03:15:59 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.