[CONJ-109] Use session time zone for ResultSet.getTimestamp(String columnLabel) Created: 2014-09-04  Updated: 2014-11-21  Resolved: 2014-10-21

Status: Closed
Project: MariaDB Connector/J
Component/s: None
Affects Version/s: 1.1.7
Fix Version/s: 1.1.8

Type: Bug Priority: Major
Reporter: Lennart Schedin Assignee: Massimo Siani (Inactive)
Resolution: Won't Fix Votes: 0
Labels: None
Environment:

Client: Windows 8, CEST. Server: Mysql server 5.5.24-0ubuntu0.12.04.1, EEST.


Attachments: Java Source File TimezoneTest.java    

 Description   

How to reproduce:
Run attached Junit test case(s) TimezoneTest.java.
Psedo code:
1. Create a table with a TIMESTAMP column
2. Insert the time (and a date) 13:20 in UTC time. It is important to make this insert correctly. You should probably not use PreparedStatement.setTimestamp(int parameterIndex, Timestamp x) because that method also may have a bug.
3. SELECT the inserted row and extract the timestamp with ResultSet.getTimestamp(String columnLabel)

Actual:
When running the JUnit computer in Swedish time (CEST) and server in Istanbul time (EEST) you get the results:

org.junit.ComparisonFailure: expected:<2014-01-01T1[3]:20:59+0000> but was:<2014-01-01T1[8]:20:59+0000>
at org.mariadb.jdbc.TimezoneTest.testGetTimestampAdvancedTimeZoneSwitching(TimezoneTest.java:124)

org.junit.ComparisonFailure: expected:<2014-01-01T1[3]:20:59+0000> but was:<2014-01-01T1[7]:20:59+0000>
	at org.mariadb.jdbc.TimezoneTest.testGetTimestampTest(TimezoneTest.java:93)

In other words the time is not 13:20, but some other time, usually shifted by the difference of the client and server.

The reason for difference between the test cases is that the method testGetTimestampTest only can insert a correct time value when the server is in UTC timezone (I think).

The test case testEnsureServerIsNotUsingAtlanticTimeZone() will always pass (unless you have put your server in a crazy time zone (GMT -2)).

Expected:
All test cases should pass (expect maybe not testGetTimestampTest() if server is using non-UTC time.). The times should always return 13:20 in UTC.

Environment:
I used an Ubuntu Virtualbox to switch time zone on my server with the command:

sudo dpkg-reconfigure tzdata;sudo service mysql restart

I tested with different time zones (Stockholm, Istanbul, UTC).

The test case is written in the context MariaDB JDBC source code. You might need to edit BaseTest to change the server connection URL.

I used Eclipse and Windows 8 in a Swedish time zone to execute the test cases. I tried my best to make sure my test cases would show an error regardless of your time zone set up. But the actual (erroneous) values may differ.

I primarily used MariaDB JDBC version 1.1.5 and 1.1.7. I also tried the current code in trunk.

I also tried with Mysql JDBC driver 5.1.26 and 5.1.31. The Mysql driver also returned wrong results.

My opinions on Java SQL API and Mysql data types:
Mysql/MariaDB server has 3 time data types (http://dev.mysql.com/doc/refman/5.1/en/datetime.html): DATE, DATETIME, and TIMESTAMP. It is my option that only TIMESTAMP should be used to store time. Because it is the only data type that stores a point in time. My definition is that a point is some seconds since an Epoch. TIMESTAMP is the number of seconds since the start of 1970 in UTC.

The DATETIME data types should be avoided since it stores a point of day. For example it stores the point of day when you normally take lunch (12:00). Since it does not stores any time zone information you cannot defer at what point in time I took my lunch, only the point of day. Of course there are exceptions to my rule when DATETIME may be used: most notably if you which to store a point of time past the year 2038 (where TIMESTAMP is out of range). But in these cases you must store the time zone in a separate column, or always know the time zone.

Java SQL API has 2 types of time methods: the ones that takes a Calendar and the ones that does not:
ResultSet.getTimestamp(int columnIndex) vs ResultSet .getTimestamp(int columnIndex, Calendar cal)
PreparedStatement.setTimestamp(int parameterIndex, Timestamp x) vs PreparedStatement .setTimestamp(int parameterIndex, Timestamp x, Calendar cal).

It is my option that the methods that takes a Calendar should never be used. My reason is found in the Javadoc of ResultSet: “…if the underlying database does not store timezone information”. Since I always use TIMESTAMP in Mysql/MariaDB I never use methods that takes a Calendar since it should be redundant since the database stores the time zone information (baked into the definition of TIMESTAMP).

Analysis of MariaDB JDBC code and suggested solution:
Using Wireshark to sniff the traffic it is easy to see that Mysql protocol used between server and JDBC is a simple protocol that often uses strings. The timestamp is transferred as a string. For example “2014-01-01 15:20:59”. This string must then be converted into a java.sql.Timestamp by the JDBC driver. So how should a client know the timezone? My understanding of the Mysql documentation is that the client and server will agree on a session timezone (http://dev.mysql.com/doc/refman/5.5/en/time-zone-support.html).

The class AbstractValueObject and method getTimestamp(Calendar cal) is the main focus. Please note that this class has changed a bit from 1.1.5 and 1.1.7. In the getTimestamp-metod a SimpleDateFormat is created. If cal is null no timezone is used for the SimpleDateFormat and the client JVM timezone is used. This may be the correct time zone for the session, but I have written my test case to demonstrate what happens when they differ.

My suggested solution is that the SimpleDateFormat should use the session time zone. I briefly looked at the code but could not see any simple way to implement this in MySQLProtocol.java. I could not find anything in my Wireshark trace or in MySQL Client/Server Protocol http://dev.mysql.com/doc/internals/en/client-server-protocol.html to specify this without executing an SQL query. One way to know the session time zone would be to force it to a specific value during the connection phase. For example always set it to UTC or the JVM default timezone.



 Comments   
Comment by Lennart Schedin [ 2014-09-11 ]

Anders Karlsson has given me a workaround that I probably will use: add this to the database URL: "sessionVariables=time_zone='Europe/Istanbul". I tested it and that will work for me.

I still feel it would be good to fix this in the MariaDB JDBC code. However it might not be possible to fix without breaking backward compatibility. One alternative would be to create some new JDBC parameter. But it is a slippery slope. Mysql JDBC has about 5 to 10 time related URL parameters.

Comment by Massimo Siani (Inactive) [ 2014-10-01 ]

You could also modify the code as provided in https://code.launchpad.net/~massimo-siani/mariadb-java-client/CONJ-109, which does not rely on the knowledge of the server timezone.
I am not sure this is a driver concern, since the server does not store the timestamp offset. Any comment is welcome.

Comment by Lennart Schedin [ 2014-10-02 ]

I’m not sure which specific change in https://code.launchpad.net/~massimo-siani/mariadb-java-client/CONJ-109 that I should look for.

I don’t agree on the statement “since the server does not store the timestamp offset”. It is my option that the data type TIMSTAMP is time zone aware, and I think the Mysql server manual backs me up: http://dev.mysql.com/doc/refman/5.1/en/datetime.html.

In other words the value of a TIMESTAMP is a point in time. From an implementation perspective I think that a TIMESTAMP is stored in the server as the number of seconds (or milliseconds) from 1970-01-01 00:00:01 UTC. In other words the “offset” from UTC is implicitly always 0 (and is not stored since it always is 0).

The DATETIME is however not time zone aware. DATETIME is a point of day (as opposed to a point in time). My attached test case used TIMESTAMP. I would recommend developers to never use DATETIME in an SQL table. There are some few cases where I see a point for DATETIME, but in the general case TIMESTAMP is preferable.

I do think that this problem is a driver concern. But since Mysql JDBC driver behaves just as bad I don’t have a “golden-bullet-solution”. In an ideal world a Java developer that uses a time zone aware data type in both SQL and Java (TIMESTAMP and java.util.Date respectively) should never have to worry about adding removing hours from a time value just because the JDBC connector does not handle it for you.

An analogy that I think fits: an integer value can be sent over the network in big endian or little endian way. In Java the data type java.lang.Integer does not trouble the developer with this. The bytes stored in RAM uses big endian in Java. But the developer never have to know this. Mysql server also stores the bytes of an integer in some way (I don’t know if it is big or little). But the java developer never have to make any special conversion to SELECT an integer value with respect to endianness (there are other conversions the developer has to know (as signed or unsigned)). The JDBC code “just deals with it” and converts (or not converts) the value to the correct endianness for Java. I feel the same should hold for time zone.

However I understand that this may not be implemented in MariaDB JDBC without breaking some backward compatibility. If I were to write a JDBC connector from scratch I would probably make sure the JDBC code is aware of the session time zone (either by asking the server for it, or setting it explicitly). That way I (The JDBC code) can convert the data from the TCP socket into the correct java.util.Date value.

How to solve this issue? The most simple solution for me is it add the “sessionVariables=time_zone='Europe/Istanbul” to the JDBC URL. But it feels wrong: I should not have to bother with this. It would be possible to add a new JDBC parameter (to avoid break backwards compatibility):

useCorrectTimeZoneConvertionInsteadOfBroken=true

With this new parameter the JDBC should behave correct. If the implementation asks the server for the session time zone or explicitly sets it is an implementation detail that does not matter. But If I must add a new parameter I might just as well add the time zone as a parameter...

Comment by Lennart Schedin [ 2014-11-21 ]

I have found a workaround that I feel is acceptable. By adding

&serverTimezone=UTC&sessionVariables=time_zone='+00:00'

as an URL parameter everything works as I expect. By using UTC formatted strings in the transfer between JDBC and server there is never a loss of "precision". Any by setting serverTimezone MariaDB JDBC will understand the data correct. This means you never have to use the methods in ResultSet and PreparedStatement that takes a Calendar parameter! And that is good, because I don't like those methods (by reasons explained in above comments)

Generated at Thu Feb 08 03:13:14 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.