[MCOL-683] ADDDATE & DATE_FORMAT combo breaks datetime Created: 2017-04-25  Updated: 2017-05-15  Resolved: 2017-05-04

Status: Closed
Project: MariaDB ColumnStore
Component/s: None
Affects Version/s: 1.0.8
Fix Version/s: 1.0.9, 1.1.0

Type: Bug Priority: Major
Reporter: Geoff Cleaves Assignee: Daniel Lee (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Environment:

centos7


Sprint: 2017-9

 Description   

Tableau generated queries which manipulate dates are broken in Columnstore but worked in InfiniDB. In the query below you can see that the first column "breaks" in that the returned value is not a date.

Query example:

SELECT 
    ADDDATE(DATE_FORMAT(datecol, '%Y-%m-%d %H:00:00'), INTERVAL 0 SECOND) AS `tableau_date`,
    cast(ADDDATE(DATE_FORMAT(datecol, '%Y-%m-%d %H:00:00'), INTERVAL 0 SECOND) as datetime) AS `casted_date`,
    datecol as raw_date
FROM
    columnstore_table
WHERE
    datecol >= CURRENT_DATE()



 Comments   
Comment by Andrew Hutchings (Inactive) [ 2017-04-25 ]

Cause is MariaDB tells the engine DATE_ADD/ADDDATE is 3 byte UTF8 and we are handling the length as if it is 1 byte. So we truncate at 1/3rd of the length.

Comment by Andrew Hutchings (Inactive) [ 2017-04-25 ]

Pull requests open for develop and develop-1.0 as well as regression suite.

For testing:

CREATE TABLE `columnstore_table` (
  `a` int(11) DEFAULT NULL,
  `datecol` date DEFAULT NULL
) ENGINE=Columnstore
 
INSERT INTO columnstore_table values (1, '2017-04-25');

Then the queries in the example, on failure the first column will have a date truncated at 6 chars. Also regression suite changes cover this.

Comment by Daniel Lee (Inactive) [ 2017-05-04 ]

[root@localhost mariadb-columnstore-server]# git show
commit 5a090c64bced6532578dd8910f15530fd37fce2c
Merge: 9efb0a7 b062156
Author: david hill <david.hill@mariadb.com>
Date: Thu Apr 27 16:13:15 2017 -0500
Merge pull request #45 from mariadb-corporation/FixPackageName
Change the package name to match engines format
[root@localhost mariadb-columnstore-server]# cd mariadb-columnstore-engine/
[root@localhost mariadb-columnstore-engine]# git show
commit cb788c0be215ad30e77e35298142fef43c61e70e
Merge: 775c605 de06c48
Author: dhall-InfiniDB <david.hall@mariadb.com>
Date: Wed May 3 09:43:29 2017 -0500
Merge pull request #163 from mariadb-corporation/MCOL-697
MCOL-697 Remove 64KB VARCHAR response limit

Reproduced the issue in 1.0.8-1
MariaDB [tpch1]> SELECT ADDDATE(DATE_FORMAT(datecol, '%Y-%m-%d %H:00:00'), INTERVAL 0 SECOND) AS `tableau_date`, cast(ADDDATE(DATE_FORMAT(datecol, '%Y-%m-%d %H:00:00'), INTERVAL 0 SECOND) as datetime) AS `casted_date`, datecol as raw_date FROM columnstore_table WHERE datecol <= CURRENT_DATE();
-------------------------------------------

tableau_date casted_date raw_date

-------------------------------------------

2017-0 2017-04-25 00:00:00 2017-04-25

-------------------------------------------
1 row in set (0.01 sec)

The same query in 1.1.0-1 returned:
--------------------------------------------------

tableau_date casted_date raw_date

--------------------------------------------------

2017-04-25 00:00:00 2017-04-25 00:00:00 2017-04-25

--------------------------------------------------
1 row in set (0.01 sec)

Comment by Daniel Lee (Inactive) [ 2017-05-04 ]

Build verified: Github source

[root@localhost mariadb-columnstore-server]# git show
commit 6e3afaaca8abf9903662878b457bdd63af6adbea
Merge: 8ef673b 719b99d
Author: dhall-InfiniDB <david.hall@mariadb.com>
Date: Thu May 4 10:31:50 2017 -0500

Merge pull request #49 from mariadb-corporation/MCOL-679-1.0

MCOL-679 Fix the vtable prep stmt parser

[root@localhost mariadb-columnstore-server]# cd mariadb-columnstore-engine/
[root@localhost mariadb-columnstore-engine]# git show
commit 85c7be75625b7b3f515b83956ee2b0acb6c550a3
Merge: b6efffb f5881ce
Author: dhall-InfiniDB <david.hall@mariadb.com>
Date: Wed Apr 26 09:42:51 2017 -0500

Merge pull request #159 from mariadb-corporation/MCOL-683-1.0

MCOL-683 Fix nested ADDDATE and DATE_FORMAT

Also verified the fix in 1.0.9-1
[root@localhost ~]# mcsmysql mytest
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 6
Server version: 10.1.22-MariaDB Columnstore 1.0.9-1

Copyright (c) 2000, 2016, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [mytest]> CREATE TABLE `columnstore_table` (
-> `a` int(11) DEFAULT NULL,
-> `datecol` date DEFAULT NULL
-> ) ENGINE=Columnstore;
Query OK, 0 rows affected (0.11 sec)

MariaDB [mytest]> INSERT INTO columnstore_table values (1, '2017-04-25');
Query OK, 1 row affected (0.19 sec)

MariaDB [mytest]> SELECT ADDDATE(DATE_FORMAT(datecol, '%Y-%m-%d %H:00:00'), INTERVAL 0 SECOND) AS `tableau_date`, cast(ADDDATE(DATE_FORMAT(datecol, '%Y-%m-%d %H:00:00'), INTERVAL 0 SECOND) as datetime) AS `casted_date`, datecol as raw_date FROM columnstore_table WHERE datecol <= CURRENT_DATE();
--------------------------------------------------

tableau_date casted_date raw_date

--------------------------------------------------

2017-04-25 00:00:00 2017-04-25 00:00:00 2017-04-25

--------------------------------------------------
1 row in set (0.05 sec)

Comment by Geoff Cleaves [ 2017-05-15 ]

Just noticed a problem which may be related:

select least(max(datefield),cast('2000-01-01' as datetime)) from cstable
results in '0000-00-00 00:00:00'

when the expected value is the maximum date in the column.

Comment by Andrew Hutchings (Inactive) [ 2017-05-15 ]

Hi Geoff,

That will be a completely separate problem. MCOL-683 is after the functions have processed and the result is going back to the MariaDB server. This new case is something happening earlier.

Can you please file a new bug report for this with some test data?

Generated at Thu Feb 08 02:23:01 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.