[MCOL-641] Full DECIMAL support in ColumnStore Created: 2017-03-27  Updated: 2021-09-10  Resolved: 2021-06-25

Status: Closed
Project: MariaDB ColumnStore
Component/s: ExeMgr
Affects Version/s: None
Fix Version/s: 6.1.1

Type: New Feature Priority: Major
Reporter: Andrew Hutchings (Inactive) Assignee: Roman
Resolution: Done Votes: 7
Labels: Compatibility

Issue Links:
Blocks
blocks MCOL-564 Aggregation overflow error Closed
is blocked by MCOL-4652 Support for wide decimals in cpimport... Closed
Duplicate
duplicates MCOL-1207 The max values for Decimal data type ... Closed
Issue split
split to MCOL-4820 Umbrella for tasks related to the dec... Open
PartOf
includes MCOL-162 CS does not support same DECIMAL prec... Closed
includes MCOL-4112 ROUND(d1,38)/TRUNCATE(d1,38) where d1... Closed
includes MCOL-4209 Decimal(38) for UDAF Closed
includes MCOL-4309 Math with aggregates on constant colu... Open
includes MCOL-4361 Replace pow(10.0, (double)scale) expr... Closed
includes MCOL-4492 Aggregated DOUBLES (AVG and SUM) do n... Closed
includes MCOL-4603 Replace long double with wide/narrow-... Closed
includes MCOL-4724 Make disk-based JOIN collation and wi... Open
includes MCOL-4743 Regression: TIME_TO_SEC(const_expr) e... Closed
Problem/Incident
causes MCOL-4553 Regression: CEIL incorrect for certai... Closed
causes MCOL-4858 Incorrect HWM check logic bulk insert... Closed
Relates
relates to MCOL-3987 ColumnStore returns opaque error mess... Closed
relates to MCOL-4121 Bitwise operations on DECIMAL(38) col... Closed
relates to MCOL-4161 Add full support for Hour/Minute/Seco... Closed
relates to MCOL-4373 UDAF + multiple DISTINCT in projectio... Open
relates to MCOL-4393 STDDEV_* family of aggregates don't h... Open
relates to MCOL-4413 ROUND() function might be a FLOOR() f... Closed
relates to MCOL-4465 UDAF with multiple arguments has a wr... Closed
relates to MCOL-4466 UDAF shouldn't use uint64_t variables... Closed
relates to MCOL-4601 MOD and DIV operators not behaving as... Open
relates to MCOL-4605 FROM_UNIXTIME(negativeDecimal) not be... Open
relates to MCOL-4607 SEC_TO_TIME(wideDecimal) returns 0 in... Open
relates to MCOL-4608 SET_TO_TIME(decimal) does not preserv... Open
relates to MCOL-4611 MOD loses precision on huge narrow de... Open
relates to MCOL-4612 A subquery with a union for DECIMAL a... Closed
relates to MCOL-4613 Garbage result of a union between hug... Closed
relates to MCOL-4622 Implicit FLOAT->INT and DOUBLE->INT c... Open
relates to MCOL-4623 sec_to_time() does not support FLOAT/... Open
relates to MCOL-4627 SET int_col=decimal_col returns an error Open
relates to MCOL-4628 SET double_col=decimal_col returns an... Open
relates to MCOL-4632 CAST(hugeNegativeWideDecimal AS SIGNE... Closed
relates to MCOL-4648 CAST(UBIGINTNULL_inWideDecimal AS UNS... Closed
relates to MCOL-4655 CAST(ROUND(magicInWideDecimal) AS SIG... Open
relates to MCOL-4667 Precision loss on AVG(decimal) Open
relates to MCOL-4671 LEFT(str, negativeInt) returns a wron... Open
relates to MCOL-4811 COS() returns different last digit pr... Closed
relates to MDEV-22502 MDB crashes in CREATE TABLE AS SELECT... Closed
relates to MDEV-22503 MDB limits DECIMAL column precision t... Closed
relates to MDEV-23032 FLOOR()/CEIL() incorrectly calculate ... Closed
relates to MDEV-23118 FORMAT(d1,dec) where dec=0/38 and d1 ... Closed
relates to MDEV-25174 DOUBLE columns do not accept large he... Open
relates to MCOL-730 cross engine join query select on dec... Closed
relates to MCOL-4377 Split DataConvert::convertColumnData() Closed
relates to MCOL-4412 Create Table with longblob column Int... Closed
relates to MCOL-4516 regr_r2 and potentially other statist... Closed
relates to MCOL-4700 Wrong result of a UNION for INT and I... Closed
relates to MDEV-22502 MDB crashes in CREATE TABLE AS SELECT... Closed
Sub-Tasks:
Key
 Summary
 Type
 Status
 Assignee
MCOL-3609 Create a framework for wide data types. Sub-Task Closed Roman  
MCOL-3610 Add support for INET6 Sub-Task Closed Roman  
MCOL-3753 Full Decimal Research Sub-Task Closed Gagan Goel  
MCOL-4167 Add support for wide-DECIMAL into Fun... Sub-Task Closed Roman  
MCOL-4171 Add support for wide-DECIMAL into Win... Sub-Task Closed Daniel Lee  
MCOL-4172 Add support for wide-DECIMAL into sta... Sub-Task Closed Roman  
MCOL-4174 Review/refactor frontend/connector code. Sub-Task Closed Alexander Barkov  
MCOL-4175 Review/refactor primitives code Sub-Task Closed Gagan Goel  
MCOL-4177 Review/refactor bulk insertion paths ... Sub-Task Closed Roman  
MCOL-4178 Review/refactor DBRM and Casual Parti... Sub-Task Closed Roman  
MCOL-4180 Review wide-DECIMAL code in dbcon/exe... Sub-Task Closed Alexey Antipovsky  
MCOL-4187 Attach Google Test as CMake external ... Sub-Task Closed  
MCOL-4313 Release builds contains ASM that eith... Sub-Task Closed Roman  
MCOL-4362 funcexp::Func_ceil::getIntVal fails w... Sub-Task Closed  
MCOL-4385 Separate Decimal and wide Decimal API... Sub-Task Closed  
MCOL-4387 Convert dataconvert::decimalToString(... Sub-Task Closed Alexander Barkov  
MCOL-4389 Add support for wide-DECIMAL into UNI... Sub-Task Closed Roman  
MCOL-4390 Overflow checks fails in decimal unit... Sub-Task Closed Roman  
MCOL-4391 Bulk insertion doesn't work for wide ... Sub-Task Closed  
MCOL-4394 Move __float128 related code into a s... Sub-Task Closed Alexander Barkov  
MCOL-4398 Add DDL test cases for wide decimal c... Sub-Task Closed Bharath Bokka  
MCOL-4409 Rename VDecimal into Decimal and make... Sub-Task Closed Alexander Barkov  
MCOL-4452 UDAF processing might return wrong re... Sub-Task Closed Roman  
MCOL-4463 Fix between for DECIMAL columns with ... Sub-Task Closed Roman  
MCOL-4469 Row::getBinaryField() might crash acc... Sub-Task Closed  
MCOL-4478 avg() on wide DECIMAL doesn't round t... Sub-Task Closed Alexey Antipovsky  
MCOL-4479 Remove libquadmath as the dependency Sub-Task Closed Alexey Antipovsky  
Epic Link: ColumnStore Compatibility Improvements
Sprint: 2020-1, 2020-2, 2020-3, 2020-4, 2020-5, 2020-6, 2020-7, 2021-1, 2021-2, 2021-3, 2021-4, 2021-5, 2021-6, 2021-7, 2021-8, 2021-9

 Description   

MariaDB ColumnStore supports DECIMAL with some limitations:

1. We do not support the full DECIMAL range that is in MariaDB

2. In several places in the code we convert the DECIMAL to DOUBLE during execution therefore losing precision

Implementing this will likely require the following:

  • Implementation of methods to handle MariaDB's DECIMAL format
  • Support for a longer than 8-byte numeric column type (there is an InfiniDB tree with work for this already)
  • Modification of the primitives processor for the math
  • Modification of the function expression processor to handle the new type
  • Version upgrade support for DECIMAL from the current form to the new form

 Comments   
Comment by Andrew Hutchings (Inactive) [ 2017-11-27 ]

As a first version this should be implemented using dictionary columns rather than fixed length when more than 18 digits are required.

Comment by Lohit Marla [ 2019-03-14 ]

Hello, my name is lohit. Looking forward to be part of this community

Comment by Vicențiu Ciorbaru [ 2019-03-25 ]

Hi! Lohitmarla

If you are interested in contributing this as part of GSoC 2019, I suggest you send an email to maria-developers mailing list
https://launchpad.net/~maria-developers and ask for more guidance. You are guaranteed to have more people notice your messages there and also plenty of folks willing to help you out.

Comment by Gagan Goel (Inactive) [ 2020-06-27 ]

The functional tests for FLOOR/CEIL distributed functions need to be updated once MDEV-23032 is fixed.

Generated at Thu Feb 08 02:22:36 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.