[MCOL-641] Full DECIMAL support in ColumnStore - Jira

XML

Word

Printable

Details

Type: New Feature
Status: Closed (View Workflow)
Priority: Major
Resolution: Done
Affects Version/s: None
Fix Version/s: 6.1.1
Component/s: ExeMgr
Labels:
- Compatibility

Epic Link:
ColumnStore Compatibility Improvements
Sprint:
2020-1, 2020-2, 2020-3, 2020-4, 2020-5, 2020-6, 2020-7, 2021-1, 2021-2, 2021-3, 2021-4, 2021-5, 2021-6, 2021-7, 2021-8, 2021-9

Description

MariaDB ColumnStore supports DECIMAL with some limitations:

1. We do not support the full DECIMAL range that is in MariaDB

2. In several places in the code we convert the DECIMAL to DOUBLE during execution therefore losing precision

Implementing this will likely require the following:

Implementation of methods to handle MariaDB's DECIMAL format
Support for a longer than 8-byte numeric column type (there is an InfiniDB tree with work for this already)
Modification of the primitives processor for the math
Modification of the function expression processor to handle the new type
Version upgrade support for DECIMAL from the current form to the new form

Attachments

Issue Links

blocks

MCOL-564 Aggregation overflow error

Closed

causes

MCOL-4553 Regression: CEIL incorrect for certain decimal numbers

Closed

MCOL-4858 Incorrect HWM check logic bulk inserting wide decimal column values

Closed

duplicates

MCOL-1207 The max values for Decimal data type is 18, how can store more bigger

Closed

includes

MCOL-162 CS does not support same DECIMAL precision as other MySQL storage engines

Closed

MCOL-4112 ROUND(d1,38)/TRUNCATE(d1,38) where d1 is DECIMAL(38)/DECIMAL(38,10) gives wrong results.

Closed

MCOL-4209 Decimal(38) for UDAF

Closed

MCOL-4309 Math with aggregates on constant columns returns incorrect results

Open

MCOL-4361 Replace pow(10.0, (double)scale) expressions with a static dictionary lookup.

Closed

MCOL-4492 Aggregated DOUBLES (AVG and SUM) do not compare properly, especially with constants and non aggregated doubles.

Closed

MCOL-4603 Replace long double with wide/narrow-decimal for avg() and sum() result type for all numerical datatypesdecimal result

Closed

MCOL-4724 Make disk-based JOIN collation and wide-decimal aware.

Open

MCOL-4743 Regression: TIME_TO_SEC(const_expr) erroneosly returns 0

Closed

is blocked by

MCOL-4652 Support for wide decimals in cpimport communication with DBRM

Closed

relates to

MCOL-3987 ColumnStore returns opaque error messages and has no means to raise SQL session warnings from PrimProc.

Closed

MCOL-4121 Bitwise operations on DECIMAL(38) column do not match results of InnoDB.

Closed

MCOL-4161 Add full support for Hour/Minute/Second to UNIX_TIMESTAMP() function

Closed

MCOL-4373 UDAF + multiple DISTINCT in projection produces an invalid answer

Open

MCOL-4393 STDDEV_* family of aggregates don't handle scale for DECIMAL arguments

Closed

MCOL-4413 ROUND() function might be a FLOOR() for DECIMAL arguments in some cases

Closed

MCOL-4465 UDAF with multiple arguments has a wrong ColType in a projection RowGroup

Closed

MCOL-4466 UDAF shouldn't use uint64_t variables storing its output as regr_count() and distinct_count() do

Closed

MCOL-4601 MOD and DIV operators not behaving as InnoDB

Open

MCOL-4605 FROM_UNIXTIME(negativeDecimal) not behaving as InnoDB

Open

MCOL-4607 SEC_TO_TIME(wideDecimal) returns 0 in DECIMAL context

Open

MCOL-4608 SET_TO_TIME(decimal) does not preserve microseconds

Open

MCOL-4611 MOD loses precision on huge narrow decimal

Open

MCOL-4612 A subquery with a union for DECIMAL and BIGINT returns zeros

Closed

MCOL-4613 Garbage result of a union between huge narrow DECIMAL and BIGINT

Closed

MCOL-4622 Implicit FLOAT->INT and DOUBLE->INT conversion is not like in InnoDB

Open

MCOL-4623 sec_to_time() does not support FLOAT/DOUBLE UNSIGNED

Open

MCOL-4627 SET int_col=decimal_col returns an error

Open

MCOL-4628 SET double_col=decimal_col returns an error

Open

MCOL-4632 CAST(hugeNegativeWideDecimal AS SIGNED) returns 0 or NULL

Closed

MCOL-4648 CAST(UBIGINTNULL_inWideDecimal AS UNSIGNED) returns 0 or NULL

Closed

MCOL-4655 CAST(ROUND(magicInWideDecimal) AS SIGNED) returns NULL

Closed

MCOL-4667 Precision loss on AVG(decimal)

Open

MCOL-4671 LEFT(str, negativeInt) returns a wrong result

Closed

MCOL-4811 COS() returns different last digit precision value for DECIMAL(38) column.

Closed

MDEV-22502 MDB crashes in CREATE TABLE AS SELECT when the precision of returning type = 0.

Closed

MDEV-22503 MDB limits DECIMAL column precision to 16 doing CTAS with floor/ceil over DECIMAL(X,Y) where X > 16

Closed

MDEV-23032 FLOOR()/CEIL() incorrectly calculate the precision of a DECIMAL(M,D) column.

Closed

MDEV-23118 FORMAT(d1,dec) where dec=0/38 and d1 is DECIMAL(38,38) gives incorrect results

Closed

MDEV-25174 DOUBLE columns do not accept large hex hybrids

Closed

MCOL-730 cross engine join query select on decimal bad precision/scale

Closed

MCOL-4377 Split DataConvert::convertColumnData()

Closed

MCOL-4412 Create Table with longblob column Internal error: Lost connection to DDLProc

Closed

MCOL-4516 regr_r2 and potentially other statistical functions can't handle tiny negative float values as variance

Closed

MCOL-4700 Wrong result of a UNION for INT and INT UNSIGNED

Closed

MDEV-22502 MDB crashes in CREATE TABLE AS SELECT when the precision of returning type = 0.

Closed

split to

MCOL-4820 Umbrella for tasks related to the decimal(38) work that needs to be done

Open

(8 includes, 1 is blocked by, 36 relates to, 1 split to)

Sub-Tasks

1.	Create a framework for wide data types.	Closed	Roman
2.	Add support for INET6	Closed	Roman
3.	Full Decimal Research	Closed	Gagan Goel (Inactive)
4.	Add support for wide-DECIMAL into FuncExp framework.	Closed	Roman
5.	Add support for wide-DECIMAL into Window Functions.	Closed	Daniel Lee (Inactive)
6.	Add support for wide-DECIMAL into statistical aggregate and regr_ UDAF functions	Closed	Roman
7.	Review/refactor frontend/connector code.	Closed	Alexander Barkov
8.	Review/refactor primitives code	Closed	Gagan Goel (Inactive)
9.	Review/refactor bulk insertion paths and code.	Closed	Roman
10.	Review/refactor DBRM and Casual Partitioning code.	Closed	Roman
11.	Review wide-DECIMAL code in dbcon/execplan classes.	Closed	Alexey Antipovsky
12.	Attach Google Test as CMake external project.	Closed	Unassigned
13.	Release builds contains ASM that either crashes processes or affects query results	Closed	Roman
14.	funcexp::Func_ceil::getIntVal fails with Thread 20 "PPBatchPrimProc" received signal SIGFPE, Arithmetic exception	Closed	Unassigned
15.	Separate Decimal and wide Decimal APIs into two classes	Closed	Unassigned
16.	Convert dataconvert::decimalToString() into VDecimal:: and TSInt128 classes methods.	Closed	Alexander Barkov
17.	Add support for wide-DECIMAL into UNION processing	Closed	Roman
18.	Overflow checks fails in decimal unit tests	Closed	Roman
19.	Bulk insertion doesn't work for wide DECIMALs.	Closed	Unassigned
20.	Move __float128 related code into a separate header file.	Closed	Alexander Barkov
21.	Add DDL test cases for wide decimal columns	Closed	Bharath Bokka (Inactive)
22.	Rename VDecimal into Decimal and make IDB_Decimal an alias from Decimal	Closed	Alexander Barkov
23.	UDAF processing might return wrong results duplicating one of the previous group's results	Closed	Roman
24.	Fix between for DECIMAL columns with certain literal values in projection.	Closed	Roman
25.	Row::getBinaryField() might crash accessing int128 from unaligned address	Closed	Unassigned
26.	avg() on wide DECIMAL doesn't round the last digit properly	Closed	Alexey Antipovsky
27.	Remove libquadmath as the dependency	Closed	Alexey Antipovsky

Activity

People

Assignee:: Roman

Reporter:: Andrew Hutchings (Inactive)

Votes:: 7 Vote for this issue

Watchers:: 15 Start watching this issue

Dates

Created:: 2017-03-27 13:39

Updated:: 2024-10-03 15:51

Resolved:: 2021-06-25 18:41

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.