Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-641

Full DECIMAL support in ColumnStore

    XMLWordPrintable

Details

    • 2020-1, 2020-2, 2020-3, 2020-4, 2020-5, 2020-6, 2020-7, 2021-1, 2021-2, 2021-3, 2021-4, 2021-5, 2021-6, 2021-7, 2021-8, 2021-9

    Description

      MariaDB ColumnStore supports DECIMAL with some limitations:

      1. We do not support the full DECIMAL range that is in MariaDB

      2. In several places in the code we convert the DECIMAL to DOUBLE during execution therefore losing precision

      Implementing this will likely require the following:

      • Implementation of methods to handle MariaDB's DECIMAL format
      • Support for a longer than 8-byte numeric column type (there is an InfiniDB tree with work for this already)
      • Modification of the primitives processor for the math
      • Modification of the function expression processor to handle the new type
      • Version upgrade support for DECIMAL from the current form to the new form

      Attachments

        Issue Links

          blocks

          Bug - A problem which impairs or prevents the functions of the product. MCOL-564 Aggregation overflow error

          • Major - Major loss of function.
          • Closed
          causes

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4553 Regression: CEIL incorrect for certain decimal numbers

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4858 Incorrect HWM check logic bulk inserting wide decimal column values

          • Major - Major loss of function.
          • Closed
          duplicates

          Bug - A problem which impairs or prevents the functions of the product. MCOL-1207 The max values for Decimal data type is 18, how can store more bigger

          • Major - Major loss of function.
          • Closed
          includes

          Bug - A problem which impairs or prevents the functions of the product. MCOL-162 CS does not support same DECIMAL precision as other MySQL storage engines

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4112 ROUND(d1,38)/TRUNCATE(d1,38) where d1 is DECIMAL(38)/DECIMAL(38,10) gives wrong results.

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4209 Decimal(38) for UDAF

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4309 Math with aggregates on constant columns returns incorrect results

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Task - A task that needs to be done. MCOL-4361 Replace pow(10.0, (double)scale) expressions with a static dictionary lookup.

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4492 Aggregated DOUBLES (AVG and SUM) do not compare properly, especially with constants and non aggregated doubles.

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. MCOL-4603 Replace long double with wide/narrow-decimal for avg() and sum() result type for all numerical datatypesdecimal result

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. MCOL-4724 Make disk-based JOIN collation and wide-decimal aware.

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4743 Regression: TIME_TO_SEC(const_expr) erroneosly returns 0

          • Major - Major loss of function.
          • Closed
          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4652 Support for wide decimals in cpimport communication with DBRM

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MCOL-3987 ColumnStore returns opaque error messages and has no means to raise SQL session warnings from PrimProc.

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4121 Bitwise operations on DECIMAL(38) column do not match results of InnoDB.

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4161 Add full support for Hour/Minute/Second to UNIX_TIMESTAMP() function

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4373 UDAF + multiple DISTINCT in projection produces an invalid answer

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4393 STDDEV_* family of aggregates don't handle scale for DECIMAL arguments

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4413 ROUND() function might be a FLOOR() for DECIMAL arguments in some cases

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4465 UDAF with multiple arguments has a wrong ColType in a projection RowGroup

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4466 UDAF shouldn't use uint64_t variables storing its output as regr_count() and distinct_count() do

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4601 MOD and DIV operators not behaving as InnoDB

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4605 FROM_UNIXTIME(negativeDecimal) not behaving as InnoDB

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4607 SEC_TO_TIME(wideDecimal) returns 0 in DECIMAL context

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4608 SET_TO_TIME(decimal) does not preserve microseconds

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4611 MOD loses precision on huge narrow decimal

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4612 A subquery with a union for DECIMAL and BIGINT returns zeros

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4613 Garbage result of a union between huge narrow DECIMAL and BIGINT

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4622 Implicit FLOAT->INT and DOUBLE->INT conversion is not like in InnoDB

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4623 sec_to_time() does not support FLOAT/DOUBLE UNSIGNED

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4627 SET int_col=decimal_col returns an error

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4628 SET double_col=decimal_col returns an error

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4632 CAST(hugeNegativeWideDecimal AS SIGNED) returns 0 or NULL

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4648 CAST(UBIGINTNULL_inWideDecimal AS UNSIGNED) returns 0 or NULL

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4655 CAST(ROUND(magicInWideDecimal) AS SIGNED) returns NULL

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4667 Precision loss on AVG(decimal)

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4671 LEFT(str, negativeInt) returns a wrong result

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4811 COS() returns different last digit precision value for DECIMAL(38) column.

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22502 MDB crashes in CREATE TABLE AS SELECT when the precision of returning type = 0.

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22503 MDB limits DECIMAL column precision to 16 doing CTAS with floor/ceil over DECIMAL(X,Y) where X > 16

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-23032 FLOOR()/CEIL() incorrectly calculate the precision of a DECIMAL(M,D) column.

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-23118 FORMAT(d1,dec) where dec=0/38 and d1 is DECIMAL(38,38) gives incorrect results

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-25174 DOUBLE columns do not accept large hex hybrids

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-730 cross engine join query select on decimal bad precision/scale

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. MCOL-4377 Split DataConvert::convertColumnData()

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4412 Create Table with longblob column Internal error: Lost connection to DDLProc

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4516 regr_r2 and potentially other statistical functions can't handle tiny negative float values as variance

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4700 Wrong result of a UNION for INT and INT UNSIGNED

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22502 MDB crashes in CREATE TABLE AS SELECT when the precision of returning type = 0.

          • Major - Major loss of function.
          • Closed
          split to

          Task - A task that needs to be done. MCOL-4820 Umbrella for tasks related to the decimal(38) work that needs to be done

          • Major - Major loss of function.
          • Open
          1.
          		 Create a framework for wide data types. Sub-Task Closed Roman  
          2.
          		 Add support for INET6 Sub-Task Closed Roman  
          3.
          		 Full Decimal Research Sub-Task Closed Gagan Goel (Inactive)  
          4.
          		 Add support for wide-DECIMAL into FuncExp framework. Sub-Task Closed Roman  
          5.
          		 Add support for wide-DECIMAL into Window Functions. Sub-Task Closed Daniel Lee (Inactive)  
          6.
          		 Add support for wide-DECIMAL into statistical aggregate and regr_ UDAF functions Sub-Task Closed Roman

          100%
          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 0.25d
          7.
          		 Review/refactor frontend/connector code. Sub-Task Closed Alexander Barkov  
          8.
          		 Review/refactor primitives code Sub-Task Closed Gagan Goel (Inactive)  
          9.
          		 Review/refactor bulk insertion paths and code. Sub-Task Closed Roman  
          10.
          		 Review/refactor DBRM and Casual Partitioning code. Sub-Task Closed Roman  
          11.
          		 Review wide-DECIMAL code in dbcon/execplan classes. Sub-Task Closed Aleksei Antipovskii  
          12.
          		 Attach Google Test as CMake external project. Sub-Task Closed Unassigned  
          13.
          		 Release builds contains ASM that either crashes processes or affects query results Sub-Task Closed Roman

          100%
          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 0.5h
          14.
          		 funcexp::Func_ceil::getIntVal fails with Thread 20 "PPBatchPrimProc" received signal SIGFPE, Arithmetic exception Sub-Task Closed Unassigned  
          15.
          		 Separate Decimal and wide Decimal APIs into two classes Sub-Task Closed Unassigned  
          16.
          		 Convert dataconvert::decimalToString() into VDecimal:: and TSInt128 classes methods. Sub-Task Closed Alexander Barkov  
          17.
          		 Add support for wide-DECIMAL into UNION processing Sub-Task Closed Roman  
          18.
          		 Overflow checks fails in decimal unit tests Sub-Task Closed Roman  
          19.
          		 Bulk insertion doesn't work for wide DECIMALs. Sub-Task Closed Unassigned  
          20.
          		 Move __float128 related code into a separate header file. Sub-Task Closed Alexander Barkov  
          21.
          		 Add DDL test cases for wide decimal columns Sub-Task Closed Bharath Bokka (Inactive)  
          22.
          		 Rename VDecimal into Decimal and make IDB_Decimal an alias from Decimal Sub-Task Closed Alexander Barkov

          100%
          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 0.25d
          23.
          		 UDAF processing might return wrong results duplicating one of the previous group's results Sub-Task Closed Roman  
          24.
          		 Fix between for DECIMAL columns with certain literal values in projection. Sub-Task Closed Roman  
          25.
          		 Row::getBinaryField() might crash accessing int128 from unaligned address Sub-Task Closed Unassigned  
          26.
          		 avg() on wide DECIMAL doesn't round the last digit properly Sub-Task Closed Aleksei Antipovskii  
          27.
          		 Remove libquadmath as the dependency Sub-Task Closed Aleksei Antipovskii  

          Activity

            People

              drrtuy Roman
              LinuxJedi Andrew Hutchings (Inactive)
              Votes:
              7 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0d
                  0d
                  Logged:
                  Time Spent - 4.5h
                  4.5h

                  Git Integration

                    Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.