[MCOL-5501] Joins on longtext using exorbitant memory when nested Created: 2023-05-25  Updated: 2023-07-01

Status: Open
Project: MariaDB ColumnStore
Component/s: None
Affects Version/s: None
Fix Version/s: Icebox

Type: Bug Priority: Minor
Reporter: Allen Herrera Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Issue Links:
Relates
relates to MCOL-5366 Research TPC-DS queries failing due t... Open

 Description   

DDL:

 CREATE TABLE `sample` (
  `WhitelabelDataSourceId` bigint(20) NOT NULL,
  `Account` longtext DEFAULT NULL,
  `AccountCurrency` longtext DEFAULT NULL,
  `AccountGroup` longtext DEFAULT NULL,
  `Asset` longtext DEFAULT NULL,
  `AssetCurrency` longtext DEFAULT NULL,
  `AssetType` int(11) NOT NULL,
  `AssetTypeOther` int(11) DEFAULT NULL,
  `Commission` double NOT NULL,
  `CommissionInCurrency` double NOT NULL,
  `Digits` int(11) NOT NULL,
  `Entry` longtext DEFAULT NULL,
  `Fees` double NOT NULL,
  `FeesInCurrency` double NOT NULL,
  `OrderNumber` longtext DEFAULT NULL,
  `NotionalAmount` double NOT NULL,
  `NotionalAmountCurrencyRate` double NOT NULL,
  `NotionalAmountCurrencyRateSource` longtext DEFAULT NULL,
  `NotionalAmountInCcy` double NOT NULL,
  `Position` longtext DEFAULT NULL,
  `Price` double NOT NULL,
  `PricePosition` double NOT NULL,
  `Profit` double NOT NULL,
  `ProfitCurrencyRate` double NOT NULL,
  `ProfitCurrencyRateAccount` double NOT NULL,
  `ProfitCurrencyRateAccountSource` longtext DEFAULT NULL,
  `ProfitCurrencyRateSource` longtext DEFAULT NULL,
  `ProfitInCurrency` double NOT NULL,
  `Quantity` double NOT NULL,
  `SecondaryConversionRate` double NOT NULL,
  `SecondaryConversionRateSource` longtext DEFAULT NULL,
  `Side` longtext DEFAULT NULL,
  `Swaps` double NOT NULL,
  `SwapsInCurrency` double NOT NULL,
  `Ticket` longtext DEFAULT NULL,
  `Timestamp` datetime(6) NOT NULL,
  `ErrorType` int(11) DEFAULT NULL,
  `Errors` longtext DEFAULT NULL,
  `CreationDate` datetime(6) NOT NULL,
  `ModificationDate` datetime(6) NOT NULL,
  `Id` bigint(20) NOT NULL
) ENGINE=Columnstore DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci

Query:

# Crashes when join is nested
SELECT Position, Ticket FROM sample AS t1 WHERE NOT EXISTS( SELECT 1 FROM sample AS t2 WHERE t2.Position = t1.Position AND t2.Entry = 'Out') LIMIT 0, 1000; 
ERROR 1815 (HY000): Internal error: (437) MCS-2001: Join or subselect exceeds memory limit.
 
# Running join alone does not crash
  SELECT  count(*)  FROM  sample AS t2 inner join Trade as t1     ON       t2.Position = t1.Position       AND t2.Entry = 'Out'

Short Term work around is to adjust datatypes from
Account longtext -> varchar(255)
Entry longtext -> char(7)
Position longtext -> varchar(100)


Generated at Thu Feb 08 02:58:21 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.