Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-5501

Joins on longtext using exorbitant memory when nested

    XMLWordPrintable

Details

    • Bug
    • Status: Open (View Workflow)
    • Minor
    • Resolution: Unresolved
    • None
    • Icebox
    • None
    • None

    Description

      DDL:

       CREATE TABLE `sample` (
        `WhitelabelDataSourceId` bigint(20) NOT NULL,
        `Account` longtext DEFAULT NULL,
        `AccountCurrency` longtext DEFAULT NULL,
        `AccountGroup` longtext DEFAULT NULL,
        `Asset` longtext DEFAULT NULL,
        `AssetCurrency` longtext DEFAULT NULL,
        `AssetType` int(11) NOT NULL,
        `AssetTypeOther` int(11) DEFAULT NULL,
        `Commission` double NOT NULL,
        `CommissionInCurrency` double NOT NULL,
        `Digits` int(11) NOT NULL,
        `Entry` longtext DEFAULT NULL,
        `Fees` double NOT NULL,
        `FeesInCurrency` double NOT NULL,
        `OrderNumber` longtext DEFAULT NULL,
        `NotionalAmount` double NOT NULL,
        `NotionalAmountCurrencyRate` double NOT NULL,
        `NotionalAmountCurrencyRateSource` longtext DEFAULT NULL,
        `NotionalAmountInCcy` double NOT NULL,
        `Position` longtext DEFAULT NULL,
        `Price` double NOT NULL,
        `PricePosition` double NOT NULL,
        `Profit` double NOT NULL,
        `ProfitCurrencyRate` double NOT NULL,
        `ProfitCurrencyRateAccount` double NOT NULL,
        `ProfitCurrencyRateAccountSource` longtext DEFAULT NULL,
        `ProfitCurrencyRateSource` longtext DEFAULT NULL,
        `ProfitInCurrency` double NOT NULL,
        `Quantity` double NOT NULL,
        `SecondaryConversionRate` double NOT NULL,
        `SecondaryConversionRateSource` longtext DEFAULT NULL,
        `Side` longtext DEFAULT NULL,
        `Swaps` double NOT NULL,
        `SwapsInCurrency` double NOT NULL,
        `Ticket` longtext DEFAULT NULL,
        `Timestamp` datetime(6) NOT NULL,
        `ErrorType` int(11) DEFAULT NULL,
        `Errors` longtext DEFAULT NULL,
        `CreationDate` datetime(6) NOT NULL,
        `ModificationDate` datetime(6) NOT NULL,
        `Id` bigint(20) NOT NULL
      ) ENGINE=Columnstore DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci
      

      Query:

      # Crashes when join is nested
      SELECT Position, Ticket FROM sample AS t1 WHERE NOT EXISTS( SELECT 1 FROM sample AS t2 WHERE t2.Position = t1.Position AND t2.Entry = 'Out') LIMIT 0, 1000; 
      ERROR 1815 (HY000): Internal error: (437) MCS-2001: Join or subselect exceeds memory limit.
       
      # Running join alone does not crash
        SELECT  count(*)  FROM  sample AS t2 inner join Trade as t1     ON       t2.Position = t1.Position       AND t2.Entry = 'Out'
      

      Short Term work around is to adjust datatypes from
      Account longtext -> varchar(255)
      Entry longtext -> char(7)
      Position longtext -> varchar(100)

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              allen.herrera Allen Herrera
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.