Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-1408

Columnstore table unable to accept writes after thousands of commits via Bulk SDK

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Critical
    • Resolution: Fixed
    • 1.1.4
    • 1.1.5
    • None
    • None
    • 2018-10, 2018-11, 2018-12

    Description

      When running the attached data generator and writing via Bulk Write SDK Columnstore it locks up around 9500 commits and puts the table into a state where no additional writes can occur. (Even if you clear tablelocks and/or restart Columnstore). The only way identified to allow writes again is to "truncate table iot" and then restart ColumnStore. This is very reproducible. The attached python program just needs to run for about 3-4 hours before this occurs.

      The python program is a bit obnoxious and does single row commits, but the actual use case would be a higher velocity of rows (10,000-100,000/second) that are committed at same frequency. If this can't do single row commits, it would not be able to do higher row counts either.

      CREATE TABLE `iot` (
      `deviecParameter` varchar(20) DEFAULT NULL,
      `deviceValue` int(11) DEFAULT NULL,
      `deviceId` char(5) DEFAULT NULL,
      `dt_Time` datetime DEFAULT NULL
      ) ENGINE=Columnstore DEFAULT CHARSET=latin1

      Attachments

        1. 1408.txt.gz
          15 kB
        2. iot.py
          3 kB

        Issue Links

          Activity

            People

              dleeyh Daniel Lee (Inactive)
              austinrutherford Austin Rutherford (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.