Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-1289

Python bulk load is slower than expected

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Won't Do
    • None
    • N/A, 1.4.5
    • None
    • None
    • 2018-07, 2018-08, 2018-09, 2018-10, 2018-11, 2018-12, 2018-13, 2018-14, 2018-15, 2018-16, 2018-17, 2018-18, 2018-19, 2018-20, 2018-21, 2019-01, 2019-02, 2019-03, 2019-04

    Description

      A user has reported that bulk loading data with the Python API is slower than expected. He said that neither the network nor the WriteEngine seem to be the bottleneck, so he suspects that performance can be improved.

      One suggestion provided was to have some functions in the Python API where he could pass the data already formatted for the table, and then have all the setColumn calls and data casting performed in the underlying C API instead of Python. He suspects that this would probably help to speed up the load process.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              GeoffMontee Geoff Montee (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.