Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-3488

chunk shifting can fail when going through SM

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • 1.4.0
    • 1.4.0
    • None
    • None

    Description

      Found that test112 fails when using storagemanager. Something to do with chunk shifting and the renaming it does. I think I know what it is. SM currently doesn't copy between filesystems, so if it's told to rename something local (one of those 'scratch' files) to something in SM, it will fail. Looking into it.

      Attachments

        Activity

          veni, vidi, vici. It was that the ChunkManager class was getting an instance of IDBFileSystem a different way than seemingly everything else. It slipped passed the rest of the tests b/c it seems only to be used in a way that wouldn't cause a test failure, unless there was a chunk expansion. On Mon, I'll grep the rest of the code looking for this method of getting a FS instance.

          pleblanc Patrick LeBlanc (Inactive) added a comment - veni, vidi, vici. It was that the ChunkManager class was getting an instance of IDBFileSystem a different way than seemingly everything else. It slipped passed the rest of the tests b/c it seems only to be used in a way that wouldn't cause a test failure, unless there was a chunk expansion. On Mon, I'll grep the rest of the code looking for this method of getting a FS instance.
          pleblanc Patrick LeBlanc (Inactive) added a comment - - edited

          Found a couple more instances of this pattern, fixed them.

          For QA, the way to test this bug is to do something similar to test112, which imports some data, then does a lot of updates that cause a compression chunk to grow and eventually need to expand. WE does some Stuff to accomplish this, including making a new copy of the file with the expanded chunk, and renaming it over top the original file. It's the renaming/moving that would not happen. So, the original file ends up being left in place. You get an error from WE saying the new file wasn't the expected size (because in fact it was looking at the original file). After the bug, the rename works, so no errors.

          pleblanc Patrick LeBlanc (Inactive) added a comment - - edited Found a couple more instances of this pattern, fixed them. For QA, the way to test this bug is to do something similar to test112, which imports some data, then does a lot of updates that cause a compression chunk to grow and eventually need to expand. WE does some Stuff to accomplish this, including making a new copy of the file with the expanded chunk, and renaming it over top the original file. It's the renaming/moving that would not happen. So, the original file ends up being left in place. You get an error from WE saying the new file wasn't the expected size (because in fact it was looking at the original file). After the bug, the rename works, so no errors.
          pleblanc Patrick LeBlanc (Inactive) added a comment - PR https://github.com/mariadb-corporation/mariadb-columnstore-engine/pull/857

          Andrew reviewed and merged but it's still assigned to me. Passing to daniel to verify this passes regression with storagemanager.

          ben.thompson Ben Thompson (Inactive) added a comment - Andrew reviewed and merged but it's still assigned to me. Passing to daniel to verify this passes regression with storagemanager.

          Build verified: 1.4.0-1

          [root@localhost alltest]# cat /data/qa/release/1.4.0-1/centos7/gitversionInfo.txt
          engine commit:
          975463c

          Performed regression test112 test on a single server installation using SM localStorage. Also did DBT3 10gb loads and many updates on tables.

          dleeyh Daniel Lee (Inactive) added a comment - Build verified: 1.4.0-1 [root@localhost alltest] # cat /data/qa/release/1.4.0-1/centos7/gitversionInfo.txt engine commit: 975463c Performed regression test112 test on a single server installation using SM localStorage. Also did DBT3 10gb loads and many updates on tables.

          People

            dleeyh Daniel Lee (Inactive)
            pleblanc Patrick LeBlanc (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.