[MCOL-3488] chunk shifting can fail when going through SM Created: 2019-09-06 Updated: 2019-09-20 Resolved: 2019-09-20 |
|
| Status: | Closed |
| Project: | MariaDB ColumnStore |
| Component/s: | None |
| Affects Version/s: | 1.4.0 |
| Fix Version/s: | 1.4.0 |
| Type: | Bug | Priority: | Major |
| Reporter: | Patrick LeBlanc (Inactive) | Assignee: | Daniel Lee (Inactive) |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Description |
|
Found that test112 fails when using storagemanager. Something to do with chunk shifting and the renaming it does. I think I know what it is. SM currently doesn't copy between filesystems, so if it's told to rename something local (one of those 'scratch' files) to something in SM, it will fail. Looking into it. |
| Comments |
| Comment by Patrick LeBlanc (Inactive) [ 2019-09-06 ] |
|
veni, vidi, vici. It was that the ChunkManager class was getting an instance of IDBFileSystem a different way than seemingly everything else. It slipped passed the rest of the tests b/c it seems only to be used in a way that wouldn't cause a test failure, unless there was a chunk expansion. On Mon, I'll grep the rest of the code looking for this method of getting a FS instance. |
| Comment by Patrick LeBlanc (Inactive) [ 2019-09-09 ] |
|
Found a couple more instances of this pattern, fixed them. For QA, the way to test this bug is to do something similar to test112, which imports some data, then does a lot of updates that cause a compression chunk to grow and eventually need to expand. WE does some Stuff to accomplish this, including making a new copy of the file with the expanded chunk, and renaming it over top the original file. It's the renaming/moving that would not happen. So, the original file ends up being left in place. You get an error from WE saying the new file wasn't the expected size (because in fact it was looking at the original file). After the bug, the rename works, so no errors. |
| Comment by Patrick LeBlanc (Inactive) [ 2019-09-09 ] |
|
PR https://github.com/mariadb-corporation/mariadb-columnstore-engine/pull/857 |
| Comment by Ben Thompson (Inactive) [ 2019-09-16 ] |
|
Andrew reviewed and merged but it's still assigned to me. Passing to daniel to verify this passes regression with storagemanager. |
| Comment by Daniel Lee (Inactive) [ 2019-09-20 ] |
|
Build verified: 1.4.0-1 [root@localhost alltest]# cat /data/qa/release/1.4.0-1/centos7/gitversionInfo.txt Performed regression test112 test on a single server installation using SM localStorage. Also did DBT3 10gb loads and many updates on tables. |