Installation with S3 storage is much slower then the last release

Build tested: 5.5.1 (Drone 1313)

Installation is successful, but create table and LDI are much slower than 5.4.3-1. There is close to 2x of the data in the S3 bucket.

5.5.1-1

Total test run time:
real 19m57.116s
user 0m6.294s
sys 0m3.207s

Create table time:
s1pm1: Query OK, 0 rows affected (1 min 10.077 sec) – orders
s1pm1: Query OK, 0 rows affected (40.523 sec) – lineitem

LDI of 1gb lineitem:
s1pm1: Query OK, 6001215 rows affected (14 min 36.892 sec)
s1pm1: Records: 6001215 Deleted: 0 Skipped: 0 Warnings: 0

S3 Bucket usage:
AWS S3 storage is used, bucket = dleeqadbroot1, objects = 153, total size = 304130229 (290 MB)

5.4.3-1

Total test run time:
real 11m14.021s
user 0m5.488s
sys 0m2.585s

Create table time:
s1pm1: Query OK, 0 rows affected (1.996 sec) – orders
s1pm1: Query OK, 0 rows affected (0.198 sec) – lineitem

LDI of 1gb lineitem:

s1pm1: Query OK, 6001215 rows affected (6 min 50.879 sec)
s1pm1: Records: 6001215 Deleted: 0 Skipped: 0 Warnings: 0

S3 bucket usage:
AWS S3 storage is used, bucket = dleeqadbroot1, objects = 131, total size = 184797331 (176 MB)

Detail info for both releases compared:

5.5.1-1

[dlee@aloha tests]$ /data/qa/installtest/test/installtest.sh s1 5.5.1-1 centos8 rpm 1pm.s3cloud.xml n root 1
Using local setup file /home/dlee/.installtest/localSetup.xml
Using local setup file /home/dlee/.installtest/localSetup.xml
Thu Dec 10 11:43:42 CST 2020 Extracting test parameters from test file
Thu Dec 10 11:43:42 CST 2020 Generating test scripts
Using local setup file /home/dlee/.installtest/localSetup.xml
Executing installation test......
StackName=s1 Release=5.5.1-1 OS=centos8 Package=rpm Test=/data/qa/installtest/ctrl/test/5.5.1-1/1pm.s3cloud.xml installMode=n TestUser=root
Release type: Enterprise
Thu Dec 10 11:43:43 CST 2020 Starting test using Vagrant boxes

real 19m57.116s
user 0m6.294s
sys 0m3.207s
s1pm1: ----------
s1pm1: 1 row in set (0.070 sec)
s1pm1:
s1pm1: Bye
s1pm1: No tables are locked in the database.
s1pm1: end runDefaultTest.sh
s1pm1: Thu Dec 10 18:03:38 UTC 2020
s1pm1: Exit status 0
AWS S3 storage is used, bucket = dleeqadbroot1, objects = 153, total size = 304130229 (290 MB)
Thu Dec 10 12:03:41 CST 2020
Correct row counts: 6001215, 1500000
Software versions matched
Expected versions: 10.5.9, 5.5.1
Tested versions= s1pm1: Server version: 10.5.9-MariaDB MariaDB Server, s1pm1: | Columnstore_version | 5.5.1 |
Passed /home/dlee/tests/5.5.1-1/centos8/root.rpm.1pm.s3.n.s1
Thu Dec 10 12:03:41 CST 2020 Pau......
[dlee@aloha tests]$

s1pm1: create table if not exists orders (
s1pm1: o_orderkey int,
s1pm1: o_custkey int,
s1pm1: o_orderstatus char (1),
s1pm1: o_totalprice decimal(12,2),
s1pm1: o_orderdate date,
s1pm1: o_orderpriority char (15),
s1pm1: o_clerk char (15),
s1pm1: o_shippriority int,
s1pm1: o_comment varchar (79)
s1pm1: ) engine=Columnstore
s1pm1: --------------
s1pm1:
s1pm1: Query OK, 0 rows affected (1 min 10.077 sec)
s1pm1:
s1pm1: --------------
s1pm1: create table if not exists lineitem (
s1pm1: l_orderkey int,
s1pm1: l_partkey int,
s1pm1: l_suppkey int,
s1pm1: l_linenumber bigint,
s1pm1: l_quantity decimal(12,2),
s1pm1: l_extendedprice decimal(12,2),
s1pm1: l_discount decimal(12,2),
s1pm1: l_tax decimal(12,2),
s1pm1: l_returnflag char (1),
s1pm1: l_linestatus char (1),
s1pm1: l_shipdate date,
s1pm1: l_commitdate date,
s1pm1: l_receiptdate date,
s1pm1: l_shipinstruct char (25),
s1pm1: l_shipmode char (10),
s1pm1: l_comment varchar (44)
s1pm1: ) engine=columnstore
s1pm1: --------------
s1pm1:
s1pm1: Query OK, 0 rows affected (40.523 sec)
s1pm1:
s1pm1: --------------
s1pm1: load data infile
s1pm1:
s1pm1: "/data/qa/source/dbt3/1g/lineitem.tbl" into table lineitem fields terminated by "|"
s1pm1: --------------
s1pm1:
s1pm1: Query OK, 6001215 rows affected (14 min 36.892 sec)
s1pm1: Records: 6001215 Deleted: 0 Skipped: 0 Warnings: 0

5.4.3-1

[dlee@aloha tests]$ /data/qa/installtest/test/installtest.sh s1 5.4.3-1 centos8 rpm 1pm.s3cloud.xml n root 1
Using local setup file /home/dlee/.installtest/localSetup.xml
Using local setup file /home/dlee/.installtest/localSetup.xml
Thu Dec 10 11:27:56 CST 2020 Extracting test parameters from test file
Thu Dec 10 11:27:57 CST 2020 Generating test scripts
Using local setup file /home/dlee/.installtest/localSetup.xml
Executing installation test......
StackName=s1 Release=5.4.3-1 OS=centos8 Package=rpm Test=/data/qa/installtest/ctrl/test/5.4.3-1/1pm.s3cloud.xml installMode=n TestUser=root
Release type: hotfix
Thu Dec 10 11:27:57 CST 2020 Starting test using Vagrant boxes

real 11m14.021s
user 0m5.488s
sys 0m2.585s
s1pm1: ----------
s1pm1: 1 row in set (0.094 sec)
s1pm1:
s1pm1: Bye
s1pm1: No tables are locked in the database.
s1pm1: end runDefaultTest.sh
s1pm1: Thu Dec 10 17:39:09 UTC 2020
s1pm1: Exit status 0
AWS S3 storage is used, bucket = dleeqadbroot1, objects = 131, total size = 184797331 (176 MB)
Thu Dec 10 11:39:13 CST 2020
Correct row counts: 6001215, 1500000
Software versions matched
Expected versions: 10.5.6-4, 5.4.3
Tested versions= s1pm1: Server version: 10.5.6-4.5.4.3-MariaDB-enterprise MariaDB Enterprise Server, s1pm1: | Columnstore_version | 5.4.3 |
Passed /home/dlee/tests/5.4.3-1/centos8/root.rpm.1pm.s3.n.s1
Thu Dec 10 11:39:13 CST 2020 Pau......

create table if not exists orders (
s1pm1: o_orderkey int,
s1pm1: o_custkey int,
s1pm1: o_orderstatus char (1),
s1pm1: o_totalprice decimal(12,2),
s1pm1: o_orderdate date,
s1pm1: o_orderpriority char (15),
s1pm1: o_clerk char (15),
s1pm1: o_shippriority int,
s1pm1: o_comment varchar (79)
s1pm1: ) engine=Columnstore
s1pm1: --------------
s1pm1:
s1pm1: Query OK, 0 rows affected (1.996 sec)
s1pm1:
s1pm1: --------------
s1pm1: create table if not exists lineitem (
s1pm1: l_orderkey int,
s1pm1: l_partkey int,
s1pm1: l_suppkey int,
s1pm1: l_linenumber bigint,
s1pm1: l_quantity decimal(12,2),
s1pm1: l_extendedprice decimal(12,2),
s1pm1: l_discount decimal(12,2),
s1pm1: l_tax decimal(12,2),
s1pm1: l_returnflag char (1),
s1pm1: l_linestatus char (1),
s1pm1: l_shipdate date,
s1pm1: l_commitdate date,
s1pm1: l_receiptdate date,
s1pm1: l_shipinstruct char (25),
s1pm1: l_shipmode char (10),
s1pm1: l_comment varchar (44)
s1pm1: ) engine=columnstore
s1pm1: --------------
s1pm1:
s1pm1: Query OK, 0 rows affected (0.198 sec)
s1pm1:
s1pm1: --------------
s1pm1: load data infile "/data/qa/source/dbt3/1
s1pm1: g
s1pm1: /lineitem.tbl" into table lineitem fields terminated by "|"
s1pm1: --------------
s1pm1:
s1pm1: Query OK, 6001215 rows affected (6 min 50.879 sec)
s1pm1: Records: 6001215 Deleted: 0 Skipped: 0 Warnings: 0