[MCOL-4989] testS3Connection missing other checks Created: 2022-02-07  Updated: 2022-06-27  Resolved: 2022-06-07

Status: Closed
Project: MariaDB ColumnStore
Component/s: None
Affects Version/s: None
Fix Version/s: 6.4.1

Type: Bug Priority: Minor
Reporter: Allen Herrera Assignee: Daniel Lee (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Sprint: 2021-17

 Description   

Video in comments.

I was able to trace down to "s3:ListBucket" not being checked by testS3Connection utility. meaning without "s3:ListBucket" permission testS3Connection can say everything is ok but a problem still exists when starting up columnstore via the debug.log you see 

S3Storage::exists(): failed to HEAD, got 'Authentication failed'.

Expected:
When using testS3Connection and it says OK. that the cluster can startup fine and store data in S3.

Actual:
Not having s3:ListBucket, you cant use S3 storage for columnstore

Minimum S3 permission needed for access key/secret S3 auth determined:

                "s3:PutObject",
 
                "s3:PutObjectAcl",
 
                "s3:GetObject",
 
                "s3:GetObjectAcl",
 
                "s3:DeleteObject",
 
                "s3:ListBucket"



 Comments   
Comment by alexey vorovich (Inactive) [ 2022-06-02 ]

ben.thompson dleeyh Test instruction should be in written from in Jira , please

Comment by Daniel Lee (Inactive) [ 2022-06-07 ]

Build verified: 6.4.1-1 (#4590)

testS3Connection now checks for the ListBucket permission.

[rocky8:root~]# testS3Connection
 
StorageManager[6417]: Using the config file found at /etc/columnstore/storagemanager.cnf
 
StorageManager[6417]: S3Storage::exists(): failed to HEAD, got 'Authentication failed'. bucket = ben-test-s3, key = 7458fc4f-b9e7-4d90-a491-f0eeeb319ad5connectivity_test.
 
StorageManager[6417]: S3Storage::exists() failed on nonexistent object. Check 'ListBucket' permissions.
 
StorageManager[6417]: S3Storage: failed to HEAD, check log files for specific error
 
S3 Storage Manager Configuration Error:
 
S3Storage: failed to HEAD, check log files for specific error

When starting ColumnStore without ListBucket, it StorageManager server now failed. DBRM is in read-only state. User attention is needed.

crit.log

Jun 7 20:52:23 rocky8 StorageManager[6417]: S3Storage::exists() failed on nonexistent object. Check 'ListBucket' permissions.
 
Jun 7 20:53:12 rocky8 StorageManager[6486]: S3Storage::exists() failed on nonexistent object. Check 'ListBucket' permissions.

systemctl status for mariadb-columnstore and Macs-storagemanager would also show failed status.

Generated at Thu Feb 08 02:54:31 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.