[MDEV-23662] S3 Storage Engine should allow to specify port to connect to. Created: 2020-09-03  Updated: 2022-06-05  Resolved: 2020-09-08

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - S3
Fix Version/s: 10.5.7

Type: Task Priority: Major
Reporter: Sergei Petrunia Assignee: Sergei Petrunia
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Relates
relates to MDEV-23650 test S3 in buildbot Closed

 Description   

S3 Storage engine should allow to specify which port to connect to.

This will make it easy to connect to e.g. minio, which is using port 9000 by default (See MDEV-23650 for details)

 Comments   
Comment by Sergei Petrunia [ 2020-09-03 ]

The S3 Storage engine part:
https://github.com/MariaDB/server/commit/cd0e4f28fd45433e98f28933bc3ed103b34b64ac
The libmarias3 part:
https://github.com/mariadb-corporation/libmarias3/pull/91/files

Comment by Sergei Petrunia [ 2020-09-03 ]

danblack would you be able to review this?

Comment by Daniel Black [ 2020-09-04 ]

Done. minor changes pushed to S3 tree.

libmarias3 looks good.

Optional enhancements:

Do you think detecting minio and mc in the path along should start minio from the test suite?

minio server (doesn't auto daemonize)

 
mkdir -p /tmp/shared
 
MINIO_ACCESS_KEY=minio MINIO_SECRET_KEY=minioadmin minio server /tmp/shared/

minio client initialization / cleanup

 
mc  alias set  local  http://127.0.0.1:9000  minio minioadmin
 
mc mb --ignore-existing local/storage-engine
 
...
 
...
 
mc rm --recursive --force  local/storage-engine
 
 mc admin service stop local

Thanks for writing it psergey.

Comment by Sergei Petrunia [ 2020-09-05 ]

Ok, got approval for https://github.com/mariadb-corporation/libmarias3/pull/91 .

Comment by Sergei Petrunia [ 2020-09-05 ]

With the patch, I have these tests fail (both on Minio and S3)

s3.no_s3                                 [ disabled ]  Other failure
 
s3.partition                             [ disabled ]  Other failure
 
s3.partition_move                        [ disabled ]  Other failure
 
s3.replication_mixed                     [ disabled ]  fails
 
s3.replication_partition                 [ disabled ]  Crashes
 
s3.replication_stmt                      [ disabled ]  fails

Comment by Sergei Petrunia [ 2020-09-05 ]

bb-10.5-mdev23662

Comment by Sergei Petrunia [ 2020-09-08 ]

danblack, re starting Minio - I think starting/stopping minio is a good next step. We could also add logic to detect if it's present and/or get AWS credentials from the environment (or somewhere else?). Need to discuss this with those who run the CI systems.
I think it's better to deal with this in a separate MDEV entry.

Comment by Sergei Petrunia [ 2020-09-08 ]

Pushed what we've had into 10.5

Generated at Thu Feb 08 09:24:06 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.