Daniel Black added a comment - 2020-09-02 23:13 Testing https://github.com/minio/minio features: single binary server for multiple architectures - http://dl.min.io/server/minio/release/ and http://dl.min.io/client/mc/release/ , multiarch container also available (docker hub) minio/minio client allows tracing of server events established and active server test changes diff --git a/mysql-test/suite/s3/my.cnf b/mysql-test/suite/s3/my.cnf index 6f8c27d4d0d..10f98631608 100644 --- a/mysql-test/suite/s3/my.cnf +++ b/mysql-test/suite/s3/my.cnf @@ -2,8 +2,16 @@ !include include/default_client.cnf [mysqld.1] +plugin-maturity = alpha plugin-load-add=ha_s3 s3=ON +s3-host-name = 127.0.0.1 +s3-protocol-version = Auto +s3-bucket = justdoit +s3-access-key = AKIAIOSFODNN7EXAMPLE +s3-secret-key = wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY +s3-region = eu-north-1 + #s3-host-name=s3.amazonaws.com #s3-protocol-version=Amazon #s3-bucket=MariaDB diff --git a/mysql-test/suite/s3/slave.cnf b/mysql-test/suite/s3/slave.cnf index 250a46e6322..7ba2fa8af7a 100644 --- a/mysql-test/suite/s3/slave.cnf +++ b/mysql-test/suite/s3/slave.cnf @@ -1,5 +1,13 @@ [mysqld.2] +plugin-maturity = alpha +plugin-load-add=ha_s3 s3=ON +s3-host-name = 127.0.0.1 +s3-protocol-version = Auto +s3-bucket = replica +s3-access-key = AKIAIOSFODNN7EXAMPLE +s3-secret-key = wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY +s3-region = eu-north-1 #s3-host-name=s3.amazonaws.com #s3-protocol-version=Amazon #s3-bucket=MariaDB Bug 1: tests need plugin-maturity = alpha Bug 2: s3 information needs so be consume in tests as environment variables consistent with other tests libmariadbs3 diff libmarias3   diff --git a/src/marias3.c b/src/marias3.c index 581b181..0ef7618 100644 --- a/src/marias3.c +++ b/src/marias3.c @@ -199,7 +199,7 @@ ms3_st *ms3_init(const char *s3key, const char *s3secret, ms3->curl = curl_easy_init(); ms3->last_error = NULL; - ms3->use_http = false; + ms3->use_http = true; ms3->disable_verification = false; ms3->first_run = true; ms3->path_buffer = ms3_cmalloc(sizeof(char) * 1024); diff --git a/src/request.c b/src/request.c index 425367b..66ec327 100644 --- a/src/request.c +++ b/src/request.c @@ -90,7 +90,7 @@ static uint8_t build_request_uri(CURL *curl, const char *base_domain, if (protocol_version == 1) { - snprintf(uri_buffer, MAX_URI_LENGTH - 1, "%s://%s/%s%s?%s", protocol, + snprintf(uri_buffer, MAX_URI_LENGTH - 1, "%s://%s:9000/%s%s?%s", protocol, domain, bucket, object, query); } @@ -111,7 +111,7 @@ static uint8_t build_request_uri(CURL *curl, const char *base_domain, if (protocol_version == 1) { - snprintf(uri_buffer, MAX_URI_LENGTH - 1, "%s://%s/%s%s", protocol, + snprintf(uri_buffer, MAX_URI_LENGTH - 1, "%s://%s:9000/%s%s", protocol, domain, bucket, object); because of https://github.com/mariadb-corporation/libmarias3/blob/master/src/marias3.c#L177 the hostname cannot contain a port hence the above hack. I also changed to http, however this was later discovered as unnecessary as minio does have a TLS implementation though configuration. Bug 3 allow s3-hostname to contain a port number I ran the container on on port 9000 consistent with the code changes hacked above: podman run -p 9000:9000 -e "MINIO_ACCESS_KEY=AKIAIOSFODNN7EXAMPLE" -e "MINIO_SECRET_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" minio/minio server /data I used web interface to create the buckets 'justdoit' and 'replica'. The command client `mc` can also create buckets. run test suite $ mysql-test/mtr --mem --parallel=5 --suite=s3 --force s3.partition_create_fail 'innodb' [ skipped ] Requires debug build s3.encryption w1 [ pass ] 44 s3.innodb 'innodb' w2 [ pass ] 48 s3.alter2 w4 [ pass ] 102 s3.partition_move 'innodb' w3 [ fail ] Test ended at 2020-09-02 08:29:50CURRENT_TEST: s3.partition_move mysqltest: At line 61: query 'alter table archive exchange partition p1 with table p1' failed: 29: Expected object 's3_test_a5d7151beca211ea8707cabbbeb0fc5c/archive#P#p1/data/000001' didn't exists3.partition w5 [ fail ] Test ended at 2020-09-02 08:29:50CURRENT_TEST: s3.partition mysqltest: At line 26: query 'ALTER TABLE t1 ADD COLUMN c INT' failed: 29: Expected object 's3_test_a5d54f15eca211ea8707cabbbeb0fc5c/t1#P#p0/data/000001' didn't exists3.amazon w4 [ skipped ] Not connected to AWS s3.no_s3 w1 [ fail ] Test ended at 2020-09-02 08:29:50CURRENT_TEST: s3.no_s3 --- /home/dan/repos/mariadb-server-10.5/mysql-test/suite/s3/no_s3.result 2020-03-26 16:13:30.887091628 +1100 +++ /home/dan/repos/mariadb-server-10.5/mysql-test/suite/s3/no_s3.reject 2020-09-02 08:29:50.203725619 +1000 @@ -9,5 +9,3 @@ rename table s3_unique_table to t1; ERROR HY000: Error on rename of './test/s3_unique_table' to './test/t1' (errno: 138 "Unsupported extension used for table")s3.backup w4 [ pass ] 31 s3.select w3 [ pass ] 33 s3.mysqldump w4 [ pass ] 59 s3.unsupported w3 [ pass ] 5 s3.replication_mixed 'mix' w5 [ fail ]Last_Errno 3 Last_Error Error 'Got error from put_object(s3_test_a6372b8feca211ea8810cabbbeb0fc5c/t1/frm): 9 File not found' on query. Default database: 's3_test_a6372b8feca211ea8810cabbbeb0fc5c'. Query: 'alter tabl(other replication tests failed in a similar way)s3.basic w3 [ pass ] 97 s3.discovery w4 [ pass ] 176 s3.arguments w1 [ pass ] 102s3.replication_partition 'innodb,mix' w5 [ fail ] Test ended at 2020-09-02 08:29:51CURRENT_TEST: s3.replication_partition mysqltest: At line 28: query 'ALTER TABLE t1 ADD COLUMN c INT' failed: 29: Expected object 's3_test_a6860a98eca211ea8919cabbbeb0fc5c/t1#P#p0/data/000001' didn't exist2020-09-02 8:29:51 8 [Warning] Slave: Got error from put_object(s3_test_a6860a98eca211ea8919cabbbeb0fc5c/t1#P#p0/aria): 9 File not found Error_code: 3 2020-09-02 8:29:51 8 [Warning] Slave: Error on rename of './s3_test_a6860a98eca211ea8919cabbbeb0fc5c/#sql-alter-1b29fe-8' to './s3_test_a6860a98eca211ea8919cabbbeb0fc5c/t1' (errno: 1 "Operation not permitted") Error_code: 1025 To configure the client to debug the traffic: mc alias set local http://127.0.0.1:9000 AKIAIOSFODNN7EXAMPLE wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY mc admin trace local This is what was used to diagnose the first error as MDEV-23648 , where the object was deleted and then attempted to access. Setting s3_protocol_version to Amazon generated no access denied errors and similar looking test resutls so it appears to support this.

Elena Stepanova added a comment - 2020-09-20 23:17 A separate S3 MTR step has been added to the old buildbot for all deb-, rpm- and zyp- builders. Among them there is a debug build rpm-centos74-amd64-debug , so it's covered as well. The test run first checks whether S3 engine has been built and exits with a warning if S3 wasn't found. If S3 engine has been found, it downloads MinIO server and client from a storage on hasky and runs the test suite. The test step takes 1-3 minutes total. Caveats: 5 tests are currently banned from buildbot as they fail reliably: replication_partition - MDEV-23730 , added to disabled.def replication_mixed, s3.replication_stmt - MDEV-23770 , added to disabled.def partition, partition_move - MDEV-23648 , disabled conditionally for emulators amazon.test never runs with the emulator on obvious reasons partition_create_fail requires debug, so it only runs on the above-mentioned debug builder RPM ppc64 builders currently don't build S3 ( MDEV-23771 ), so the test run gets skipped on them If the suite is run with all tests enabled, the order which MTR chooses to run them (in non-parallel mode) causes MDEV-23769 to occur. It is not anyhow related to S3, but makes result checking more difficult. Obviously it won't be a problem when S3 is fixed and all tests pass. MinIO have dropped support for i386. The builds they offer are from 2018; and they didn't just stop building the binaries, the recent code refuses to build on x86 saying it's not supported. It only affects deb builders, as we don't have any 32-bit RPM or Zypper builders. Due to this, some logic is not available there, for example it doesn't have mc alias command, so mc config host add local ... needs to be used instead. A bigger problem is that these builds seem to be less stable, I've seen some sporadic failures already. I've added some retries in buildbot, it remains to be seen whether they help. If not, we may have to drop 32-bit tests.