Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-35633

Setting s3_protocol_version on MariaDB Server 10.5 has no effect

Details

    Description

      MariaDB Server 10.5 is using a buggy version of libmarias3 (3846890) * in which the value of *MS3_OPT_FORCE_PROTOCOL_VERSION is overwriting the wrong variable hence setting s3_protocol_version won't have any effect.

      When setting s3_host_name to an actual hostname instead of an IP, protocol_version in libmarias3 will default to 2, which is not the case with every S3 provider resulting in wrong request headers.
      The error one will encounter is "Couldn't resolve host name" when the server will send requests to the S3 provider.

      This bug was solved in libmarias3 -> 289108f.

      $ git diff 3846890 289108f
      		 
      diff --git a/src/marias3.c b/src/marias3.c
      		 
      index 74d7233..7a07bdd 100644
      		 
      --- a/src/marias3.c
      		 
      +++ b/src/marias3.c
      		 
      @@ -617,7 +617,7 @@ uint8_t ms3_set_option(ms3_st *ms3, ms3_set_option_t option, void *value)
      		 
               return MS3_ERR_PARAMETER;
      		 
             }
      		 
       
      -      ms3->list_version = protocol_version;
      		 
      +      ms3->protocol_version = protocol_version;
      		 
             break;
      		 
           }

      Steps to reproduce the issue on 10.5

      docker run -it --rm quay.io/mariadb-foundation/bb-worker:ubuntu20.04 bash
      		 
      clone MariaDB Server -> branch==10.5
      		 
       
      		 
      sh -c 'cmake . -DCMAKE_BUILD_TYPE=Debug -DCMAKE_C_COMPILER_LAUNCHER=ccache -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER_LAUNCHER=ccache -DCMAKE_CXX_COMPILER=g++ -DPLUGIN_TOKUDB=NO -DPLUGIN_MROONGA=NO -DPLUGIN_SPIDER=YES -DPLUGIN_OQGRAPH=NO -DPLUGIN_PERFSCHEMA=YES -DPLUGIN_SPHINX=NO -DWITH_DBUG_TRACE=OFF -DEXTRA_DEBUG=ON && make -j24'
      		 
       
      		 
      export S3_PROTOCOL_VERSION=Original
      		 
      export S3_BUCKET=amd64-ubuntu-2004-35971
      		 
      export S3_HOST_NAME=minio.mariadb.org
      		 
      export S3_PORT=443
      		 
      export S3_USE_HTTP=OFF
      		 
      export S3_ACCESS_KEY=#access_key#
      		 
      export S3_SECRET_KEY=#secret_key#
      		 
       
      		 
      sh -c '
      		 
                 cd mysql-test &&
      		 
                 exec perl mysql-test-run.pl --verbose-restart --force --retry=3 --max-save-core=2 --max-save-datadir=10 --parallel=$(expr 7 \* 2) --suite=s3;
      		 
                 '
      		 
      Every test yields "Couldn't resolve host name".
      		 
       
      		 
      CURRENT_TEST: s3.basic
      		 
      mysqltest: At line 18: query 'alter table t1 engine=s3' failed: 3: Got error from put_object(s3_test_ad103c5bb88311efa5520242ac110002/t1/frm): 5 Couldn't resolve host name

      Attachments

        Activity

          rvarzaru Varzaru Razvan-Liviu added a comment - - edited

          monty

          My first attempt was to only update the submodule for libmarias3 to the earliest commit in time where this bug is resolved,
          which is 289108f9f296d0ce5fe8ea5c13ec4bc3edc24dcc (Fix MS3_OPT_FORCE_PROTOCOL_VERSION)
          and this is what you see in the pull request at the moment.

          What I don't like about this approach is that libmarias3 version bump in my PR still doesn't match the one in 10.6 onwards, potentially causing merge conflicts and if someone doesn't pay attention to it, causing a downgrade in 10.6, 10.11 and so on.

          What I tried locally:
          -> backported your commit 4bd94afbd3cecdcca9edb87834b41de5494fbbea (MDEV-32884 Make s3_debug dynamic)

          • this will bump libmarias3 to a81724ab07bd28e16bf431419c24b6362d5894fc
          • this causes the s3.debug test to fail
          • the above bug was solved in bc5e904043d4c50fb0cc0cbdce4149866bf4e6e0 (MDEV-32884 Improve S3 options comaptibility)
            -> backported bc5e904043d4c50fb0cc0cbdce4149866bf4e6e0 also and now the S3 tests PASS.

          My concern is that bc5e904043d4c50fb0cc0cbdce4149866bf4e6e0 changes the expectations for the user in 10.5,
          meaning Legacy, Path and Domain options were introduced and Original now means v2 protocol.

          Do you have any advice what would be the best approach to solve the bug in 10.5 with least changes?

          rvarzaru Varzaru Razvan-Liviu added a comment - - edited monty My first attempt was to only update the submodule for libmarias3 to the earliest commit in time where this bug is resolved, which is 289108f9f296d0ce5fe8ea5c13ec4bc3edc24dcc (Fix MS3_OPT_FORCE_PROTOCOL_VERSION) and this is what you see in the pull request at the moment. What I don't like about this approach is that libmarias3 version bump in my PR still doesn't match the one in 10.6 onwards , potentially causing merge conflicts and if someone doesn't pay attention to it, causing a downgrade in 10.6, 10.11 and so on. What I tried locally: -> backported your commit 4bd94afbd3cecdcca9edb87834b41de5494fbbea ( MDEV-32884 Make s3_debug dynamic) this will bump libmarias3 to a81724ab07bd28e16bf431419c24b6362d5894fc this causes the s3.debug test to fail the above bug was solved in bc5e904043d4c50fb0cc0cbdce4149866bf4e6e0 ( MDEV-32884 Improve S3 options comaptibility) -> backported bc5e904043d4c50fb0cc0cbdce4149866bf4e6e0 also and now the S3 tests PASS. My concern is that bc5e904043d4c50fb0cc0cbdce4149866bf4e6e0 changes the expectations for the user in 10.5, meaning Legacy, Path and Domain options were introduced and Original now means v2 protocol. Do you have any advice what would be the best approach to solve the bug in 10.5 with least changes?

          People

            monty Michael Widenius
            rvarzaru Varzaru Razvan-Liviu
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.