The easiest way to run tests with S3 locally is to use MinIO. You can find the tips in the description of MDEV-31031, or follow steps of mtr-s3 tests from buildbot logs (which should be more or less the same). Alternatively, you can try to use the instruction from the KB, but it's possible that it's no longer applicable if the currently available MinIO version changed too much. The buildbot uses an old MinIO release.
Once you have started MinIO, you can put the testcase under suite/s3 and run it normally as ./mtr s3.testname.
This error normally happens upon DML on a table which is already S3.
If we follow the logic that concurrent online alter should work as if ALTER was executed after DML, then we shouldn't have the error, since DML is executed on the table while it's still Aria, and then Aria table is converted into S3 – both are permitted operations.