Since this report itself turned out to be a feature request, I will put my suggestions here.
1. Durability
Overlays should sustain any level of nesting, as long as it is supported in MTR itself. By "sustain" I mean that MTR should not die running an internal suite (with subsuites) due to the presence of its overlays, full or partial. Lets say we have in MTR
mysql-test/suite/engine/test1.test
mysql-test/suite/engine/basic/test2.test
mysql-test/suite/engine/basic/types/test3.test
Now we can run
perl ./mtr --suite=engine
perl ./mtr --suite=engine/basic
perl ./mtr --suite=engine/basic/types
We should still be able to do so, no matter whether an engine or plugin has any of
mysql-test/suite/engine
mysql-test/suite/engine/basic
mysql-test/suite/engine/basic/types
folders.
Now it is not so – if I have storage/my_engine/mysql-test/engine/basic folder and try to run MTR as
perl ./mtr --suite=engine/basic
(even without the dash), it dies.
2. Supported level of nesting
At the same time, I don't think it's strictly necessary to actually support the same level of nesting in overlays as it is done in MTR. It is somewhat nice to have, mainly to avoid confusion, but if there is any problem implementing it, I believe that 2 levels (parent and child suite) should be enough for the most real-life purposes.
3. Syntax
I do find it important whether nested overlays should be triggered as
--suite=engine-my_engine/basic or
--suite=engine/basic-my_engine
Whichever is easier to implement should be fine.
It would be good to avoid making a user launch it as
--suite=engine-my_engine/basic-my_engine
as it seems redundant (especially if we decide to support more than 2 levels), but it is not critical either.
4. Add-ons
I previously suggested to make MTR execute all subsuites of 'engine' if it is run as
--suite=engine
but thinking about it, it is probably not a good idea, since the 'engine' suite can have tests of its own, and if we implement such a feature, it will be impossible to execute only tests from the 'engine' suite, without subsuites.
A nice-to-have (but low priority) feature could be to improve currently half-supported syntax --suite=engine/*, to print correct test full names in the output, and not to die on middle-level expansions.
To exend the example above, now, if we have
mysql-test/suite/engine/test1.test
mysql-test/suite/engine/basic/test2.test
mysql-test/suite/engine/basic/types/test3.test
executing mtr --suite=engine/* would run test2, and mtr --suite=engine/basic/* would run test3. But it's a corner case only that works – if there are more than one subsuite on either level, MTR would die, e.g. if we have
mysql-test/suite/engine/test1.test
mysql-test/suite/engine/basic/test2.test
mysql-test/suite/engine/basic/types/test3.test
mysql-test/suite/engine/basic/indexes/test4.test
then mtr --suite=engine/* would still run test2, but mtr --suite=engine/basic/* would die.
More consistent behavior would be nice: -suite=suite1/* to execute all subsuites of suite1 (but not their subsuites), and --suite=suite1-my_engine/* to work accordingly, as well as --suite=suite1/*.
Also, for the same nested suites, the following works in MySQL, but makes MariaDB version die at the same line 383 (regardless whether I have overlays for engines subsuites or not):
perl ./mtr --suite=engines/*
It is not perfect in MySQL either, as it produces dirty output like
engines/*.db_create_drop [ pass ] 24
(see the asterisk instead of the subsuite name). Still, dying is not good.
Otherwise, I am going to create a separate feature request for providing a convenient way to launch nested subsuites.