Rough ideas by AV:

1. support multinode tests that include DDL/DML commands directed at various multiple nodes
2. support mulitnode tests where DML commands are interspersed with various OS level events such as host reboot/docker restart
3. support at least 3 deployment mode (on prem(xbench), docker-compose , k8s (sky)
4. chose the language to write individual tests ( sql script, shell , python, other ??) . The test developer should have abstractions that work for all deployment modes. For example " connect to PM2 " or " restart PM2" . Test developer should not have to code if/then/else to support deployment modes.
5. tests should collect all artifacts such as logs ad config files from each node , at least all files from /var/log/mariadb/columnstore and /etc/columnstore
6. any crash info should be collected as well
7. There should be some meta language (set of yaml ??) that define which tests run in each deployment mode. we may chose to run a subset for some deployment to save time