[MXS-475] Repeatability of test scenario Created: 2015-11-16  Updated: 2017-12-04  Resolved: 2016-03-23

Status: Closed
Project: MariaDB MaxScale
Component/s: test
Affects Version/s: None
Fix Version/s: 1.4.1

Type: Task Priority: Major
Reporter: Johan Wikman Assignee: Timofey Turenko
Resolution: Done Votes: 0
Labels: None


 Description   

Definition: Here is with a maintenance release meant a release that fixes a number (may be 1) specific issues in a previous release.

In order to lower the threshold to make such a release, we need to be able to reproduce as exactly as possible the test-suite and test environment that was used when the immediately preceding previous release was made. This in order to ensure that we do not find issues that have been revealed due to, for instance, new and improved test-cases.

When making a maintenance release we are only interested in it being better than the previous release, not that it fixes all known problems at the point the maintenance release is made, as that would make the threshold much higher and effectively turn the maintenance release into a regular release.

The test suite used for testing the new maintenance release should be the very same test suite used for testing the previous release, plus test cases for the specific issues the maintenance release is supposed to address.

One approach would be to tag the test-suite using the same tags that are used for tagging the actual Maxscale release.



 Comments   
Comment by Johan Wikman [ 2016-03-23 ]

The maxscale-system-test repository is now tagged with the same tag as the MaxScale repository.

Generated at Thu Feb 08 03:59:33 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.