www.weisserth.net

Nondeterministic tests are a common problem in test setups, especially with integration tests. For example, imagine you are doing an integration test of two components that exchange messages where receiving a message in one of the systems changes its internal state which you want to verify. However, the message exchange can be delayed due to external factors (network latency, load etc.). When running tests (for example using the excellent Cucumber framework), testing steps are usually run sequentially. It may happen that the tested system is lagging behind the testing steps, failing your test in some (but not all) cases. You need to make your tests more tolerant for these situations. Often, that means retrying for a couple of times.