Pattern of test

I want to share summary of Selenium Design Patterns and Best Practices book in Chapter 2 The Spaghetti Pattern & Chapter 3 Refactoring Test . The Spaghetti Pattern, The Chain Linked Pattern & The Big Ball Of Mud Pattern in Chapter 2. The DRY testing pattern, the Hermetic Test Pattern, The Randon Run Order Principle in Chapter 3.

1. The Spaghetti pattern

The Spaghetti pattern is characterized by lack of perceived architecture and design. Tests in this pattern not only depend on the execution order of all the tests, but also tend to over-share internal private components with each other. Furthermore, variables in the test suite are shared on a global level, allowing individual tests too much control over the whole test suite. The Spaghetti pattern is a close relative to the Chain Linked pattern and the Big Ball of Mud pattern.

Advantages of the Spaghetti pattern

- Quick start: Record and Playback to record one long test session and split it up into smaller chunks.

- Smaller code base: As a result, each individual test is smaller in code size, and we do not need to have unique test data for each test.

- Smoke tests: smoke tests need to be fast, brief, and leave as small a test data footprint as possible.

Disadvantages of the Spaghetti pattern

- Anti-pattern: context of test automation, this leads to long-term maintainability problems. Thus, it is considered an anti-pattern.

- Tight coupling: The more tightly integrated individual parts of the application are, the more indistinguishable they are from each other.

- No random order: Dependence on the strict order of execution leads to the inability to run our tests in a random order.

- No parallel test runs: Having each test depend on the execution order of the whole suite will prevent us from accomplishing this task.

- Covers up failures: A failure in the beginning of the test suite can prevent the execution of the entire test suite.

- No resilience: Certain tests are not able to fulfill their target goals.

2. The Chain Linked pattern

The Chain Linked pattern is an improvement on the Spaghetti pattern. Each link in the chain is an individual test and is an entity on its own. Even though each test is self contained and does not share too much with its neighbors, it still relies on a rigid order of execution. Most tests in this pattern rely on previous tests to set up the environment to be just right. This pattern is a huge improvement on the Spaghetti pattern in its long-term maintainability.

3. The Big Ball of Mud pattern

Big Ball of Mud does not have any formal structures that will allow a distinction between any individual components. Test data and results are promiscuously shared amongst most distant and unrelated components until everything is global and mutable without warning. Unintentional test failures occur when a component is changed for a new test without the realization that hundreds of other tests depend on it.In comparison to the Spaghetti pattern, this state of affairs is in much more dire need of repair.

4. The DRY testing pattern

One of these principles is the Don’t Repeat Yourself (DRY) principle; the most basic idea behind the DRY principle is to reduce long-term maintenance costs by removing all unnecessary duplication.

Advantages of the DRY testing pattern

• Modular tests: Tests and test implementations are self-sufficient. Any test can run in any order.
• Reduced duplication: All actions such as filling out a form are neatly kept in a single place instead of having multiple copies peppered all over the suite.
• Fast updates: Having unique actions in a single place makes it easy to update the tests to mimic new growth of the application.
• No junk code: Constant upkeep of the test suite, with deletion of duplicates, prevents the test suite from having code that is no longer used.

Disadvantages of the DRY testing pattern

• Complicated project structure: Some test actions will be logically grouped with other similar actions.
• Lack of a good IDE: There aren’t many good IDEs that will notify the test developer if a test action has already been implemented.

5. The Hermetic test pattern

The Hermetic test pattern is the polar opposite of the Spaghetti pattern; it states that each test should be completely independent and self-sufficient.

Advantages of the Hermetic test pattern

• Clean start: Each test has a cleaned up environment to work in. This prevents accidental test pollution from previous tests, such as a created user that should not be present.
• Resilience: Each test is responsible for its own environment, so it has no need for everything to go perfectly right somewhere else in the suite.
• Modular: Each test is standalone and can be rearranged into smaller test suites such as a smoke suite, or can run as a standalone test.
• Random run order practice: Since tests do not depend on successful completion of any other test in the suite, they can be executed in any order.
• Parallel testing: If our test suite can run in any random execution order, it can also be executed in parallel.

Disadvantages of the Hermetic test pattern

• Upfront design: Each test needs to be designed to be self-sufficient. Each test can reuse methods used by other tests, but cannot reuse data and test results generated by other tests.
• Runtime increase: Since each test has to set up the environment before it starts, the runtime of each individual test is increased.
• Resource usage increase: Increased runtime of individual tests means that the test suite will need more resources, such as RAM, to run tests in parallel.

6. The random run order principle

Random run order is more of a principle than a pattern. It applies to the test execution. This execution is usually performed on a Continuous Integration (CI) environment. The random run order principle states that the order of the test suite execution should be randomized every time the suite is executed.

Advantages of the random run order principle

· Prevents test interdependence: Any test that depends on another test to set up the environment will be exposed rather quickly. This forces us to maintain good Hermetic integrity within each test.

· Flushes out data pollution: Sometimes test flakiness does not come from the setup stage of the test. Often we will find a test that intentionally puts the test environment into a bad state on purpose, to test the application resilience.

· Built in: Some test frameworks not only support test randomization out of the box, but also have the random run order as a default setting.

Disadvantages of the random run order principle

· A lot of refactoring: If we have a large and mature test suite with copious usage of the Spaghetti pattern and the Big Ball of Mud pattern, making tests compatible with a random run order is tremendous amount of work.

· Random run audits: If the test run is completely random for every single build, it can be difficult to know the sequence of tests that causes the instability. One way to solve this difficulty is to have an audit trail for each build that will allow us to know the exact sequence of tests that led us to failure.

· Team frustration: Running tests in a random order can create a lot of frustration and resentment for the whole team.

· No built-in support: A lot of test frameworks do not support test randomization. Implementing this functionality might be very difficult.

Refference

Kovalenko, Dima.(2014).Selenium Design Patterns and Best Practices.UK:Packt Publishing