Automation Testing Methodology
The framework describes how test data will be handled, script naming conventions, test script creation standards, how scripts will be organized, and more.
Automation Framework = Guidelines for automation testing
Why Use a Framework?
- Facilitates streamlined, efficient automation testing
- Makes testers more productive
- Aids in QA organization and efficiency
- Increases speed and accuracy
- Adaptable for new applications
- Offers high level summaries for managers
QA Mentor builds frameworks by taking many client-specific factors into consideration, such as:
- What is the maturity level of the QA process?
- What’s the expected time to market for the project?
- What are the budget constraints?
- What kind of skills and experience do the test engineers have?
- What are the automation objectives?
- What is the size of the project?
- How many applications or systems are involved?
- How stable is the application?
Types of Frameworks
A modular framework breaks down full processes into individual sections and creates independent scripts for each. These modules are then combined hierarchically as needed to create full business scenarios. The same module may be used in multiple scenarios, or even all of them. This type of framework is ideal for large, stable applications.
Similar to the modular framework, this one divides common actions into functions or procedures. Instead of creating independent scripts for each, libraries are created that represent these sections, which are then called from the scripts. Ideal for small to medium applications that are considered stable, the test library framework does require a little more technical knowledge to create and maintain.
Also called table-driven, keyword-driven frameworks are completely application-independent. Keywords are essentially actions that can be performed on a component. Data tables are created with these keywords, along with step-by-step instructions. The data tables look remarkably like manual test scripts, but the keywords actually trigger events in the application. This framework is scalable and highly reusable, ideal for small applications.
In data-driven frameworks, all of the logic is in the test scripts but the data used is kept in external files. As the tests are executed, input and output values are read from associated data files and loaded into the variables in the scripts. Using this method a single script can be executed with multiple sets of data, reducing the need for a larger number of scripts. Less code is written and the tests are flexible, albeit very similar. This type of framework is suitable for any application, including those that are still under development and considered unstable.
The most commonly used framework is the hybrid. Historically, it’s a combination of keyword and data-driven, but realistically it can be a combination of any or all of the previously mentioned frameworks. The hybrid uses the best of each to create a more ideal framework, but does increase the complexity. This framework is ideal for medium to large applications.
Components of a Well Designed Framework
- Environment and application variables and constants – used in the test scripts
- Configuration files – Access controls and global settings are stored here
- Object repository – Objects for the GUI validation testing are contained here
- Test Data – Application specific data is crucial to all tests
- Recovery scenarios- These handle unexpected errors that cause scripts to stop running and help restart test execution
- Clean up scripts – These help to return application back to a base state
- Generic – usable across applications
- Business – specific to an application
- Extended – specific to an application
Reporting and Logging
- Test reports – Reports the steps completed, data used, and outcome
- Error and exception logs – These identify problems with the script, data, or environment
- Defect reports – Reports of any conditions that did not meet expectations
- Debug logs – This information is crucial to fixing defects quickly
The actual application used for the automation.