USA Toll Free 1-800-622-2602 | Main Office 1-212-960-3812
Automation Testing Methodology
The framework describes how test data will be handled, script naming conventions, test script creation standards, how scripts will be organized, and more.
Automation Framework = Guidelines for automation testing
Request More Information
Automation Framework Describes
- How test data will be handled
- Script naming conventions
- Test script creation standards
- Script organization
Why Use a Framework?
- Facilitates streamlined, efficient automation testing
- Makes testers more productive
- Aids in QA organization and efficiency
- Increases speed and accuracy
- Adaptable for new applications
- Offers high level summaries for managers
QA Mentor builds frameworks by taking many client-specific factors into consideration, such as:
- What is the maturity level of the QA process?
- What’s the expected time to market for the project?
- What are the budget constraints?
- What kind of skills and experience do the test engineers have?
- What are the automation objectives?
- What is the size of the project?
- How many applications or systems are involved?
- How stable is the application?
Types of Frameworks
Modular
A modular framework breaks down full processes into individual sections and creates independent scripts for each. These modules are then combined hierarchically as needed to create full business scenarios. The same module may be used in multiple scenarios, or even all of them. This type of framework is ideal for large, stable applications.
Functional Library
Similar to the modular framework, this one divides common actions into functions or procedures. Instead of creating independent scripts for each, libraries are created that represent these sections, which are then called from the scripts. Ideal for small to medium applications that are considered stable, the test library framework does require a little more technical knowledge to create and maintain.
Keyword-driven
Also called table-driven, keyword-driven frameworks are completely application-independent. Keywords are essentially actions that can be performed on a component. Data tables are created with these keywords, along with step-by-step instructions. The data tables look remarkably like manual test scripts, but the keywords actually trigger events in the application. This framework is scalable and highly reusable, ideal for small applications.
Data-driven
In data-driven frameworks, all of the logic is in the test scripts but the data used is kept in external files. As the tests are executed, input and output values are read from associated data files and loaded into the variables in the scripts. Using this method a single script can be executed with multiple sets of data, reducing the need for a larger number of scripts. Less code is written and the tests are flexible, albeit very similar. This type of framework is suitable for any application, including those that are still under development and considered unstable.
Hybrid
The most commonly used framework is the hybrid. Historically, it’s a combination of keyword and data-driven, but realistically it can be a combination of any or all of the previously mentioned frameworks. The hybrid uses the best of each to create a more ideal framework, but does increase the complexity. This framework is ideal for medium to large applications.
Components of a Well Designed Framework
Resources
- Environment and application variables and constants – used in the test scripts
- Configuration files – Access controls and global settings are stored here
- Object repository – Objects for the GUI validation testing are contained here
- Test Data – Application specific data is crucial to all tests
- Recovery scenarios- These handle unexpected errors that cause scripts to stop running and help restart test execution
- Clean up scripts – These help to return application back to a base state
Driver Script
Function Library
- Generic – usable across applications
- Business – specific to an application
- Extended – specific to an application
Reporting and Logging
- Test reports – Reports the steps completed, data used, and outcome
- Error and exception logs – These identify problems with the script, data, or environment
- Defect reports – Reports of any conditions that did not meet expectations
- Debug logs – This information is crucial to fixing defects quickly
Automation Tool
The actual application used for the automation.
Checklist for Success
Completing the plan of attack is critical to understanding the goals of the application and the automation effort.
Understanding the percentage of the total test cases to be automated will help determine a suitable framework, number of resources, and timeline.
Test scripts should have minimal scripting involved, making it easier for QA teams to understand and visualize each step of the scenario.
Data-driven tests will increase coverage of both the application and data specific scenarios, as well as increase the re-usability of specific scripts.
Whether the verification is at the front-end or the database, confirmation of accuracy and the integrity of the data should be a high priority.
By structuring the scripts with minimal dependencies, unattended execution becomes much easier.
