Select Page

Automation Testing Methodology

The framework describes how test data will be handled, script naming conventions, test script creation standards, how scripts will be organized, and more.

Industry Awards

Global Locations

Certified QA Professionals

Happy Clients

Completed Projects

Test automation framework is an application that allows you to write a series of tests without worrying about the constraints or limitations of the underlying test tools. Regardless of the automation tool, you would be able to achieve your automation desires and objectives. Our automation expert building framework to include the following:

• Common codebase and database
• Exception and error handling
• Modularized reusable, and maintainable code and data
• Standardized test idioms
• Configurable test suites
• Global test properties

One of the main advantages of our proprietary framework is

• Optimization of testing efforts through unattended execution
• Customized reporting and Test evidence recording
• Multi-environment run with minimal configuration changes
• Metrics data collation and analysis
• Comprehensive documentation facilitating quick learning and easy maintenance

Automation Framework = Guidelines for automation testing

Request More Information

Test automation has undergone several stages of evolution, both in the development of marketable test tool technologies and in the development of test automation processes and frameworks within individual QA organizations. The typical path followed is described below:

Record and Playback: monitoring of an active user session, recording user inputs related to objects encountered in the user interface, and storing all steps and input data in a procedural script. This method is useful in learning how to use a test tool, but the scripts produced are difficult to maintain after the application under test changes and do not produce reliable, consistent results.

Test Script Modularity: creating small, independent scripts that represent modules, sections, and functions of the application-under-test; then combining them in a hierarchical fashion to construct larger tests. This represents the first step toward creating reusable test assets.

Test Library Architecture: dividing the application under test into procedures and functions – also known as objects and methods depending on your implementation language – instead of a series of unique scripts. This requires the creation of library files that represent modules, sections, and functions of the application under test. These files, often referred to as function libraries, are then called directly from within test case scripts. Thus, as elements of the application change, only the common library components which reference them must be changed, not multiple test scripts with hard-coded references which might be difficult to locate and validate.

Data-Driven Testing: reading input and output values from data files or tables into the variables used in recorded or manually coded test scripts. These scripts include navigation through the application and logging of test status. This abstraction of data from the test script logic allows testers with limited knowledge of the test tool to focus on developing, executing and maintaining larger and more complex sets of test data. This increase in organizational efficiency fosters enhanced test coverage with shorter test cycles.

Keyword-Driven Testing: including test step functionality in the data driven process by using data tables and keywords to trigger test events. Test steps are expressed as Object → Action →Expected Result. The difference between data-driven and keyword-driven testing is that each line of data in a keyword script includes a reference that tells the framework what to do with the test data on that line. The keyword attached to the test step generally maps to a call to a library function using parameters read in from the data file or table. One major benefit is the improved maintainability of the test scripts: by fully modularizing automation of each step, it’s easier to accommodate any user interface changes in the application under test.

Automation Framework Describes

How test data will be handled

Script naming conventions

Test script creation standards

Script organization

As noted earlier, one of the challenges facing test automation is to speed up testing processes while increasing the accuracy and completeness of tests. The evolution of test automation frameworks has been driven by accepting this challenge.

Data-Driven Frameworks

This type of functional test automation framework abstracts the data layer from the test script logic. Ideally, only data used as inputs to test objects and outputs from test events would need to change from one iteration to the next. The types of test scripts used in this architecture are described below.

Driver Script

• Performs initialization of the test environment (as required)
• Calls each Test Case Script in the order specified by the Test Plan
• Controls the flow of test set execution

Test Case Script

• Executes application test case logic using Business Component Function scripts
• Loads test data inputs (function parameters) from data files and tables
• Evaluates actual result based on the expected result loaded from data files and tables

Business Component Function Script

• Exercises specific business process functions within an application
• Issues a return code to indicate result or exception
• Uses parameter (input) data derived from data files and tables

Common Subroutine Function Script

• Performs application specific tasks required by two or more business component functions
• Issues a return code to indicate result or exception
• Uses parameter (input) data derived from data files and tables

User-Defined Function Script

• Contains logic for generic, application-specific, and screen-access functions
• Can include code for test environment initialization, debugging and results logging

In this architectural model, the “Business Component” and “Common Subroutine” function scripts invoke “User Defined Functions” to perform navigation. The “Test Case” script would call these two scripts, and the “Driver” script would call this “Test Case” script the number of times required to execute Test Cases of this kind. In each case, the only change between iterations is in the data contained in the files that are read and processed by the “Business Function” and “Subroutine” scripts.

Keyword-driven Frameworks

This type of framework builds on the data-driven framework by including business component functionality in the data tables. Keywords are used within each test step to trigger specific actions performed on application objects.

Driver Script

• Governs text execution workflow
• Performs initialization of the test environment (as required)
• Calls the application-specific Action (“Controller”) Script, passing to it the names of the business process test cases. These test cases can be stored in spreadsheets, delimited text files, or database records.

Action Script

• Acts as the “controller” for test case execution
• Reads and processes the business process test case name received from the Driver Script
• Matches on “key words” contained in the input dataset
• Builds a list of parameters from values included with the test data record
• Calls “Utility” scripts associated with the “key words”, passing the created list of parameters

Utility Scripts

• Process the list of input parameter received from the Action Script
• Perform specific tasks (e.g. press a key or button, enter data, verify data, etc.), calling “User Defined Functions” as required
• Record any errors encountered during test case execution to a Test Report (e.g. data sheet, table, test tool UI, etc.)
• Return to the Action Script, passing a result code for processing status (e.g. pass, fail, incomplete, error)

User-Defined Function Libraries

• Contain code for general and application-specific functions
• May be called by any of the above script-types in order to perform specific tasks
• Can contain business rules

 Mercury Quality Center with TestDirector and QuickTest Professional

Business Process Testing uses a role-based model, allowing collaboration between non-technical Subject Matter Experts and QA Engineers versed in QuickTest Pro.

Why Use a Framework?

Facilitates streamlined, efficient automation testing

Makes testers more productive

Aids in QA organization and efficiency

Increases speed and accuracy

Adaptable for new applications

Offers high level summaries for managers

Business process tests are composed of business components. The information in the business component’s outer layer — the description, status, and implementation requirements, together with the steps that make up the component — is defined in Quality Center by the SME, who then runs and analyzes the associated tests and test sets. A QuickTest Engineer populates a shared repository with the different objects in the application being tested and encapsulates all activities and scripted steps into operations, essentially using function libraries in a keyword based automation framework.

When QuickTest Professional is connected to a Quality Center project with Business Process Testing support, the objects defined by the QuickTest Engineer in the object repository are available for use by the SME. In addition, all the business component information is visible in QuickTest. This integration and visibility between the two applications enables the SME to implement the testing steps for the business components that are defined in business process tests, and also enables the QuickTest Engineer to effectively maintain the set of objects in the object repository and the operations in the function libraries.

In addition to creating and maintaining the object repository, the QuickTest Engineer defines a set of elements that comprise an Application Area, created in QuickTest Professional and containing all of the settings and resources required to create the content of a business component. These include all the objects from the application under test contained in the shared object repository, and the user-defined operations contained in function library files.

Each business component can be associated with a specific Application Area, or can share an Application Area with other components. Application area settings are automatically inherited by the business components that are based on that application area. An application area includes:

Resources: resource settings include associated library files and the shared object repository.

Add-Ins: the add-ins associated with the first business component in a business process test (inherited from the application area used by the component) are automatically loaded in QuickTest Professional when Quality Center runs the test. Quality Center assumes that add-ins are required for all the business components in the same business process test.

Windows-Based Applications: if you are creating a business component to test a Windows-based application, you must specify the application on which the business component can run. Other environments are supported by the appropriate QuickTest Add-In.

Recovery Scenarios: activated during execution of a business component test when an unexpected event occurs.

QA Mentor builds frameworks by taking many client-specific factors into consideration, such as:

  1. What is the maturity level of the QA process?
  2. What’s the expected time to market for the project?
  3. What are the budget constraints?
  4. What kind of skills and experience do the test engineers have?
  5. What are the automation objectives?
  6. What is the size of the project?
  7. How many applications or systems are involved?
  8. How stable is the application?

QA Mentor Automation Methodology

Types of Frameworks

Modular

A modular framework breaks down full processes into individual sections and creates independent scripts for each. These modules are then combined hierarchically as needed to create full business scenarios. The same module may be used in multiple scenarios, or even all of them. This type of framework is ideal for large, stable applications.

Keyword-driven

Also called table-driven, keyword-driven frameworks are completely application-independent. Keywords are essentially actions that can be performed on a component. Data tables are created with these keywords, along with step-by-step instructions. The data tables look remarkably like manual test scripts, but the keywords actually trigger events in the application. This framework is scalable and highly reusable, ideal for small applications.

Hybrid

The most commonly used framework is the hybrid. Historically, it’s a combination of keyword and data-driven, but realistically it can be a combination of any or all of the previously mentioned frameworks. The hybrid uses the best of each to create a more ideal framework, but does increase the complexity. This framework is ideal for medium to large applications.

Functional Library

Similar to the modular framework, this one divides common actions into functions or procedures. Instead of creating independent scripts for each, libraries are created that represent these sections, which are then called from the scripts. Ideal for small to medium applications that are considered stable, the test library framework does require a little more technical knowledge to create and maintain.

Data-driven

In data-driven frameworks, all of the logic is in the test scripts but the data used is kept in external files. As the tests are executed, input and output values are read from associated data files and loaded into the variables in the scripts. Using this method a single script can be executed with multiple sets of data, reducing the need for a larger number of scripts. Less code is written and the tests are flexible, albeit very similar. This type of framework is suitable for any application, including those that are still under development and considered unstable.

Components of a Well Designed Framework

Resources
  • Environment and application variables and constants – used in the test scripts
  • Configuration files – Access controls and global settings are stored here
  • Object repository – Objects for the GUI validation testing are contained here
  • Test Data – Application specific data is crucial to all tests
  • Recovery scenarios- These handle unexpected errors that cause scripts to stop running and help restart test execution
  • Clean up scripts – These help to return application back to a base state
Function Library
  • Generic – usable across applications
  • Business – specific to an application
  • Extended – specific to an application
Automation Tool

The actual application used for the automation.

Driver Script
  • Environment and application variables and constants – used in the test scripts
  • Configuration files – Access controls and global settings are stored here
  • Object repository – Objects for the GUI validation testing are contained here
  • Test Data – Application specific data is crucial to all tests
  • Recovery scenarios- These handle unexpected errors that cause scripts to stop running and help restart test execution
  • Clean up scripts – These help to return application back to a base state
Reporting and Logging
  • Test reports – Reports the steps completed, data used, and outcome
  • Error and exception logs – These identify problems with the script, data, or environment
  • Defect reports – Reports of any conditions that did not meet expectations
  • Debug logs – This information is crucial to fixing defects quickly

Checklist for Success

Complete the Test Plan

Completing the plan of attack is critical to understanding the goals of the application and the automation effort.

Identify Candidates for Automation

Understanding the percentage of the total test cases to be automated will help determine a suitable framework, number of resources, and timeline.

Keep it Simple

Test scripts should have minimal scripting involved, making it easier for QA teams to understand and visualize each step of the scenario.

Create External Test Data

Data-driven tests will increase coverage of both the application and data specific scenarios, as well as increase the re-usability of specific scripts.

Identify and Incorporate Verification Checkpoints

Whether the verification is at the front-end or the database, confirmation of accuracy and the integrity of the data should be a high priority.

Keep Scripts Independent

By structuring the scripts with minimal dependencies, unattended execution becomes much easier.

We’re here when you need us. If you have questions about anything on our site or our services, or if you are ready to start a consultation, we want you to contact us so we’ve tried to make it easy.

Awards & Recognitions in Software Testing Services

awards

GET IN TOUCH

Please complete the form and one of our QA Expert Specialists will be in contact within 24 hours.
Alternatively, drop us an email at support@qamentor.com or give us a call at 212-960-3812

Form Submitted Successfully.