Test automation framework is an application that allows you to write a series of tests without worrying about the constraints or limitations of the underlying test tools. Regardless of the automation tool, you would be able to achieve your automation desires and objectives. Our automation expert building framework to include the following:

• Common codebase and database
• Exception and error handling
• Modularized reusable, and maintainable code and data
• Standardized test idioms
• Configurable test suites
• Global test properties

One of the main advantages of our proprietary framework is

• Optimization of testing efforts through unattended execution
• Customized reporting and Test evidence recording
• Multi-environment run with minimal configuration changes
• Metrics data collation and analysis
• Comprehensive documentation facilitating quick learning and easy maintenance

Test automation has undergone several stages of evolution, both in the development of marketable test tool technologies and in the development of test automation processes and frameworks within individual QA organizations. The typical path followed is described below:

• Record and Playback: monitoring of an active user session, recording user inputs related to objects encountered in the user interface, and storing all steps and input data in a procedural script. This method is useful in learning how to use a test tool, but the scripts produced are difficult to maintain after the application under test changes and do not produce reliable, consistent results.

• Test Script Modularity: creating small, independent scripts that represent modules, sections, and functions of the application-under-test; then combining them in a hierarchical fashion to construct larger tests. This represents the first step toward creating reusable test assets.

• Test Library Architecture: dividing the application under test into procedures and functions – also known as objects and methods depending on your implementation language – instead of a series of unique scripts. This requires the creation of library files that represent modules, sections, and functions of the application under test. These files, often referred to as function libraries, are then called directly from within test case scripts. Thus, as elements of the application change, only the common library components which reference them must be changed, not multiple test scripts with hard-coded references which might be difficult to locate and validate.

• Data-Driven Testing: reading input and output values from data files or tables into the variables used in recorded or manually coded test scripts. These scripts include navigation through the application and logging of test status. This abstraction of data from the test script logic allows testers with limited knowledge of the test tool to focus on developing, executing and maintaining larger and more complex sets of test data. This increase in organizational efficiency fosters enhanced test coverage with shorter test cycles.

• Keyword-Driven Testing: including test step functionality in the data driven process by using data tables and keywords to trigger test events. Test steps are expressed as Object → Action →Expected Result. The difference between data-driven and keyword-driven testing is that each line of data in a keyword script includes a reference that tells the framework what to do with the test data on that line. The keyword attached to the test step generally maps to a call to a library function using parameters read in from the data file or table. One major benefit is the improved maintainability of the test scripts: by fully modularizing automation of each step, it’s easier to accommodate any user interface changes in the application under test.

Automation Framework Describes

As noted earlier, one of the challenges facing test automation is to speed up testing processes while increasing the accuracy and completeness of tests. The evolution of test automation frameworks has been driven by accepting this challenge.

Data-Driven Frameworks

This type of functional test automation framework abstracts the data layer from the test script logic. Ideally, only data used as inputs to test objects and outputs from test events would need to change from one iteration to the next. The types of test scripts used in this architecture are described below.

Driver Script

• Performs initialization of the test environment (as required)
• Calls each Test Case Script in the order specified by the Test Plan
• Controls the flow of test set execution

Test Case Script

• Executes application test case logic using Business Component Function scripts
• Loads test data inputs (function parameters) from data files and tables
• Evaluates actual result based on the expected result loaded from data files and tables

Business Component Function Script

• Exercises specific business process functions within an application
• Issues a return code to indicate result or exception
• Uses parameter (input) data derived from data files and tables

Common Subroutine Function Script

• Performs application specific tasks required by two or more business component functions
• Issues a return code to indicate result or exception
• Uses parameter (input) data derived from data files and tables

User-Defined Function Script

• Contains logic for generic, application-specific, and screen-access functions
• Can include code for test environment initialization, debugging and results logging

In this architectural model, the “Business Component” and “Common Subroutine” function scripts invoke “User Defined Functions” to perform navigation. The “Test Case” script would call these two scripts, and the “Driver” script would call this “Test Case” script the number of times required to execute Test Cases of this kind. In each case, the only change between iterations is in the data contained in the files that are read and processed by the “Business Function” and “Subroutine” scripts.

Keyword-driven Frameworks

This type of framework builds on the data-driven framework by including business component functionality in the data tables. Keywords are used within each test step to trigger specific actions performed on application objects.

Driver Script

• Governs text execution workflow
• Performs initialization of the test environment (as required)
• Calls the application-specific Action (“Controller”) Script, passing to it the names of the business process test cases. These test cases can be stored in spreadsheets, delimited text files, or database records.

Action Script

• Acts as the “controller” for test case execution
• Reads and processes the business process test case name received from the Driver Script
• Matches on “key words” contained in the input dataset
• Builds a list of parameters from values included with the test data record
• Calls “Utility” scripts associated with the “key words”, passing the created list of parameters

Utility Scripts

• Process the list of input parameter received from the Action Script
• Perform specific tasks (e.g. press a key or button, enter data, verify data, etc.), calling “User Defined Functions” as required
• Record any errors encountered during test case execution to a Test Report (e.g. data sheet, table, test tool UI, etc.)
• Return to the Action Script, passing a result code for processing status (e.g. pass, fail, incomplete, error)

User-Defined Function Libraries

• Contain code for general and application-specific functions
• May be called by any of the above script-types in order to perform specific tasks
• Can contain business rules

 Mercury Quality Center with TestDirector and QuickTest Professional

Business Process Testing uses a role-based model, allowing collaboration between non-technical Subject Matter Experts and QA Engineers versed in QuickTest Pro.

Business process tests are composed of business components. The information in the business component’s outer layer — the description, status, and implementation requirements, together with the steps that make up the component — is defined in Quality Center by the SME, who then runs and analyzes the associated tests and test sets. A QuickTest Engineer populates a shared repository with the different objects in the application being tested and encapsulates all activities and scripted steps into operations, essentially using function libraries in a keyword based automation framework.

When QuickTest Professional is connected to a Quality Center project with Business Process Testing support, the objects defined by the QuickTest Engineer in the object repository are available for use by the SME. In addition, all the business component information is visible in QuickTest. This integration and visibility between the two applications enables the SME to implement the testing steps for the business components that are defined in business process tests, and also enables the QuickTest Engineer to effectively maintain the set of objects in the object repository and the operations in the function libraries.

In addition to creating and maintaining the object repository, the QuickTest Engineer defines a set of elements that comprise an Application Area, created in QuickTest Professional and containing all of the settings and resources required to create the content of a business component. These include all the objects from the application under test contained in the shared object repository, and the user-defined operations contained in function library files.

Each business component can be associated with a specific Application Area, or can share an Application Area with other components. Application area settings are automatically inherited by the business components that are based on that application area. An application area includes:

• Resources: resource settings include associated library files and the shared object repository.

• Add-Ins: the add-ins associated with the first business component in a business process test (inherited from the application area used by the component) are automatically loaded in QuickTest Professional when Quality Center runs the test. Quality Center assumes that add-ins are required for all the business components in the same business process test.

• Windows-Based Applications: if you are creating a business component to test a Windows-based application, you must specify the application on which the business component can run. Other environments are supported by the appropriate QuickTest Add-In.

• Recovery Scenarios: activated during execution of a business component test when an unexpected event occurs.

QA Mentor Automation Methodology