Common Test

User's Guide

Version 1.15.4

- User's Guide
- Reference Manual
- Release Notes
- PDF
- Top

- Expand All
- Contract All

### Chapters

- Introduction
  - Top of chapter
  - Scope
  - Prerequisites
- Common Test Basics
  - Top of chapter
  - General
  - Test Suite Organisation
  - Support Libraries
  - Suites and Test Cases
  - External Interfaces
- Getting Started
  - Top of chapter
  - Introduction for Newcomers
  - Test Case Execution
  - A Simple Test Suite
  - A Test Suite with Configuration Functions
  - Questions and Answers
- Installation
  - Top of chapter
  - General Information
- Writing Test Suites
  - Top of chapter
  - Support for Test Suite Authors
  - Test Suites
  - Init and End per Suite
  - Init and End per Test Case
  - Test Cases
  - Test Case Information Function
  - Test Suite Information Function
  - Test Case Groups
  - Parallel Property and Nested Groups
  - Parallel Test Cases and I/O
  - Repeated Groups
  - Shuffled Test Case Order
  - Group Information Function
  - Information Functions for Init- and End-Configuration
  - Data and Private Directories
  - Execution Environment
  - Timetrap Time-Outs
  - Logging - Categories and Verbosity Levels
  - Illegal Dependencies
- Test Structure
  - Top of chapter
  - General
  - Skipping Test Cases
  - Definition of Terms
- Examples and Templates
  - Top of chapter
  - Test Suite Example
  - Test Suite Templates
- Running Tests and Analyzing Results
  - Top of chapter
  - Using the Common Test Framework
  - Automatic Compilation of Test Suites and Help Modules
  - Running Tests from the OS Command Line
  - Running Tests from the Erlang Shell or from an Erlang Program
  - Test Case Group Execution
  - Running the Interactive Shell Mode
  - Step-by-Step Execution of Test Cases with the Erlang Debugger
  - Test Specifications
  - Running Tests from the Web-Based GUI
  - Log Files
  - HTML Style Sheets
  - Repeating Tests
  - Silent Connections
- External Configuration Data
  - Top of chapter
  - General
  - Syntax
  - Requiring and Reading Configuration Data
  - Using Configuration Variables Defined in Multiple Files
  - Encrypted Configuration Files
  - Opening Connections Using Configuration Data
  - User-Specific Configuration Data Formats
  - Examples of Configuration Data Handling
  - Example of User-Specific Configuration Handler
- Code Coverage Analysis
  - Top of chapter
  - General
  - Use
  - Stopping the Cover Tool When Tests Are Completed
  - The Cover Specification File
  - Cross Cover Analysis
  - Logging
- Using Common Test for Large-Scale Testing
  - Top of chapter
  - General
  - Use
  - Test Specifications
  - Automatic Startup of Test Target Nodes
- Event Handling
  - Top of chapter
  - General
  - Use
- Dependencies between Test Cases and Suites
  - Top of chapter
  - General
  - Saving Configuration Data
  - Sequences
- Common Test Hooks
  - Top of chapter
  - General
  - Installing a CTH
  - CTH Scope
  - Manipulating Tests
  - Synchronizing External User Applications with Common Test
  - Example CTH
  - Built-In CTHs
- Some Thoughts about Testing
  - Top of chapter
  - Goals
  - What to Test

# 1 Introduction

### 1.1 Scope

Common Test is a portable application for automated testing. It is suitable for:

- Black-box testing of target systems of any type (that is, not necessarily implemented in Erlang). This is performed through standard O&M interfaces (such as SNMP, HTTP, CORBA, and Telnet) and, if necessary, through user-specific interfaces (often called test ports).

- White-box testing of Erlang/OTP programs. This is easily done by calling the target API functions directly from the test case functions.

Common Test also integrates use of the OTP cover tool in application Tools for code coverage analysis of Erlang/OTP programs.

Common Test executes test suite programs automatically, without operator interaction. Test progress and results are printed to logs in HTML format, easily browsed with a standard web browser. Common Test also sends notifications about progress and results through an OTP event manager to event handlers plugged in to the system. This way, users can integrate their own programs for, for example, logging, database storing, or supervision with Common Test.

Common Test provides libraries with useful support functions to fill various testing needs and requirements. There is, for example, support for flexible test declarations through test specifications. There is also support for central configuration and control of multiple independent test sessions (to different target systems) running in parallel.

### 1.2 Prerequisites

It is assumed that the reader is familiar with the Erlang programming language.