DevOps Glossary of Terms

A/B Testing

A technique in which a new feature, or different variants of a feature, are made available to different sets of users and evaluated by comparing metrics and user behavior.

Acceptance Testing

Typically high-level testing of the entire system carried out to determine whether the overall quality of both new and existing features is good enough for the system to go to production.


A precursor to DevOps. Agile is a software development and, more broadly, business methodology, that emphasizes short, iterative planning and development cycles to provide better control and predictability and to support changing requirements as projects evolve.

Application Release Orchestration (ARO)

Tools, scripts, or products that automatically install and correctly configure a given version of an application in a target environment, ready for use. Also referred to as “Application Release Automation” (ARA) or “Continuous Delivery and Release Automation” (CDRA).

Behavior-Driven Development (BDD)

A development methodology that asserts software should be specified in terms of the desired behavior of the application, and with syntax that is readable for business managers.

Black Box Testing

A testing or quality assurance practice that assumes no knowledge of the inner workings of the system being tested, and which thus attempts to verify external rather than internal behavior or state.

Build Agent

A type of agent used in Continuous Integration that can be installed locally or remotely in relation to the Continuous Integration server. It sends and receives messages about handling software builds.

Build Artifact Repository

A tool used to organize artifacts with metadata constructs and to allow automated publication and consumption of those artifacts.

Build Automation

Tools or frameworks that allow source code to be automatically compiled into releasable binaries. Usually includes code-level unit testing to ensure individual pieces of code behave as expected.

Canary Release

A go-live strategy in which a new application version is released to a small subset of production servers and heavily monitored to determine whether it behaves as expected. If everything seems stable, the new version is rolled out to the entire production environment.

Configuration Drift

A term for the general tendency of software and hardware configurations to drift, or become inconsistent, with the template version of the system due to manual ad hoc changes (like hotfixes) that are not introduced back into the template.

Configuration Management

A term for establishing and maintaining consistent settings and functional attributes for a system. It includes tools for system administration tasks, such as IT infrastructure automation.


Similar but more lightweight than a virtual machine, containers are stand-alone, executable packages containing everything needed to run a piece of software: code, runtime, system tools, system libraries, settings, and so on.

Continuous Delivery

A set of processes and practices that radically remove waste from your software production process, enable faster delivery of high- quality functionality, and set up a rapid and effective feedback loop between your business and your users.

Continuous Integration (CI)

A development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is then verified by an automated build, allowing teams to detect problems early.

Dark Launch

A go-live strategy in which code implementing new features is released to a subset of the production environment but is not visibly, or only partially, activated. The code is exercised, however, in a production setting without users being aware of it.

Delivery Pipeline

A sequence of orchestrated, automated tasks implementing the software delivery process for a new application version. Each step in the pipeline is intended to increase the level of confidence in the new version to the point where a go/no-go decision can be made. A delivery pipeline can be considered the result of optimizing an organization’s release process.


A portmanteau of development and operations, DevOps is a set of processes, practices, and tools that improve communication, collaboration, and processes between the various roles in the software development cycle, resulting in delivery of better software with speed and stability.

DevOps Intelligence

An approach to continually improving software development practices by analyzing the data produced by the tools and development process itself. This allows organizations to improve efficiency, lower risk, and create better software.


The practice of integrating security into the DevOps process.

Functional Testing

Testing of the end-to-end system to validate functionality. With executable specifications, Functional Testing is carried out by running the specifications against the application.

Infrastructure as a Service (IaaS)

Cloud-hosted virtualized machines, usually billed on a “pay as you go” basis. Users have full control of the machines but need to install and configure any required middleware and applications themselves.

Infrastructure as Code

A system configuration management technique in which machines, network devices, operating systems, middleware, and so on are specified in a fully automatable format. The specification, or “blueprint,” is regarded as code that is executed by provisioning tools, kept in version control, and generally subject to the same practices used for application code development.


“Lean manufacturing,” or “lean production,” is an approach or methodology that aims to reduce waste in a production process by focusing on preserving value. Largely derived from practices developed by Toyota in car manufacturing, lean concepts have been applied to software development as part of agile methodologies. The Value Stream Map (VSM), which attempts to visually identify valuable and wasteful process steps, is a key lean tool.


A software architecture design pattern in which complex applications are composed of small, independent processes communicating with each other using language-agnostic APIs. These services are small, highly decoupled, and focus on doing a small task.

Non-functional Requirements (NFRs)

The specification of system qualities, such as ease-of-use, clarity of design, latency, speed, and ability to handle large numbers of users, that describe how easily or effectively a piece of functionality can be used, rather than simply whether it exists. These characteristics can also be addressed and improved using the Continuous Delivery feedback loop.


A type of organization in which the management of systems on which applications run is either handled completely by an external party (such as a PaaS vendor) or fully automated. A NoOps organization aims to maintain little or no in-house operations capability or staff.

Orchestration Pipeline

Tools or products that enable the various automated tasks that make up a Continuous Delivery pipeline to be invoked at the right time. They generally also record the state and output of each of those tasks and visualize the flow of features through the pipeline.

Platform as a Service (PaaS)

Cloud-hosted application runtimes, usually billed on a “pay as you go” basis. Customers provide the application code and limited configuration settings, while the middleware, databases, and so on are part of the provided runtime.

Product Owner

A person or role responsible for the definition, prioritization, and maintenance of the list of outstanding features and other work to be tackled by a development team. Product Owners are common in agile software development methodologies and often represent the business or customer organization. Product Owners need to play a more active, day-to-day role in the development process than their counterparts in more traditional software development processes.


The process of preparing new systems for users. In a Continuous Delivery scenario, this work is typically done by development or test teams. The systems are generally virtualized and instantiated on demand. Configuration of the machines to install operating systems, middleware, and so on is handled by automated system configuration management tools, which also verify that the desired configuration is maintained.

Regression Testing

Testing of the end-to-end system to verify that changes to an application did not negatively impacted existing functionality.

Release Coordination

The definition and execution of all the actions required to take a new feature or set of features from code check-in to go-live. In a Continuous Delivery environment, this is largely or entirely automated and carried out by the pipeline.

Release Management

The process of managing software releases from the development stage to the actual software release itself.

Release Orchestration

The use of tools for defining, automating, securing, tracking, and controlling manual and automated tasks in application releases.

Test-Driven Development (TDD)

A development practice in which small tests to verify the behavior of a piece of code are written before the code itself. The tests initially fail, and the aim of the developer(s) is then to add code to make them succeed.

Unit Testing

Code-level (i.e., does not require a fully installed end-to-end system to run) testing to verify the behavior of individual pieces of code. Test-driven development makes extensive use of unit tests to describe and verify intended behavior.

Value Stream Mapping

A process visualization and improvement technique used heavily in lean manufacturing and engineering approaches. Value Stream Maps are used to identify essential process steps vs. “waste” that can be progressively eliminated from the process.


A systems management approach in which users and applications do not use physical machines, but simulated systems running
on actual, “real” hardware. Such “virtual machines” can be automatically created, started, stopped, cloned, and discarded in a matter of seconds, giving operations tremendous flexibility.


A software development methodology based on a phased approach to projects, from “Requirements Gathering” through “Development” and so on, to “Release.” Phases late in the process (typically related to testing and QA) tend to be squeezed, as delays put projects under time pressure.

White Box Testing

A testing or quality assurance practice that is based on verifying the correct functioning of the internals of a system by examining its (internal) behavior and state as it runs.