This document describes the deployment framework that is part of the component-oriented delivery process of Intershop 7. Before reading this document, please make sure to get a glimpse at the big picture in the Concept - Continuous Delivery Tools (valid to 7.10).This document is primarily targeted at administrators and developers, who need to create the deployment of an Intershop 7 based solution. Following the spirit of the DevOps-movement in general and Gradle in particular, configuration and development strongly blend into each other. Therefore this document also provides an entry point for administrators, who need to customize the deployment beyond most basic needs. A more practical and more basic entry to configuring and running the deployment is provided in the Cookbook - Gradle Deployment Tools (7.4 CI - ICM 7.7). The majority of recipes are intended to be used without prior knowledge of this document.
In the component-oriented life cycle the deployment is located between the phases build/assembly and runtime. The following diagram shows what the deployment does on a low technical and on a high conceptual level.
The framework emphasizes the following features:
Also known as source control, source code management systems (SCM), or revision control systems (RCS). VCS is a mechanism for keeping multiple versions of your files, so that when you modify a file you can still access the previous revisions. Place, where build and package software components are located. Provide a common interface to a dependency management system.
Phrase Meaning Version Control System (VCS) Artifact Repository Code Analysis Process to analyze source code to calculate metrics, find bugs, etc. Continuous Delivery Pipeline Sometimes called Deployment Pipeline, describes the stages, which code artifacts runs through source to production system. System Component A software package of different code artifacts and files, that have to be deployed together. System Component Set Is a container for system components, that needs to be build and branched together. Assembly An assembly references one or more system components residing in the same or a configured artifact repository in order to deploy or deliver them together. Build Process Compiles and packages files and code artifacts from a source project to deployable artifacts. Publish Process The process which transfers the deployable artifacts to a configured artifact repository. Assembly Process This process combines several system components to an assembly. Deployment Process This process extracts files and code artifacts from an artifact repository and applies the configuration. Project Gradle Distribution This is a customized Gradle distribution with the preconfigured artifact repositories and Gradle plugins. Gradle Plugin A Gradle plugin packages up reusable pieces of build logic, which can be used across many different projects and builds. Project Gradle Plugin This is a Gradle plugin which contains special corporate respectively project settings. Corporate Plugin The term is used as a synonym for Project Gradle Plugin. Gradle Extension Object Java Bean compliant class holding configurations for Gradle plugins. Gradle Wrapper The Gradle Wrapper is the preferred way of starting a Gradle build. The wrapper is a batch script on Windows, and a shell script for other operating systems. When you start a Gradle build via the wrapper, Gradle will be automatically downloaded and used to run the build. See for more information The Gradle Wrapper in the Gradle documentation (2.11, 2.7, 2.3, 2.0, 1.8) Intershop Cluster A number of hosts of different types serving an Intershop 7. Cluster Node One separately deployable part of an Intershop cluster. A host can run multiple nodes of one Intershop cluster.
Also known as source control, source code management systems (SCM), or revision control systems (RCS). VCS is a mechanism for keeping multiple versions of your files, so that when you modify a file you can still access the previous revisions.
Place, where build and package software components are located. Provide a common interface to a dependency management system.
|A goal to be reached. In the domain of deployment this goal can consist of a file structure and certain file contents, but also operating system configuration. The phrase is used to put emphasis on the goal rather then the way to reach it, which can be determined automatically.
|A process is idempotent, if executing it repeatedly results in the same state as executing it once. Processes aiming to reach a desired state must be at least idempotent.
|A process is convergent, if it does not change more than necessary to reach a desired state. If the current state is already the desired state it should therefore do nothing. Convergence is stronger than idempotence.
|General term for everything the deployment may create or change, like files, directories or links.
The Intershop 7 deployment process is based on a concept known as desired state or target state.
The deployment process is able to reach that desired state from any current state. This can be a virgin system, the state after a previous deployment, or previously deployed state has been modified or broken. Thus providing a single description the deployment can handle use cases like installation, upgrade, downgrade (between any two versions), repair and undeployment. It also covers re-installation in case of failed hardware – even if the system that failed has gone through some very complicated upgrade-path, which is hard or inefficient to repeat.
The overall desired state for a deployment consists of desired states for resources. Resources are files, directories, links and services. Resources have an identity, a type and properties (content, permissions, etc).
This is similar to what the configuration management system Puppet offers. However, Intershop does not try to reimplement full configuration management. Instead we concentrate on lower level resources, scaling better with large number of those than most configuration management systems do. For higher level or more OS-centric types of resources, like users and groups, 3rd party packages and OS configuration, you can easily wrap a configuration management system around the deployment.
When applying a desired state the deployment creates not only resources and determine their properties (e.g., a file and its contents and permissions), but it will also delete resources created by previous deployments that are no longer desired. An undeployment is then simply applying an empty desired state.
In order to allow reaching a desired state a process must at least be idempotent: Executed with the same input repeatedly, it has to lead to the same output. Translated to our scenario, feeding the same desired state to the deployment process repeatedly must result in the same system state (e.g., file contents, permissions, etc).
Idempotence can also be satisfied by a simple full-remove and start-fresh approach for upgrades. The deployment process however tries to only perform operations that are really necessary to reach the desired state. There are a few reasons:
Only applying the minimum set of operations is called convergence. The concepts of desired state management and convergence originate from the DevOps movement. The deployment process is designed to play nicely with configuration management systems, like Puppet or Chef, which base upon the same principle.
The effort to declare the desired state is split between development and administration. On a high conceptual level the workflow is:
Technically developers create components to be deployed and assemblies grouping their own and third party components and publish them to repositories. They contain (directly or indirectly by pointing to deployment plugins):
The administrator creates a Gradle settings script in the local filesystem specifying:
For details on how to bootstrap the deployment see the Cookbook - Gradle Deployment Tools (7.4 CI - ICM 7.7).
This section assumes that you are familiar with Concept - Continuous Delivery Tool | Gradle-based Automation.
The desired state is declared in:
Unifying the language used during development and administration allows for free distribution of efforts between both.
We provide different Gradle plugins for deployment, covering different levels of abstraction and granularity.
Every plugin configures a Gradle
Project in three ways:
The following table shows the most important plugins from low to high level:
|handles the declaration of a desired state for all built-in resource types on a low level and provides tasks to apply them
knows how to deploy a cartridge to a specified directory using
This plugin is automatically applied to all projects, that do not provide their own deploy.gradle script.
|contains the re-usable part of deploying a low level Intershop 7 component using
knows how to deploy an assembly with all its components by:
provides tasks to trigger the deployment of all components by depending on their tasks for applying the desired state
For deploying multiple components at once the deployment framework uses Gradle's multi-project build feature. The assembly is represented by the root project. All other deployed components are represented by direct child projects. Therefore root project is typically configured using the
AssemblyDeploymentPlugin and its children by the
The project hierarchy must be specified during Gradle's initialization phase by configuring the
Settings object. For this purpose we provide two plugins:
Handles selection of components to deploy based on an environment and host-type and creates an according project structure, triggers application of
The assembly itself can also provide an assembly-specific deployment script in an artifact of type
|Allows to bootstrap the deployment from a Gradle settings script in the local filesystem using a standard Gradle distribution. It must be provided with an assembly to start. The assembly must contain an artifact of type
DeploymentBootstrapPlugin will apply this script to the Gradle
Settings and the script will typically apply the
The following diagram shows a typical distribution of build logic across plugins and scripts. It also depicts the control flow and where dependencies to components based on Ivy-meta-data are declared.
To improve support for automated upgrades (and downgrades), we encourage a strong separation of code from data, meta-data from configuration.
To perform an upgrade automatically, applying a desired state, the deployment must be in charge of the deployed resources exclusively. Meaning that no other process, no manual action of the administrator, nor any automatically running process including the deployed application itself should change the resource's state. Changing a properties-file after it has been created by the deployment can lead to loss of data as it is overwritten by future upgrades. While it would be possible to try to merge these changes and those coming from an upgrade, this approach is rarely automatic nor delivers reproducible results.
Two more reliable approaches are:
The next two sections cover these approaches and their applicability in more detail.
Another physical storage that must merge information from development and runtime are database tables: It must meet a schema and must contain some minimum entries. This is a much more complex use case and is continued to be handled by DBInit and DBMigrate.
A prominent example for runtime merging of configuration sources is the Intershop configuration framework. Based on a single entry file, the
configuration.xml any number of property files and other physical storages, like the database, can be combined to a single source of configuration values. Prior to its introduction the common and necessary workflow was:
Copy this properties-file to the host / the shared file system during deployment.
With the introduction of the configuration framework instead of step 3 we recommend to add an entry in the
configuration.xml(an additional file/an external configuration source) and leave the original properties-file untouched. The properties-files provided from development remain as a limited form of meta-data: Name of possible properties, together with a description in form of comments and a default value, which will be effective at runtime if not overridden.
Other consumers of configuration do not use the configuration framework, but still offer splitting configuration into multiple physical sources. The Intershop logging framework based on logback is an example.
In some physical resources it is still necessary to merge information from development and deployment / administration time. Besides legacy reasons, there are a few reasons remaining.
There is a technological gap between some consumers of the configuration and the configuration framework:
For configuration files that do not use the Java properties format, but some more complex format like the
httpd.conf or XML it is not straightforward to merge information from multiple sources. Trying to do so at runtime can lead to hard to understand misconfiguration. The
configuration.xml itself is a good example.
The deployment offers content filters on deploying files to solve this problem. The deployment can write a modified version of a file delivered by the build process, e.g., can add properties, modify XML on the fly. Details of this modification - like which values to give to properties - are part of the desired state. To change these details the desired state must be changed and the deployment must be rerun. (Because of convergence the deployment scales down nicely for performing small changes.)
Some of the files will get modified after deployment, so that they do no longer correspond to the indexed state. This may either be an accidental misconfiguration, that should be repaired in the next deployment process, or it may represent a desired change in itself, so that it must not be overwritten by future deployments.
This problem does not impact convergence, as the deployment process is able to distinguish between updating an unmodified file from a previous desired state to a new desired state and an external modification, that has not been performed by deployment itself. In the former case, the existing file can be overwritten safely, whereas the latter case cannot be handled equally for all conflicts and requires configuration on a per-file basis.
This section lists differences between the Gradle deployment tools and the previously used solutions for deploying Intershop 7.