Related Documents
Document Properties
KbidX27327
Last Modified03-Feb-2020
Added to KB23-Nov-2015
Public AccessEveryone
StatusOnline
Doc TypeGuidelines, Concepts & Cookbooks
Product
  • Gradle Tools
  • ICM 7.6
  • ICM 7.7

Cookbook - Setup CI Infrastructure (valid to Gradle Tools 2.7)

Note

Intershop recommends to use Gradle Tools 2.11 and ICM 7.8 for new projects. For active projects Intershop recommends the update to Gradle Tools version 2.11. Please see Public Release Notes - Gradle Tools - Version 2.11 for migration steps.

1 Introduction


This cookbook describes in full detail how to create the required CI infrastructure. This document provides detailed examples based on Jenkins as CI server and Subversion for version control. In addition, the cookbook also describes how to set up the required users and permissions.

This document describes the setup and configuration of the base processes in a Continuous Integration Environment (CI).


Note

This cookbook requires Intershop Gradle tools version 2.7. Previous versions of this document show how to deal with the

This document replaces Cookbook - Setup CI Infrastructure (valid to Gradle Tools 2.3).

Since Intershop 7.6 delivers maven artifacts, it is necessary to reconfigure existing infrastructure. Read more at Migration.

The Intershop Gradle Tools and this document make no strong assumptions which artifact repository server, continuous integration server (CI server) or version control system (VCS) have to be used. This cookbook nonetheless provides examples for using Sonatype Nexus, Jenkins and Subversion (SVN).

Note

The recipes in this cookbook are made for a specific ICM version in combination with specific Intershop Gradle Tools.

Since the bootstrap of the Intershop Gradle Tools defines your project's directory structure and different versions of the Intershop Gradle Tools differ regarding their plugins you have to ensure to use a valid combination of ICM and Intershop Gradle Tools.

For detailed version dependency information refer to Reference - CI Dependencies of Intershop Commerce Management.

For more detailed information on which plugins are available/deprecated for which Intershop Gradle Tools version refer to:

This is an overview of all necessary components:

overview-ci-setup-v2

The order of the recipes describes the way to set up a CI environment with all the necessary components. Some recipes can be executed in parallel or may be executed multiple times.

CI Setup Workflow

  1. Prepare the necessary server for the process
    1. Artifact repository server (see recipe Configure Artifact Repository Server)
      This recipe describes the setup of an artifact repository based on Sonatype Nexus. 
    2. CI Server (see recipe Configure CI Server)
      This description is based on Jenkins.
    3. Version Control System
      All examples and descriptions are based on Apache Subversion. Intershop does not require any special configurations on this server.
  2. Generate Sources/Configuration (see recipe Create Sources From Intershop Templates)
    For the build processes of Intershop special source artifacts are necessary. Intershop provides the possibility to generate these sources from templates.
  3. Setup CI Process for Corporate Artifacts (see recipe Setup CI Build for Corporate Artifacts)
    This recipe describes how to build artifacts on the CI server only needed once per environment.
  4. Setup CI Process for Project Artifacts (see recipe Setup CI Build for Corporate Artifacts)
    This recipe describes how to build artifacts on the CI server needed separately for each project.
  5. Setup a developer environment (see recipe Setup Developer Environment in cookbook Gradle Developer Workflow)
    This document describes the setup steps on a developer machine. This can be done by the developer self, but it is also possible to provide VMs for the project developers.

1.1 Glossary


Phrase
Meaning
Version Control System (VCS)

Also known as source control, source code management systems (SCM), or revision control systems (RCS). VCS is a mechanism for keeping multiple versions of your files, so that when you modify a file you can still access the previous revisions.

Artifact Repository

Place, where build and package software components are located. Provide a common interface to a dependency management system.

Code AnalysisProcess to analyze source code to calculate metrics, find bugs, etc.
Continuous Delivery PipelineSometimes called Deployment Pipeline, describes the stages, which code artifacts runs through source to production system.
System ComponentA software package of different code artifacts and files, that have to be deployed together.
System Component SetIs a container for system components, that needs to be build and branched together.
AssemblyAn assembly references one or more system components residing in the same or a configured artifact repository in order to deploy or deliver them together.
Build ProcessCompiles and packages files and code artifacts from a source project to deployable artifacts.
Publish ProcessThe process which transfers the deployable artifacts to a configured artifact repository.
Assembly ProcessThis process combines several system components to an assembly.
Deployment ProcessThis process extracts files and code artifacts from an artifact repository and applies the configuration.
Project Gradle DistributionThis is a customized Gradle distribution with the preconfigured artifact repositories and Gradle plugins.
Gradle PluginA Gradle plugin packages up reusable pieces of build logic, which can be used across many different projects and builds.
Project Gradle PluginThis is a Gradle plugin which contains special corporate respectively project settings.
Corporate PluginThe term is used as a synonym for Project Gradle Plugin.
Gradle Extension ObjectJava Bean compliant class holding configurations for Gradle plugins.
Gradle WrapperThe Gradle Wrapper is the preferred way of starting a Gradle build. The wrapper is a batch script on Windows, and a shell script for other operating systems. When you start a Gradle build via the wrapper, Gradle will be automatically downloaded and used to run the build. See for more information The Gradle Wrapper in the Gradle documentation (2.11, 2.7, 2.3, 2.0, 1.8)
Intershop ClusterA number of hosts of different types serving an Intershop 7.
Cluster NodeOne separately deployable part of an Intershop cluster. A host can run multiple nodes of one Intershop cluster.

1.2 References

2 Migration

This section is a short overview what is changed in context of this document. For migrations related to the product's structure itself please see the migration guide Guide - 7.6 Build Changes.

Note

To use the bootstrap package for the CI setup in version 2.7.0 or newer (delivered with Gradle Tools 2.7 or newer), these changes are also required.

As Intershop 7.6 contains Maven artifacts, it is necessary to reconfigure existing infrastructure. Follow the steps described in recipe Configure Artifact Repository Server, to provide the needed repositories.

There is a new CI Bootstrap version which comes with changes for a project setup. Generate these files and merge them into existing projects that should run on Intershop Commerce Management 7.6. To get the generated files follow the recipe Create Sources From Intershop Templates. Copy the content of the generated assembly and component set project into your existing projects to use the updated corporate Gradle distribution.

Finally the CI build plan configuration needs also be adapted. Please see recipe Setup CI Build For Project Artifacts.

3 Recipe: Configure Artifact Repository Server

3.1 Problem

The artifact repository server stores artifacts. Artifacts are uploaded by the build server and downloaded by development environments, the build server and demo/production deployments. The build engineer resp. developer should set up and configure this server at the beginning of a project. So, which artifact repository servers are available and which configuration is required to run the Intershop build, assembly and deployment tools?

3.2 Solution

3.2.1 Supported Servers

The following artifact repository servers are available and work with the Intershop continuous delivery tools:

Example

 See the discussion for an example using Sonatype Nexus.

3.2.2 Configuration Steps

Choose an artifact repository server and perform the following steps:

  1. Install a repository server according to the vendors documentation.
  2. A repository server can hold multiple repositories. Each repository is reachable via a URL of the form <repository base URI>/<repository ID>, e.g., http://nexus/repositories/releases.
    Create/configure the following repositories:

    IDUsageComment
    jcenterFor Maven artifacts available from Bintray JCenter. (Proxy for https://jcenter.bintray.com) These are artifact proxies to repositories in the internet, which cache all already retrieved artifacts. Please check the detailed configuration below.
    centralFor Maven artifacts available from Maven Central. (Proxy for https://secure.central.sonatype.com/maven2)
    ishrepoFor binary artifacts provided by Intershop. (Proxy for https://repository.intershop.de/)
    Please have a look on you contract page in the Intershop KB area for more information. 
    distributionsRepository for Gradle distributions. 
    gradleFor corporate/ project Gradle configuration. 
    releasesFor corporate/ project releases. 
    snapshotsFor all artifacts from snapshot builds. This is the only repository that should allow
    overwriting of existing artifacts with the same name and version.
     
    componentsThis repository units gradle, releases, ishreleases (if present) and ishrepo in one group.These are repository groups. Please check the detailed configuration below.
    mavenallThis repository units central and ishrepo in one group.

    ishreleases

    ishreleases is the predecessor to the proxy-based ishrepo and was used to host Intershop releases that were not delivered via Intershop's public repository. You do not need to create this repository when working only with Intershop releases that are delivered via public repository (e.g., Intershop Commerce Suite 7.6).

  3. Configure permissions:
    1. All users should have read access to all repositories of the project (Intershop releases, releases, components, Gradle, snapshots, Central and distributions).
    2. At least one user must be granted with permissions to publish new artifacts.
      In Maven the process of pushing artifacts to the repository is called deploy and release, while in the gradle world it is called publish.

3.3 Discussion

3.3.1 Example Using Sonatype Nexus

Download and install Sonatype Nexus:

This is the overview of available repositories in an installation before any changes:

The following repositories are necessary for all build and deployment processes:

3.3.1.1 Proxy Repositories

NameIDRepostiory TypeRemote Storage LocationFormatRemarks
JCenterjcenterProxyhttps://jcenter.bintray.com/Maven2 
CentralcentralProxyhttp://repo1.maven.org/maven2/Maven2 
IntershopishrepoProxyhttps://repository.intershop.de/content/repositories/<contract-id>Maven2See Setup the Intershop Proxy below for details on how to get your contract-id and authorization information.

If you need additional remote repositories, all these repositories should be configured in one group. This makes it easier to configure this kind of repositories in the Gradle configuration, because all different repositories are usable via one URL. These repositories provide third party artifacts from public Maven repositories. The repository server needs internet access.

3.3.1.2 Set Up the Intershop Proxy Repository

The Intershop Proxy Repository in your repository makes available the artifacts according to your contract with Intershop. So you need to log on to the Intershop Knowledge Base and retrieve your credential tokens as well as the contract-id from your contract details. The contract-id has to be appended to the proxy URL in Remote Storage Location textfield. Then you need to check the Authentication (optional) box and provide your credential tokens. 

Note

Login failed for repository.intershop.de

Although Intershop's public repository server shows a login screen when viewed in a web browser, those credentials do not entitle you to log into the Artifactory UI. A login attempt yields the following error message:

Incorrect username, password or no permission to use the Artifactory User Interface.

This is entirely intentional by design. Your credentials are suitable only for resolving artifacts using a proxy repository.

Note

Possible pitfall, when attempting to access files via this proxy repository. If the following message appears:
404 - Automatic routing filter rejected remote request for path /com/intershop/build/infrastructure/intershop-ci-setup-bootstrap/<version>/intershop-ci-setup-bootstrap-<version>.zip from M2Repository(id=ishrepo)

Apply the following workaround.

Switch to the tab Routing and make sure Discovery is unchecked.

Note

In case the error message 404 - Automatic routing filter rejected remote request persists, please verify the configuration of the Nexus -> Views/Repositories -> Routing and adapt or delete the existing rules.

3.3.1.3 Hosted Repositories

The following repositories must be created and configured as hosted repositories:

NameIDRepository TypeRepository PolicyDeployment PolicyUsage
DistributionsdistributionshostedReleaseDisable Redeployfor project Gradle distribution
GradlegradlehostedReleaseDisable Redeployfor project Gradle plugins
ReleasesreleaseshostedReleaseDisable Redeployfor corporate / project releases
SnapshotssnapshotshostedSnapshotAllow Redeployfor all artifacts from snapshot builds

3.3.1.4 Groups

The following repository forms a repository group of different repositories. There are hosted repositories as well as the Intershop proxy repository in there. Create it according to the following table:

NameIDRepository TypeFormatRepository Names
ComponentscomponentsgroupMaven2Intershop, Releases, Gradle
Maven allmavenallgroupMaven2JCenter, Central, Intershop

Once you created the new repository group, switch to the configuration tab an move the needed repository from Available Repositories to Ordered Group Repositories. 

Note

The order of the assigned repositories is very important.

  1. JCenter
  2. Central
  3. Intershop 

Components Configuration

Maven All Configuration

Note

Older releases of Intershop were imported to a repository 'Intershop Releases'. If you need this for migration projects or for other use cases, it is necessary to add this repository also to the group "Components". Please add this repository to the end of the list.

3.3.1.5 User, Roles and Permissions

In the example we use the default admin user with the default password.

To create a deploy user:

  1. Create the role Project Upload with the following roles and privileges:
    Nexus Deployment Role, All Repositories (create, read, update, view), Artifact Download, Artifact Upload
  2. Create the user deploy and assign the new role.

3.3.1.6 Summary

The format of all hosted and proxy repositories is always Maven 2.

Note

The ID is case sensitive, because it is part of the URL.

It is necessary to configure these repositories in your artifact repository installation. After the configuration the following URLs are available.

The actual path nexus/content/repositories/central depends on the actual installation of your repository server.
NameURL
JCenterhttp://<Hostname:Port>/nexus/content/repositories/jcenter
Centralhttp://<Hostname:Port>/nexus/content/repositories/central
Intershophttp://<Hostname:Port>/nexus/content/repositories/ishrepo
Maven allhttp://<Hostname:Port>/nexus/content/groups/mavenall
Distributionshttp://<Hostname:Port>/nexus/content/repositories/distributions
Gradlehttp://<Hostname:Port>/nexus/content/repositories/gradle
Releaseshttp://<Hostname:Port>/nexus/content/repositories/releases
Intershop Releaseshttp://<Hostname:Port>/nexus/content/repositories/ishreleases
Componentshttp://<Hostname:Port>/nexus/content/groups/components
Snapshotshttp://<Hostname:Port>/nexus/content/repositories/snapshots

The resulting repository overview should be something like this:

4 Recipe: Configure CI Server

4.1 Problem

For the automatic execution of all build process and the automatic installation of a new build in an integration environment it is required to have a special CI server.

How can an administrator set up this kind of server?

4.2 Solution

This is an example list of CI servers:

The listed CI servers always support remote processes, so there are different options for the execution on the server (the CI-host) or on a remote-agent (a connected client-host).

These are the extremes of all options:

  • Undistributed: All processes are running on one single machine.
  • Distributed: Component set builds, assembly builds, integration and the demo servers will be started on different bare metal machines or VMs.

Example

 See the discussion for an example using Jenkins.

4.2.1 Prerequisites

Hardware requirements for server processes

  • Each active build of a component set process needs approx 1GB RAM and one CPU.
  • An assembly build needs approx two CPUs and approx 3GB RAM.
  • A single INTERSHOP instance requires in minimum two CPUs and approx 4GB RAM.
  • The space on the hard disc depends on the size of the custom component set.

Software requirements for server processes

  • The hosts (server and remote-agent) are part of the CI Environment and need to perform build- and deployment-tasks. They must meet minimum software requirements.
  • Oracle JDK must be installed for Jenkins and all other processes (See also requirements of the used Intershop version)

Database

  • For each assembly build, integration and demo server a database schema is necessary. The sizes of the table spaces depend on the amount of the init data of the custom extension.

4.2.2 Steps

Choose a CI server and perform the following steps:

Note

The CI server must support the type of your VCS. There are also corresponding plugins for special use cases of interacting with artifact repositories.

  1. Install a CI server according to the vendors documentation.
  2. Configure permissions to the servers in the CI Infrastructure, that are:
    1. VCS read credentials – server needs read privilege
    2. Artifact repository credentials – the user with permission to publish new artifacts (see Recipe: Configure Artifact Repository Server > Configure permissions)

(optional)

  1. For long running processes and also for demo and integration test machines it is recommended to use external agents. It is also possible to use ssh scripts and commands for the access to external machines if the usage of agents is limited.
  2. For a faster analysis of some automatic build steps it is helpful to have a web front end for your VCS. Most CI servers provide support for those kind of interfaces.
  3. Special tasks like assembly builds, the setup of the integration server and the demo servers should run on different bare metal machines or VMs.

4.2.3 Folder for Additional Build Directories

Gradle uses a directory for caches and it is also possible to store additional configuration files in this directory. This directory can be set with the environment variable GRADLE_USER_HOME or with an additional parameter (See Gradle Command Line). The build and assembly process provided by Intershop uses this directory for a special publishing configuration on the CI server. Therefore it is necessary to provide a directory where the CI server process can read and write files. The owner of this directory should be the CI server user.

Make that path available as environment variable BASE_GRADLE_USER_HOME to your CI builds.

4.3 Discussion

4.3.1 Example Using Jenkins

Download and Install Jenkins:

You may follow the instructions from https://wiki.jenkins-ci.org/display/JENKINS/Use+Jenkins, reps. https://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins.

4.3.1.1 Plugin Recommendations

After the server is started, you should change the admin password. Furthermore it is necessary to activate the following set of plugins:

Plugin
Description
Email-ext plugin

The CI server should be able to send e-mails, when an error has occurred in a build.

This plugin extends Jenkins built in email notification functionality by giving you more control. It provides customization of 3 areas.

  • Triggers - Select the conditions that should cause an email notification to be sent.
  • Content - Specify the content of each triggered email's subject and body.  
  • Recipients - Specify who should receive an email when it is triggered.
Dashboard View

The CI server should provide the possibility for different views of the build plans.

This plugin contributes a new view implementation that provides a dashboard / portal-like view for Jenkins.

Build Environment Plugin*This plugin shows information about the environment of a build and gives the option to compare the environments of two builds.
All Changes Plugin

Shows all changes which influenced the builds of a project.

  • Shows changes by dependent builds (via fingerprinting)
  • Shows changes by subprojects added via a BuildStep from the parameterized-trigger-plugin
EnvInject Plugin*

For configurations it is necessary to specify some environment variables. These settings must be isolated for a specific job.

This plugin makes it possible to have an isolated environment for your jobs.

The plugins marked with a star are mandatory for the following documentation.

4.3.1.2 Configuration Hints

It is necessary to add / change the following configuration settings:

  1. JDK
    1. Login with an admin account.
    2. Select Manage Jenkins.
    3. Select Configure System.
    4. Adapt the JDK configuration.
  2. Subversion (or SCM - Source Code Management)
    1. Select the preferred Subversion version.
  3. E-mail Notification and Extended E-mail Notification
    1.  Set SMTP server:
    2. Add default recipients:
  4. Add global properties:

    VariableValue Linux
    BASE_GRADLE_USER_HOME/opt/intershop/build
    DEPLOY_USER_NAMEdeploy
  5. Add global password for deployment user (see Recipe: Configure Artifact Repository Server > User, Roles and Permissions)

    1. Go to Manage Jenkins | Configure Systems.
    2. Select Global Passwords | Add.
    3. Add password.

      NameDEPLOY_USER_PASSWORD
      Passworde.g., admin123

5 Recipe: Create Sources From Intershop Templates

5.1 Problem

For a project setup or development environment setup it is necessary to have scripts, configuration files and also some basic source artifacts. The sources for the following artifacts must be generated from provided templates.

How can a developer or administrator create the source artifacts from the provided Intershop template?

5.2 Solution

5.2.1 Prerequisites

Meet the minimum system requirements.

  • Windows or Linux machine with:
    • Installed Oracle JDK (use a version specified in the system requirements of your Intershop version)
    • Access to https://services.gradle.org/
    • Access to the VCS (this documentation uses Subversion as the version control system)
     

See Recipe: Configure Artifact Repository Server.    

5.2.2 Steps

  1. Check your active Java version:

    >java -version
    java version "1.8.0_51"
    Java(TM) SE Runtime Environment (build 1.8.0_51-b16)
    Java HotSpot(TM) 64-Bit Server VM (build 25.51-b03, mixed mode)
  2. Download the latest version of the bootstrap package for the CI setup via your proxy.

    Note

    Newer versions of the Intershop Bootstrap Package

    Find the version mapping table and download links for a corresponding Intershop Commerce Management version in the Reference - CI Dependencies of Intershop Commerce Management.

    Fetch CI Bootstrap Package on Linux
     $ wget http://<your-reposerver:port>/nexus/content/repositories/ishrepo/com/intershop/build/infrastructure/intershop-ci-setup-bootstrap/3.3.0/intershop-ci-setup-bootstrap-3.3.0.zip
    Fetch CI Bootstrap Package on Windows
    (new-object System.Net.WebClient).DownloadFile('http://<your-reposerver:port>/nexus/content/repositories/ishrepo/com/intershop/build/infrastructure/intershop-ci-setup-bootstrap/3.3.0/intershop-ci-setup-bootstrap-3.3.0.zip', '<target dir>\intershop-ci-setup-bootstrap-3.3.0.zip')

    Note

    Browsing the Proxy Repository

    Due to the way the proxy mechanism is implemented by Nexus, browsing the repository may not work reliably. Do not browse the repository to find the artifact, but enter the URL directly.

    This downloads the release through your previously defined proxy repository of the Intershop artifacts. Accordingly it will be also available in your repository. Alternatively, you may just copy the URL into your browser and fetch it this way. The URL above might differ from your repository configuration. On the default installation the repository base path (/nexus/content/repositories) contains the /nexus/ path, if you change it, change it here accordingly.

  3. Unzip the the contents to a directory of your choice.

    Example for unzipping (Linux)
    unzip intershop-ci-setup-bootstrap-*.zip -d <target directory>
  4. Open the build.gradle file in the intershop-ci-setup-bootstrap subdirectory in an editor. This file configures the generation of artifacts.
    It contains lines marked with //TODO. Go through these lines and provide the necessary information.
    Also see the discussion for required and optional adjustments of the build.gradle.
  5. Run the following command inside the intershop-ci-setup directory to generate all source artifacts:

    Note

    It is important to specify the JAVA_HOME before you start the command.

    Run generation of all sources (Windows)
    gradlew intershopCISetupAll
    Run generation of all sources (Linux)
    ./gradlew intershopCISetupAll

    See the discussion for a directory tree of the generated sources. You may also generate (or regenerate) only some of the source artifacts by running dedicated Gradle tasks, see discussion.

  6. Add jar files to the prepared Oracle JDBC Driver component set:
    1. Download file ojdbc7.jar, ucp.jar and ons.jar from the Oracle web by select the required Version (http://www.oracle.com/technetwork/database/features/jdbc/index-091264.html), agreeing to license conditions.
    2. Copy files to <setupDirectory>/projects/oracleDriver/p_oracle/3rd_oracle/staticfiles/cartridge/lib.
  7. Add all generated sources to your VCS.
    It is recommended to add them in a way that the different source artifacts can be branched independently. How to do this depends on the VCS.
    Also add the intershop-ci-setup directory to your VCS for later regeneration of sources.

See discussion for example commands if you use SVN as VCS.

5.3 Discussion

Problems resolving artifacts

In you encounter problems resolving artifacts in this recipe, please verify your artifact repository server's configuration, including the proxy repository's settings.

5.3.1 Required Adjustments of build.gradle

  1. Corporate names: Specify names that identify your company/department. If you use the same CI environment for multiple projects, these can stay the same across projects.
    1. Technical corporate name: Specify a reverse DNS name for your company/department in property corporateName.
  2. Project names: Specify names that identify your project.
    1. Technical project name: Specify a technical identifier for your project in property projectName.
  3. Output directory: Specify a directory to generate sources into in the property setupDirectory.

  4. Repository base URL: Specify the repository base URL of your artifact repository server in property repoBaseURL.

  5. Intershop Proxy Repository: Provide the URL to the ishrepo repository (your proxy repository) on the marked line in the buildscript section.

5.3.2 Optional Adjustments of build.gradle

5.3.2.1 Output Directories

 By default all sources are generated into subdirectories of the setupDirectory. You may also configure independent output directories for different types of source artifacts. See the directories section of the build.gradle file and the following examples.

Example for adjusting output directories (Linux)
apply plugin: 'ci-setup'
IntershopCISetup {
	directories {
		setupDirectory = '/home/developer/intershop-ci/sources'
		devOpsDir = '/home/developer/devops-resources'
	
		projectsDir = "${setupDirectory}/projects"
	}
}
Example for adjusting output directories (Windows)
apply plugin: 'ci-setup'
IntershopCISetup {
	directories {
		setupDirectory = 'C:/Users/developer/intershop-ci/sources'
		devOpsDir = 'C:/Users/developer/intershop-ci/devops-resources'
	
		projectsDir = "${setupDirectory}/projects"
	}
}

5.3.2.2 Repositories

The build.gradle file comes with default IDs for all important repositories. It is possible to rename them and add new ones. See the repository section of the build.gradle file.

Documentation assumes default repository IDs

Use the ability to rename repositories with care. The configured IDs will be taken into account for all generated sources. This documentation however assumes the default IDs, especially for the ishrepo repository.

5.3.2.3 Corporate Distribution

The repository section allows to adjust the version of your corporate distribution in property distributionVersion.

It also allows to configure a different URL. So your corporate distribution can be downloaded by the Gradle wrapper in property distributionURL. This way you can place your corporate distribution on any web server you want.

5.3.2.4 Versions

The versions section configures which versions of Intershop 7, the Gradle Tools and third party components should be used. By default the latest available version of Intershop 7 available in your repository is used. You may specify different versions available in your repository.

5.3.2.5 Component Set and Assembly

The sections assembly and componentSet define names and versions of component set and assembly. Most properties have sensible default values based on projectName and corporateName, but you may change them. You may also adjust the initial version numbers.

The component set also contains an example component. You may adjust name and description.

5.3.3 Overview of Tasks and Output

Running gradlew tasks shows all available tasks:

:tasks
------------------------------------------------------------
All tasks runnable from root project
------------------------------------------------------------
...
CI Server Setup tasks
---------------------
ciSetup - Create all source artefacts of a CI server setup

Corporate Configuration tasks
-----------------------------
corporateSetup - Create all source artefacts of the corporate configuration
createCorporateConfiguration - Creates a structure of the 'CorporateConfiguration' plugin.
createCorporateDistribution - Creates a structure of a corporate distribution package project.
createOracleComponentSet - Creates a special component set for publishing Oracle JDBC drivers

Intershop CI Source tasks
-------------------------
intershopCISetupAll - Create all necessary source artefacts

Project Setup tasks
-------------------
createDeploymentConfig - Creates a structure of a deployment configuration.
createProject - Creates a structure of a project configuration with assembly, componentset and developer home.
projectSetup - Create all source artefacts of a project setup

...

The resulting sources (assuming default output directories) look like this:

Generated Sources
sources
├── devops
│   ├── ci_server
│   │   ├── host_configs
│   │   │   └── ciserver
│   │   │       └── environment.properties
│   │   └── user_home
│   │       ├── gradle.properties
│   │       └── init.gradle
│   ├── deployments
│   │   └── intershop-project
│   │       ├── gradle
│   │       │   └── wrapper
│   │       │       ├── gradle-wrapper.jar
│   │       │       └── gradle-wrapper.properties
│   │       ├── gradlew
│   │       ├── gradlew.bat
│   │       └── settings.gradle
│   └── gradle
│       ├── corporate-distribution
│       │   ├── build.gradle
│       │   ├── gradle
│       │   │   └── wrapper
│       │   │       ├── gradle-wrapper.jar
│       │   │       └── gradle-wrapper.properties
│       │   ├── gradle.properties
│       │   ├── gradlew
│       │   ├── gradlew.bat
│       │   └── src
│       │       └── init.d
│       │           └── intershop-init.gradle
│       └── corporate-plugins
│           ├── build.gradle
│           ├── corporate-configuration
│           │   └── src
│           │       └── main
│           │           └── resources
│           │               └── configuration
│           │                   └── corporate.properties
│           ├── gradle
│           │   └── wrapper
│           │       ├── gradle-wrapper.jar
│           │       └── gradle-wrapper.properties
│           ├── gradle.properties
│           ├── gradlew
│           ├── gradlew.bat
│           └── settings.gradle
└── projects
    ├── intershop-project
    │   ├── assembly
    │   │   └── gradle
    │   │       └── wrapper
    │   │           ├── gradle-wrapper.jar
    │   │           └── gradle-wrapper.properties
    │   ├── componentset
    │   │   └── gradle
    │   │       └── wrapper
    │   │           ├── gradle-wrapper.jar
    │   │           └── gradle-wrapper.properties
    │   └── developer_home
    │       ├── gradle_environment.bat
    │       └── gradle_environment.sh
    └── oracleDriver
        └── p_oracle
            ├── 3rd_oracle
            │   ├── build.gradle
            │   └── staticfiles
            │       ├── cartridge
            │       │   └── lib
            │       └── share
            │           └── system
            │               └── config
            │                   └── cartridges
            ├── build.gradle
            ├── gradle
            │   └── wrapper
            │       ├── gradle-wrapper.jar
            │       └── gradle-wrapper.properties
            ├── gradle.properties
            ├── gradlew
            ├── gradlew.bat
            ├── init.gradle
            └── settings.gradle

The following table shows which task generates which source artifacts. Multiple projects may share the same CI environment. Some source artifacts are global to the CI environment (corporate artifacts), others are project specific (project artifacts).

LineGenerated byProject or corporate artifactUsed inDescription
4-10ciSetupcorporateSetup CI Build For Project ArtifactsConfiguration specific to running the CI builds (snapshot + release) including host-specific configuration
11-19createDeploymentConfigproject Configuration template for running a deployment outside of the CI and development environment, e.g., for demo, integration tests and production
21-32createCorporateDistributioncorporateSetup CI Build For Corporate ArtifactsSee Concept - Gradle Build Tools
33-48createCorporateConfigurationcorporate

Setup CI Build For Corporate Artifacts

See Concept - Gradle Build Tools
50-63createProjectprojectSetup CI Build For Project Artifacts, Cookbook - Gradle Developer Workflow (valid to Gradle Tools 2.7)See Concept - Continuous Delivery Tools, Concept - Gradle Assembly Tools, Concept - Gradle Build Tools
64-84createOracleComponentSetcorporateSetup CI Build For Corporate ArtifactsFor legal reasons Intershop cannot deliver the Oracle JDBC drivers. This component set contains scripts to package Oracle JDBC drivers as component. Once published to the corporate repository, the Oracle JDBC drivers can be handled like any other component.

5.3.4 Examples If Using SVN As VCS

If you use SVN as your VCS you may use the following commands to add your intershop-ci-setup directory and the generated sources to it.

This example assumes that:

  • You work on Linux (Windows commands are similar).
  • The intershop-ci-setup directory is located at /home/developer/intershop-ci/intershop-ci-setup.
  • Your SVN root is http://ciserver/svn/intershop.
  • Your output directory (property setupDirectory in intershop-ci-setup/build.gradle) is /home/developer/intershop-ci/sources and you use the default output directories structure.

Perform the following steps:

  1. Create the following file for a list of common SVN ignores (assumed location is /home/developer/ignore.txt):

    /home/developer/ignore.txt
    build
    .gradle
    bin
    target
  2. Add your intershop-ci-setup-bootstrap directory by executing:

    SVN command
    cd /home/developer/intershop-ci
    
    # Create SVN structure
    svn mkdir --parents http://ciserver/svn/intershop/devops/intershop-ci-setup-bootstrap -m "create path for template project" 
    svn mkdir --parents http://ciserver/svn/intershop/devops/intershop-ci-setup-bootstrap/branches -m "create branches folder"
    svn mkdir --parents http://ciserver/svn/intershop/devops/intershop-ci-setup-bootstrap/tags -m "create tags folder"
     
    # Import sources to SVN
    svn import intershop-ci-setup-bootstrap http://ciserver/svn/intershop/devops/intershop-ci-setup-bootstrap/trunk -m "add intershop-ci-setup configuration"
     
    # Make your local folder a working copy
    svn co --force http://ciserver/svn/intershop/devops/intershop-ci-setup-bootstrap/trunk intershop-ci-setup-bootstrap
    svn revert -R intershop-ci-setup
     
    # Add ignore properties (see ignore.txt)
    svn propset svn:ignore -F /home/developer/ignore.txt intershop-ci-setup
    svn commit intershop-ci-setup-bootstrap -m "add svn ignore"
  3. Add your corporate plugin configuration and corporate distribution by executing:

    SVN Commands
    cd /home/developer/intershop-ci/sources/devops/gradle
    
    # Create SVN structure
    svn mkdir --parents http://ciserver/svn/intershop/devops/gradle -m "create path for Gradle projects"
    svn mkdir --parents http://ciserver/svn/intershop/devops/gradle/corporate-distribution/branches -m "create svn structure"
    svn mkdir --parents http://ciserver/svn/intershop/devops/gradle/corporate-distribution/tags -m "create svn structure"
    svn mkdir --parents http://ciserver/svn/intershop/devops/gradle/corporate-plugins/branches -m "create svn structure"
    svn mkdir --parents http://ciserver/svn/intershop/devops/gradle/corporate-plugins/tags -m "create svn structure"
    
    # Import sources to SVN
    svn import corporate-distribution http://ciserver/svn/intershop/devops/gradle/corporate-distribution/trunk -m "add corporate distribution"
    svn import corporate-plugins http://ciserver/svn/intershop/devops/gradle/corporate-plugins/trunk -m "add multiproject corporate-plugins"
     
    # Make your local folder a working copy
    svn co --force http://ciserver/svn/intershop/devops/gradle/corporate-plugins/trunk corporate-plugins
    svn revert -R corporate-plugins
    
    svn co --force http://ciserver/svn/intershop/devops/gradle/corporate-distribution/trunk corporate-distribution
    svn revert -R corporate-distribution
      
    # Add ignore properties (see ignore.txt) 
    svn propset svn:ignore -F /home/developer/ignore.txt corporate-plugins
    svn commit corporate-plugins -m "add svn ignore"
     
    svn propset svn:ignore -F /home/developer/ignore.txt corporate-distribution
    svn commit corporate-distribution -m "add svn ignore"
  4. Add the project artifacts by executing (assuming project name is 'corp_project'):

    SVN Commands
    cd /home/developer/intershop-ci/sources/projects
    
    # Create SVN structure
    svn mkdir --parents http://ciserver/svn/intershop/projects/corp_project -m "create path for project"
    svn mkdir --parents http://ciserver/svn/intershop/projects/corp_project/branches -m "create svn structure"
    svn mkdir --parents http://ciserver/svn/intershop/projects/corp_project/tags -m "create svn structure"
    
    # Import sources to SVN
    svn import corp_project http://ciserver/svn/intershop/projects/corp_project/trunk -m "add project"
    
    # Make your local folder a working copy
    svn co --force http://ciserver/svn/intershop/projects/corp_project/trunk corp_project
    svn revert -R corp_project 
    
    # Add ignore properties (see ignore.txt) 
    svn propset svn:ignore -F /home/developer/ignore.txt corp_project
    svn commit corp_project -m "add svn ignore"
    
  5. Add the Oracle JDBC Drivers Component Set by executing:

    SVN Commands
    cd /home/developer/intershop-ci/sources/projects/oracleDriver
    
    # Create SVN structure
    svn mkdir --parents http://ciserver/svn/intershop/projects/oracleDriver -m "create path for project oracle driver"
    svn mkdir --parents http://ciserver/svn/intershop/projects/oracleDriver/branches -m "create svn structure"
    svn mkdir --parents http://ciserver/svn/intershop/projects/oracleDriver/tags -m "create svn structure"
    
    # Import sources to SVN
    svn import p_oracle http://ciserver/svn/intershop/projects/oracleDriver/trunk -m "add Oracle JDBC driver project"
    
    # Make your local folder a working copy
    svn co --force http://ciserver/svn/intershop/projects/oracleDriver/trunk .
    svn revert -R .
    
    # Add ignore properties (see ignore.txt)
    svn propset svn:ignore -F /home/developer/ignore.txt .
    svn commit . -m "add svn ignore"
  6. Add demo deployment scripts (assuming projectName is 'corp_project') by executing:

    SVN Commands
    cd /home/developer/intershop-ci/sources/devops/deployments
    
    # Create SVN structure
    svn mkdir --parents http://ciserver/svn/intershop/devops/deployments -m "create path for deployment projects"
    svn mkdir --parents http://ciserver/svn/intershop/devops/deployments/corp_project/branches -m "create svn structure"
    svn mkdir --parents http://ciserver/svn/intershop/devops/deployments/corp_project/tags -m "create svn structure"
     
    # Import sources to SVN
    svn import corp_project http://ciserver/svn/intershop/devops/deployments/corp_project/trunk -m "add deployment configuration"
     
    # Make your local folder a working copy
    svn co --force http://ciserver/svn/intershop/devops/deployments/corp_project/trunk corp_project
    svn revert -R corp_project
    
    # Add ignore properties (see ignore.txt)
    svn propset svn:ignore -F /home/developer/ignore.txt corp_project
    svn commit corp_project -m "add svn ignore"
    
  7. Add CI server user home scripts and host configuration by executing:

    SVN Commands
    cd /home/developer/intershop-ci/sources/devops/ci_server
    
    # Create SVN structure
    svn mkdir --parents http://ciserver/svn/intershop/devops/ci_server/user_home -m "create path for ci_server user home files and directories" 
    svn mkdir --parents http://ciserver/svn/intershop/devops/ci_server/host_configs -m "create path for ci_server host_configs files and directories" 
    svn mkdir --parents http://ciserver/svn/intershop/devops/ci_server/host_configs/branches -m "create svn structure"
    svn mkdir --parents http://ciserver/svn/intershop/devops/ci_server/host_configs/tags -m "create svn structure"
    svn mkdir --parents http://ciserver/svn/intershop/devops/ci_server/user_home/branches -m "create svn structure"
    svn mkdir --parents http://ciserver/svn/intershop/devops/ci_server/user_home/tags -m "create svn structure"
    
    # Import sources to SVN
    svn import user_home http://ciserver/svn/intershop/devops/ci_server/user_home/trunk -m "add GRADLE user home configuration for ci build processes"
    svn import host_configs http://ciserver/svn/intershop/devops/ci_server/host_configs/trunk -m "add host specific configuration for ci assembly process"
    
    # Make your local folder a working copy
    svn co --force http://ciserver/svn/intershop/devops/ci_server/user_home/trunk user_home
    svn revert -R user_home
    svn co --force http://ciserver/svn/intershop/devops/ci_server/host_configs/trunk host_configs
    svn revert -R host_configs
    
    # Add ignore properties (see ignore.txt)
    svn propset svn:ignore -F /home/developer/ignore.txt user_home
    svn commit user_home -m "add svn ignore" 
    svn propset svn:ignore -F /home/developer/ignore.txt host_configs
    svn commit host_configs -m "add svn ignore"

6 Recipe: Set Up Project Based on the Responsive Starter Store

6.1 Problem

Starting with Intershop Commerce Management Suite 7.6  (B2C, B2X) the delivered assemblies no longer include cartridges that define the storefront applications and do not provide the storefront functionality, meaning the view layer with the content model. Instead Intershop provides the Responsive Starter Store. This includes the storefront parts as a source cartridges component set and the according assemblies that can be used as a blue print to set up customer projects.

After creating the project source structure from the Intershop templates it is necessary to fill the project with the specific artifacts, especially those that make up the storefront. The Responsive Starter Store is a good starting point, that provides the functionality, code artifacts and demo content of a standard responsive storefront.

How can a build engineer or developer set up a project based on the Responsive Starter Store?

6.2 Solution

6.2.1 Prerequisites

See the following recipes:

6.2.2 Steps

Note

The following listings contain diff-style where "<DIFF>" leads the listing title.

This describes changes on files where "-String=original line of code;" represents the original line of code that should be replaced with "+String=replaced line of code'".

  1. Download the latest version of the Responsive Starter Store.

  2. Copy the contents of the Responsive Starter Store component set into your projects component set folder.

  3. Set up the initialization cartridges.

  4. Copy the contents of the fitting Responsive Starter Store assembly to your projects assembly folder.

  5. Commit changes to your VCS.

See the following sections for details.

6.2.3 Download the Latest Version of the Responsive Starter Store

Note

Newer versions of the responsive starter store
Find the Responsive Starter Store version and download links for a corresponding Intershop Commerce Management version in the Reference - CI Dependencies of Intershop Commerce Management.

Example: Responsive Starter Store version 1.0.1:

  1. Download the Responsive Starter Store component set (a_responsive).

    Responsive Starter Store - Component Set
    http://<your-reposerver:port>/nexus/content/repositories/ishrepo/com.intershop.public.source/a_responsive/1.0.1/zips/a_responsive-zip-src-1.0.1.zip
  2. Download an assembly of the Responsive Starter Store.

    Either the the assembly for B2C (inspired-b2c), or

    Responsive Starter Store - inSPIRED B2C Assembly
    http://<your-reposerver:port>/nexus/content/repositories/ishrepo/com.intershop.public.source/inspired-b2c/1.0.1/zips/inspired-b2c-zip-src-1.0.1.zip

    The assembly that includes B2C and B2B (inspired-b2x)

    Responsive Starter Store - inSPIRED B2X Assembly
    http://<your-reposerver:port>/nexus/content/repositories/ishrepo/com.intershop.public.source/inspired-b2x/1.0.1/zips/inspired-b2x-zip-src-1.0.1.zip

6.2.4 Copy the Contents of the Responsive Starter Store Component Set into the Projects Component Set Folder

Content Overview of the Responsive Starter Store component set:

a_responsive
├── .settings
├── app_sf_responsive
├── app_sf_responsive_b2b
├── app_sf_responsive_b2c
├── app_sf_responsive_cm
├── app_sf_responsive_costcenter
├── app_sf_responsive_smb
├── as_responsive
├── as_responsive_b2b
├── demo_responsive
├── demo_responsive_b2b
├── demo_responsive_catalog
├── demo_responsive_content
├── demo_responsive_ocst
├── demo_responsive_search
├── .classpath
├── .project
├── build.gradle
├── extbuild.gradle
├── gradle.properties
├── gradlew
├── gradlew.bat
└── settings.gradle
 
 
base storefront cartridge
additional B2B functionality cartridge
base storefront cartridge
base storefront cartridge
additional B2B functionality cartridge
base storefront cartridge
base storefront cartridge
additional B2B functionality cartridge
demo content cartridge
additional B2B demo content cartridge
demo content cartridge
demo content cartridge
demo content cartridge
demo content cartridge
 
 
 
 
 
 
 
 

Place the content into the working copy of your projects component set folder (that was generated via CI bootstrap before). Rest assured that there are no conflicting files in your projects component set folder that otherwise need to be overwritten.

Long path names on Windows

On Windows certain tools have trouble dealing with path names longer than 255 characters. When in doubt, please use 7-Zip 15.12 or newer to uncompress the source packages.

After copying the contents of the a_responsive component set into your projects component set folder some files need to be changed to fit your projects configuration.

  1. Replace "a_responsive" with your projects component set name in the following files:

    <DIFF> settings.gradle
    // define root project name
    -rootProject.name = 'a_responsive'
    +rootProject.name = 'corp_componentset'
    <DIFF> .project
    <projectDescription>
    -   <name>a_responsive</name>
    +   <name>corp_componentset</name>
  2. Replace the "com.intershop.responsive" group with your projects component set group in the following file:

    <DIFF> build.gradle
    description 'Components Responsive Starter Store Applications'
    -group = 'com.intershop.responsive'
    +group = 'com.company'
    
    subprojects {
    -   group = 'com.intershop.responsive'
    +   group = 'com.company'
  3. In the same file delete the repositories definition in the buildscript {}-block (the repositories are set via your corporate distribution):

    <DIFF> build.gradle
    buildscript {
    -   repositories {
    -       maven {
    -           url 'http://nexus/nexus/content/repositories/gradle-releases/'
    -       }
    -   }
    ...
    }
  4. Replace the version number with a version number fitting to your projects component set and change the Commerce Management version in the following file:

    <DIFF> gradle.properties
    # project settings
    -version=1.0.1
    +version=1.0.0
    -filter.com.intershop.assembly.commerce_management_b2x = 7.6.0.0
    +filter.com.intershop.assembly.commerce_management_b2x = 7.6.1.1
  5. Remove not needed cartridges.

    The table gives an overview of the needed cartridges for the different assembly types.

    Assembly TypeStorefront CartridgesDemo/Init Cartridges
    B2C

    app_sf_responsive
    app_sf_responsive_cm
    app_sf_responsive_b2c
    app_sf_responsive_smb
    as_responsive

    demo_responsive
    demo_responsive_catalog
    demo_responsive_content
    demo_responsive_search
    demo_responsive_ocst

    B2X

    B2C Storefront Cartridges +
    app_sf_responsive_b2b

    app_sf_responsive_costcenter
    as_responsive_b2b

    B2C Demo/Init Cartridges +
    demo_responsive_b2b

6.2.5 Setup the Initialization Cartridges

The Responsive Starter Store provides a set of demo cartridges that can be kept as starting point for the projects initialization cartridges or they can be removed completely.

CartridgeDescription
demo_responsiveDemo organization structure for inSPIRED with inTRONICS, inTRONICS Business and Myers
demo_responsive_catalogCatalogs with categories and products for the inSPIRED organization
demo_responsive_contentCMS demo content (including campaigns and promotions) for the inSPIRED organization
demo_responsive_searchSearch demo configuration
demo_responsive_ocstAdditional demo content of the OCST (Contact Center)
demo_responsive_b2bB2B specific content for the inTRONICS Business channel


In case they are used as init cartridges it is probably useful to rename the cartridges from "demo_..." to "init_..." or some other project naming scheme.
The renaming needs to be done for the cartridges folder name and in the .project file of the cartridge. The display name and description of the cartridge can be renamed in the build.gradle.

6.2.6 Copy the Contents of the Fitting Responsive Starter Store Assembly to the Projects Assembly Folder

Content Overview of the Responsive Starter Store assembly:

inspired-[b2c|b2x]
├── src
├── build.gradle
├── extbuild.gradle
├── gradle.properties
├── gradlew
├── gradlew.bat
└── settings.gradle

Place the content into the working copy of your projects assembly folder (that was generated via CI bootstrap before). Rest assured that there are no conflicting files in your projects assembly folder that otherwise need to be overwritten.

With the Responsive Starter Store two different assemblies are provided inspired-b2c and inspired-b2x. Each of the two assemblies can be used as the blue print for the custom project. Its content needs to be copied to your projects assembly folder. After copying some files need to be changed to fit your projects configuration.

  1. Replace "inspired-b2c" or "inspired-b2x" with your projects assembly name in the following file:

    <DIFF> settings.gradle
    -rootProject.name= 'inspired-b2c'
    +rootProject.name= 'corp_assembly'
  2. Replace the "com.intershop.responsive" group with your projects group and the version number fitting to your project in the following file:

    <DIFF> build.gradle
    -group = 'com.intershop.responsive'
    +group = 'com.company'
    -version = '1.0.1'
    +version = '1.0.0'
    
    assembly {
        cartridges {
    
    -       include(*(storefrontCartridges.collect {"com.intershop.responsive:$it"}), in:[development, test, production])
    +       include(*(storefrontCartridges.collect {"com.company:$it"}), in:[development, test, production])
    
    -       include (*(initCartridges.collect {"com.intershop.responsive:$it"}), in: init)
    +       include (*(initCartridges.collect {"com.company:$it"}), in: init)
        }
    }
  3. In the same file delete the repositories definition in the buildscript {}-block (the repositories are set via your corporate distribution):

    <DIFF> build.gradle
    buildscript {
    -   repositories {
    -       maven {
    -           url 'http://nexus/nexus/content/repositories/gradle-releases/'
    -       }
    -   }
    ...
    }
  4. Replace the "com.intershop.responsive" group with your projects group, the component set name "a_responsive" with your projects component set name with the correct version number and change the Commerce Management version in the following file:

    <DIFF> gradle.properties
    # componentsets with the same live cycle
    -filter.com.intershop.responsive.a_responsive = 1.0.1%suffix%
    +filter.com.company.corp_componentset = 1.0.0%suffix%
    -filter.com.intershop.assembly.commerce_management_b2x = 7.6.0.0
    +filter.com.intershop.assembly.commerce_management_b2x = 7.6.1.1

    %suffix% – will be substituted with "-local", "-SNAPSHOT" or ".<BuildNo>" depending on the environment where it is built.

6.2.7 Commit the Changes to the VCS

6.3 Discussion

6.3.1 Different Options of Customizing the Responsive Starter Store

As mentioned earlier the demo cartridges of the Responsive Starter Store can be used as blue print for the projects initialization cartridges. For that they should be renamed and fitted to your projects needs.

Regarding the storefront cartridges of the Responsive Starter Store (app_sf_responsive, app_sf_responsive_cm, ...) there are several options on how you can use and customize them in your projects. Those options are described below.

In any case it might be necessary to not only adapt the app_sf_responsive cartridges but also the application type definitions of as_responsive. If additional project cartridges are added they need to be added to the application type definition too.

6.3.1.1 Use Responsive Starter Store Cartridges As Is

This is the above described approach of copying the relevant storefront cartridges of the Responsive Starter Store to your projects component set.

Any customization can either be done within additional project cartridges, with the possibilities overloading and overwriting provides. This is pretty much the same as before, except for the fact that also the Responsive Starter Store storefront cartridges are part of the project cartridges set.

This fact, that the Responsive Starter Store storefront cartridges are part of your projects sources, leads to the second customization option. It is now possible to make any customization directly in the cartridges that were provided by Intershop since they are now project cartridges. You have pretty much the same control over these sources as an Intershop developer has when developing the Responsive Starter Store. Any parts can be changed or removed or added within the cartridges of the Responsive Starter Store. This gives you for example full control over the provided CMS content model, making it possible to simply add new call parameters where needed or removing not wanted components without the need of using the overwrites mechanism.

Using the Responsive Starter Store storefront cartridges as they are is probably the simplest approach. And in case the customization is done within additional project cartridges this is probably the easiest approach for later migrations to new Responsive Starter Store versions too.

In any case newer versions of the Responsive Starter Store cartridges are not getting updated automatically but will be provided as sources too and it is up to the project to decide if and how they will update their currently used Responsive Starter Store cartridges. Less customization in those cartridges will probably make the update/migration easier.

6.3.1.2 Copy and Rename the Responsive Starter Store Cartridges

The Responsive Starter Store is intended to be a blue print for a storefront project. The provided cartridges can be used the way they are regarding their naming and their content or they can be customized in any way wanted.

In case you want to create several independent application types that are initially based on the Responsive Starter Store or if you want to create your own storefront cartridge set based on the Responsive Starter Store but with your own naming conventions, you have to copy and rename the app_sf_responsive cartridges to transform them them into your own cartridges.

For this you basically have to search for all references to app_sf_responsive and alike in the Responsive Starter Store cartridges and replace all with your own cartridge naming. This goes for references in the content model files (*.pagelet2), in pipelines, in ISML templates and configuration files.

Also the CMS import files located in sites folder of demo_responsive_content needs to be adapted to the new naming.

After this everything should work as before but with the new naming.

6.3.1.3 Use Own Storefront Cartridges

In case you want to use or create your own storefront cartridge set that is totally different from the Responsive Starter Store you can also remove the app_sf_responsive cartridges completely from your project component set. In that case you might only use the component set configuration files of the Responsive Starter Store but not the cartridges to set up your project.

You would build your storefront only on the basic storefront functionality that is provided by the Commerce Management assembly you are using. Basically this gives you only the storefront functionality of the sld_ch_sf_base cartridge. The whole storefront view layer would need to be provided by your own cartridges.

7 Recipe: Setup CI Build for Corporate Artifacts

7.1 Problem

Multiple projects may share the same CI environment. Some artifacts are required once for the CI environment and independently of projects (corporate artifacts).

Which steps are necessary to provide these artifacts for all other processes?

7.1.1 References

7.2 Solution

7.2.1 Prerequisites

See the following recipes:

7.2.2 Steps

  1. Provide host-specific configuration for deployments during assembly build.
  2. Set up the following automated processes on your CI Server:
    1. Process for building the corporate distribution
    2. Process for building the corporate plugin configuration
    3. Process for building the Oracle JDBC Drivers Component Set
  3. Trigger the processes a first time in the order they were set up, so the gradle user home is prepared and artifacts are uploaded to your artifact repository server.

See the following sections for details.

Example

 See the discussion for an example using Jenkins and SVN.

7.2.3 Provide Host-specific Configuration

Provide host-specific configuration for deployments during assembly build:

  1. Checkout the host configuration (<sources>/devops/ci_server/host_configs/<hostname>) from your VCS.
  2. Adjust the environment.properties file.
    Depending on the operating system it is necessary to specify the user settings in this file. Furthermore, it is necessary to define the database connection parameter and used ports and interfaces.
  3. Commit the changes to your VCS.
  4. Duplicate the folder <hostname> for each remote-CI agent and repeat the three steps above.

7.2.4 Process for Building Corporate Distribution

Set up an automated process for building the corporate distribution. Configure the process to:

  1. Check out the sources of the corporate distribution from your VCS ( <sources>/devops/gradle/corporate-distribution ) to the folder corporate-distribution.
  2. Execute the following commands:

    Publish corporate distribution (Linux)
    cd corporate-distribution
    sh ./gradlew clean publish -PRepoUserLogin=<deploy user login> -PRepoUserPasswd=<deploy user password>
    Publish corporate distribution (Windows)
    cd corporate-distribution
    gradlew clean publish -PRepoUserLogin=<deploy user login> -PRepoUserPasswd=<deploy user password>

    This process should be triggered only manually (e.g., not automatically by polling the VCS), see discussion.

7.2.5 Process for Building Corporate Plugin Configuration

Set up an automated process for building the corporate plugin configuration. Configure the process to:

  1. Check out the sources of the corporate plugin configuration from your VCS (<sources>/devops/gradle/corporate-plugins) to the folder corporate-plugins.
  2. Execute the following commands: 

    Publish corporate plugin configuration (Linux)
    cd corporate-plugins
    sh ./gradlew clean publish -PRepoUserLogin=<deploy user login> -PRepoUserPasswd=<deploy user password>
    Publish corporate plugin configuration (Windows)
    cd corporate-plugins
    gradlew clean publish -PRepoUserLogin=<deploy user login> -PRepoUserPasswd=<deploy user password>

    This process should be triggered automatically upon detecting changes in the VCS.

7.2.6 Process for Building the Oracle JDBC Drivers Component Set

Set up an automated process for building the Oracle JDBC drivers component set. Configure the process to:

  1. Check out the sources of the component set from the VCS (<sources>/projects/oracleDriver/p_oracle) to the folder p_oracle.
  2. Execute the following commands:

    Publish release of p_oracle (Linux)
    cd p_oracle
    sh ./gradlew clean publish -I init.gradle -PRepoUserLogin=<deploy user login> -PRepoUserPasswd=<deploy user password>
    Publish release of p_oracle (Windows)
    cd p_oracle
    gradlew clean publish -I init.gradle -PRepoUserLogin=<deploy user login> -PRepoUserPasswd=<deploy user password>

    This process should be triggered only manually (e.g., not automatically by polling the VCS), see discussion.

7.3 Discussion

7.3.1 Gradle User Home

The build processes on the CI server need a GRADLE_USER_HOME with a configuration for publishing to a remote repository (see Publishing to a Remote Repository in Cookbook - Gradle Build Tools). Intershop provides a template of the necessary files for publishing to a remote repository: init.gradle and gradle.properties. These files are located in <sources>/devops/ci_server/user_home.

The CI job will copy these files to a directory on the machine outside CI standard directories. So all other processes can use this directory for the GRADLE_USER_HOME. Gradle uses this directory also for caching of artifacts. There is no automatic deletion functionality for this kind of caches. Therefore, it is necessary to run periodically an automatic clean up process.

7.3.1.1 Example Script for Automated Process Managing the Gradle User Home

The following scripts replaces the contents of <BASE_GRADLE_USER_HOME>/gradle_user_home by the contents of the folder user_home. Processes holding handles to <BASE_GRADLE_USER_HOME>/gradle_user_home will keep using the same directory.

Replace <build number> by a number provided by the CI server that grows in subsequent executions of the automated process. (Jenkins and Bamboo offer a variable ${BUILD_NUMBER} for that purpose.)

Create gradle user home/Purge old ones (Linux)
export HOMEDIR=${BASE_GRADLE_USER_HOME}/gradle_user_home
mkdir ${HOMEDIR}_<build number>
cp ${WORKSPACE}/user_home/* ${HOMEDIR}_<build number>
if [ -d ${HOMEDIR} ]; then
    mv ${HOMEDIR} ${HOMEDIR}_before_<build number> \
    && mv ${HOMEDIR}_<build number> ${HOMEDIR} \
    && rm -rf ${HOMEDIR}_before_<build number>
else 
    mv ${HOMEDIR}_<build number> ${HOMEDIR} 
fi
Create gradle user home/Purge old ones (Windows)
set HOMEDIR=%BASE_GRADLE_USER_HOME%\gradle_user_home
mkdir "%HOMEDIR%_<build number>"
copy "user_home" "%HOMEDIR%_<build number>"
IF EXIST "%HOMEDIR%" (
    move /Y "%HOMEDIR%" "%HOMEDIR%_before_<build number>"
    move /Y "%HOMEDIR%_<build number>" "%HOMEDIR%"
    rmdir /s /Q "%HOMEDIR%_before_<build number>"
) ELSE (
    move /Y "%HOMEDIR%_<build number>" "%HOMEDIR%"
)
LineDescription
1Create a variable for the directory.
BASE_GRADLE_USER_HOME is a global variable. See "Directory for Additional Build Directories" in Recipe - Configure CI Server.
2Create a new directory.
3Copy the files from source directory to new directory.
5 - 7The old home directory will be replaced with the new one. If this step finished successfully, the old directory will be removed.
9Runs only if there is a home directory before.

7.3.2 Rebuilding  Corporate Artifacts

Before rebuilding the corporate distribution always increase the version number (located in <sources>/devops/gradle/corporate-distribution/gradle.properties). After rebuilding, change the version number in the wrapper configuration (gradle-wrapper.properties) of all your projects.

Changes in the corporate plugin configuration are effective 24h hours after rebuilding at latest (this is Gradles default cache timeout). You can force it by passing --refresh-dependencies to gradlew when starting the Gradle process accessing the configuration.

Before rebuilding the Oracle JDBC Drivers component set always increase the version or build number (located in <sources>/projects/oracleDriver/p_oracle/gradle.properties).

7.3.3 Example Using Jenkins and SVN

This example assumes that:

7.3.3.1 Build Item for Corporate Distribution

Create a new build item Corporate Distribution:

ParameterValueDescription
Item namecorporate-distributionThe item name is also used for the directory path. Do not use spaces in this configuration.
Build typeBuild a free-style software project 
Advanced Project Options  
Display NameCorporate DistributionThis is used for the item name on the web frontend
DescriptionCreates a corporate distribution for the main project 
Source Code ManagementSubversion 
Repository URLhttp://ciserver/svn/intershop/devops/gradle/corporate-distribution/trunkURL of the corporate distribution project
- CredentialsUsername / PasswordSelect Add and configure this parameter in the dialog.
Local module directorycorporate-distributionThis is the target path in the workspace directory for the the Subversion checkout.
- Repository browserCollabnetType of a repository browser
- - URLhttp://ciserver/viewvc/intershop/devops/gradle/corporate-distribution/trunkURL for the repository browser of this project
Build Triggers[no selection] 
Build Environment  
- Global passwords[selected]Global configured password will be used
BuildExecute Shell / Execute Windows batch commandOn a Linux server use Execute Shell. On Windows it is necessary to select Execute Windows batch command.
- Command (Linux)
Publish corporate distribution
export GRADLE_USER_HOME=${WORKSPACE}/gradle_user_home
cd corporate-distribution
sh ./gradlew clean publish \
-PRepoUserLogin=$DEPLOY_USER_NAME \
-PRepoUserPasswd=$DEPLOY_USER_PASSWORD

Before the build runs the old build should be deleted. Therefore, the target clean is also called.
Username and password of the repository deploy user must be specified with the project properties RepoUserLogin and RepoUserPasswd.
 

DEPLOY_USER_PASSWORD was specified as global password.

WORKSPACE is a environment-varialbe of Jenkins.

- Command (Windows)
Publish corporate distribution
set GRADLE_USER_HOME=%WORKSPACE%/gradle_user_home
cd corporate-distribution
gradlew clean publish ^
-PRepoUserLogin=%DEPLOY_USER_NAME% ^
-PRepoUserPasswd=%DEPLOY_USER_PASSWORD%
Jenkins build log
...
:clean UP-TO-DATE
:downloadGradle
:customGradleDistribution
:generateDescriptorFileForIvyPublication
:publishIvyPublicationToIvy Distribution RepositoryRepository
Upload http://ciserver/nexus/content/repositories/distributions/gradle-dist/corporate_gradle_2.7/1.0.0.0/corporate_gradle_2.7-1.0.0.0-bin.zip
Upload http://ciserver/nexus/content/repositories/distributions/gradle-dist/corporate_gradle_2.7/1.0.0.0/corporate_gradle_2.7-1.0.0.0-bin.zip.sha1
Upload http://ciserver/nexus/content/repositories/distributions/gradle-dist/corporate_gradle_2.7/1.0.0.0/ivy.xml
Upload http://ciserver/nexus/content/repositories/distributions/gradle-dist/corporate_gradle_2.7/1.0.0.0/ivy.xml.sha1
:publish

BUILD SUCCESSFUL

Total time: 18.544 secs
Finished: SUCCESS

Corporate distribution in the artifact repository if using Nexus:

7.3.3.2 Build Item for Corporate Plugin Configuration

Create a new build item Corporate Plugin Configuration:

ParameterValueDescription
Item namecorporate-pluginsThe item name is also used for the directory path. Do not use spaces in this configuration.
Build typeBuild a free-style software project 
Advanced Project Options  
Display NameCorporate Plugin ConfigurationThis is used for the item name on the web frontend
DescriptionCreates a corporate plugin configuration for the main project 
Source Code ManagementSubversion 
Repository URLhttp://ciserver/svn/intershop/devops/gradle/corporate-plugins/trunkURL of the corporate distribution project
- CredentialsUsername / PasswordSelect Add and configure this parameter in the dialog.
Local module directorycorporate-pluginsThis is the target path in the workspace directory for the the Subversion checkout.
- Repository browserCollabnetType of a repository browser
- - URLhttp://ciserver/viewvc/intershop/devops/gradle/corporate-plugins/trunkURL for the repository browser of this project
Build Triggers[no selection] 
Build Environment  
- Global passwords[selected]Global configured password will be used
BuildExecute Shell / Execute Windows batch commandOn a Linux server use Execute Shell. On Windows it is necessary to select Execute Windows batch command.
- Command (Linux)
Publish corporate plugin configuration
export GRADLE_USER_HOME=${WORKSPACE}/gradle_user_home
cd corporate-plugins
sh ./gradlew clean publish \
-PRepoUserLogin=$DEPLOY_USER_NAME \
-PRepoUserPasswd=$DEPLOY_USER_PASSWORD

Before the build runs the old build should be deleted. Therefore, the target clean is also called.
Username and password of the repository deploy user must be specified with the project properties RepoUserLogin and RepoUserPasswd.

DEPLOY_USER_PASSWORD was specified as global password.

DEPLOY_USER_NAME was specified as global property.

WORKSPACE is a environment-varialbe of Jenkins.

- Command (Windows)
Publish corporate plugin configuration
set GRADLE_USER_HOME=%WORKSPACE%\gradle_user_home
cd corporate-plugins
gradlew clean publish ^
-PRepoUserLogin=%DEPLOY_USER_NAME% ^
-PRepoUserPasswd=%DEPLOY_USER_PASSWORD%
Log output
...
:corporate-configuration:clean UP-TO-DATE
:corporate-configuration:generateDescriptorFileForIvyPublication
:corporate-configuration:compileJava UP-TO-DATE
:corporate-configuration:compileGroovy UP-TO-DATE
:corporate-configuration:processResources
:corporate-configuration:classes

:corporate-configuration:jar

:corporate-configuration:publishIvyPublicationToIvyRepository

Upload http://ciserver/nexus/content/repositories/gradle/com.intershop.build.gradle/corporate-configuration/1.0.0.0.20140702141143/jars/corporate-configuration-jar-1.0.0.0.20140702141143.jar
Upload http://ciserver/nexus/content/repositories/gradle/com.intershop.build.gradle/corporate-configuration/1.0.0.0.20140702141143/jars/corporate-configuration-jar-1.0.0.0.20140702141143.jar.sha1
Upload http://ciserver/nexus/content/repositories/gradle/com.intershop.build.gradle/corporate-configuration/1.0.0.0.20140702141143/ivys/ivy-1.0.0.0.20140702141143.xml
Upload http://ciserver/nexus/content/repositories/gradle/com.intershop.build.gradle/corporate-configuration/1.0.0.0.20140702141143/ivys/ivy-1.0.0.0.20140702141143.xml.sha1
:corporate-configuration:publish

BUILD SUCCESSFUL

Total time: 11.662 secs

Finished: SUCCESS

Corporate plugin configuration in the artifact repository if using Nexus:

7.3.3.3 Build Item for Gradle User Home

Create a new build item Gradle User Home:

ParameterValueDescription
Item namegradle_user_homeThe item name is also used for the directory path. Do not use spaces in this configuration.
Build typeBuild a free-style software project 
Prepare an environment for the run[selected] 
Keep Jenkins Environment Variables[selected] 
Keep Jenkins Build Variables[selected] 
Advanced Project Options  
Display NameGRADLE_USER_HOMEThis is used for the item name on the web frontend
DescriptionCreates/Updates GRADLE_USER_HOME folder 
Source Code ManagementSubversion 
Repository URLhttp://ciserver/svn/intershop/devops/ci_server/user_home/trunkURL of the CI server home configuration files.
- CredentialsUsername / PasswordSelect Add and configure this parameter in the dialog.
Local module directoryuser_homeThis is the target path in the workspace directory for the the Subversion checkout.
- Repository browserCollabnetType of a repository browser
- - URLhttp://ciserver/viewvc/intershop/devops/ci_server/user_home/trunkURL for the repository browser of this project
Build TriggersBuild periodically 
- ScheduleH 3 * * 2-6The job will run every night from Tuesday to Saturday
BuildExecute Shell / Execute Windows batch commandOn a Linux server use Execute Shell. On Windows it is necessary to select Execute Windows batch command.
- Command (Linux)
Create gradle user home/Purge old ones
export HOMEDIR=${BASE_GRADLE_USER_HOME}/gradle_user_home
mkdir ${HOMEDIR}_${BUILD_NUMBER}
cp ${WORKSPACE}/user_home/* ${HOMEDIR}_${BUILD_NUMBER}
if [ -d ${HOMEDIR} ]; then
    mv ${HOMEDIR} ${HOMEDIR}_before_${BUILD_NUMBER} \
    && mv ${HOMEDIR}_${BUILD_NUMBER} ${HOMEDIR} \
    && rm -rf ${HOMEDIR}_before_${BUILD_NUMBER}
else 
    mv ${HOMEDIR}_${BUILD_NUMBER} ${HOMEDIR} 
fi

Creates the gradle_user_home-directory.

BUILD_NUMBER is a environment-variable of the Jenkins-item.

WORKSPACE is a environment-varialbe of Jenkins.

- Command (Windows)
Create gradle user home/Purge old ones
set HOMEDIR=%BASE_GRADLE_USER_HOME%\gradle_user_home
mkdir "%HOMEDIR%_%BUILD_NUMBER%"
copy "%WORKSPACE%\user_home" "%HOMEDIR%_%BUILD_NUMBER%"
IF EXIST "%HOMEDIR%" (
    move /Y "%HOMEDIR%" "%HOMEDIR%_before_%BUILD_NUMBER%"
    move /Y "%HOMEDIR%_%BUILD_NUMBER%" "%HOMEDIR%"
    rmdir /s /Q "%HOMEDIR%_before_%BUILD_NUMBER%"
) ELSE (
    move /Y "%HOMEDIR%_%BUILD_NUMBER%" "%HOMEDIR%"
)

7.3.3.4 Build Item for Oracle JDBC Driver Component Set

Create a new build item Oracle JDBC Driver component set:

ParameterValueDescription
Item namep_oracleThe item name is also used for the directory path. Do not use spaces in this configuration.
Advanced Project Options  
Display NameOracle JDBC Driver component setThis is used for the item name on the web frontend
Build typeBuild a free-style software project 
DescriptionCreate Oracle JDBC Driver component set 
Source Code ManagementSubversion 
Repository URLhttp://ciserver/svn/intershop/projects/oracleDriver/trunk/p_oracleURL of the corporate distribution project
- CredentialsUsername / PasswordSelect Add and configure this parameter in the dialog.
Local module directoryp_oracleThis is the target path in the workspace directory for the the Subversion checkout.
- Repository browserCollabnetType of a repository browser
- - URLhttp://ciserver/svn/intershop/projects/oracleDriver/trunk/p_oracleURL for the repository browser of this project
Build TriggersPoll SCM 
- ScheduleH/15 * * * *The build is triggered manually only. Before the build can be restarted, the module must be deleted from the artifact repository.
Build Environment  
Inject passwords to the build as environment variables[selected]Password will be defined for the environment
-- Global passwords[selected]Global configured password will be used
BuildExecute Shell / Execute Windows batch commandOn a Linux server use Execute Shell. On Windows it is necessary to select Execute Windows batch command.
- Command (Linux)
Publish release of p_oracle
export GRADLE_USER_HOME=${WORKSPACE}/gradle_user_home
cd p_oracle
sh ./gradlew clean publish -I init.gradle \
-PRepoUserLogin=$DEPLOY_USER_NAME \
-PRepoUserPasswd=$DEPLOY_USER_PASSWORD

Before the build runs the old build should be deleted. Therefore the target clean is also called.
Username and password of the repository deploy user must be specified with the project properties RepoUserLogin and RepoUserPasswd. It is also possible to run this command manually.

DEPLOY_USER_PASSWORD was specified as global password.

DEPLOY_USER_NAME was specified as global property.

WORKSPACE is a environment-varialbe of Jenkins.

- Command (Windows)
Publish release of p_oracle
set GRADLE_USER_HOME=%WORKSPACE%/gradle_user_home
cd p_oracle
gradlew clean publish -I init.gradle ^
-PRepoUserLogin=%DEPLOY_USER_NAME% ^
-PRepoUserPasswd=%DEPLOY_USER_PASSWORD%
Log output
...
:3rd_oracle:clean
:3rd_oracle:zipShare UP-TO-DATE
:3rd_oracle:generateDescriptorFileForIvyPublication

:3rd_oracle:publishIvyPublicationToReleasesRepository

Upload http://ciserver/nexus/content/repositories/releases/com.intershop/3rd_oracle/12.1.0.2.0.0/jars/ucp-jar-12.1.0.2.0.0.jar
Upload http://ciserver/nexus/content/repositories/releases/com.intershop/3rd_oracle/12.1.0.2.0.0/jars/ucp-jar-12.1.0.2.0.0.jar.sha1
Upload http://ciserver/nexus/content/repositories/releases/com.intershop/3rd_oracle/12.1.0.2.0.0/jars/ons-jar-12.1.0.2.0.0.jar
Upload http://ciserver/nexus/content/repositories/releases/com.intershop/3rd_oracle/12.1.0.2.0.0/jars/ons-jar-12.1.0.2.0.0.jar.sha1
Upload http://ciserver/nexus/content/repositories/releases/com.intershop/3rd_oracle/12.1.0.2.0.0/jars/ojdbc6-jar-12.1.0.2.0.0.jar
Upload http://ciserver/nexus/content/repositories/releases/com.intershop/3rd_oracle/12.1.0.2.0.0/jars/ojdbc6-jar-12.1.0.2.0.0.jar.sha1
Upload http://ciserver/nexus/content/repositories/releases/com.intershop/3rd_oracle/12.1.0.2.0.0/ivys/ivy-12.1.0.2.0.0.xml
Upload http://ciserver/nexus/content/repositories/releases/com.intershop/3rd_oracle/12.1.0.2.0.0/ivys/ivy-12.1.0.2.0.0.xml.sha1
:3rd_oracle:publish
:generateDescriptorFileForIvyPublication
Download http://ciserver/nexus/content/repositories/releases/com.intershop/3rd_oracle/12.1.0.2.0.0/ivys/ivy-12.1.0.2.0.0.xml

:publishIvyPublicationToReleasesRepository
Upload http://ciserver/nexus/content/repositories/releases/com.intershop.set/p_oracle/12.1.0.2.0.0/ivys/ivy-12.1.0.2.0.0.xml
Upload http://ciserver/nexus/content/repositories/releases/com.intershop.set/p_oracle/12.1.0.2.0.0/ivys/ivy-12.1.0.2.0.0.xml.sha1
:publish

BUILD SUCCESSFUL


Total time: 13.974 secs

Finished: SUCCESS

Oracle JDBC Driver component set in the artifact repository if using Nexus:

7.3.3.5 Jenkins Overview with All Build Items

8 Recipe: Setup CI Build for Project Artifacts

8.1 Problem

After changes on the code base it is a good practice to run an integration build with all automatic tests to provide fast feedback for the developer team. Furthermore, it is necessary to check the impact of the changes on the initialization of the database.

How can a administrator or developer set up a continuous integration process for all project artifacts? 

8.2 Solution

8.2.1 Prerequisites

See the following recipes:

8.2.2 Steps

  1. Set up the following automated processes on your CI Server:
    1. Process for building a snapshot of the component set.
    2. Process for building a snapshot of the assembly.
    3. Process for building a release of the component set and assembly.
  2. Trigger the processes a first time in the order they were set up, so the Gradle user home is prepared and artifacts are uploaded to your artifact repository server.

Example

See the discussion for an example using Jenkins and SVN.

8.2.3 Process for Building a Snapshot of the Component Set

Set up up an automated process for a snapshot build of the component set. Configure the process to:

  1. Check out the sources of the component set from the VCS (<sources>/projects/<project name>/<component set name>) to the folder <component set name>.
  2. Execute the following commands:

    Publish snapshot of component set (Linux)
    export GRADLE_USER_HOME=${BASE_GRADLE_USER_HOME}/gradle_user_home
    cd <component set name>
    sh ./gradlew clean build publish --refresh-dependencies -PRepoUserLogin=<deploy user login> -PRepoUserPasswd=<deploy user password>
    Publish snapshot of component set (Windows)
    set GRADLE_USER_HOME=%BASE_GRADLE_USER_HOME%\gradle_user_home
    cd <component set name>
    gradlew clean build publish --refresh-dependencies -PRepoUserLogin=<deploy user login> -PRepoUserPasswd=<deploy user password>

    This process should be triggered automatically upon detecting changes in the VCS.

8.2.4 Process for Building a Snapshot of the Assembly

Set up an automated process for a snapshot build of the assembly. Configure the process to:

  1. Check out the sources of the assembly set from the VCS (<sources>/projects/<project name>/<assembly name>) to the folder <assembly name>.
  2. Check out the host configuration (<sources>/devops/ci_server/host_configs/) into the folder host_configs.
  3. Execute the following commands:

    Publish snapshot of assembly (Linux)
    export COMPONENT_SET=<organization of component set>.<name of component set>
    export COMPONENT_SET_VERSION=<version of component set without build number/snapshot suffix>
    export GRADLE_USER_HOME=${BASE_GRADLE_USER_HOME}/gradle_user_home
    export ENVIRONMENT_PROPS=<workspace directory>/host_configs/<name of executing host>/environment.properties
    
    cd <assembly name>
    sh ./gradlew clean dbinit build publish --refresh-dependencies -PreleaseWithDump=true -PRepoUserLogin=<deploy user login> \
    -PRepoUserPasswd=<deploy user password> -PbuildEnvironmentProperties=${ENVIRONMENT_PROPS} -PtestEnvironmentProperties=${ENVIRONMENT_PROPS}
    Publish snapshot of assembly (Windows)
    set COMPONENT_SET=<organization of component set>.<name of component set>
    set COMPONENT_SET_VERSION=<version of component set without build number/snapshot suffix>
    set GRADLE_USER_HOME=%BASE_GRADLE_USER_HOME%\gradle_user_home
    set ENVIRONMENT_PROPS=<workspace directory>\host_configs\<name of executing host>\environment.properties
    
    cd <assembly name>
    gradlew clean dbinit build publish --refresh-dependencies -PreleaseWithDump=true -PRepoUserLogin=<deploy user login> ^
    -PRepoUserPasswd=<deploy user password> -PbuildEnvironmentProperties="%ENVIRONMENT_PROPS%" -PtestEnvironmentProperties="%ENVIRONMENT_PROPS%"

    This process should be triggered automatically upon detecting changes in the VCS.

8.2.5 Process for Building a Release of the Component Set and Assembly

Set up an automated process for a release build of component set and assembly. Configure the process to:

  1. Check out the sources of the component set from the VCS (<sources>/projects/<project name>/<component set name>) to the folder <component set name>.
  2. Check out the sources of the assembly set from the  VCS (<sources>/projects/<project name>/<assembly name>) to the folder <assembly name>.
  3. Check out the host configuration (<sources>/devops/ci_server/host_configs/) into the folder host_configs.
  4. Execute the following commands:

    Publish release of component set and assembly (Linux)
    export BUILD_NO=.`date +%Y%m%d%H%M%S`
    export GRADLE_USER_HOME=${BASE_GRADLE_USER_HOME}/gradle_user_home
    export COMPONENT_SET=<organization of component set>.<name of component set>
    export COMPONENT_SET_VERSION=<version of component set without build number/snapshot suffix>
    export ENVIRONMENT_PROPS=<workspace directory>/host_configs/<name of executing host>/environment.properties
    export PUBLISH_STATUS=release
    
    cd <component set name>
    sh ./gradlew clean build publish --refresh-dependencies -PRepoUserLogin=<deploy user login> -PRepoUserPasswd=<deploy user password>
      
    cd <assembly name>
    sh ./gradlew clean dbinit build publish --refresh-dependencies -PreleaseWithDump=true -PRepoUserLogin=<deploy user login> \
    -PRepoUserPasswd=<deploy user password> -PbuildEnvironmentProperties=${ENVIRONMENT_PROPS} -PtestEnvironmentProperties=${ENVIRONMENT_PROPS}
    Publish release of component set and assembly (Windows)
    set BUILD_NO=.%date:~6,4%%date:~3,2%%date:~0,2%%time:~0,2%%time:~3,2%%time:~6,2%
    set GRADLE_USER_HOME=%BASE_GRADLE_USER_HOME%\gradle_user_home
    set COMPONENT_SET=<organization of component set>.<name of component set>
    set COMPONENT_SET_VERSION=<version of component set without build number/snapshot suffix>
    set ENVIRONMENT_PROPS=<workspace directory>\host_configs\<name of executing host>\environment.properties
    set PUBLISH_STATUS=release
    
    cd <component set name>
    gradlew clean build publish --refresh-dependencies -PRepoUserLogin=<deploy user login> -PRepoUserPasswd=<deploy user password>
      
    cd <assembly name>
    gradlew clean dbinit build publish --refresh-dependencies -PreleaseWithDump=true -PRepoUserLogin=<deploy user login> ^
    -PRepoUserPasswd=<deploy user password> -PbuildEnvironmentProperties="%ENVIRONMENT_PROPS%" -PtestEnvironmentProperties="%ENVIRONMENT_PROPS%"

    Note

    The output of the BUILD_NO expression depends on the date/time configuration of the Windows system. The example-expression is based on the following output of %date% and %time%:

    + echo %date%
    31.12.2015
    + echo %time%
    08:45:40,20

    So the expression (%date:~6,4%%date:~3,2%%date:~0,2%%time:~0,2%%time:~3,2%%time:~6,2%) returns the value "20151231084540".

    This process should be triggered only manually (e.g., not automatically by polling the VCS).

8.3 Discussion

8.3.1 Assembly Builds

Long running processes with higher resource requirements (dbinit, dbmigrate)  should run on remote agents. That makes monitoring of this processes more efficient.

It is possible to disable the test of the database during the assembly build. To do so remove the parameter releaseWithDump from the script or change the value of this parameter to false. This reduces the time for the build, but the database initialization is not tested. Furthermore, the database export file will not be published and developers can not use this artifact for faster initialization of the database of their environment.

8.3.2 Snapshot Build

For a fast developer feedback we suggest to use snapshot builds. Project component set and assembly will be build in different steps.

  • The build of the component set ensures, that there are no compile errors and all JUnit tests were finished successfully after a code change. It takes some minutes (< 10 min).
  • The assembly build ensures, that the database initialization (dbinit) or database migration (dbmigrate) runs without any errors. The assembly build runs with a database initialization, therefore, it takes much more time than the build of the component set. The run time depends on the size of the database objects.

The result of these build steps represents a temporary status. It is not possible to deploy this on a server for further testing (see section Release Build below).

The feedback time after changes depends on the performance of the build machine, the database and the amount of data (products, catalogs, customer) for the database initialization.

For the snapshot process it is possible to publish the artifacts to a local file based repository like a developer, but this makes it impossible to distribute processes over different machines.

It is also possible to use only the release process and instead of this to run the release process more often. In this case more artifacts will be published to the artifact repository and a clean up process is necessary.

8.3.3 Release Build 

The release build runs time triggered once per day or per week or on request. All project artifacts of this release should use the same version and build number. Cartridges, components and assembly use the same version. This kind of build can be used for the installation on the integration machine or for long running manual tests.

It is possible to run a temporary installation based on the snapshot build, but it is not possible to reproduce this build result. The result of a release build is reproducible. For the storage of this more build information the init.script contains a task writeBuildInformation. This information can be used for hand over to the next processes or for build documentation. It is also possible to extend this script.

8.3.4 Example Using Jenkins and SVN

This example assumes that:

  • Your SVN root is http://ciserver/svn/intershop.
  • You use the default output directories structure.
  • The name of the component set is corp_componentset.
  • The name of the assembly is corporate_assembly.

8.3.5 Build Item for Snapshot of Component Set

Create a new build item Component set corp_componentset:

ParameterValueDescription
Item namecomponentset_webshopThe item name is also used for the directory path. Do not use spaces in this configuration.
Build typeBuild a free-style software project 
Prepare an environment for the run[selected] 
Keep Jenkins Environment Variables[selected] 
Keep Jenkins Build Variables[selected] 
Advanced Project Options  
Display NameComponent Set corp_componentsetThis is used for the item name on the web front end.
DescriptionCreate a snapshot build of the component set. 
Source Code ManagementSubversion 
Repository URL

http://ciserver/svn/intershop/projects/corp_project/trunk/corp_componentset 

URL of the project component set
- CredentialsUsername / PasswordSelect Add and configure this parameter in the dialog.
Local module directorycorp_componentsetThis is the target path in the workspace directory for the the Subversion checkout.
- Repository browserCollabnetType of a repository browser
- - URLhttp://ciserver/viewvc/intershop/projects/corp_project/trunk/corp_componentset URL for the repository browser of this project
Build TriggersPoll SCM 
- ScheduleH/5 * * * *The job will check the VCS for changes every 5 minutes.
Build Environment  
Inject passwords to the build as environment variables[selected]Password will be defined for the environment.
-- Global passwords[selected]Global configured password will be used.
BuildExecute Shell / Execute Windows batch commandOn a Linux server use Execute Shell. On Windows it is necessary to select Execute Windows batch command.
- Command (Linux)
Publish snapshot of component set
export GRADLE_USER_HOME=${BASE_GRADLE_USER_HOME}/gradle_user_home
cd corp_componentset
sh ./gradlew clean build publish --refresh-dependencies \
-PRepoUserLogin=$DEPLOY_USER_NAME \
-PRepoUserPasswd=$DEPLOY_USER_PASSWORD

Provide the GRADLE_USER_HOME (Global property BASE_GRADLE_USER_HOME is used).

Change to the component set folder.
Run the publish target of the build with project parameter for deploy user and deploy user password.

DEPLOY_USER_PASSWORD was specified as global password.

DEPLOY_USER_NAME was specified as global property.

- Command (Windows)
Publish snapshot of component set
set GRADLE_USER_HOME=${BASE_GRADLE_USER_HOME}/gradle_user_home
cd corp_componentset
gradlew clean build publish --refresh-dependencies ^
-PRepoUserLogin=%DEPLOY_USER_NAME% ^
-PRepoUserPasswd=%DEPLOY_USER_PASSWORD%
Log output
Building in workspace /var/lib/jenkins/jobs/componentset_webshop/workspace
Updating http://ciserver/svn/intershop/projects/corp_project/trunk/corp_componentset at revision '2014-07-16T16:04:20.333 +0200'
At revision 43
no change for http://ciserver/svn/intershop/projects/corp_project/trunk/corp_componentset since the previous build
[workspace] $ /bin/sh -xe /tmp/hudson2340063613026622480.sh
+ export GRADLE_USER_HOME=/opt/intershop/build/gradle_user_home
+ GRADLE_USER_HOME=/opt/intershop/build/gradle_user_home
+ cd corp_componentset
+ sh ./gradlew clean build publish --refresh-dependencies -PRepoUserLogin=admin -PRepoUserPasswd=****
Downloading http://ciserver/nexus/content/repositories/distributions/gradle-dist/corporate_gradle_1.8/1.0.0.0/corporate_gradle_1.8-1.0.0.0-bin.zip
..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Unzipping /opt/intershop/build/gradle_user_home/wrapper/dists/corporate_gradle_1.8-1.0.0.0-bin/4kqi2io27c0nscl7hdce2ddnqr/corporate_gradle_1.8-1.0.0.0-bin.zip to /opt/intershop/build/gradle_user_home/wrapper/dists/corporate_gradle_1.8-1.0.0.0-bin/4kqi2io27c0nscl7hdce2ddnqr
Set executable permissions for: /opt/intershop/build/gradle_user_home/wrapper/dists/corporate_gradle_1.8-1.0.0.0-bin/4kqi2io27c0nscl7hdce2ddnqr/gradle-1.8/bin/gradle
Download http://ciserver/nexus/content/repositories/gradle/com.intershop.build.gradle/corporate-configuration/1.0.0.0.20140712231610/ivys/ivy-1.0.0.0.20140712231610.xml
Download http://ciserver/nexus/content/repositories/ishreleases/com.intershop.build.gradle/corporate-plugin/1.2.0.0.20140711152402/ivys/ivy-1.2.0.0.20140711152402.xml
Download http://ciserver/nexus/content/repositories/ishreleases/com.intershop.build.gradle/versioning/1.2.0.0.20140711152402/ivys/ivy-1.2.0.0.20140711152402.xml
...
Download http://ciserver/nexus/content/repositories/ishreleases/com.intershop/sld_system_app/7.5.0.0.20140711190011/jars/sld_system_app-jar-7.5.0.0.20140711190011.jar
...
:publish

BUILD SUCCESSFUL

Total time: 49.762 secs
Finished: SUCCESS

Jenkins overview page:

For a better overview of the existing projects in Jenkins a separate tab Intershop Project was created.

8.3.6 Build Item for Snapshot of Assembly

Create a new build item Assembly corporate_assembly:

Parameter
Value
Description
Item nameassembly_webshopThe item name is also used for the directory path. Do not use spaces in this configuration.
Build typeBuild a free-style software project 
Prepare an environment for the run[selected] 
Keep Jenkins Environment Variables[selected] 
Keep Jenkins Build Variables[selected] 
- Properties Content

COMPONENT_SET=com.corporate.corp_componentset
COMPONENT_SET_VERSION=1.0.0

 
Advanced Project Options  
Display NameAssembly corporate_assemblyThis is used for the item name on the web front end.
DescriptionCreate a snapshot build of the assembly. 
Source Code ManagementSubversion 
Repository URLhttp://ciserver/svn/intershop/projects/corp_project/trunk/corp_assemblyURL of the project assembly.
- CredentialsUsername / PasswordSelect Add and configure this parameter in the dialog.
Local module directorycorporate_assemblyThis is the target path in the work space directory for the the Subversion checkout.
Repository URLhttp://ciserver/svn/intershop/devops/ci_server/host_configs/trunkURL of the configurations for the CI server.
- CredentialsUsername / PasswordSelect Add and configure this parameter in the dialog.
Local module directoryhost_configsThis is the folder with names of hosts on which the assembly build can be executed.
- Repository browserCollabnet 
- - URLhttp://ciserver/viewvc/intershop 
Build TriggersPoll SCM 
- ScheduleH/5 * * * *The job will check the VCS for changes every 5 minutes.
Build Environment  
Inject passwords to the build as environment variables[selected]Password will be defined for the environment.
-- Global passwords[selected]Global configured password will be used.
BuildExecute Shell / Execute Windows batch commandOn a Linux server use Execute Shell. On Windows it is necessary to select Execute Windows batch command.

Command (Linux)

Publish snapshot of assembly
export GRADLE_USER_HOME=${BASE_GRADLE_USER_HOME}/gradle_user_home
export ENVIRONMENT_PROPS="${WORKSPACE}/host_configs/ciserver/environment.properties"


# JENKINS-19222 (NODE_NAME is not defined on master)
if [ "${NODE_NAME}" != "master" -a "${NODE_NAME}" != "" ]
then
    export ENVIRONMENT_PROPS="${WORKSPACE}/host_configs/${NODE_NAME}/environment.properties"
fi

cd corporate_assembly
sh ./gradlew clean dbinit build publish --refresh-dependencies -PreleaseWithDump=true -PRepoUserLogin=$DEPLOY_USER_NAME \
-PRepoUserPasswd=$DEPLOY_USER_PASSWORD -PbuildEnvironmentProperties=${ENVIRONMENT_PROPS} -PtestEnvironmentProperties=${ENVIRONMENT_PROPS}

Command (Windows)

Publish snapshot of assembly
set GRADLE_USER_HOME=%BASE_GRADLE_USER_HOME}%\gradle_user_home
set ENVIRONMENT_PROPS=%WORKSPACE%\host_configs\ciserver\environment.properties

:: JENKINS-19222 (NODE_NAME is not defined on master)
IF NOT "%NODE_NAME%" == "master" IF NOT "%NODE_NAME%" == "" (
    set ENVIRONMENT_PROPS=%WORKSPACE%\host_configs\%NODE_NAME%\environment.properties
)

cd corporate_assembly
gradlew clean dbinit build publish --refresh-dependencies -PreleaseWithDump=true -PRepoUserLogin=%DEPLOY_USER_NAME% ^
-PRepoUserPasswd=%DEPLOY_USER_PASSWORD% -PbuildEnvironmentProperties="%ENVIRONMENT_PROPS%" -PtestEnvironmentProperties="%ENVIRONMENT_PROPS%"
LineComment
1Provide the GRADLE_USER_HOME (Global property BASE_GRADLE_USER_HOME is used).
2

The default value of ENVIRONMENT_PROPS is set to the configuration of the build machine.

WORKSPACE is a environment-varialbe of Jenkins.

5If the process runs on a remote agent, the configuration should be changed.
10Change the working directory to the assembly folder.
11

Run gradle including the following tasks:

buildCreate and test all artifacts.

publish

Publish artifacts of the assembly to the repository. The database export will be also published.

The following project properties are defined:

-PreleaseWithDump=true
The database export will be published with the result of the project.
Therefore, a deployment and dbinit runs before.
-PRepoUserLogin=$DEPLOY_USER_NAME
-PRepoUserPasswd=$DEPLOY_USER_PASSWORD
User with password for the publishing on the artifact repository
-PbuildEnvironmentProperties=${ENVIRONMENT_PROPS} 
-PtestEnvironmentProperties=${ENVIRONMENT_PROPS}
Settings for the deployment and database initialization
Log output
...
:packEncryptionConfig
:publishIvyPublicationToBuildRepository SKIPPED
:publishIvyPublicationToSnapshotsRepository

...
:publishIvyPublicationToTestRepository SKIPPED
:publishTestAssemblyPublicationToBuildRepository SKIPPED
:publishTestAssemblyPublicationToSnapshotsRepository SKIPPED
:publish

BUILD SUCCESSFUL

Total time: 40 mins 2.232 secs

Finished: SUCCESS

Jenkins overview page:

For a better overview of the existing projects in Jenkins a separate tab Intershop Project was created. The duration of the assembly build depends on the performance of the build machine and the database.

8.3.7 Build Item for Release Build

Create a new build item Release webshop:

Parameter
Value
Description
Item namewebshop_releaseThe item name is also used for the directory path. Do not use spaces in this configuration.
Build typeBuild a free-style software project 
Prepare an environment for the run[selected] 
Keep Jenkins Environment Variables[selected] 
Keep Jenkins Build Variables[selected] 
- Properties Content

PUBLISH_STATUS=release

COMPONENT_SET=com.corporate.corp_componentset
COMPONENT_SET_VERSION=1.0.0

 
Advanced Project Options  
Display NameRelease "webshop"This is used for the item name on the web front end.
DescriptionCreate the release of all artifacts of the "webshop" project. 
Source Code ManagementSubversion 
Repository URL

http://ciserver/svn/intershop/projects/corp_project/trunk/corp_componentset

URL of the project component set.
- CredentialsUsername / PasswordSelect Add and configure this parameter in the dialog.
Local module directorycorp_componentsetThis is the target path in the workspace directory for the the Subversion checkout.
Source Code ManagementSubversion 
Repository URLhttp://ciserver/svn/intershop/projects/corp_project/trunk/corp_assemblyURL of the project assembly
- CredentialsUsername / PasswordSelect user and password.
Local module directorycorporate_assemblyThis is the target path in the workspace directory for the the Subversion checkout.
Repository URLhttp://ciserver/svn/intershop/devops/ci_server/host_configs/trunkURL of the configurations for the CI server.
- CredentialsUsername / PasswordSelect Add and configure this parameter in the dialog.
Local module directoryhost_configsThis is the folder with names of hosts on which the assembly build can be executed.
- Repository browserCollabnet 
- - URLhttp://ciserver/viewvc/intershop 
Build Triggers[empty] 
Build Environment  
Inject passwords to the build as environment variables[selected]Password will be defined for the environment.
-- Global passwords[selected]Global configured password will be used.

The following build steps must be configured:

8.3.7.1 Build-Step 1

Linux:

Execute shell - Create build no
NEW_BUILD_NO=`date +%Y%m%d%H%M%S`
echo "BUILD_NO = .${NEW_BUILD_NO}" > ${WORKSPACE}/buildno.properties

Windows:

Execute Windows batch command - Create build no
set NEW_BUILD_NO=%date:~6,4%%date:~3,2%%date:~0,2%%time:~0,2%%time:~3,2%%time:~6,2%
echo BUILD_NO = .%NEW_BUILD_NO% > %WORKSPACE%\buildno.properties

8.3.7.2 Build-Step 2

both:

Inject environment variables - Set build no for environment
Properties File Path${WORKSPACE}/buildno.properties

8.3.7.3 Build-Step 3

Linux:

Execute shell - Publish project component set
export GRADLE_USER_HOME=${BASE_GRADLE_USER_HOME}/gradle_user_home
cd corp_componentset
sh ./gradlew clean publish -PRepoUserLogin=$DEPLOY_USER_NAME -PRepoUserPasswd=$DEPLOY_USER_PASSWORD

Windows:

Execute Windows batch command - Publish project component set
set GRADLE_USER_HOME=${BASE_GRADLE_USER_HOME}\gradle_user_home
cd corp_componentset
gradlew clean publish -PRepoUserLogin=%DEPLOY_USER_NAME% -PRepoUserPasswd=%DEPLOY_USER_PASSWORD%

8.3.7.4 Build-Step 4

Linux:

Execute shell - Publish project assembly
export GRADLE_USER_HOME=${BASE_GRADLE_USER_HOME}/gradle_user_home
export ENVIRONMENT_PROPS="${WORKSPACE}/host_configs/ciserver/environment.properties"

# JENKINS-19222 (NODE_NAME is not defined on master)
if [ "${NODE_NAME}" != "master" -a "${NODE_NAME}" != "" ]
then
    export ENVIRONMENT_PROPS="${WORKSPACE}/host_configs/${NODE_NAME}/environment.properties"
fi

cd corporate_assembly
sh ./gradlew clean dbinit build publish --refresh-dependencies -PreleaseWithDump=true -PRepoUserLogin=$DEPLOY_USER_NAME \
-PRepoUserPasswd=$DEPLOY_USER_PASSWORD -PbuildEnvironmentProperties=${ENVIRONMENT_PROPS} -PtestEnvironmentProperties=${ENVIRONMENT_PROPS}

Windows:

Execute Windows batch command - Publish project assembly
set GRADLE_USER_HOME=${BASE_GRADLE_USER_HOME}\gradle_user_home
set ENVIRONMENT_PROPS=%WORKSPACE%\host_configs\ciserver\environment.properties

:: JENKINS-19222 (NODE_NAME is not defined on master)
IF NOT "%NODE_NAME%" == "master" IF NOT "%NODE_NAME%" == "" (
    set ENVIRONMENT_PROPS=%WORKSPACE%\host_configs\%NODE_NAME%\environment.properties
)

cd corporate_assembly
gradlew clean dbinit build publish --refresh-dependencies -PreleaseWithDump=true -PRepoUserLogin=%DEPLOY_USER_NAME% ^
-PRepoUserPasswd=%DEPLOY_USER_PASSWORD% -PbuildEnvironmentProperties="%ENVIRONMENT_PROPS%" -PtestEnvironmentProperties="%ENVIRONMENT_PROPS%"
Log output
...
:generateDescriptorFileForIvyPublication
:packEncryptionConfig
:publishIvyPublicationToBuildRepository SKIPPED
:publishIvyPublicationToReleasesRepository

...
:publishIvyPublicationToTestRepository SKIPPED
:publishTestAssemblyPublicationToBuildRepository SKIPPED
:publishTestAssemblyPublicationToReleasesRepository SKIPPED
:publish

BUILD SUCCESSFUL

Total time: 39 mins 54.281 secs

Finished: SUCCESS

That is the final Jenkins overview page:

9 Additional Recipe: Configuration of a Secured Repository

9.1 Problem

If the repository is accessible over the public internet, it is necessary to configure the security of the repository. This recipe describes a security configuration for Sonatype Nexus.

9.2 Solution

9.2.1 Repository Configuration

Once you configured the basic repository the following configuration on the repository is required.

  1. Change the password of the standard admin account.
    1. Switch to the Profile tab of the user menu.
    2. Click Change Password and provide your own password.
  2. Add Repository Target Privileges for repositories.
    1. Select Privileges.
    2. Select Add | Repository Target Privilege.

       
    3. Add target privileges

      NameDescriptionRepositoryRepository Target
      DistributionsTarget Privilege DistributionsDistributions (Repo)All (Maven 2)
      ComponentsTarget Privilege Components Components (Group)All (Maven 2)
      Maven allTarget Privilege Maven allMaven all (Group)All (Maven 2)
      GradleTarget Privilege GradleGradle (Repo)All (Maven 2)
      ReleasesTarget Privilege Releases Releases (Repo)All (Maven 2)
      Intershop ReleasesTarget Privilege Intershop Releases Intershop Releases (Repo)All (Maven 2)
      SnapshotsTarget Privilege Snapshots Snapshots (Repo)All (Maven 2)
      CentralTarget Privilege CentralCentral (Repo)All (Maven 2)

  3. Create roles for the access to the repositories.
    1. Select Roles.
    2. Select Add | Nexus Role.

      If you use LDAP or an other user database like Atlassian Crowd it is possible to create External Role Mapping instead.
    3. Add the role configuration and finish this configuration with Save.

      Role IDNameRole/Privilege Management
      maven2-distributions-readRole: Distributions (read)Distributions - (read)
      Distributions - (view)
      maven2-components-readRole: Components (read)Components - (read)
      Components - (view)
      maven2-mavenall-readRole: Maven all (Read)Maven all - (read)
      Maven all - (view)
      maven2-snapshots-readRole: Snapshots (read)Snapshots - (read)
      Snapshots - (view)
      maven2-central-readRole: Central (read)Central - (read)
      Central - (view)
      maven2-gradle-fullRole: Gradle (full)Distributions - (create)
      Distributions - (delete)
      Distributions - (read)
      Distributions - (update)
      Distributions - (view)
      Gradle - (create)
      Gradle - (delete)
      Gradle - (read)
      Gradle - (update)
      Gradle - (view)
      maven2-projectbuilds-fullRole: Projectbuilds (full)Releases - (create)
      Releases - (delete)
      Releases - (read)
      Releases - (update)
      Releases - (view)
      Snapshots - (create)
      Snapshots - (delete)
      Snapshots - (read)
      Snapshots - (update)
      Snapshots - (view)
      maven2-ishreleases-fullRole: Intershop Releases (full)Intershop Releases - (create)
      Intershop Releases - (delete)
      Intershop Releases - (read)
      Intershop Releases - (update)
      Intershop Releases - (view)


  4. Add role for project developer.
    1. Select Roles.
    2. Select Add | Nexus Role.
    3. Add the role configuration for a project developer and finish this configuration with Save

      Role IDNameRole/Privilege Management
      nexus-project-developerProject DeveloperNexus Developer Role
      Role: Distributions (read)
      Role: Components (read)
      Role: Maven all (read)
      Role: Snapshots (read)
      Role: Central (read)

      Instead of separate roles for the repositories it also possible to use "Repo: All Maven2 Repositories (View)" and "Repo: All Maven2 Repositories (Read)". 

  5. Add role for project deployment user.
    1. Select Roles.
    2. Select Add | Nexus Role.
    3. Add the role configuration for a project  deployment user and finish this configuration with Save.

      Role IDNameRole/Privilege Management
      nexus-project-deploymentProject Deployment User

      Nexus Deployment Role
      Role: Projectbuilds (full)
      Role: Components (read)
      Role: Maven all (read)
      Project Developer

      The role "Project Developer" is used to ensure that Project Deployment Users have the same repositories available as developers.

  6. Add role for a special deployment user.
    For special artifacts (Intershop releases, Gradle artifacts (distribution, plugins) it is possible to create an special deployment user.

    1. Select Roles.
    2. Select Add | Nexus Role.
    1. Add the role configuration for a project  deployment user and finish this configuration with Save.

      Role IDNameRole/Privilege Management
      nexus-intershop-deploymentIntershop Deployment UserNexus Deployment Role
      Role: Gradle (full)
      Role: Intershop Releases (full) 
  7. Remove read privileges for all repositories from anonymous user.
    1. Select Users.
    2. Select user anonymous.
    3. Select Repo: All Repositories (Read).
    4. Select Remove.
    5. Save the configuration.

  8. Add read privileges for the distributions repository to anonymous user.
    1. Select Users.
    2. Select user anonymous.
    1. Select Add | Role: Distributions (read) from the list of roles.
       
  9. Add deployment users.
    1. Configure deployment user for corporate artifacts (Corporate Distribution, Corporate Configuration Plugin, Intershop releases).
      1. Select Users.
      2. Select Add | Nexus User.
      3. Specify the configuration and finish the configuration with Save.

        PropertyValue
        User ID (Login)corp_deployment
        Email<admin account of this user>
        StatusActive
        Password<password for all processes>
        Role ManagementIntershop Deployment User
    2. Configure deployment user for project artifacts.
      1. Select Users.
      2. Select Add | Nexus User.
      3. Specify the configuration and finish the configuration with Save.

        PropertyValue
        User ID (Login)prj_deployment
        Email<admin account of this user>
        StatusActive
        Password<password for all processes>
        Role ManagmentProject Deployment User
  10. Create and configure developer accounts.
    1. Select Users.
    2. Select Add | Nexus User.
    3. Specify the configuration and finish the configuration with Save.

      PropertyValue
      User ID (Login)<developer account login>
      Email<email of the user>
      StatusActive
      Password<password>
      Role ManagmentProject Developer

9.2.2 Build and Deployment Configuration

It is necessary to extend the local gradle.properties or the environment settings with the necessary credentials.

Build and Deployment on CI Server

Process 
  • Import Intershop Delivery
  • Setup CI Build For Corporate Artifacts
    • Corporate Distribution
    • Corporate Plugin Configuration

For this task it is necessary to use the deployment user for corporate artifacts. Username and password is part of the command line.

Example
gradlew clean publish -PRepoUserLogin=corp_deployment -PRepoUserPasswd=<passwd>
  • Oracle JDBC Drivers Component Set

The result of the build will be published to the releases repository. Therefore, it is necessary to use the project deployment user.

Example
// this configuration is necessary for the read access
export REPO_USER=prj_deployment
export REPO_USER_PASSWD=<deploy user password>
 
gradlew clean publish -I init.gradle -PRepoUserLogin=prj_deployment -PRepoUserPasswd=<passwd> 
  • Component Set Build
  • Assembly Build

The result of the build will be published to the releases or snapshots repository. Therefore, it is necessary to use the project deployment user. But this process needs also read access to other repositories.

Example
// this configuration is necessary for the read access
export REPO_USER=prj_deployment
export REPO_USER_PASSWD=<deploy user password>


// Project properties will be used for the write access 
gradlew build publish -PRepoUserLogin=prj_deployment -PRepoUserPasswd=<deploy user password> 

It is also possible to specify Java system properties.

Example
// System properties are used for read access
// Project properties will be used for the write access 
gradlew build publish -DrepoUser=prj_deployment -DrepoUserPasswd=<deploy user password> -PRepoUserLogin=prj_deployment -PRepoUserPasswd=<deploy user password> 

Build and Deployment on Development Machine

The developer can also specify username and password on the command line or as an environment variable. But this is not really user friendly.

It is possible to add the login credentials in the gradle.properties of Gradle user home.

<GRADLE_USER_HOME>/gradle.properties
systemProp.repoUser = <developer account login>
systemProp.repoUserPasswd = <developer account password>

9.3 Discussion

User names and passwords are sent to the repository server for authentication. For security reasons the transport should be secured. Therefore, it is necessary to configure https for the server communication. It is recommended to use a certificate created by a certification authority. For private certificates it is necessary to import this in the key-store of the used JDK. This must be done on all machines, which have access to the repository - developer, build, test and production machines also. 

User name  and password of users must be stored as plain text on the developer machine as well on the build machine. This login and password can be used also for the login on the repository server directly. So it is also possible to change the password on this way. Sonatype provides a professional version of Nexus with costs. This version can use special tokens for the build and deployment access to the repository. It is not possible to log in to the server with this credentials.

Disclaimer

The information provided in the Knowledge Base may not be applicable to all systems and situations. Intershop Communications will not be liable to any party for any direct or indirect damages resulting from the use of the Customer Support section of the Intershop Corporate Web site, including, without limitation, any lost profits, business interruption, loss of programs or other data on your information handling system.

Customer Support
Knowledge Base
Product Resources
Support Tickets