Related Documents
Document Properties
KbidA25599
Last Modified03-Feb-2020
Added to KB12-Mar-2014
Public AccessEveryone
StatusOnline
Doc TypeGuidelines, Concepts & Cookbooks
Product
  • Gradle Tools
  • ICM 7.6
  • ICM 7.7

Cookbook - Gradle Developer Workflow (valid to Gradle Tools 2.7)

1 Introduction

This cookbook gives practical advise on how to develop using Gradle and Intershop build and deployment tools. The target audience are developers of Intershop customer projects and partners. Knowledge of Concept - Continuous Delivery Tools is assumed.The cookbook presents a directory setup and workflow, that is optimized for a single system component set and assembly.

 

Names in angle brackets

Upper case names in angle brackets like <COMPONENTSET> are references to environment variables. Replace them by the proper OS-dependent syntax, e.g., %COMPONENTSET% on Windows or $COMPONENTSET on Linux. The environment variables are set by the gradle_environment.bat or gradle_environment.sh scripts (see recipes Setup a Development Environment and Run Gradle Tasks).

1.1 Preconditions

Before a single developer can set up an environment, a few things have to be set up first for the team. The required team setup is out of scope of this cookbook. Such information is included in:

1.2 Glossary


Phrase
Meaning
Version Control System (VCS)

Also known as source control, source code management systems (SCM), or revision control systems (RCS). VCS is a mechanism for keeping multiple versions of your files, so that when you modify a file you can still access the previous revisions.

Artifact Repository

Place, where build and package software components are located. Provide a common interface to a dependency management system.

Code AnalysisProcess to analyze source code to calculate metrics, find bugs, etc.
Continuous Delivery PipelineSometimes called Deployment Pipeline, describes the stages, which code artifacts runs through source to production system.
System ComponentA software package of different code artifacts and files, that have to be deployed together.
System Component SetIs a container for system components, that needs to be build and branched together.
AssemblyAn assembly references one or more system components residing in the same or a configured artifact repository in order to deploy or deliver them together.
Build ProcessCompiles and packages files and code artifacts from a source project to deployable artifacts.
Publish ProcessThe process which transfers the deployable artifacts to a configured artifact repository.
Assembly ProcessThis process combines several system components to an assembly.
Deployment ProcessThis process extracts files and code artifacts from an artifact repository and applies the configuration.
Project Gradle DistributionThis is a customized Gradle distribution with the preconfigured artifact repositories and Gradle plugins.
Gradle PluginA Gradle plugin packages up reusable pieces of build logic, which can be used across many different projects and builds.
Project Gradle PluginThis is a Gradle plugin which contains special corporate respectively project settings.
Corporate PluginThe term is used as a synonym for Project Gradle Plugin.
Gradle Extension ObjectJava Bean compliant class holding configurations for Gradle plugins.
Gradle WrapperThe Gradle Wrapper is the preferred way of starting a Gradle build. The wrapper is a batch script on Windows, and a shell script for other operating systems. When you start a Gradle build via the wrapper, Gradle will be automatically downloaded and used to run the build. See for more information The Gradle Wrapper in the Gradle documentation (2.11, 2.7, 2.3, 2.0, 1.8)
Intershop ClusterA number of hosts of different types serving an Intershop 7.
Cluster NodeOne separately deployable part of an Intershop cluster. A host can run multiple nodes of one Intershop cluster.

2 Recipe: Set Up a Development Environment

2.1 Problem

I want to develop on Intershop 7.4 CI or later. How get I a Gradle-based development environment?

2.2 Solution

Ensure the preconditions are fulfilled before starting the development.

Setting up an development environments consists of the following steps:

  1. Set up a Gradle user home.
  2. Check out system component set and assembly using your VCS.
  3. Build system component set and publish to local repository.
  4. Deploy a server.
  5. Initialize the database.
  6. Set up an Intershop Studio workspace.

The following steps provide you with a jump start without further ado. See discussion for further details.

Since command line syntax on Windows and Linux slightly differ, the steps are provided once for Windows and once for Linux. Except for the command line syntax both are equivalent.

2.2.1 Windows

#ActionExplanation
1

Check if the minimum System Requirements are fulfilled.

  • Check the Software Requirements
  • Provide a valid License File

Additionally to the general requirements:

  • Install a client for your VCS
  • Install Intershop Studio
For gradle_environment.bat created with Gradle Tools >= 2.0: The gradle_environment.bat (see next step) by default expects the Intershop Studio executable to be installed at C:\intershop\IntershopStudio\IntershopStudio.exe. If your Intershop Studio is located elsewhere adapt the line defining openStudio in gradle_environment.bat accordingly.
2

For Gradle Tools version 1.0-1.1:

Create an empty directory to contain your development environment.

Check out preconfigured Gradle user home ( gradle_user_home ) from your VCS to the subdirectory gradle_user_home.

The partition/drive should have enough space/speed to hold your sources, a deployed server and the size of a deployed server caches for remote repositories.

 

2

For Gradle Tools version >= 2.0:

Create an empty directory to contain your development environment.

Check out the developer home from your VCS to the subdirectory developer_home (as generated at <sources>/devops/developer/<project name> in Cookbook - Setup CI Infrastructure (valid to GradleTools 2.1)).

The partition/drive should have enough space/speed to hold your sources, a deployed server and the size of a deployed server caches for remote repositories.

3

Open a console, change into the checked out folder and execute gradle_environment.bat.

This sets several environment variables, including GRADLE_USER_HOME, COMPONENTSET, ASSEMBLY, SERVER, REPO inside this console. We will refer to this console as developer console. Keep using it through the remaining steps.
4

Check out your system component set to the folder pointed to by %COMPONENTSET% using your VCS.

 
5Check out your assembly to the folder pointed to by %ASSEMBLY% using your VCS. 
6

Go to <COMPONENTSET> (run cdSet)* and run gradlew useSetAsFilter publish

This builds your system component sets. The results can be found at <COMPONENTSET>\<component>\target(replaces IS_TARGET). It also packages the results and publishes them to your local repository at <REPO>. This is necessary to offer a unified deployment across development and production.

The useSetAsFilter task ensures that your locally built cartridges are deployed, like instead of released versions created by the continuous integration server. It does so by changing <GRADLE_USER_HOME>\gradle.properties.

7

Go to <ASSEMBLY> (run cdAssembly)* and run gradlew

This creates the file <ASSEMBLY>\environment.properties.sample. It is a template for all deployment configuration required to deploy a server for development.
8

Copy <ASSEMBLY>\environment.properties.sample to <ASSEMBLY>\environment.properties and fill in the gaps/make necessary adjustments.

At least adjust the following sections: #License and #Database configurations.

9Go to <ASSEMBLY> (run cdAssembly)* and run gradlew deployServer

This deploys the assembly as your test server into <SERVER>. Note that IS_HOME now resides at <SERVER>\local and IS_SHARE at <SERVER>\share.

10

Run as adminstrator/with root privileges (as requested in the console output): <SERVER>\postInstall\installServices.bat

This creates the services. The services run under your user. You will be prompted for your password.
11

Go to <ASSEMBLY> (run cdAssembly)* and

  • run gradlew dbinit -PskipDeployment to init your database by running DBInit or
  • run gradlew importDump to import your database using a dump generated by the CI server.

See the Recipe: Initialize the Database for details.

12

At the developer console go to your Intershop Studio installation and run IntershopStudio -data %WORKSPACE%

(or run openStudio)*

It is important to start Intershop Studio from a developer console with all environment variables set. No more initial configuration in the preference menu is necessary. Using Gradle requires Intershop Studio version >= 4.3.4.

You may run into a problem when opening the preference dialog the first time, see the box Issue with Intershop Studio Preferences below.

13Import the system component set using the wizard Component Set projects into Workspace

See the section Intershop Studio on how to import a component set.

* Commands provided by gradle_environment.bat created with Gradle Tools >= 2.0

Starting services and the application server are out of scope of this document.

2.2.2 Linux

#ActionExplanation
1

Check if the minimum System Requirements are fulfilled.

  • Check the Software Requirements
  • Provide a valid License File

Additionally to the general requirements:

  • Install a client for your VCS
  • Install Intershop Studio
For gradle_environment.sh created with Gradle Tools >= 2.0: The gradle_environment.sh (see next step) by default expects the Intershop Studio executable to be installed at /opt/intershop/IntershopStudio/IntershopStudio. If your Intershop Studio is located elsewhere adapt the line defining openStudio in gradle_environment.sh accordingly.
2

For Gradle Tools version 1.0-1.1:

Create an empty directory to contain your development environment.

Check out preconfigured Gradle user home (gradle_user_home) from your VCS to the subdirectory gradle_user_home.

The partition/drive should have enough space/speed to hold your sources, a deployed server and the size of a deployed server caches for remote repositories.

 

2

For Gradle Tools version >= 2.0:

Create an empty directory to contain your development environment.

Check out the developer home from your VCS to the subdirectory developer_home (as generated at <sources>/devops/developer/<project name> in Cookbook - Setup CI Infrastructure (valid to GradleTools 2.1)).

The partition/drive should have enough space/speed to hold your sources, a deployed server and the size of a deployed server caches for remote repositories.

3

Open a console, change into the checked out folder and execute:
source ./ gradle_environment.sh

This sets several environment variables, including GRADLE_USER_HOME, COMPONENTSET, ASSEMBLY, SERVER, REPO inside this console. We will refer to this console as developer console. Keep using it through the remaining steps.
4

Check out your system component set to the folder pointed to by $COMPONENTSET using your VCS.

 
5Check out your assembly to the folder pointed to by $ASSEMBLY using your VCS. 
6

Go to<COMPONENTSET> (run cdSet)* and run ./gradlew useSetAsFilter publish

This builds your system component sets. The results can be found at <COMPONENTSET>/<component>/target(replaces IS_TARGET). It also packages the results and publish them to your local repository at <REPO>. This is necessary to offer a unified deployment across development and production.

The useSetAsFilter task ensures that your locally built cartridges are deployed, like instead of released versions created by the continuous integration server. It does so by changing <GRADLE_USER_HOME>/gradle.properties.

7Go to <ASSEMBLY> (run cdAssembly)* and run ./gradlew
This creates the file <ASSEMBLY>/environment.properties.sample. It is a template for all deployment configuration required to deploy a server for development.
8

Copy <ASSEMBLY>/environment.properties.sample to <ASSEMBLY>/environment.properties and fill in the gaps/make necessary adjustments.

At least adjust the following sections: #License and #Database configurations.

9Go to <ASSEMBLY>(run cdAssembly)* and run ./gradlew deployServer

This deploys the assembly as your test server into <SERVER>. Note that IS_HOME now resides at <SERVER>/local and IS_SHARE at <SERVER>/share.

10

Run as adminstrator/with root privileges (as requested in the console output): <SERVER>/postInstall/installServices.sh

This creates the init.d-scripts. The services will run under your user.
11

Go to <ASSEMBLY> (run cdAssembly )* and

  • run ./gradlew dbinit -PskipDeployment to init your database by running DBInit or
  • run ./gradlew importDump to import your database using a dump generated by the CI server.

See the Recipe: Initialize the Database for details.

12

At the developer console go to your Intershop Studio installation and run IntershopStudio -data $WORKSPACE

(or run openStudio)*

It is important to start Intershop Studio from a developer console with all environment variables set. No more initial configuration in the preference menu is necessary. Using Gradle requires Intershop Studio version >= 4.3.4.

You may run into a problem when opening the preference dialog the first time, see the box Issue with Intershop Studio Preferences below.

13Import the system component set using the wizard Component Set projects into Workspace

See the section Intershop Studio on how to import a component set.

* Commands provided by gradle_environment.bat created with Gradle Tools >= 2.0

Starting services and the application server are out of scope of this document.

2.3 Discussion

2.3.1 Gradle User Home and Developer Home

Gradle tools version 1.0-1.1:

The Gradle user home (<GRADLE_USER_HOME>) is a folder that stores all files associated with a certain development context. It contains:

  • An environment script gradle_environment.bat/ gradle_environment.sh
  • Developer specific configuration in gradle.properties
  • Caches for downloads from remote artifact repositories

Gradle tools version >= 2.0:

The developer home is a folder containing an environment script (gradle_environment.bat/ gradle_environment.sh), see next section. The developer home is checked into the VCS.

The Gradle user home (<GRADLE_USER_HOME>) is a folder that stores temporary files associated with a certain development context. It contains:

  • Developer specific configuration in gradle.properties
  • Caches for downloads from remote artifact repositories

The Gradle user home is not checked into the VCS.

2.3.2 Developer Console

When setting up your CI environment a script gradle_environment.bat | gradle_environment.sh is generated. This script configures the directory layout you work with, by setting a few environment variables.

We refer to a console with these environment variables in place as a developer console. Run all gradlew commands and Intershop Studio from a developer console.

To start a developer console:

  1. Open a console or use an existing one
  2. Go to
    1. Gradle tools 1.0-1.1: Gradle user home folder
    2. Gradle tools >= 2.0: Developer home folder
  3. Run
    1. gradle_environment.bat on Windows or
    2. source ./gradle_environment.sh on Linux

The following table lists the most important environment locations provided from gradle_environment.bat | gradle_environment.sh represented by environment variables.

NameContentsEnvironment variable
Gradle user homeDeveloper configuration in gradle.properties, as well as caches for downloads from remote repositoriesGRADLE_USER_HOME
System Component set directoryVCS working copy of component set sourcesCOMPONENTSET
Assembly directoryVCS working copy of assembly sourcesASSEMBLY
Server directoryServer deployed for manual and automatic testing (contains both IS_HOME and IS_SHARE as subfolders)SERVER
Javadoc directoryJavadoc of cartridges (excluding own cartridges)JAVADOC
Intershop Studio workspaceWorkspace dataWORKSPACE
Local repositoryLocally published components – used internally as a deployment sourceREPO

These environment variables are partly for your convenience and partly to configure Gradle/Intershop Studio.

with Gradle Tools >= 2.0:

The environment script provides the following alias commands:

CommandFunction
cdRepoChange directory to local repository
cdSetChange directory to project component set
cdAssemblyChange directory to project assembly
cdServerChange directory to server directory
cdDocChange directory to java doc directory
openStudioOpen Intershop Studio

2.3.3 Build, Publish and Repositories

Running gradlew publish in <COMPONENTSET> actually consists of of two steps, first build, then publish. It is necessary to not only build components, but also to publish them to a repository (e.g., copy them to a folder with predefined structure) for two reasons:

  1. Gradle needs to find dependencies for compiling Java. Gradle knows two kinds of dependencies:
    1. External dependencies are all dependencies across component set boundaries (e.g., the cartridge bc_foundation in the component set p_business to the cartridge core in the component set p_core). They are denoted by compile group: 'com.intershop', name: '<name>' in a component's build.gradle file. They must be resolved from (a local or remote) repository.
    2. Dependencies inside a component set (e.g., core to isml, both inside p_core) are called project dependencies and are denoted by compile project(':<name>'). (Project dependencies resolved directly from the target directories.)
  2. The deployment needs to find all components to deploy in a (local or remote) repository.

The useSetAsFilter task of the component set adds a filter property in the <GRADLE_USER_HOME>/gradle.properties:

filter.<organization of component set>.<name of component set> = <version>-local

This property ensures that the assembly process uses local versions for all cartridges from this component set. (So you deploy local versions of these in the initial and later re-deployments.) See also Concept - Gradle Build Tools | Configuring Dependency Versions.

2.3.4 Initial Deployment

The developer deployment first generates a <SERVER>/settings.gradle file from the environment.properties configuration. Based on this file it then runs standard deployment procedure as describes in Concept - Gradle Deployment Tools specifying the host type all, e.g., deploying all host types of the assembly. As deployment source it uses the repositories declared in your corporate plugin as well as your local repository at <REPO>.

Besides being based on the same deployment scripts, the developer deployment is also tailored for development in a few ways:

    • The properties IS_SOURCE in <SERVER>/local/intershop.properties and cartridges.source in <SERVER>/share/system/cartridges/cartridgelist.properties. This ensures that the application server uses the cartridges checkout directory to load static artifacts and classes.

    • Deploy development.properties from <ASSEMBLY>/development.properties to <SERVER>/share/system/config/cluster/development.properties . This way reload and no-preload behavior is preconfigured for the application server.
    • Deploy all cartridges exclusively to the assembly environment development (i.e., that are not included in the production environment), like the dev_lilith cartridge .
    • Open debug ports in tomcat.bat / tomcat.sh and dbinit.bat / dbinit.sh as configured in <ASSEMBLY>/environment.properties. The configuration of debug ports depends on the configuration of the source directories in the gradle.properties in the developer home.

2.3.5 Post-install Scripts and Services

The post-install scripts perform all deployment operations for which administrator / root privileges are required. Currently this is only the creation of services. The developer deployment uses the current user as deployment user and– on Windows– also as user for executing the deployed applications / services. Therefore, you have to enter your password for every service to be created.

After the post-install scripts, running services / applications is completely up to you and works mostly as before. If starting any service fails on Windows, please refer to the Recipe (Windows) Starting a Service Fails in the Cookbook - Gradle Deployment Tools (7.4 CI - ICM 7.7)

2.3.6 Intershop Studio

Intershop Studio will find your sources, the deployed server etc. by relying on the environment variables in a developer console. Therefore, start Intershop Studio from a developer console. No more initial configuration in the preference menu is necessary.

To import your system component set into the workspace:

  1. From the menu select File | Import...
    The Import dialog opens.
  2. Choose Intershop Studio | Component Set Projects into Workspace (Gradle) wizard and click Next.
    The Import Component Set (Gradle) wizard is started.
  3. Browse... for the directory of the system component set.
  4. Click Build Model.
  5. Select the desired projects (cartridges).
  6. Click Select Required to also select all required projects.
  7. Click Finish.
  8. Wait till background actions have finished.

To import your assembly into the workspace, use the same steps, but instead select the import wizard Gradle | Gradle Project. I.e:

  1. From the menu select File | Import...
    The Import dialog opens.
  2. Choose Gradle | Gradle Project wizard and click Next.
    The Import Gradle Project wizard is started.
  3. Browse... for the directory of the assembly.
  4. Click Build Model.
  5. Click Select All.
  6. Click Finish.
  7. Wait till background actions have finished.

Note

Issue in Intershop Studio Preferences

There is a known issue in Intershop Studio 4.3.4:

When accessing the preference dialog (menu Window | Preferences) and the preference page Intershop Studio there may be red cross next to Development Gradle Properties. Hovering of this cross a text similar to the following will be displayed: The key 'serverDirectory' is not found in the properties file. Also in this case you cannot select another preference page on the right nor close the dialog with the OK button.

First: Make sure that you started Intershop Studio from a developer console with all environment variables set.

If you have and the problem still occurs, you may have run into the issue ISTUDIO-123. The issue is fixed in higher versions. For a workaround in the meantime, please see the steps below the picture.

This problem is mostly cosmetic and will not affect you in your work. (Intershop Studio will find serverDirectory using environment variables set by the gradle_environment.bat/gradle_environment.sh.)

To be able to work with the preference dialog again, use the following workaround:

  1. Select any other preference page on the right.
  2. A popup with the message The currently displayed page contains invalid values will occur. Close this popup with OK.
  3. Close the preference dialog with Cancel.
  4. Reopen the preference dialog using the menu Window | Preferences.

The preference page General will be selected and you can continue working.

Avoid selecting the preference page Intershop Studio. (If you select it, you will have to repeat the workaround.) It is not necessary to configure anything there.

Preference pages nested under Intershop Studio are not affected and can and should still be used.

2.3.7 Process / Data Flow

The following diagram shows how data flows between directories (blue) during the different steps (red).

Note

The application server can now read (denoted by dotted lines) static artifacts like ISML templates, pipelines, queries etc directly from the SVN working copies. No more linking / copying is necessary. (Also it reads classes directly from the target directories.)

developer_process_data_flow

3 Recipe: Run Gradle Tasks

3.1 Problem

Gradle tasks are the units of work you can trigger. This recipe describes how. For an introduction beyond this recipe, please refer to the Gradle user guide.

3.2 Solution

You may run Gradle tasks using Intershop Studio or the console. Generally you have to:

  1. Specify which Gradle project to run tasks in.
  2. Specify which Gradle task(s) to execute.

The assembly, all cartridges and the system component set itself are Gradle projects. The component set and the cartridges form a Gradle multi-project build. The component set works as the root project and cartridges are (direct) child-projects of the component set.

Tasks can have dependencies to other tasks. Executing a certain task will always also execute its dependencies.

You may specify multiple tasks to run in one execution of Gradle. The order of depending tasks is mostly determined automatically by Gradle.

  • Depending tasks will always be executed after their dependencies.
  • Tasks that do not depend on each other may still have a preconfigured order, if you run them together.

Tasks independent of each other are executed in the order in which you are passing the tasks in.

Note

Be aware that the order of independent tasks is important. E.g., executing a cleaning task after the build task leads to an empty directory.

3.2.1 Running Tasks from Intershop Studio

In Intershop Studio each Gradle project is represented by an Eclipse project.

To execute tasks in a cartridge/assembly/component set:

  1. Right-click the corresponding project in the Package Explorer.
    The context menu opens.
  2. Click Run As | Gradle Build...
    For cartridges you can also reach that dialog by clicking Build Cartridge (Gradle Build) in the menu.
  3. In the dialog Edit Configuration select the tasks to be executed.
  4. Optionally, click Order to sort the tasks.
  5. Click Apply to confirm your settings.
  6. Click Run to execute the task(s).

You may also specify additional parameters in the dialog.

To reuse and switch between multiple configurations per project (e.g., different deploy or database tasks in the assembly), give them different names. Thus they can be found in the list of the external tool configurations .

3.2.2 Running Tasks via Console

To execute tasks via Console:

  1. Open a developer console
  2. Go to <ASSEMBLY> directory for executing tasks in the assembly or
    to the <COMPONENTSET> for executing tasks in the component set or a single cartridge
  3. Run
    1. On Windows: gradlew :<task1> :<task2>, e.g., gradlew :deployServer :importDump
    2. On Linux: ./gradlew :<task1> :<task2>, e.g., ./gradlew :deployServer :importDump

Syntax difference on Linux

In the remainder of this section we will ignore the minor difference between execution on Linux and Windows. Always run ./gradlew instead of gradlew if you work on Linux.

To execute a task in a certain cartridge (i.e., a child-project), pass in :<cartridge>:<task>, e.g., gradlew :core:clean :core:build.

You can also combine tasks from different cartridges, e.g., gradlew :core:build :bc_foundation:build.

Tasks names with or without colon

To execute a task in all cartridges of the component set execute gradlew <task> (without colon), e.g., gradlew build. Running the command with a colon, e.g., gradlew :build, leads to an error if the task does only exist for cartridges, but not for the component set:

FAILURE:
Could not determine which tasks to execute.

* What went wrong:
Task 'build' not found in root project 'projectComponentSet'.

* Try:
Run gradlew
tasks to get a list of available tasks. 

3.3 Discussion

The following is a loose collection of helpful hints to get you started with Gradle. Run all of them in a developer console.

To display all cartridges in the component set execute the following inside <COMPONENTSET>:

gradlew :projects

To display tasks of a (root) project (component set or assembly):

gradlew :tasks

To display the tasks of a certain subproject (e.g., cartridge) run:

gradlew :<name>:tasks

To find out which tasks are actually going to be executed (e.g., which dependencies are automatically pulled in) and in which order:

gradlew <tasks> --dry-run

4 Recipe: Initialize the Database

Note

The installation of the Oracle database software and database instance especially a XE instance is not part of the Intershop deployment tools, but is performed by the Oracle setup routine. The Guide - Setup Oracle XE as Intershop Development Database describes, how the Oracle XE edition as well as Intershop 7 need to be configured for use as development database.

4.1 Problem

The database must be initialized before testing (or as part of the test, i.e., to see if the database can be initialized).

4.2 Solution

There are two ways to initialize your database.

4.2.1 Import Database Dump of Release

You may import the database dump generated for the released version of your assembly. This is only useful if your local changes have no effect on the initial database state. To import the dump run the task importDump on the assembly.

The release version from which the dump is taken is determined by the property dump.<assembly organization>.<assembly name> in <GRADLE_USER_HOME>/gradle.properties. This way rerunning the importDump task always imports the same dump, unless you change the property.

The dump property is automatically created upon first run of the importDump task. To update the property to the latest released version run the updateDumpVersion task.

4.2.2 Execute DBInit

You may run the DBInit through assembly. To do so:

  1. Open a developer console and go to <ASSEMBLY>
  2. Run
    1. On Windows: gradlew dbinit -PskipDeployment
    2. On Linux: ./ gradlew dbinit -PskipDeployment

The task dbinit has a dependency to deployServer, i.e., if you execute the DBInit the server is automatically be deployed before that. If you want to run dbinit without prior execution of deployServer, pass in -PskipDeployment.

4.3 Discussion

4.3.1 Import Database Dump of Release

The dump is downloaded from the release repository and handed over to ant import at <SERVER>/local/tools/misc.

Also branding installations, which are usually created by running DBInit are downloaded from the release repository and extracted into <SERVER>/share/sites.

You can still use ant import at <SERVER>/local/tools/misc to import other dumps.

Quick Start

The Intershop delivery/DVD comes with two dumps that might be useful until your own assembly was built by a CI server.

Find them in the delivery/on the DVD at:

  • Dump without reference organization Primetech: Intershop-<version>/repositories/components/com.intershop.assembly/intershop7/<version>/dmps/database-dump-dump.dmp
  • Dump with reference organization Primetech: Intershop-<version>/repositories/components/com.intershop.assembly/primetech/<version>/dmps/database-dump-dump.dmp

You can also find them in a repository (like a Nexus) filled with the delivery contents in a similar path.

To use them:

  1. Copy one of these files to <SERVER>/local/tools/misc/database.dmp
  2. Open a developer console and go to <SERVER>/local/tools/misc
  3. Execute ant import

4.3.2 Execute DBInit

The command first executes ant dbschema-clean at <SERVER>/local/tools/misc to purge your database account. After that it executes dbinit --classic in <SERVER>/local/bin, i.e., a full DBInit over all cartridges without interactions.

You may still run these commands manually if you like - or need to pass in other options.

5 Recipe: Make Changes Effective on the Server

5.1 Problem

After the initial setup, the developer changes the sources of the component set. This may happen because of changes made by the present developer or updates to the VCS working copies of assembly and component set. The developer must be able to see this changes in the deployed server.

5.2 Solution

Depending on the kind of change, different actions are necessary to make the changes visible in the running server.

Kind of changeAction
The changed code artifact resides in <COMPONENTSET>/<cartridge>/staticfiles/cartridge.Reload the page. The application server automatically reloads the files directly from this directory (and recompile ISML templates if necessary).
The changed code artifact is a Java class.Rebuild the affected cartridges and restart the application server. The application server loads the classes from <COMPONENTSET>/<cartridge>/target/classes.
The changed code artifact resides in <COMPONENTSET>/<cartridge>/staticfiles/share.

Publish and redeploy the affected cartridge(s) and restart the application server / rerun the dbinit. (Publishing automatically includes rebuilding.)

You are unsure which kinds of changes have been made - e.g., you updated your VCS working copy.Publish and redeploy the component set. You may also want to reinitialize your database, see Recipe: Initialize the Database.
Changes in the assembly, including changes in <ASSEMBLY>/environment.properties.Redeploy the whole assembly. You may also want to reinitialize your database, see Recipe: Initialize the Database.

See discussion below for how to perform these actions.

5.3 Discussion

Please, refer to the Recipe: Run Gradle Tasks on how to run these tasks. This recipe also mentions how to execute a task on multiple or all cartridges at once.

5.3.1 Rebuild

To rebuild (e.g., recompile) a cartridge, run the build task on that cartridge.

The buildDependents task builds the cartridge itself and all cartridges in your component set that depend on it. This is useful to see if Java changes cause compilation errors in depending components.

When not cleaning the build results before building / publishing, Gradle performs an incremental build. Gradle only re-execute Gradle tasks whose inputs have changed (or whose outputs have been tampered with). This is somewhat safer and more reliable than re-executing an Ant build. Using the non-cleaning, incremental build should suffice for developers most of the time.

To execute a clean build run the tasks clean and build on the cartridge(s).

Using these tasks does not touch the contents of your local repository. If you later redeploy your changes, make sure to publish them before.

5.3.2 Publish

To republish a cartridge to your local repository, run the publish task on that cartridge. This overwrites the local version of that cartridge inside your local repository at <REPO>.

5.3.3 Redeploy

All deployment tasks are run on the assembly project. Before you can deploy any change, you must have published it.

To redeploy all cartridges of your component set run the task deployCartridges.

To redeploy only a few cartridges:

  1. Open a developer console and go to <ASSEMBLY>
  2. Run
    1. On Windows: gradlew deployCartridges -Pcartridges="<space separated list of cartridges>", e.g., gradlew deployCartridges -Pcartridges="core bc_foundation"
    2. On Linux: ./gradlew deployCartridges -Pcartridges="<space separated list of cartridges>", e.g., ./gradlew deployCartridges -Pcartridges="core bc_foundation"

To redeploy the whole server run the task deployServer.

Note

The deployment also works incrementally and touches only files that have been changed (it does not delete all files). In most cases, it should be possible to keep services / the application server running. Restart after the deployment, if the application server only reloads the changed files upon restart (like for most Java classes).

Running the post-install scripts again is not necessary after redeployment.

6 Recipe: Run Automatic Tests

6.1 Problem

A developer needs to execute and debug tests.

6.2 Solution

There are currently three ways to execute tests.

6.2.1 Run Tests Inside Intershop Studio

This should be the preferred way to execute tests for the developer.

Before being able to run the system or integration tests, you must set up your workspace by:

  1. From the menu select Window | Preferences.
    You may run into a problem when opening the preference dialog the first time, see the box Issue with Intershop Studio Preferences above.
  2. Go to preference page Intershop Studio | JUnit Cartridge Test | Classpath.
  3. Check both Add Intershop Suite Server entries and All components.
  4. Click Apply to confirm your settings.
  5. Click Ok.

To run or debug a single test:

  1. Select a test class, a Java package containing test classes or a method of a test class
  2. Open the context menu and select
    1. Run as | JUnit Test for unit tests (located in the tested cartridge, see discussion) or
      Run as | JUnit Cartridge Test for integration or system tests or
    2. Debug as | JUnit Test for unit tests or
      Debug as | JUnit Cartridge Test for integration or system tests

To run multiple tests that are not in the same Java package, please create a JUnit4 test suite first and run that suite.

This way of execution applies to all types of tests.

6.2.2 Execute Tests with Gradle

You may run all unit tests of a cartridge (i.e., not a _test-cartridge) by running the Gradle test task on that cartridge. A report is generated per cartridge at <COMPONENTSET>/<cartridge>/target/reports/tests/index.html.

You may also run the Gradle test task on the component set. This executes the test tasks of all cartridges and generates an overall report. The report can be found at <COMPONENTSET>/target/reports/allTests/index.html.

Note

This approach only executes unit tests!

6.2.3 Execute Tests with Test Runner

You may run multiple integration or system tests with the Intershop Test-Runner by:

  1. Open a developer console.
  2. If your prompt does not yet start with ES<n>|...(e.g., ES1|D:\is7\trunk\), run <SERVER>/local/bin/environment.bat.
  3. Go to <SERVER>/local/tools/misc.
  4. Run ant test.

Unit tests are not available in the test runner anymore! See the discussion below for rationale.

6.3 Discussion

There are three kinds of tests that are treated differently.

6.3.1 Unit Tests

Unit tests are tests that require nothing but a properly populated class path to run. Especially, they cannot require a pre-created files or directories (like through deployment) or a database. (They may however start up a pretty complete server in their initialization - as long as this server does not access any file system resources, like configuration files.)

Super classes/annotations for Unit tests are:

  • Base class junit.framework.TestCase (legacy test cases written for JUnit version <= 4)
  • Classes with @RunWith annotation (newly written test cases starting with JUnit 4)

Unit tests are located in the same cartridge as the functionality they test.

Run unit tests using:

  • the Intershop Studio or
  • via Gradle (developer tools)

Unit tests are not deployed

Unit tests are solely for execution by the developer and during build by the continuous integration server. Since unit tests are not needed after the build process any longer, they are not packaged up and cannot be deployed.

A consequence of this simplification is that they are not available for execution in the test runner any more.

6.3.2 Integration Tests

Integration tests require a fully deployed server and are executed within that server, so they have direct access to the classes and objects in the JVM.

Super classes for Integration tests are:

  • com.intershop.tools.etest.server.UnitTestCase(sic!)
  • com.intershop.tools.etest.remote.CAPITestCase

Integration tests are located in the accompanying test cartridge of the cartridge being tested (bc_pricing_test for bc_pricing).

Run integration tests using:

  • the Intershop Studio
  • the Test Runner

6.3.3 System Tests

System tests run outside the server and access it via any remote interface (e.g., via HTTP). Technically they are run like unit tests, but they require additional Java properties to know how to connect to a server.

Super classes for system tests are:

  • tests.remote.com.intershop.testfw.util.ucm.UCMHTTPTestCase

System tests are currently all located in ucm_tests in component set tests.

Run integration tests using:

  • the Intershop Studio
  • the Test Runner

7 Recipe: Create a Cartridge

7.1 Problem

Developers need to add new cartridges to an existing component set.

7.2 Solution

To add a new cartridge to an existing component set perform the following overall steps:

  1. Check out an existing component set and import it to Intershop Studio workspace (see according recipes).
  2. Create a new cartridge using Intershop Studio.
  3. Fill the build.gradle of new cartridge.
  4. Add the cartridge to build.gradle file of the assembly and specify the order.
  5. Publish the component set and redeploy the assembly

Please see discussion below for details on how to perform these steps.

It is not necessary to register the cartridge inside the component set in any way. Instead all sub-folders not explicitly excluded in <COMPONENTSET>/settings.gradle are treated as components and included in the build. This way upon commit they are also included in the build of the continuous integration server automatically.

Other non-Gradle related steps, like adding the cartridge to an application type may still be necessary.

7.3 Discussion

7.3.1 Create a New Cartridge Using Intershop Studio

  1. From the menu select File | New | Cartridge.
  2. Enter cartridge and Java package name.
  3. Click Next.
  4. Use the template Default (Gradle Support) (should be already selected).
    Properties like organization and version are now specified by the component set.
  5. Click Finish.

Note

Unique Cartridge Names

Besides publishing cartridges with a qualified name, e.g., with an organization and a name, for legacy reasons the unqualified names still have to be unique. I.e., you cannot create a cartridge with the organization com.customer and name core, because there already is core in com.intershop. Instead keep adding prefixes to the name like customer_core.

7.3.2 Fill build.gradle

The build.gradle file of a cartridge is both, the build script and the build configuration file for that cartridge. Reusable build functionality comes in Gradle build plugins. Different kinds of artifacts are handled by different plugins. The Gradle build will fail, if you apply a plugin, but there are no matching source files to process.

Intershop Studio generates an initial build.gradle file that is suitable for cartridges containing Java source code and static files. Please refer to Cookbook - Gradle Build Tools | Configure the Build of a Component / Cartridge on how to configure your build.gradle file.

Note

Cartridges that contain ISML templates must declare a few dependencies:

dependencies {

compile group: 'com.intershop' , name: 'tools'

compile group: 'com.intershop' , name: 'isml'

compile group: 'com.intershop' , name: 'servletengine'

compile group: 'com.intershop' , name: 'core'

}

If you fail to do so an error similar to this one will occur when building / publishing the component set:

...
Execution failed for task ':<cartridge>:isml2classMain'
> taskdef class com.intershop.beehive.isml.capi.ISML2JSP cannot be found using the classloader AntClassLoader
....

7.3.2.1 Declare Dependencies

When declaring dependencies you need to distinguish between:

  1. Dependencies to cartridges in the same component set.
    These are also called project dependencies. Declare them using the syntax compile project(':<cartridge name>') inside the dependencies block.
  2. Dependencies to cartridges in another component set.
    These are called external dependencies. Declare them using the syntax compile group: '<cartridge organization>', name: '<cartridge name>'

The following example demonstrates the different declarations.

build.gradle (excerpt)
dependencies {
    compile group: 'com.intershop', name: 'tools'
	compile project(':customer_base')
} 

As already used from dependency management with Ivy the compile classpath is resolved intransitively: If you specify a dependency to the core cartridge you will not automatically have a dependency to tools. So you will not have tools in your classpath unless you declare the dependency explicitly.

No versions in build.gradle file

Note that external dependencies do not have to declare versions directly. Instead Intershop recommends to declare them using version and filter properties in a gradle.properties file. See Concept - Gradle Build Tools | Configuring Dependency Versions for details.

7.3.3 Add New Cartridge to Assembly

Please follow the Recipe in Cookbook - Gradle Assembly Tools | Add Cartridges to an Assembly.

7.3.4 Publish and Deploy

To let the changes take effect locally, publish the component set and redeploy your assembly, as described in Recipe: Make Changes Effective on the Server. Make sure to publish the whole system component set by running the publish task on the system component set, not just the added cartridge.

8 Recipe: Remove a Development Environment

8.1 Problem

The developer needs to remove the development environment when finished to free system resources.

8.2 Solution

  1. Stop processes running on the deployed server. (Optional for services - as they are stopped automatically later.)
  2. Run Gradle task undeploy on the assembly. This first execution generates an uninstall script for the services.
  3. Run as administrator / with root privileges (as requested in the console output): <SERVER>\postInstall\uninstallServices.bat. This stops and removes the services.
  4. Run Gradle task undeploy on the assembly again. This purges your server directory.
  5. Delete the directories <REPO>, <WORKSPACE>, <JAVADOC>, your VCS working copies <ASSEMBLY> and <COMPONENTSET> and finally <GRADLE_USER_HOME>.

9 Recipe: Clean Gradle Cache

9.1 Problem

The Gradle cache located <GRADLE_USER_HOME>/cache currently grows indefinitely and may eventually fill the hard disk.

9.2 Solution

On Windows:

  1. Close Intershop Studio if open.
  2. Open a developer console and go to <COMPONENTSET>.
  3. Run gradlew --stop.
  4. Run SET GRADLE_USER_HOME=%USERPROFILE%\.gradle.
  5. Run gradlew --stop (see discussion for why this has do be done twice).
  6. Reset GRADLE_USER_HOME by running gradle_environment.bat.
  7. Delete the folder <GRADLE_USER_HOME>/cache using an OS command.

On Linux:

  1. Close Intershop Studio if open.
  2. Open a developer console and go to <COMPONENTSET>.
  3. Run ./gradlew --stop.
  4. Run export GRADLE_USER_HOME=$HOME/.gradle.
  5. Run ./gradlew --stop (see discussion for why this has do be done twice).
  6. Reset GRADLE_USER_HOME by running gradle_environment.sh.
  7. Delete the folder <GRADLE_USER_HOME>/cache using an OS command.

9.3 Discussion

9.3.1 Cache Contents and Cleaning

The Gradle cache contains:

  • Artifacts downloaded from remote repositories, like binaries of cartridges used as dependencies, but also deployment and build plugins. This is by far the largest part.
  • Precompiled gradle scripts.

Gradle has no feature for selectively cleaning the artifact cache so far: http://forums.gradle.org/gradle/topics/managing_artifact_cache_growth.

In the meantime, you have to revert to completely deleting the Gradle cache occasionally as described above. The disadvantage of this approach is that all artifacts required for further build and deployment processes will be downloaded again.

9.3.2 Stopping Daemons

Before deleting the cache directory all processes that hold file handles into it must be stopped, i.e., all Gradle daemons. You can do so by calling gradlew --stop.

Note

This command only stops daemons started with the Gradle user home specified by GRADLE_USER_HOME environment variable. Due to a bug in the Gradle Tooling API (http://issues.gradle.org/browse/GRADLE-2414) IntershopStudio will partially use the default Gradle user home at <OS_USER_HOME>/.gradle. That is why you have to run the command twice with different values of GRADLE_USER_HOME.

You can see all running Gradle daemons (not just of the current Gradle user home) by running jps. Example of output:

19452 lilith.jar
22244 Jps
17448 GradleDaemon
13520
15852 GradleDaemon
16772
3468 GradleDaemon
22276

Disclaimer

The information provided in the Knowledge Base may not be applicable to all systems and situations. Intershop Communications will not be liable to any party for any direct or indirect damages resulting from the use of the Customer Support section of the Intershop Corporate Web site, including, without limitation, any lost profits, business interruption, loss of programs or other data on your information handling system.

Customer Support
Knowledge Base
Product Resources
Support Tickets