Introduction

It is assumed that you are familiar with the Concept - Gradle Assembly Tools.

Topics not covered in this document are:

1. How to work with an existing (and therefore configured) assembly as part of the developer workflow. This includes execution of the assembly process. 

2. How to deploy an assembly. This is either part of the developer workflow (see above) or covered in the following documents:

• Concept - Gradle Deployment Tools
• Cookbook - Gradle Deployment Tools (7.4 CI - ICM 7.7)

Glossary

References

• Concept - Gradle Build Tools
• Concept - Gradle Assembly Tools
• Concept - Gradle Deployment Tools
• Cookbook - Gradle Deployment Tools (7.4 CI - ICM 7.7)

Recipe: Create a New Assembly Inheriting From an Existing Assembly

Problem

Developers need to create custom assemblies to deploy custom cartridges or a custom combination of existing cartridges.

An assembly can also be created with the scripts in the development-setup of an Intershop delivery package (See Cookbook - Setup CI Infrastructure).

This recipe provides a more in-depth explanation on how to create an assembly manually.

Solution

This covers the more common case of creating an assembly that inherits from an existing one.

Create a single root folder and the following files. Add them to your source repository, once you are done. See Concept - Gradle Assembly Tools for the purposes of these files.

To verify that your initial assembly works, publish and/or deploy it.

Discussion

build.gradle

Copy the following minimum build.gradle file and follow through the TODOs:

Gradle tools version 1.0:

// Make the assembly plugin available to the build.gradle script
buildscript { 
    dependencies {
        classpath 'com.intershop.build.gradle:ish-assembly'
		classpath 'com.intershop.build.gradle:plugin-tests'
	 }    
}

// Apply the 'ish-assembly' plugin to the project
apply plugin: 'ish-assembly'

// Specify the namespace in which the assembly is published into the repository. (The name inside this namespace is automatically derived from the folder name.)
// TODO: Change to your desired namespace
group = 'com.example.assembly'

assembly {
	// Inherit from another assembly
	// TODO: Replace by <org>:<name> of the inherited assembly
	inheritFrom('<org>:<name>') {		
		// Inherit assembly deploy.gradle and deploy-settings.gradle scripts
		includeArtifacts type:['deploy-gradle', 'deploy-settings-gradle']

		// By default will also inherit all host-type, environments
		// and the contained components/cartridges
	}

	// Supply additional meta-data that will be displayed in the SMC/at logon screens
    extraAttributes = [
		//TODO: Change to appropriate values
	    'productName': 'Intershop 7',
        'copyrightOwner': 'Intershop Communications',
        'copyrightFrom': '2005'
   ]
}

Gradle tools version >= 1.1:

// Make the assembly plugin available to the build.gradle script
buildscript { 
    dependencies {
        classpath 'com.intershop.build.gradle:ish-assembly'
		classpath 'com.intershop.build.gradle:plugin-tests'
	 }    
}

// Apply the 'ish-assembly' plugin to the project
apply plugin: 'ish-assembly'

// Specify the namespace in which the assembly is published into the repository. (The name inside this namespace is automatically derived from the folder name.)
// TODO: Change to your desired namespace
group = 'com.example.assembly'

assembly {
	// Inherit from another assembly
	// TODO: Replace by <org>:<name> of the inherited assembly
	inheritFrom('<org>:<name>') {		
		// Inherit assembly deploy.gradle and deploy-settings.gradle scripts
		includeArtifacts type:['deploy-gradle', 'deploy-settings-gradle']

		// By default will also inherit all host-type, environments
		// and the contained components/cartridges
	}

	cartridges {
		// Inherit cartridge order from another assembly
		// TODO: Replace by <org>:<name> of the inherited assembly
		order = listFromAssembly('<org>:<name>')
	}

	// Supply additional meta-data that will be displayed in the SMC/at logon screens
    extraAttributes = [
		//TODO: Change to appropriate values
	    'productName': 'Intershop 7',
        'copyrightOwner': 'Intershop Communications',
        'copyrightFrom': '2005'
   ]
}

The ish-assembly plugin extends the Gradle project by:

• Adding the DSL extension assembly for configuring the assembly's contents. (See other recipes.)
• Adding the DSL extension assemblyBuild for configuring how a database dump is created during build. (See other recipes.)
• Adding tasks to
  
  • publish the assembly
  • deploy the assembly
  • run the DBInit

You have created an assembly that is basically the same like the inherited one, except for a different name. To customize the assembly, see the other recipes.

gradle.properties File

Copy the following minimum gradle.properties file and follow through the TODOs:

# TODO: Provide version number of your assembly (excluding build number/local/snapshot suffix)
version = 7.5.0.0

# build plugin versions
filter.com.intershop.build.set.cartridge-plugins = 1.0.0.0.+
filter.com.intershop.build.set.extension-plugins = 1.0.0.0.+

Recipe: Add Custom Deployment Logic to an Assembly

Problem

I need to add custom deployment logic to my inherited assembly.

When adding my deploy.gradle, I get an exception like:

Execution failed for task ':publishBuildAssemblyPublicationToBuildRepository'.
> Failed to publish publication 'buildAssembly' to repository 'build'
   > Invalid publication 'buildAssembly': multiple artifacts with the identical name, extension, type and classifier ('deploy', 'gradle', 'deploy-gradle', 'null').

Solution

1. Create the file <ASSEMBLY>/deploy.gradle.
2. Add the parent assembly's deployment logic to this file.
3. Add your own custom deployment logic to this file.
4. Exclude the artifact when inheriting from the parent assembly.

Discussion

Example until Gradle Deployment Tools 2.7

assembly {  
    inheritFrom('com.intershop.assembly:commerce_management_b2x') {
        includeArtifacts
        type:['deploy-settings-gradle']
        //     Exclude the artifact when inheriting from
        the parent assembly....
        //
        includeArtifacts type:['deploy-gradle', 'deploy-settings-gradle']
    }
}

To find the parent assembly's deployment logic, browse your software repository and look for the deploy-gradle artifact of the according assembly.

To exclude the artifact from inheritance, see Recipe: Customize Assembly Inheritance.

Recipe: Add Cartridges to an Assembly

Problem

The developer needs to add cartridges to an assembly.

Solution

See the discussion for details on the steps.

For Gradle tools version 1.0:

1. Add cartridges to one or more assembly subsets by using the method assembly.cartridges.include in the build.gradle file.
2. Ensure that the added cartridges (and their transitive dependencies, see discussion) are included in the cartridge-order.txt file.
3. Ensure that a version for the added cartridges is declared in the gradle.properties file of the assembly.

For Gradle tools version >= 1.1:

1. Add cartridges to one or more assembly subsets by using the method assembly.cartridges.include in the build.gradle file. 
2. Ensure that the added cartridges (and their transitive dependencies, see discussion) are included in the property assembly.cartridges.order. This ensures that the cartridgelist.properties file is appropriately generated.
3. Ensure that a version for the added cartridges is declared in the gradle.properties file of the assembly.

Discussion

Add Cartridges to Subsets in build.gradle File

To add cartridges to one or more environments, add the following declaration:

assembly {
    cartridges {    
		//TODO: Specify organization and name of the cartridges, also choose which environments to include them in   
        include 'com.example:cartridge1', 
				'com.example:cartridge2', 
				in: [production, development, test]
	}
}

To add a cartridge as init-only-cartridge (see Concept - Gradle Assembly Tools for more details) use the same method, but specify in:init.

assembly {
    cartridges {
		//TODO: Specify organization and name of the cartridges
        include 'com.example:cartridge1',
				'com.example:cartridge2',
				in: init
	}
}

Transitive vs. Intransitive Dependencies

By default adding a cartridge also adds all its transitive dependencies, i.e., all dependencies declared in the configuration default of its ivy.xml file, those cartridges' dependencies etc.

For most applications this makes sense, since you do not want to deploy a cartridge without its dependencies. Also you can minimize declaration effort by including the application suite cartridge. Such a cartridge already has dependencies to all application type cartridges, which in turn should already have dependencies to all cartridges contained in that application.

For more control you may include a cartridge without its dependencies using the following declaration:

assembly {
    cartridges {
        //TODO: Specify organization and name of the cartridges, also choose which environments to include them in
        include(
            'com.example:cartridge1',
            'com.example:cartridge2',
            in: [production, development, test]
        ) {
            transitive = false
        }
    }
}

Ensure Cartridges Are Listed in cartridge-order.txt

(For Gradle tools version 1.0)

To generate the <SHARE>/system/cartridges/cartridgelist.properties file, which serves as basis for look-up precedence of ISML templates, pipelines, queries, etc. and also for the order in which DBInit preparers are executed, the assembly process needs to order cartridges. This is performed by looking up the order of cartridges in the cartridge-order.txt file.

The format of the cartridge-order.txt file is:

• Each line contains the name of a cartridge (unqualified, e.g., without organization)
• Empty lines are ignored
• Lines starting with a hash mark '#' are comments and are ignored as well

The cartridge-order.txt must contain all cartridges that are contained in the assembly, but may contain any number of additional cartridges. It is only used to determine the order of cartridges, not which cartridges to include in the assembly.

If you use multiple assemblies it is therefore recommended to share the  cartridge-order.txt file across them.

When adding cartridges with their transitive dependencies, it is necessary that all these dependencies are also included in cartridge-order.txt.

Ensure Cartridges Are Listed in assembly.cartridges.order

(For Gradle tools version >= 1.1)

To generate the <SHARE>/system/cartridges/cartridgelist.properties file, which serves as basis for look-up precedence of ISML templates, pipelines, queries, etc. and also for the order in which DBInit preparers are executed, the assembly process needs to order cartridges. This is performed by looking up the order of cartridges in the property assembly.cartridges.order.

The property assembly.cartridges.order is of type List<String>. Each entry of the list should be the unqualified name of a cartridge, e.g., bc_foundation.

The property assembly.cartridges.order can be filled in several ways.

Best Practice

The following way is considered best practice to fill the assembly.cartridges.order property:

• Store a list of your own cartridges with the desired sorting
• Use that list to include the cartridges into the assembly
• Reuse the order of cartridges from the inherited assembly and concatenate the list of your cartridges

assembly {
    cartridges {
        def productionCartridges = ['customer_cartridge_1', 'customer_cartridge_2']
        
		// TODO: Replace by organization of your cartridges
        include(*(productionCartridges.collect {"<org>:$it"}), in:[development, test, production]) {
            transitive = false
        }

		// TODO: Replace by <org>:<name> of the inherited assembly       
        order = listFromAssembly('<org>:<name>') + productionCartridges
    }
}

Order From an Existing Assembly

Read the order of cartridges from an existing assembly using the method listFromAssembly:

assembly {
	cartridges {
		//TODO: Adapt name and organization of assembly
		order = listFromAssembly('com.intershop.assembly:intershop7')
	}
}

Order From a File

Read the order from an external file using the method listFromFile:

assembly {
	cartridges {
		//TODO: Adapt name/location of the file. In this example the file is expected in the same directory as the build.gradle file
		order = listFromFile(file('cartridge-order.txt'))
	}
}

The expected file format is:

• Each line contains the name of a cartridge
• Leading and trailing spaces are trimmed
• Empty lines are ignored
• Lines starting with a hash mark '#' are comments and are ignored as well

Order From Groovy List Literal

Create the list from a Groovy list literal directly written in the build.gradle file:

assembly {
	cartridges {
		//TODO: Adapt names of cartridges
		order = ['core', 'bc_foundation', 'bc_marketing']
	}
}

You may also create a list using a list literal once and pass it to both property assembly.cartridges.order and method assembly.cartridges.include. This assumes that you include your cartridges intransitively.

assembly {
    cartridges {
        def productionCartridges = ['core', 'bc_foundation']
        
        include(*(productionCartridges.collect {"com.intershop:$it"}), in:[development, test, production]) {
            transitive = false
        }
        
        order = productionCartridges
    }
}

Order From Complex Expression

Create the list from any combination of the above, by concatenating multiple other lists, inserting or removing items. You may use any method/operation offered by the JDK or GDK (Groovy extensions to JDK) on java.util.List.

assembly {
    cartridges {
        // Order cartridges:
        // - Cartridge 'customer_java_patches' is first
        // - Then all cartridges inherited from intershop7
        // - Then all cartridges mentioned in 'custom-cartridge-order.txt'
        // - Order cartridge 'storefront_app_patch' directly after cartridge 'storefront_app'
        def intermediaryOrder = ['customer_java_patches'] +  listFromAssembly('com.intershop.assembly:intershop7') + listFromFile(file('custom-cartridge-order.txt'))
        order = intermediaryOrder.plus(intermediaryOrder.find('storefront_app')+1, 'storefront_app_patch')
    }
}

Default Order

To be compatible with Gradle tools version 1.0 the property assembly.cartridges.order is filled with the contents of a file cartridge-order.txt if it exists. If it does not it is empty by default.

Declare Version of Added Cartridges

To adjust versions more easily, they should be declared in a properties-file instead of the build.gradle file directly. Alter the gradle.properties file of your assembly by either:

• Adding a property version.<organization of cartridge>.<name of cartridge>=<version> for each added cartridge, or
• Adding a property filter.<organization of assembly>.<name of assembly>=<version> of an assembly or component set containing the added cartridges.
  This is already the case for all cartridges in your own component set.

Also add the declaration to the gradle.properties of your component set if your cartridges have dependencies to the added cartridges.

See section Configuring Dependency Versions in Concept - Gradle Build Tools for details.

Recipe: Add Infrastructure Components to an Assembly

Problem

The developer needs to add infrastructure components to an assembly.

Solution

Add infrastructure components to a host-type by using the method assembly.hostTypes.<host-type name>.include.

assembly {
    hostTypes {
		//TODO: replace 'appserver' by the host-type's name
		appserver {
			// TODO: Specify organization and names of the cartridges
            include(
                'com.example:component1',
                'com.example:component2',
            ) {
                transitive = false
            }
		}
	}
}

Discussion

The same consideration like in the recipe Cookbook - Gradle Assembly Tools (valid to 7.8)#Add Cartridges to an Assembly regarding transitive vs. intransitive dependencies applies here.

To add infrastructure components including their transitive dependencies use:

assembly {
    hostTypes {
		//TODO: replace 'appserver' by the host-type's name
		appserver {
			// TODO: Specify organization and names of the cartridges
        	include 'com.example:component1',
                    'com.example:component2'
		}
	}
}

Recipe: Add an Environment to an Assembly

Problem

To offer a different set of cartridges in a new deployment scenario, the developer needs to define a new environment within an assembly.

Solution

Use the method assembly.environments.create:

assembly {
	environments {
		//TODO: Specify the name of the new environment
		create 'preproduction'
	}
}

The new environment initially contains no cartridges. Refer to the recipe Cookbook - Gradle Assembly Tools (valid to 7.8)#Add Cartridges to an Assembly for adding cartridges to this environment.

Discussion

Example: Add a new environment to your assembly that inherits all the cartridges from the production environment.

Solution: Add the environment prelive to your assembly build.gradle.

assembly {   
 …
    environments {
       create ('prelive') {
          production.cartridgesConfiguration.dependencies.all {
             cartridgesConfiguration.dependencies.add(it)
          }
       }
    }
    …
}

Recipe: Add a Host Type to an Assembly

Problem

To offer a different set of infrastructure components in a new deployment scenario, the developer needs to create a new host type in an assembly.

Solution

Add a declaration in the assembly.hostTypes:

assembly {
	hostTypes {
		//TODO: Replace 'imageServer' by the name of your host-type
		imageServer {
			//TODO: Set the flags (see Concept for meaning of the flags)
			includeLocal = false
			includeShare = false
			includeCartridges = false
			includeJavadoc = false
		}
	}
}

The new host type initially contains no components. Refer to the recipe Cookbook - Gradle Assembly Tools (valid to 7.8)#Add Infrastructure Components to an Assembly for adding components to this environment.

Recipe: Customize Handling of Third-Party Libraries

Problem

I am using dependency management for third-party libraries with Intershop 7 version 7.6 or later, but the default behavior does not work the way I need it.

Solution

When including cartridges transitively (see Recipe: Add Cartridges to an Assembly), the depending libraries become system components themselves to the assembly. Therefore dealing with libraries equals the dealing with any other system component.

Discussion

Transitive Dependency Management

When including cartridges transitively, Gradle will not only include their direct dependencies, but also the dependencies of those dependencies and so on.

You can use standard Gradle mechanisms if you need to exclude libraries that come in via this transitive closure:

configurations.all {
    exclude group:'com.example', module:'excluded-lib'
}

Version Conflicts

The assembly does not allow conflicting versions for any system component. In case some library gets included in multiple versions, you will see an exception like:

> Could not resolve all dependencies for configuration ':development-cartridges'.
   > A conflict was found between the following modules:
      - com.example:conflicting-lib:1.2.3
      - com.example:conflicting-lib:1.0.0

You may either

• Resolve the version conflict by using version properties, e.g.:

  Assembly's gradle.properties (snippet)

  version.com.example.conflicting-lib=1.2.3

  Overriding Library Versions

  While it is technically possible to override versions of libraries used by Intershop 7, be advised that the resulting system cannot be supported unless explicitly stated by Intershop.

  or

• Exclude all but the desired dependency as described in the section Transitive Dependency Management above.

Class Collisions

A new verification has been added, that inspects the Jar files of all libraries in the assembly. In case some Java classes are contained in more than one dependency an error is thrown, e.g.:

> There are class collisions in your dependencies
   > Collision between com.google.code.findbugs:annotations:2.0.0 and com.google.code.findbugs:jsr305:2.0.1
      > javax.annotation.CheckForNull
      > [...]

You can inspect the origin of different dependencies as described below and can use Transitive Dependency Management to exclude unwanted dependencies to resolve collisions.

gradlew checkClassCollisions

Introspecting Transitive Dependencies

In case you are wondering about the origin of some unwanted dependency, you can use Gradle to get some insight, e.g.:

gradlew dependencyInsight --dependency slf4j-api

You do not need to specify the configuration to introspect, as the assembly process defaults to the environment development.

Assembly Inheritance

In general libraries are inherited from the parent assembly just like other system components. See Recipe: Customize Assembly Inheritance for more details.

Libraries excluded in the parent assembly will not be inherited to the child assembly. However, if you add cartridges to your child assembly, that yield the same unwanted transitive dependencies, you have to exclude those dependencies again.

The same holds true for version conflict resolutions.They are inherited for the transitive dependencies of inherited system components, but may reappear for newly added system components.

Recipe: Customize Assembly Inheritance

Problem

The developer needs to create an assembly inheriting contents from an existing assembly.

Solution

Use the method assembly.inheritFrom to declare inheritance from an existing assembly:

assembly {
	inheritFrom('com.example.assembly:base-assembly') {		
		// Control which dependencies/artifacts to inherit, see discussion
		// By default all dependencies in all component subsets and all artifacts are inherited
	}
}

Discussion

To select which artifacts and dependencies should be inherited, pass a closure to assembly.inheritFrom and inside it call includeArtifacts/ excludeArtifacts as well as includeDependencies/ excludeDependencies any number of times.

This API is largely inspired from Gradle's way to declare which files to copy in a copy operation, called a "copy spec". See the Gradle user guide for drawing parallels. To stay in the naming, the closure passed to the inheritFrom method is called an "assembly inheritance spec".

Selection of Dependencies

Each call to  includeDependencies/ excludeDependencies defines a predicate, i.e., a function that returns true or false given a dependency from the inherited assembly. A certain dependency is inherited if and only if it matches:

1. Any include and
2. None of the excludes

As an exception to this rule - if there are no includes declared, all dependencies are considered included. (Excludes are still effective in this case.)

This process is performed for each assembly subset (e.g., host-type, environment, configuration - see Concept - Gradle Assembly Tools). For example, if a cartridge is included in the environments production and development in the inherited assembly, you may select to inherit it only for environment development.

The following listing shows the different ways to define a predicate using different overloadings of includeDependencies. The same overloadings are available for excludeDependencies.

assembly {
	inheritFrom('com.example.assembly:base-assembly') {
		// Include all dependencies in the given component subsets.
		includeDependencies from:[production, development]

		 // Include dependencies by their (unqualified) name in all component subsets they were originally in
	    includeDependencies 'cartridge1', 'cartridge2'

		// Include dependencies by their (unqualified) name, but only in the given component subsets
	    includeDependencies 'cartridge1', 'cartridge2', from:[production, development]

		// Include dependencies based on the return value of a closure for arbitrary complex predicates
		includeDependencies { ModuleVersionIdentifier dependency, componentSubset -> 
			//TODO: Return true or false, based on properties of the dependency.
		} 
	}
}

There are two pitfalls here that may not be obvious:

//TODO: Replace 'production' by name of the environment and 'cartridge1' by the name of the of the cartridge
assert project.configurations.'production-cartridges'.dependencies.find { Dependency dependency ->  dependency.name == 'cartridge1' }

Scoping for includeDependencies and excludeDependencies

If you want to declare includes explicitly for some known subset of dependencies, but just inherit everything in others, you can limit the effect of includes and excludes by scopes. This is very similar to Gradle's nesting of copy specs, like providing different include patterns for different sub-directories.

Available scopes are:

• All component subsets (i.e., host-types, environments, configurations)
• A special scope by the name of environments, that contains all environments
• A special scope by the name of hostTypes, that contains all host types

Scopes are opened by calling the method within in the assembly inheritance spec. This method must be passed a closure, which is an assembly inheritance spec itself, i.e., it can also call includeDependencies and  excludeDependencies.

Consider the following examples:

assembly {
	inheritFrom('com.example.assembly:base-assembly') {
		includeDependencies 'cartridge1', from:production
	}
}

assembly {
	inheritFrom('com.example.assembly:base-assembly') {
		within(production) {
			 includeDependencies 'cartridge1'
		}
	}
}

In both examples the environment only contains the cartridge1 inherited from the environment production. In example A however, cartridge1 is the only dependency inherited from the assembly. No dependencies are inherited from other environments or host-types.

In example B, the includeDependencies call does not affect any other component subset than production. So - because this is default when there are no explicit includes - other environments, host-types and configurations are inherited with all their dependencies.

Scoping also works for  excludeDependencies calls, but calls within a scope and calls using the  from notation are equivalent.

The following example shows, how to:

• inherit production and development as is (i.e., with all their dependencies)
• inherit no other environments
• in host-type appserver only inherit runtime
• inherit all other host-types and configurations as is

assembly {
	inheritFrom('com.example.assembly:base-assembly') {
		within(environments) {
			includeDependencies from:[production, development]
		}
 
		within(appserver) {
			includeDependencies 'runtime'
		}
	}
}

Selection of Artifacts

The selection of artifacts works very similar to those of dependencies. Currently all artifacts are associated with the Ivy configuration default, so it is not necessary to look for artifacts by their assembly subset.

The following listing shows the different ways to define a statement using different overloadings of includeArtifacts. The same overloadings are available for excludeArtifacts.

assembly {
	inheritFrom('com.example.assembly:base-assembly') {
		// Include artifact by their type
	    includeArtifacts type:['deploy-gradle', 'deploy-settings-gradle']

		// Include artifacts by their name (excluding their extension)
	    includeArtifacts name:'branding'
		
		// Include artifacts based on the return value of a closure for arbitrary complex predicates
		includeArtifacts { ResolvedArtifact artifact, componentSubset -> 
			//TODO: Return true or false, based on properties of the artifact.
		}
	}
}

Selection of Assembly Subsets

Host-types, environments and configurations that exist in the assembly are automatically copied if any dependency or artifact is inherited from them. If no dependency / artifact is inherited, they are omitted.

Recipe: Generate a Database Dump

(For Gradle tools version 1.0-1.1, for Gradle tools version >= 2.0 see Recipe: Run DBInit/DBMigrate )

Problem

The Continuous Integration server should create a database dump during the assembly process and include it as artifact of the assembly, so it can be used for faster setup of demos / developer environments. The maintainer of the assembly needs to declare this.

Solution

Perform the following steps, see discussion for details:

1. Include database dump as artifact, which automatically adds its generation to the assembly process.
2. To speed up the assembly process, optionally configure to generate the dump incrementally based on the dump of another assembly.
3. Make sure branding installations are handled correctly.

Discussion

Include Database Dump as Artifact

To include the database dump as an artifact, add the following line to gradle.properties file in your Gradle user home:

releaseWithDump = true

If this property is set, the database dump will be generated before publishing the assembly and the published assembly will include the database dump as artifact of type dump. Typically this is used only in the Gradle user home of the CI server, as publishing assemblies for developers must be fast and developers do not need to generate dumps themselves. You may still set it for a developer temporarily for testing the behavior.

gradlew publish -PskipDeployment

Generate the Dump Incrementally

You may choose between:

1. Creating a dump from scratch: This cleans the database, runs a full DBInit and exports the database contents as new dump. This is the default behavior.
2. Create a dump incrementally: Import a database dump from another assembly, run a partial DBInit over additional cartridges and export the database content as new dump.

A typical use case for incremental dumps is adding demo data in a special assembly with a demo cartridge, that otherwise inherits from a non-demo assembly.

To create a dump incrementally insert the following block in to the build.gradle file of the assembly (note that this uses an assemblyBuild block as root element):

assemblyBuild {
    database {
		//TODO: Replace by <org>:<name> of the assembly you want to inherit database dump from
        inherit('com.example.assembly:base-assembly')
        
		//TODO: Insert list of cartridges for which the partial DBInit should be executed
        initCartridges = ['cartridge1', 'cartridge2']
    }
}

For this to work, you have to find an assembly that contains a suitable dump (e.g., containing a subset of the cartridges in your assembly). This is typically the same assembly that you inherit dependencies from.

If an incremental generation of the database dump is not possible because of some reason, you can always revert to the slower generation from scratch.

Handle Branding Installations

The  BrandingPreparer in  bc_foundation is used to install branding packages during DBInit. It extracts branding files into the IS_SHARE/sites folder. When initializing a database with a pregenerated dump, instead of running DBInit, these files are missing.

To work around this, the assembly process can also package the extracted branding installations from the sites folder and include them for deployment. There are two cases to consider:

1. Case: The DBInit in your assembly runs the BrandingPreparer, but the DBInit of the base assembly does not. (Or you do not have a base assembly / perform a full DBInit anyway.) Add the following line to build.gradle:

   build.gradle

    apply plugin: 'ish-assembly-branding'

   This activates packaging of branding installations after generation of the dump and publishs them as an artifact branding of type share. The package will be automatically deployed by standard Intershop 7 build plugins.

2. Case: The DBInit in your assembly does not run the BrandingPreparer, but the DBInit of the base assembly did. (If unsure, check out the ivy.xml file of the base assembly in the repository for an artifact of name branding and type share.) Add the following line to the build.gradle file (or just add an include to the existing inheritFrom):

   build.gradle

   assembly {
   	// TODO: Replace by <org>:<name> of your inherited assembly
   	inheritFrom('com.example.assembly:base-assembly') {		
   		// Control which dependencies/artifacts to inherit, see discussion
   		// By default all dependencies in all component subsets and all artifacts are inherited
   		includeArtifacts name:'branding'
   	}
   }

   This will inherit the packaged branding installations from the base assembly.

   
   

Encryption

Some of the database content is encrypted. To decrypt the content again:

1. The previously used encryption must be declared using the same ID in <IS_SHARE>/system/config/cluster/encryption.properties.

2. Also the keystore file <IS_SHARE>/system/config/cluster/intershop.keystore has to contain the same keys or use  algorithm=PLAIN, e.g., no encryption at all.

Before running the DBInit for creating a dump, the assembly process will automatically modify the encryption.properties to use PLAIN as encryption algorithm:

intershop.encryption.0.id = demo
intershop.encryption.0.algorithm = PLAIN
intershop.encryption.0.passphrase = topsecret
intershop.encryption.0.default=true

Recipe: Run DBInit/DBMigrate

(For Gradle tools version >= 2.0, for Gradle tools version 1.0-1.1 see Recipe: Generate a Datatabase Dump)

Problem

The Continuous Integration server should run DBInit and/or DBMigrate during the assembly process for two reasons:

1. Test that DBInit can be run successfully or that a dump of an older version can be migrated. (Also use the resulting database content for further tests, like that the application server starts.)
2. Export a database dump after running DBInit/DBMigrate and publish it as as artifact of the assembly, so it can be used for faster setup of demos / developer environments.

The maintainer of the assembly needs to declare this.

Solution

1. Configure how to run DBInit/DBMigrate:
   
   1. For DBInit choose between running incrementally based on the dump of another assembly, or running a full DBInit over all cartridges.
   2. For DBMigrate specify which dump to import.
   3. Make sure branding installations are handled correctly.
2. Trigger DBInit/DBMigrate:
   1. Specify to export a datatabase dump and publish it as an artifact by setting the project property releaseWithDump.
   2. Alternatively explicitly run the dbinit or dbmigrate task.

You may configure DBInit and DBMigrate in the same assembly, but never run both in the same Gradle execution.

Discussion

Configure DBInit

You may choose between:

1. Creating a dump from scratch: This cleans the database, runs a full DBInit. This is the default behavior.
2. Create a dump incrementally: Import a database dump from another assembly, run a partial DBInit over additional cartridges.

A typical use case for incremental dumps is adding demo data in a special assembly with a demo cartridge, that otherwise inherits from a non-demo assembly.

To create a dump incrementally insert the following block in to the build.gradle file of the assembly (note that this uses an assemblyBuild block as root element):

assemblyBuild {
    dbinit {
		//TODO: Replace by <org>:<name> of the assembly you want to inherit database dump from
        importFrom('com.example.assembly:base-assembly')
        
		//TODO: Insert list of cartridges for which the partial DBInit should be executed
        initCartridges = ['cartridge1', 'cartridge2']
    }
}

For this to work you have to find an assembly, that contains a suitable dump (e.g., containing a subset of the cartridges in your assembly). This is typically be the same assembly that you inherit dependencies from.

If an incremental generation of the database dump is not possible because of some reason, you can always revert to the slower generation from scratch.

Handle Branding Installations

The BrandingPreparer in bc_foundation is used to install branding packages during DBInit. It extracts branding files into the IS_SHARE/sites folder. When initializing a database with a pre-generated dump, instead of running DBInit, these files are missing.

To work around this, the assembly process can also package the extracted branding installations from the sites folder and include them for deployment. There are two cases to consider:

1. Case: The DBInit in your assembly runs the BrandingPreparer, but the DBInit of the base assembly does not (or you do not have a base assembly / perform a full DBInit anyway). Add the following line to build.gradle:

   build.gradle

    apply plugin: 'ish-assembly-branding'

   This activates packaging of branding installations after generation of the dump and publishes them as an artifact branding of type share. The package will be automatically deployed by standard Intershop 7 build plugins.

2. Case: The DBInit in your assembly does not run the BrandingPreparer, but the DBInit of the base assembly did (if unsure, check out the ivy.xml file of the base assembly in the repository for an artifact of name branding and type share). Add the following line to the build.gradle file (or just add an include to the existing  inheritFrom):

   build.gradle

   assembly {
   	// TODO: Replace by <org>:<name> of your inherited assembly
   	inheritFrom('com.example.assembly:base-assembly') {		
   		// Control which dependencies/artifacts to inherit, see discussion
   		// By default all dependencies in all component subsets and all artifacts are inherited
   		includeArtifacts name:'branding'
   	}
   }

   This will inherit the packaged branding installations from the base assembly.

   
   

Configure DBMigrate

Before running DBMigrate a database dump must be imported as start point of the migration.

To import a dump created by the CI server in an older version of the assembly insert the following block in to the build.gradle file of the assembly (note that this uses an assemblyBuild block as root element):

assemblyBuild {
    dbmigrate {
		//TODO: Replace by <org>:<name>:<version> of assembly to import old dump from
        importFrom('com.example.assembly:assembly:1.0.0.0.+')        
    }
}

Alternatively provide a dump file explicitly:

assemblyBuild {
    dbmigrate {
		//TODO: Replace by dump file to import. This example assumes that the dump file is located in the source directory of the assembly
        importFile = file('dump-1.0.0.0.dmp')
    }
}

Include Database Dump as Artifact

To include the database dump as an artifact, decide whether it should originate from running DBInit or DBMigrate.

Add one the following lines to gradle.properties file in your Gradle user home:

# include DBInit dump
releaseWithDump=dbinit
# include DBMigrate dump
releaseWithDump=dbmigrate

If this property is set and you execute the task publish:

1. The dbinit/dbmigrate task and their dependencies (see next section) are executed.
2. The database dump is be exported to a file.
3. The dump file is included as artifact of type dump.

Typically this is used only in the Gradle user home of the CI server. The default value of releaseWithDump is false.

If releaseWithDump is false and you execute the task publish:

• Neither dbinit nor dbmigrate are executed and
• The resulting assembly does not contain an artifact.

This way publishing assemblies for developers is fast. You may still set it for a developer temporarily for testing the behavior.

Run DBInit/DBMigrate Explicitly

You may also run dbmigrate/ dbinit tasks explicitly. This will also trigger their dependencies:

1. Deploy a server
2. Clean the database
3. Import a dump if configured/necessary

If any of the prerequisites or dbmigrate/ dbinit itself fail, the build will fail as well. This can be used to test dbinit/dbmigrate.

It will not automatically export the dump, nor include it as an artifact in the assembly.

You may skip the deployment if you are sure that the last deployment succeeded and nothing relevant has changed. To do so add the project property skipDeployment to your execution. For example:

gradlew dbinit -PskipDeployment

Recipe: Add Solr Search Service

Problem

To use Solr-based search in the Commerce Management application and storefront, it is necessary to deploy the Solr search service and a Solr server.

Solution

1. Add the cartridge com.intershop.services:ac_search_solr to your assembly (follow the recipe Recipe - Add Solr Search Service (valid to 7.8)#Add Cartridges to an Assembly).
2. Deploy a Solr server. To achieve this, add logic to your assembly. There are different deployment scenarios:
   
   1. For a single machine deployment (e.g., for development/demo): Deploy the Solr war-file into the Tomcat of an Intershop 7 application server.
   2. For cluster deployments (e.g., in production): Deploy a separate Solr server in production environments.
3. Configure ac_search_solr to connect to the deployed Solr server

See discussion for details.

Discussion

Deploy Solr Server

The Intershop 7 delivery comes with a component 3rd_solr_war_int that contains the Solr war-file and a deployment script.

Until Intershop Version 7.4.6

The deployment script of 3rd_solr_war_int registers the Solr war-file in the Tomcat of the Intershop 7 appserver instance appserver0.

To deploy this only in some scenarios, create a new host-type appserverWithSolr. Add the following components to the host-type appserverWithSolr:

• com.intershop.services:3rd_solr_war_int
• All components that are contained in the host-type appserver

To do so use the following code in the build.gradle file of your assembly:

...
assembly {
	...
	// Before your existing inheritFrom-block
    hostTypes {
        appserverWithSolr {
            includeLocal = true
            include('com.intershop.services:3rd_solr_war_int') {
                transitive = false
            }
        }
    }

	// In your existing inheritFrom-block
    inheritFrom('<org>:<name>') {
        ...        
        eachDependency { dependency ->
            // // include all components in host type 'appserver' also in host type 'appserverWithSolr'
            if (dependency.configurations.contains(appserver)) {
                dependency.configurations << appserverWithSolr
            }
        } 
    }     
}

Single machine deployments (host type all) include all host types including appserverWithSolr. This way the Solr war-file is deployed into the Apache Tomcat server of an Intershop 7 application server.

When deploying in a cluster, simply use the host type appserver instead of appserverWithSolr to avoid deployment of the Solr war-file. Deploy your Solr server manually (e.g., without the Intershop Deployment Tools) using a standard Apache Tomcat and Solr war-file.

Since Intershop Version 7.5

Intershop 7.5 introduces a new assembly intershop7-solr. This assembly contains Solr search functionality in addition to the intershop7 base components.

This assembly defines the additional host type solr, that deploys a stand-alone Solr search server inside a Tomcat servlet container. Furthermore, host type all also contains Solr search, which will install a Solr server inside the first appserver's Tomcat by default.

Configure ac_search_solr to Connect to Solr Server

Until Intershop Version 7.4.6

Provide a custom deploy.gradle for your assembly. In this deploy.gradle file add the following code:

if (deploymentBootstrap.hostType.hostType.name == 'all') {
    apply plugin: com.intershop.deploy.intershop.SolrDeploymentPlugin
}

The SolrDeploymentPlugin configures <IS_SHARE>/system/config/cartridges/ac_search_solr.properties to connect to the Solr running the Apache Tomcat of Intershop 7. It is only applied for single machine deployments.

For clustered deployments, configure the connection to the separate Solr server during deployment in your settings.gradle. See recipe Configure Intershop 7 in the Cookbook - Gradle Deployment Tools (7.4 CI - ICM 7.7) on how to specify properties read by the configuration framework.

Properties relevant for the Solr based search service are:

Since Intershop version 7.5

Gradle Tools 2.0 introduces a new deployment extension for configuring the Solr cluster for Intershop 7, see Cookbook - Gradle Deployment Tools.

Make sure, that com.intershop.deploy.intershop.SolrDeploymentPlugin is being applied during deployment of the assembly project by either:

• Inheriting the deploy.gradle script from intershop7-solr

• Adding the following line to your custom assembly deployment script:

  applying SolrDeploymentPlugin in custom deploy.gradle

  [...]
  apply plugin: com.intershop.deploy.intershop.SolrDeploymentPlugin

Recipe: Merge Host Types into a New One

Problem

Two or more host types should be used to form a new one.

Solution

• The new host type contains all components of the two previous host types.
• To obtain the values of the attributes includeJavadoc, includeCartridges, includeShare, includeLocal for the new new host type, the corresponding values of the previous host types are OR linked

The following example refers to the merge of the host types:

• appserver
• discoveryserver

to a new host type

• discoveryappserver

assembly {
    hostTypes {
		appserver{
			include (
                'com.intershop:3rd_tomcat',
                'com.intershop:3rd_ant',
                // further components from appserver
            ) {
                transitive = false
            }
            includeLocal = true
            includeCartridges = false          
        }
	}
}

assembly {
    hostTypes {
		discoveryserver
		{
			include (
                'com.intershop.microservice:DiscoveryServer',
            ) {
                transitive = false
            }
		
            includeLocal = true
            includeShare = false
            includeCartridges = false     
        }
	}
}

assembly {
    hostTypes {
		discoveryappserver
		{
			include (
				'com.intershop:3rd_tomcat',
                'com.intershop:3rd_ant',
                // further components from appserver
                'com.intershop.microservice:DiscoveryServer',
            ) {
                transitive = false
            }
		
            includeLocal = true
            includeShare = false
            includeCartridges = false      
        }
	}
}