Migrate Tools Version

To use Gradle tools version 2.0 with Intershop 7 version 7.4.6.1 or 7.4.6.2, perform the following steps to migrate your environment:

1. Import the delivery containing the Gradle tools 2.0 into your repository, see Recipe: Import Intershop Delivery in Cookbook - Setup CI Infrastructure (valid to GradleTools 2.1).
2. Regenerate corporate plugin configuration and corporate distribution by following the Recipe: Create Sources From Intershop Templates in Cookbook - Setup CI Infrastructure (valid to GradleTools 2.1). Run the tasks createCorporateConfiguration, createCorporateDistribution, createDeveloperHome.
3. Recreate/adapt all automated CI processes following the recipes Setup CI Build for Corporate Artifacts and Setup CI Build for Project Artifacts.
4. Build the corporate plugin configuration and the corporate distribution by triggering the automated CI processes.
5. For both, your assembly and component set, perform the following steps:
   
   • Change the Gradle distribution used by the Gradle Wrapper by modifying the property distributionUrl in gradle/wrapper/gradle-wrapper.properties.

   • Add/Change the following properties in the gradle.properties file:

     gradle.properties for updated Gradle Tools

     filter.com.intershop.build.set.cartridge-plugins=2.0.+
     filter.com.intershop.build.set.extension-plugins=2.0.+
     version.com.intershop.deployment-bootstrap=2.0.+

6. Adapt the developer environment by migrating your custom changes to gradle_environment.bat/sh to the newly generated scripts from step 2.
7. In settings.gradle deployment files raise the version of deployment-bootstrap to 'com.intershop:deployment-bootstrap:2.0.+'.

New and Noteworthy

Raised Gradle Version from 1.8 to 2.0

Intershop Gradle tools 2.0 are based on Gradle 2.0. Previous versions were based on Gradle 1.8.

See Gradle's own release notes for changes:

• http://www.gradle.org/docs/1.9/release-notes
• http://www.gradle.org/docs/1.10/release-notes
• http://www.gradle.org/docs/1.11/release-notes
• http://www.gradle.org/docs/1.12/release-notes
• http://www.gradle.org/docs/2.0/release-notes

Deploy Cartridges on Appserver Host

It is possible to deploy cartridges on each appserver host instead of the shared file system. This reduces the downsides of a shared file system as a possible single point of failure and performance bottleneck.

See:

• Recipe: Configure Cartridge Location in Cookbook - Gradle Deployment Tools (7.4 CI - ICM 7.7)
• TargetExtension in Reference - Gradle Deployment Tools

Migration

To deploy cartridges on the appserver, see Recipe: Configure Cartridge Location in Cookbook - Gradle Deployment Tools (7.4 CI - ICM 7.7).

Preinstalled JDK Used by Intershop 7

Intershop 7 no longer deploys its own Java Development Kit (JDK). Instead it uses an existing JDK on your system. This has the following advantages:

1. With Gradle based deployment a JDK was required to start the deployment itself, which makes the second JDK redundant (if both use the same JDK version).
2. Upgrading the JDK, e.g., for security reasons, is simpler and can be performed with standard tools of your operating system.

See:

• Recipe: Configure the Deployment in Cookbook - Gradle Deployment Tools (7.4 CI - ICM 7.7)
• TargetExtension in Reference - Gradle Deployment Tools

Migration

The JDK in <IS_HOME>/engine/jdk will be removed upon first deployment of an assembly based on Intershop 7 version >= 7.5. Some empty directories may be left over – delete them manually.

By default Intershop 7 uses the JDK used to run the deployment. (It may also point to the JRE located inside a JDK.) You may override this, i.e., run Intershop 7 with a different JDK than the deployment. To do so:

1. For development environments specify a property javaHome in <ASSEMBLY>/environment.properties (see <ASSEMBLY>/environment.properties.sample).

In both cases make sure that:

1. The JDK matches the version mentioned in System Requirements Sheet according to your Intershop version.The JCE Unlimited Strength Policy Files are deployed (a manual step required after installing the JDK). 

Host Type for Solr Server Deployment

Clustered and automated deployment is now fully supported for the Solr search server. Intershop 7 has gained an additional host type 'solr' and an assembly 'intershop7-solr' defining it. This host type includes the Intershop nodemanager, tomcat application server and the Solr server application.

See:

• Recipe: Add Solr Search Service in Cookbook - Gradle Assembly Tools (valid to 7.8)
• Recipe: Configure the Deployment in Cookbook - Gradle Deployment Tools (7.4 CI - ICM 7.7)
• SolrDeploymentPlugin in Reference - Gradle Deployment Tools

Migration

To make the new host type available in your assembly perform the following steps in a development environment:

1. Replace the version filter version.com.intershop.assembly.intershop7 by version.com.intershop.assembly.intershop7-solr in the gradle.properties file.
2. Inherit from assembly intershop7-solr instead of intershop7 by replacing the name in all places in the build.gradle file of your assembly.
3. Remove the host type 'appserverWithSolr' created by following the Recipe: Add Solr Search Service in Cookbook - Gradle Assembly Tools (valid to 7.8) for previous versions of Intershop 7 from the build.gradle file.
4. Publish the assembly and check that <ASSEMBLY>/target/publications/ivy/ivy.xml contains the host type 'solr'.

Single machine deployments (like those of developers) will automatically run a Solr server in the first Intershop 7 appserver instance.

For a clustered deployment set up an additional deployment for the new host type, see Cookbook - Gradle Deployment Tools (7.4 CI - ICM 7.7).

Tomcat Servlet Container Deployment

There is a new extensible API for creating and configuring Tomcat servlet container instances. This API enables configuration of settings like JVM arguments or contained Web Applications.

See:

• Recipe: Configure the Deployment in Cookbook - Gradle Deployment Tools (7.4 CI - ICM 7.7)
• Recipe: Deploy a Custom Web Application in Cookbook - Gradle Deployment Tools (7.4 CI - ICM 7.7)
• TomcatPlugin in Reference - Gradle Deployment Tools

Generation and Deployment for Encryption Configuration

Intershop 7 encrypts sensitive parts of the database content (and optionally the database password itself). The accompanying encryption configuration (including the keystore) is now generated before or during the deployment and copied to the deployed server as is.

This eases detection of tampering and recovering it. Also the target directory is configurable, allowing for easier administration of file permissions.

See:

• Recipe: Configure Encryption in Cookbook - Gradle Deployment Tools (7.4 CI - ICM 7.7)
• Recipe: Encrypt Database Password in Cookbook - Gradle Deployment Tools (7.4 CI - ICM 7.7)
• EncryptionAdminPlugin in Reference - Gradle Deployment Tools
• EncryptionDeploymentPlugin in Reference - Gradle Deployment Tools

Migration

For development environments no migration is necessary.

For production/demo environments:

1. Create a backup of your existing encryption configuration files (intershop.keystore, encryption.properties and random) from <IS_SHARE>/system/config/cluster.
2. Copy your existing encryption configuration files to the folder of your settings.gradle deployment file.
   Like your settings.gradle deployment file these files should become part of your storage for deployment configuration. (Version control system or Configuration Management Tool.)

3. Depending on when you want to modify your encryption configuration, perform one of the following alternatives:

   1. If you want to modify configuration during the deployment, add a configuration block to your settings.gradle containing at least the algorithm to use for future encryptions:

      settings.gradle

      [...]
      deploymentBootstrap {
      	[...]
      	config {
      		[...]
      		encryption {
      			// For compatibility reasons the default is the outdated "Password based encryption with MD5 and triple DES" algorithm
      			algorithm = 'PBEWithMD5AndTripleDES'
      		}
      	}
      } 

      The next deployment will adapt your encryption configuration in the folder of the settings.gradle file and the server and make the new algorithm effective for all future encryptions.

   2. If you want to modify configuration in a pre-deployment step, add the following line to your settings.gradle:

      settings.gradle

      [...]
      deploymentBootstrap {
      	[...]
      	config {
          	[...]
      		generateEncryptionConfig.enabled = false
      	}
      } 

      The deployment will not touch the encryption configuration and only deploy it.

4. Remove any workaround introduced in your settings.gradle file. See the Knowledge Base Article Re-deployment of Intershop 7.4 CI overwrites the valid "encryption.properties" for details.
5. Rerun the deployment.

See Recipe: Configure Encryption in Cookbook - Gradle Deployment Tools (7.4 CI - ICM 7.7) for more details.

Enhanced Tooling to Generate Sources

The tool to generate sources to set up a CI Infrastructure and project has been enhanced. It generates sources for:

• Corporate plugin configuration and corporate distribution
• Project assemblies and component sets
• Environment scripts for developers (developer home)
• Environment scripts for CI server
• Templates for deploying a demo/production system

The documentation in the Cookbook - Setup CI Infrastructure (valid to GradleTools 2.1) now covers setting up a CI server, including a full example if using Jenkins as CI Server, Subversion as VCS and Nexus as Repository Server.

The corporate specific repository configuration for the corporate plugin has been reduced to a single properties-file, which makes it easier to maintain and upgrade with future Gradle tools versions. See also section Project/Corporate Specific Artifacts and Configuration in Concept - Gradle Build Tools.

The development environment scripts have been improved (see Recipe: Setup a Development Environment in Cookbook - Gradle Developer Workflow (valid to Gradle Tools 2.7)):

• The developer console includes command aliases for switching between important directories.
• The developer console includes a command alias to start Intershop Studio.
• Environment configuration files that should be checked into a VCS are separated from those that should not. (Environment scripts are relocated to a directory developer_home. Gradle caches and temporary configuration remain in directory gradle_user_home.)

See:

• Cookbook - Setup CI Infrastructure (valid to GradleTools 2.1)

Migration

It is required to use the new tooling to upgrade Gradle tools to version 2.0 (see first chapter of this document).

Run DBMigrate During Assembly Build

In the past the assembly build process on the CI server included running the DBInit. There are two reasons to do so:

• Validate that the DBInit runs without errors.
• Provide the resulting database content as dump for the developers.

It is now also possible to alternatively import an existing database dump and run DBMigrate for the same purpose. (You can also do both for the same assembly in two separate build processes.)

The existing DSL for configuring DBInit has been deprecated and replaced by a new one.

See:

• Recipe: Run DBInit/DBMigrate in Cookbook - Gradle Assembly Tools (valid to 7.8)
• IshAssemblyBuildExtension in Reference - Gradle Assembly Tools

Migration

The following DSL syntax in the build.gradle of the assembly is deprecated:

assemblyBuild {
    database {
        inherit('com.intershop.assembly:intershop7')
        initCartridges = ['ucm_demo']
    }
}

the following syntax:

assemblyBuild {
    dbinit {
        importFrom('com.intershop.assembly:intershop7')
        initCartridges = ['ucm_demo']
    }
    dbmigrate {
        importFrom('com.intershop.assembly:primetech:7.4.6.1.+')       
    }
}

SSL Box Deployment

You may configure the webserver to work inside an external SSL box through the deployment DSL.

See:

• Recipe: Configure the Deployment in Cookbook - Gradle Deployment Tools (7.4 CI - ICM 7.7)
• SSLBoxDeploymentPlugin in Reference - Gradle Deployment Tools

Migration

To run the webserver inside an SSL box add the following to the settings.gradle deployment file for the webserver host.

[...]
deploymentBootstrap {
	[...]
	config {
		[...]
		webadapter {
			[...]
			useSSLBox         = true
		    sslBoxSecuredPort = 12345
		}
	}
}

Replication Aware Deployment

The default behavior of the deployment is to overwrite/revert changes introduced by non-deployment processes (like the application server) when rerun. This can cause problems when using mass data replication for file content.

This is now handled correctly. Files deployed to <IS_SHARE>/sites/*/1 are also deployed to <IS_SHARE>/sites/*/2 for live systems. Also staged content in <IS_SHARE>/sites/*/1 and <IS_SHARE>/sites/*/2  will not be overwritten by deployment.

See:

• Recipe: Configure Mass Data Replication in Cookbook - Gradle Deployment Tools (7.4 CI - ICM 7.7)
• StagingDeploymentPlugin in Reference - Gradle Deployment Tools

Migration

Remove modification handler added as a workaround as described in Recipe: Configure Mass Data Replication in Cookbook - Gradle Deployment Tools (7.4 CI - ICM 7.7) for Gradle tools version 1.0-1.1.

Oracle Service Name in Deployment

To configure the database instance you can now specify an Oracle service name instead of an Oracle SID.

See:

• Recipe: Configure the Deployment in Cookbook - Gradle Deployment Tools (7.4 CI - ICM 7.7)
• DatabaseExtension in Reference - Gradle Deployment Tools

Migration

For development environments specify a property databaseServiceName in <ASSEMBLY>/environment.properties instead of property databaseSid (see <ASSEMBLY>/environment.properties.sample).

For production/demo environments the following DSL syntax in the settings.gradle deployment file is deprecated:

database {
  sid = '<FILL IN>'
}

Replace it with the following syntax:

database {
  serviceName = '<FILL IN>'
}

Build Plugin for JAXB Schema Generator

You may now run the JAXB schema generator in Gradle, generating an XSD from annotated Java classes.

See:

• jaxbSchemaGen plugin in Reference - Gradle Build Tools

Incremental Build for JAXB Generator Fixed

The task jaxb<name> for generating Java classes from xsd-files using JAXB sometimes showed up 'UP-TO-DATE', not regenerating Java classes, even if xsd-files where changed. This is now fixed.

See:

• jaxb plugin in Reference - Gradle Build Tools

Resource Files Generated From WSDL Are Included

The wsdl plugin now includes non-Java files generated from WSDL as resource files in the created jar-file.

See:

• wsdl plugin in Reference - Gradle Build Tools

Import Does Not Republish Existing Modules

Importing Intershop deliveries does no longer cause to publish a module if it already exists in the target repository (with the same organization, name and version).

This improves the speed and safety of imports. Configuring nexus repositories to allow "redeploy" during import is no longer necessary.

See:

• Recipe: Import Delivery in Cookbook - Setup CI Infrastructure (valid to GradleTools 2.1)

Inheriting Host Types Overrides Flags

Inheriting host types in the assembly process now overrides host type flags (includeShare, includeLocal, etc) if the host type was created before. This is closer to the behavior intended in most cases.

Assume the following snippet in the build.gradle of the assembly:

assembly {    
    hostTypes {       
        webserver {            
            include('com.example:component') {
                transitive = false
            }            
        }
    }

    inheritFrom('com.intershop.assembly:intershop7') 
}

The host type webserver in the inherited assembly intershop7 has the flag includeLocal = true.

In Gradle tools version 1.0-1.1 publishing above assembly will result in a host type definition with includeLocal = false in the ivy.xml file of the assembly.

With Gradle tools version 2.0 includeLocal is inherited as true.

See:

• Recipe: Add Infrastructure Component to an Assembly in Cookbook - Gradle Assembly Tools (valid to 7.8)

Host Type javadoc Is Inherited

The host type javadoc for deploying only the Javadoc was only available in assembly intershop7. It is now also available in all assemblies inheriting from it, unless explicitly excluded (or explicitly only including some environments or host types).

Fixed Redeployment of Symbolic Links

Deployment of symbolic links failed if the link existed, but the link target did not. This is now fixed.

See:

• Recipe: Create a Custom Symbolic Link in Cookbook - Gradle Deployment Tools (7.4 CI - ICM 7.7)

Generated Files Are Part of the Index

Files whose complete content is generated during the deployment can now be part of the index. This allows the deployment to detect modifications and collisions like for all other deployed files.

Examples of generated files are:

• cartridgelist.properties (by default in <IS_SHARE>/system/cartridges)

Experimental: Faster Up-To-Date Checks

Like Gradle for copying the deployment uses checksums for checking whether a file must be updated or not. Gradle tools 2.0 try to avoid calculating checksums for repository files based on the files metadata. This feature is still experimental, but can help speeding up redeployments in development of Intershop 7 and custom deployment logic tremendously.

To activate this experimental feature either set the Java system property FastSourceChecksum to true when running the deployment (or an assembly process running the deployment). To set the system property:

• Pass -DFastSourceChecksum=true when running gradlew of assembly or deployment to activate it for a single run.
• Alternatively add the line systemProp.FastSourceChecksum=true in the gradle.properties file of the Gradle user home to activate it permanently (for all processes using this Gradle user home).

Known Issues

Redeployment Resets the System's Solr Configuration

For systems that are not part of a replication environment the deployment overwrites changes made to the file <IS_SHARE>/system/config/cluster/solr/solr.xml. As this is the place, where Solr stores its index configurations, this basically resets all Solr data.

Workaround

Define a custom modification handler, e.g.:

deployment {
    modification {
        if (target.includeShare) {
            keep('solrXml') {
                priority 'myProject' // TODO: adjust
                dir target.shareDirectory
                include 'system/config/cluster/solr/solr.xml'
            }
        }
    }
}

Limitations

This issue only covers the given solr.xml configuration file, not the index data itself. If you deploy pregenerated solr indexes (into the <IS_SHARE>/sites folder), you have to define own modification handling rules for those files.

Versioning Issues

The Gradle Tools 2.0.1 Patch Release fixes the following issues:

Snapshot Dependencies Are Not Automatically Refreshed

Dependencies to a snapshot components (whose version ends with -SNAPSHOT) will not get refreshed for each build, which may result in Gradle using outdated artifacts from its local cache.

Project Dependencies Use Outdated Files When a Version Filter Is Applied

If you have a project depending on another project in your component set, and you make changes to both projects that depend on one another, the build may fail because the depending project will use the outdated version of its dependency.

Gradle May Incorrectly Use Obsolete Versions of Local Modules

When upgrading local component set sources to a newer version, obsolete artifacts of the old version may still be used for the assembly.

Known Issues

Generated Dependency Versions

The intershop-ci-bootstrap will generate projects with dependencies to Gradle Tools plugins in version 2.+. This may break the build once newer releases of Gradle Tools are made available, because they may not be compatible with older Gradle base versions anymore.

Please change the following properties in all your gradle.properties files from

filter.com.intershop.build.set.cartridge-plugins=2.+
filter.com.intershop.build.set.extension-plugins=2.+
version.com.intershop.deployment-bootstrap=2.+

to

filter.com.intershop.build.set.cartridge-plugins=2.0.+
filter.com.intershop.build.set.extension-plugins=2.0.+
version.com.intershop.deployment-bootstrap=2.0.+