Public Release Notes - Gradle Tools - Version 2.0

Note

This Gradle tools version is included in Intershop 7 version 7.4.6.3 and 7.5.0.0. It can also be used along with Intershop 7 version 7.4.6.1 and 7.4.6.2.

This Gradle tools version is based on Gradle 2.0.

1 Migrate Tools Version

To use Gradle tools version 2.0 with Intershop 7 version 7.4.6.1 or 7.4.6.2, perform the following steps to migrate your environment:

  1. Import the delivery containing the Gradle tools 2.0 into your repository, see Recipe: Import Intershop Delivery in Cookbook - Setup CI Infrastructure (valid to GradleTools 2.1).
  2. Regenerate corporate plugin configuration and corporate distribution by following the Recipe: Create Sources From Intershop Templates in Cookbook - Setup CI Infrastructure (valid to GradleTools 2.1). Run the tasks createCorporateConfiguration, createCorporateDistribution, createDeveloperHome.
  3. Recreate/adapt all automated CI processes following the recipes Setup CI Build for Corporate Artifacts and Setup CI Build for Project Artifacts.
  4. Build the corporate plugin configuration and the corporate distribution by triggering the automated CI processes.
  5. For both, your assembly and component set, perform the following steps:
    • Change the Gradle distribution used by the Gradle Wrapper by modifying the property distributionUrl in gradle/wrapper/gradle-wrapper.properties.
    • Add/Change the following properties in the gradle.properties file:

      gradle.properties for updated Gradle Tools
      filter.com.intershop.build.set.cartridge-plugins=2.0.+
      filter.com.intershop.build.set.extension-plugins=2.0.+
      version.com.intershop.deployment-bootstrap=2.0.+
  6. Adapt the developer environment by migrating your custom changes to gradle_environment.bat/sh to the newly generated scripts from step 2.
  7. In settings.gradle deployment files raise the version of deployment-bootstrap to 'com.intershop:deployment-bootstrap:2.0.+'.

Other Migration Steps

Besides these general migration steps to get to use version 2.0, none of the migration steps in this document is mandatory. (They are only required if you want to take advantage of new features.)

Some new features (and their migration steps) only work in conjunction with migrating Intershop 7 to version >= 7.5. This is noted where relevant.

2 New and Noteworthy

2.1 Raised Gradle Version from 1.8 to 2.0

Intershop Gradle tools 2.0 are based on Gradle 2.0. Previous versions were based on Gradle 1.8.

See Gradle's own release notes for changes:

Compatibility

We raised the major version of Intershop Gradle tools to indicate that compatibility could be broken because of the update to Gradle 2.0. However, besides the upgrade to Gradle 2.0, all changes introduced in Intershop Gradle tools are backwards compatible. (Especially, no deprecated features were removed.)

2.2 Deploy Cartridges on Appserver Host

It is possible to deploy cartridges on each appserver host instead of the shared file system. This reduces the downsides of a shared file system as a possible single point of failure and performance bottleneck.

See:

Note

This only works in along with Intershop 7 version >= 7.5. In earlier versions the cartridges must still be deployed in the shared file system.

Migration

To deploy cartridges on the appserver, see Recipe: Configure Cartridge Location in Cookbook - Gradle Deployment Tools (7.4 CI - ICM 7.7).

2.3 Preinstalled JDK Used by Intershop 7

Intershop 7 no longer deploys its own Java Development Kit (JDK). Instead it uses an existing JDK on your system. This has the following advantages:

  1. With Gradle based deployment a JDK was required to start the deployment itself, which makes the second JDK redundant (if both use the same JDK version).
  2. Upgrading the JDK, e.g., for security reasons, is simpler and can be performed with standard tools of your operating system.

See:

Note

This only works in conjunction with Intershop 7 version >= 7.5. Earlier versions still deploy a JDK to <IS_HOME>/engine/jdk and rely on that location.

Migration

The JDK in <IS_HOME>/engine/jdk will be removed upon first deployment of an assembly based on Intershop 7 version >= 7.5. Some empty directories may be left over – delete them manually.

By default Intershop 7 uses the JDK used to run the deployment. (It may also point to the JRE located inside a JDK.) You may override this, i.e., run Intershop 7 with a different JDK than the deployment. To do so:

  1. For development environments specify a property javaHome in <ASSEMBLY>/environment.properties (see <ASSEMBLY>/environment.properties.sample).

In both cases make sure that:

  1. The JDK matches the version mentioned in System Requirements Sheet according to your Intershop version.The JCE Unlimited Strength Policy Files are deployed (a manual step required after installing the JDK). 

2.4 Host Type for Solr Server Deployment

Clustered and automated deployment is now fully supported for the Solr search server. Intershop 7 has gained an additional host type 'solr' and an assembly 'intershop7-solr' defining it. This host type includes the Intershop nodemanager, tomcat application server and the Solr server application.

See:

Note

This only works in conjunction with Intershop 7 version >= 7.5 and a compatible version of the Solr search service. Earlier versions still only support solr in single machine deployments.

Migration

To make the new host type available in your assembly perform the following steps in a development environment:

  1. Replace the version filter version.com.intershop.assembly.intershop7 by version.com.intershop.assembly.intershop7-solr in the gradle.properties file.
  2. Inherit from assembly intershop7-solr instead of intershop7 by replacing the name in all places in the build.gradle file of your assembly.
  3. Remove the host type 'appserverWithSolr' created by following the Recipe: Add Solr Search Service in Cookbook - Gradle Assembly Tools (valid to 7.8) for previous versions of Intershop 7 from the build.gradle file.
  4. Publish the assembly and check that <ASSEMBLY>/target/publications/ivy/ivy.xml contains the host type 'solr'.

Single machine deployments (like those of developers) will automatically run a Solr server in the first Intershop 7 appserver instance.

For a clustered deployment set up an additional deployment for the new host type, see Cookbook - Gradle Deployment Tools (7.4 CI - ICM 7.7).

2.5 Tomcat Servlet Container Deployment

There is a new extensible API for creating and configuring Tomcat servlet container instances. This API enables configuration of settings like JVM arguments or contained Web Applications.

See:

Note

This only works in conjunction with Intershop 7 version >= 7.5. In earlier versions usage of this API will have no effect.

2.6 Generation and Deployment for Encryption Configuration

Intershop 7 encrypts sensitive parts of the database content (and optionally the database password itself). The accompanying encryption configuration (including the keystore) is now generated before or during the deployment and copied to the deployed server as is.

This eases detection of tampering and recovering it. Also the target directory is configurable, allowing for easier administration of file permissions.

See:

Migration

For development environments no migration is necessary.

For production/demo environments:

  1. Create a backup of your existing encryption configuration files (intershop.keystore, encryption.properties and random) from <IS_SHARE>/system/config/cluster.
  2. Copy your existing encryption configuration files to the folder of your settings.gradle deployment file.
    Like your settings.gradle deployment file these files should become part of your storage for deployment configuration. (Version control system or Configuration Management Tool.)
  3. Depending on when you want to modify your encryption configuration, perform one of the following alternatives:

    1. If you want to modify configuration during the deployment, add a configuration block to your settings.gradle containing at least the algorithm to use for future encryptions:

      settings.gradle
      [...]
      deploymentBootstrap {
      	[...]
      	config {
      		[...]
      		encryption {
      			// For compatibility reasons the default is the outdated "Password based encryption with MD5 and triple DES" algorithm
      			algorithm = 'PBEWithMD5AndTripleDES'
      		}
      	}
      } 

      The next deployment will adapt your encryption configuration in the folder of the settings.gradle file and the server and make the new algorithm effective for all future encryptions.

    2. If you want to modify configuration in a pre-deployment step, add the following line to your settings.gradle:

      settings.gradle
      [...]
      deploymentBootstrap {
      	[...]
      	config {
          	[...]
      		generateEncryptionConfig.enabled = false
      	}
      } 

      The deployment will not touch the encryption configuration and only deploy it.

  4. Remove any workaround introduced in your settings.gradle file. See the Knowledge Base Article Re-deployment of Intershop 7.4 CI overwrites the valid "encryption.properties" for details.
  5. Rerun the deployment.

See Recipe: Configure Encryption in Cookbook - Gradle Deployment Tools (7.4 CI - ICM 7.7) for more details.

2.7 Enhanced Tooling to Generate Sources

The tool to generate sources to set up a CI Infrastructure and project has been enhanced. It generates sources for:

  • Corporate plugin configuration and corporate distribution
  • Project assemblies and component sets
  • Environment scripts for developers (developer home)
  • Environment scripts for CI server
  • Templates for deploying a demo/production system

The documentation in the Cookbook - Setup CI Infrastructure (valid to GradleTools 2.1) now covers setting up a CI server, including a full example if using Jenkins as CI Server, Subversion as VCS and Nexus as Repository Server.

The corporate specific repository configuration for the corporate plugin has been reduced to a single properties-file, which makes it easier to maintain and upgrade with future Gradle tools versions. See also section Project/Corporate Specific Artifacts and Configuration in Concept - Gradle Build Tools.

The development environment scripts have been improved (see Recipe: Setup a Development Environment in Cookbook - Gradle Developer Workflow (valid to Gradle Tools 2.7)):

  • The developer console includes command aliases for switching between important directories.
  • The developer console includes a command alias to start Intershop Studio.
  • Environment configuration files that should be checked into a VCS are separated from those that should not. (Environment scripts are relocated to a directory developer_home. Gradle caches and temporary configuration remain in directory gradle_user_home.)

See:

Note

The documentation refers to a new repository ishreleases. In previous versions we used only releases for Intershop releases as well for the project releases. This will be an issue, when Intershop provides a public repository. In this case it is no longer necessary to import a release from DVD, but it is necessary to configure a proxy repository. With different repositories it is possible to change the configuration for the repository with Intershop releases. Therefore we added this new repository ishreleases.

It is also possible to use the old configuration, in this case ishreleases can be replaced with releases.

Migration

It is required to use the new tooling to upgrade Gradle tools to version 2.0 (see first chapter of this document).

2.8 Run DBMigrate During Assembly Build

In the past the assembly build process on the CI server included running the DBInit. There are two reasons to do so:

  • Validate that the DBInit runs without errors.
  • Provide the resulting database content as dump for the developers.

It is now also possible to alternatively import an existing database dump and run DBMigrate for the same purpose. (You can also do both for the same assembly in two separate build processes.)

The existing DSL for configuring DBInit has been deprecated and replaced by a new one.

See:

Migration

The following DSL syntax in the build.gradle of the assembly is deprecated:

build.gradle of assembly
assemblyBuild {
    database {
        inherit('com.intershop.assembly:intershop7')
        initCartridges = ['ucm_demo']
    }
}

the following syntax:

build.gradle of assembly
assemblyBuild {
    dbinit {
        importFrom('com.intershop.assembly:intershop7')
        initCartridges = ['ucm_demo']
    }
    dbmigrate {
        importFrom('com.intershop.assembly:primetech:7.4.6.1.+')       
    }
}

2.9 SSL Box Deployment

You may configure the webserver to work inside an external SSL box through the deployment DSL.

See:

Migration

To run the webserver inside an SSL box add the following to the settings.gradle deployment file for the webserver host.

[...]
deploymentBootstrap {
	[...]
	config {
		[...]
		webadapter {
			[...]
			useSSLBox         = true
		    sslBoxSecuredPort = 12345
		}
	}
}

2.10 Replication Aware Deployment

The default behavior of the deployment is to overwrite/revert changes introduced by non-deployment processes (like the application server) when rerun. This can cause problems when using mass data replication for file content.

This is now handled correctly. Files deployed to <IS_SHARE>/sites/*/1 are also deployed to <IS_SHARE>/sites/*/2 for live systems. Also staged content in <IS_SHARE>/sites/*/1 and <IS_SHARE>/sites/*/2  will not be overwritten by deployment.

See:

Migration

Remove modification handler added as a workaround as described in Recipe: Configure Mass Data Replication in Cookbook - Gradle Deployment Tools (7.4 CI - ICM 7.7) for Gradle tools version 1.0-1.1.

2.11 Oracle Service Name in Deployment

To configure the database instance you can now specify an Oracle service name instead of an Oracle SID.

See:

Migration

For development environments specify a property databaseServiceName in <ASSEMBLY>/environment.properties instead of property databaseSid (see <ASSEMBLY>/environment.properties.sample).

For production/demo environments the following DSL syntax in the settings.gradle deployment file is deprecated:

Config DSL block
database {
  sid = '<FILL IN>'
}

Replace it with the following syntax:

Config DSL block
database {
  serviceName = '<FILL IN>'
}

2.12 Build Plugin for JAXB Schema Generator

You may now run the JAXB schema generator in Gradle, generating an XSD from annotated Java classes.

See:

2.13 Incremental Build for JAXB Generator Fixed

The task jaxb<name> for generating Java classes from xsd-files using JAXB sometimes showed up 'UP-TO-DATE', not regenerating Java classes, even if xsd-files where changed. This is now fixed.

See:

2.14 Resource Files Generated From WSDL Are Included

The wsdl plugin now includes non-Java files generated from WSDL as resource files in the created jar-file.

See:

2.15 Import Does Not Republish Existing Modules

Importing Intershop deliveries does no longer cause to publish a module if it already exists in the target repository (with the same organization, name and version).

This improves the speed and safety of imports. Configuring nexus repositories to allow "redeploy" during import is no longer necessary.

See:

2.16 Inheriting Host Types Overrides Flags

Inheriting host types in the assembly process now overrides host type flags (includeShare, includeLocal, etc) if the host type was created before. This is closer to the behavior intended in most cases.

Assume the following snippet in the build.gradle of the assembly:

build.gradle of assembly
assembly {    
    hostTypes {       
        webserver {            
            include('com.example:component') {
                transitive = false
            }            
        }
    }

    inheritFrom('com.intershop.assembly:intershop7') 
}

The host type webserver in the inherited assembly intershop7 has the flag includeLocal = true.

In Gradle tools version 1.0-1.1 publishing above assembly will result in a host type definition with includeLocal = false in the ivy.xml file of the assembly.

With Gradle tools version 2.0 includeLocal is inherited as true.

See:

2.17 Host Type javadoc Is Inherited

The host type javadoc for deploying only the Javadoc was only available in assembly intershop7. It is now also available in all assemblies inheriting from it, unless explicitly excluded (or explicitly only including some environments or host types).

Note

When using Gradle tools in conjunction with Intershop 7 version < 7.5, the assemblies primetech and primetech-solr still miss host type javadoc. After rebuilding your custom assembly it will contain the host-type.

Deployment of symbolic links failed if the link existed, but the link target did not. This is now fixed.

See:

2.19 Generated Files Are Part of the Index

Files whose complete content is generated during the deployment can now be part of the index. This allows the deployment to detect modifications and collisions like for all other deployed files.

Examples of generated files are:

  • cartridgelist.properties (by default in <IS_SHARE>/system/cartridges)

2.20 Experimental: Faster Up-To-Date Checks

Like Gradle for copying the deployment uses checksums for checking whether a file must be updated or not. Gradle tools 2.0 try to avoid calculating checksums for repository files based on the files metadata. This feature is still experimental, but can help speeding up redeployments in development of Intershop 7 and custom deployment logic tremendously.

To activate this experimental feature either set the Java system property FastSourceChecksum to true when running the deployment (or an assembly process running the deployment). To set the system property:

  • Pass -DFastSourceChecksum=true when running gradlew of assembly or deployment to activate it for a single run.
  • Alternatively add the line systemProp.FastSourceChecksum=true in the gradle.properties file of the Gradle user home to activate it permanently (for all processes using this Gradle user home).

3 Known Issues

3.1 Redeployment Resets the System's Solr Configuration

For systems that are not part of a replication environment the deployment overwrites changes made to the file <IS_SHARE>/system/config/cluster/solr/solr.xml. As this is the place, where Solr stores its index configurations, this basically resets all Solr data.

Workaround

Define a custom modification handler, e.g.:

Custom modification handler
deployment {
    modification {
        if (target.includeShare) {
            keep('solrXml') {
                priority 'myProject' // TODO: adjust
                dir target.shareDirectory
                include 'system/config/cluster/solr/solr.xml'
            }
        }
    }
}

Limitations

This issue only covers the given solr.xml configuration file, not the index data itself. If you deploy pregenerated solr indexes (into the <IS_SHARE>/sites folder), you have to define own modification handling rules for those files.

3.2 Versioning Issues

The Gradle Tools 2.0.1 Patch Release fixes the following issues:

Snapshot Dependencies Are Not Automatically Refreshed

Dependencies to a snapshot components (whose version ends with -SNAPSHOT) will not get refreshed for each build, which may result in Gradle using outdated artifacts from its local cache.

Project Dependencies Use Outdated Files When a Version Filter Is Applied

If you have a project depending on another project in your component set, and you make changes to both projects that depend on one another, the build may fail because the depending project will use the outdated version of its dependency.

Gradle May Incorrectly Use Obsolete Versions of Local Modules

When upgrading local component set sources to a newer version, obsolete artifacts of the old version may still be used for the assembly.

4 Known Issues

4.1 Generated Dependency Versions

The intershop-ci-bootstrap will generate projects with dependencies to Gradle Tools plugins in version 2.+. This may break the build once newer releases of Gradle Tools are made available, because they may not be compatible with older Gradle base versions anymore.

Please change the following properties in all your gradle.properties files from

problematic gradle.properties
filter.com.intershop.build.set.cartridge-plugins=2.+
filter.com.intershop.build.set.extension-plugins=2.+
version.com.intershop.deployment-bootstrap=2.+

to

safe gradle.properties
filter.com.intershop.build.set.cartridge-plugins=2.0.+
filter.com.intershop.build.set.extension-plugins=2.0.+
version.com.intershop.deployment-bootstrap=2.0.+

Disclaimer

The information provided in the Knowledge Base may not be applicable to all systems and situations. Intershop Communications will not be liable to any party for any direct or indirect damages resulting from the use of the Customer Support section of the Intershop Corporate Web site, including, without limitation, any lost profits, business interruption, loss of programs or other data on your information handling system.

Customer Support
Knowledge Base
Product Resources
Support Tickets