Introduction

Due to technological development and security issues it is sometimes necessary to solve a problem in a production installation - very fast and with a minimum impact to other functions. For these cases Intershop introduced custom fixes. A custom fix contains only a minimum set of files that must be changed to solve a particular issue. To allow immediate use in production it must not contain changes of the API. The solution provided in a custom fix can be customer specific. The usage of a custom fix is an exception to handle an emergency issue in a production environment.

This document describes the workflow of building and using custom fixes with the Gradle-based build and deployment.

Custom Fix as a Component

Within the Gradle build and deploy tools a custom fix is represented by a component. It shares most of the properties of a cartridge or infrastructure component:

• It has a qualified name and a version number.
• It is stored in the repository.
• Its deployment is an automated process.
• To deploy a custom fix it must be added to the assembly.

A custom fix contains the following Ivy artifacts:

The whole custom fix including all artifacts can be delivered as a single *.zip file.

Custom Fix Deployment and Conflicts

A custom fix typically patches code artifacts (pipelines, ISML templates, Java classes, shell scripts) from other components, i.e., it contains code artifacts with the same ID (file name/qualified class name etc.) as other deployed components. This requires some kind of conflict handling, i.e., it has to be determined which of the differing versions of the same code artifact to prefer at deployment or runtime. The deploy.gradle file of a custom fix makes sure that code artifacts from the custom fix will be preferred.

An unresolved conflict occurs if multiple custom fixes contain the same code artifact.

Java Classes

For Java classes the jar-file of the custom fix is deployed additionally to the original jar-file(s) and added to the beginning of the Java class path. While the original classes still reside in the original jar-files in the deployed system, class loaders will prefer classes from custom fixes.

There is no detection of collisions between two custom fixes for Java classes at deployment time.

Other Code Artifacts

All other code artifacts are physically replaced on the file level, i.e., the file from the custom fix is deployed instead of the original file. For this the deploy.gradle file of the custom fix declares to exclude the file from the deployment of the original component.

If two custom fixes try to deploy to the same target file however, the deployment will issue a conflict and fail. To solve the conflict you must either:

• Replace the conflicting fixes by a single merged fix or
• If the file content from one of the custom fixes supersedes the others already – make its file be preferred by declaring exclusions for the other fixes in the deploy.gradle file of your assembly.

Custom Fix Tool

Part of the Intershop delivery is a template for a custom fix tool. This template is prepared for the project environment during the setup of development source artifacts (see Cookbook - Setup CI Infrastructure (valid to GradleTools 1.1)). These scripts can also be generated separately:

cd <deliveries>\Intershop-<version>\development-setup
 ..\gradlew :createCustomFixTool

It is possible to store these scripts in an VCS. Part of this directory structure is also an example of a custom fix, which can be removed from the VCS. 

 custom-fix
 |   Readme
 |
 +---builder 					scripts for publishing
 |   |   build.gradle 				Gradle build file
 |   |   gradle.properties 			configuration file of the custom fix
 |   |   gradlew[.bat]          	Gradle starter
 |   |   run.[bat|sh] 				start script for publishing
 |   |
 |   +---artifacts
 |   |   \---example 			example custom fix (name of the custom fix)
 |   |           deploy.gradle 		deploy script
 |   |           lib.jar			library archive
 |   |           share.zip 			package for the shared directory
 |   |
 |   \---gradle 				Gradle wrapper
 |
 +---exporter                   scripts for the export 
 |       build.gradle 				Gradle build file
 |       transfer.gradle        	Gradle build file
 |       gradle.properties  		configuration file of the custom fix
 |       run.[bat|sh]           	export start script
 |
 +---home-builder 				Gradle user home of the publishing process
 |       gradle.properties	    	configuration file with deployment user settings
 |       init.gradle 				Gradle initialization script
 |
 +---home-expimp 				Gradle user home of the export and import
 |       gradle.properties      	configuration file with deployment user settings
 |       init.gradle            	Gradle initialization script
 |
 \---importer                   scripts for the import of a custom fix
     |   build.gradle 			Gradle build file
     |   import.gradle          Gradle build file
     |   import.properties      configuration file (see 'Cookbook - Setup CI Infrastructure')
     |   run.[bat|sh]           import start script
     |
     +---artifacts				folder for the unzipped custom fix 
     \---repository	

Recipe: Publishing Artifacts to a Remote Repository

Problem

A developer created a jar file, a zip file for the shared file system and/or the local part of the deployed system containing patched and new code artifacts. The deploy.gradle file of the custom fix was prepared and adapted to exclude files from the deployment of the original components.

How can a developer or build engineer publish these artifacts as a custom fix to a remote artifact repository?

Solution

1. Store the necessary artifacts in the folder <custom fix tool set>/builder/artifacts/<custom fix name>

2. Edit the configuration file <custom fix tool set>/builder/gradle.properties

   gradle.properties

   # custom fix configuration
   
   # name or id of the custom fix
   # note: this is also used for file names
   customfixName = <custom fix name>
   
   
   # group or organization of custom fix
   # this is used in the repository
   customfixGroup = com.intershop.customfix
   
   
   # version of the custom fix
   customfixVersion = 1.0.0.0

3. Start the command for publishing

   Working directory	<custom fix tool set>/builder
   Windows command	run.bat
   Linux command	./run.sh

Discussion

The artifacts for this process must be prepared by a developer manually. There is currently no automatic process for this step.

Recipe: Export a Custom Fix From a Remote Repository

Problem

A custom fix is stored in a remote artifact repository. It is necessary to store it in a zip file, so that it can be sent to another location or environment via email.

How can a developer or build engineer export the custom fix from a remote repository to a zip file?

Solution

1. Edit the configuration file <custom fix tool set>/exporter/gradle.properties. See Recipe: Publishing of Artifacts to a Remote Repository.

2. Start the command for export

   Working directory	<custom fix tool set>/exporter
   Windows command	run.bat
   Linux command	./run.sh

The zip file of the delivery package is located in <custom fix tool set>/exporter/target.

Discussion

This step is only necessary for transferring custom fixes across separate environments. In most cases one repository is used for one company or project and different environments, so exporting the custom fix is not necessary.

Recipe: Import a Custom Fix to a Remote Repository

Problem

The custom fix was received in a zip-file as part of an email.

How can a build engineer or administrator import this custom fix to a remote repository of a certain environment?

Solution

1. Copy the zip-file of the custom fix to <custom fix tool set>/importer/artifacts.


2. Unzip it in the same directory without any changes.

   Note

   On a Windows system the path length can be limited. Move the directory of the custom fix tool set to a shorter path, so all Windows tools can handle it.

   
   

3. Check the import.properties for the correct settings. This file was also prepared in the setup of source artifacts for the CI environment (see Cookbook - Setup CI Infrastructure (valid to GradleTools 1.1)).

   import.properties

   # import properties for the Intershop artifacts
   # Which repository will be used?
   # remote - All artifacts will be transfered to a remote repostiors
   #          like Sonatype Nexus or JFrog Artifactore or others.
   #          This is the most recommended setting for a standard setup.
   # local  - All artifacts will be transfered to a local repository.
   repo = remote
   
   # These are the settings for the remote repository.
   #
   # URL of the repository for Intershop releases
   # example:
   # 	target.repo.url = http://documentation:8081/nexus/content/repositories/releases
   target.repo.url = http://documentation:8081/nexus/content/repositories/releases
   
   # These are the user settings of an user with the permission to 
   # upload artifacts to the remote repository.
   # example
   #   deploy.user.login = admin
   #   deploy.user.password = admin123
   deploy.user.login = admin
   deploy.user.password = admin123
   
   # This is the path for a other local repository.
   target.repo.dir = /local/repository

   
   

4. Start the command for import.

   Working directory	<custom fix tool set>/importer
   Windows command	run.bat
   Linux command	./run.sh

5. The custom fix is now copied to the repository.

Discussion

It is also possible to transfer the custom fix to a local repository. For this use case set the property  repo in the import.properties to local.

Recipe: Deploy a Custom Fix

Problem

The custom fix is uploaded to a repository in the CI environment of the customer.

How can a build engineer or adminstrator deploy this custom fix?

Solution

Deploying a custom fix involves two high-level steps, described in the next two sections.

Add the Custom Fix to the Assembly

The assembly must be modified to contain the custom fix. This is a (simple) development task.

group=com.intershop.hotfix
name=IS-1234
version=1.2.3.4

1. To test the custom fix locally you may setup a full development environment as described at Cookbook - Gradle Developer Workflow (valid to Gradle Tools 2.7). Alternatively you can just check out the assembly sources from your VCS.
   In the VCS working copy of the assembly follow the recipe Add Infrastructure Components To An Assembly  in the Cookbook - Gradle Assembly Tools (valid to 7.8). Add the custom fix to the host-type appserver and share.

   build.gradle

   assembly {
   	...
   	hostTypes {
           appserver {
   			include '<group of custom fix>:<name of the custom fix>'       
           }
           share {
   			include '<group of custom fix>:<name of the custom fix>'       
           }
       }
   	...
   }

2. Add version.<group of custom fix>.<name of the custom fix> = <version number of custom fix> to the gradle.properties of the assembly. By using a fixed version number instead of a version expression like '+' you make sure that newer versions are not unexpectedly used upon next release of the assembly. See section Configuring Dependency Versions in the Concept - Gradle Build Tools.

3. To test your changes locally you may optionally redeploy the assembly and test it. See recipe Make Changes Effective on the Server  in the Cookbook - Gradle Developer Workflow (valid to Gradle Tools 2.7).
4. Commit your changes to the assembly.

After committing trigger your CI server to re-release the assembly. You may want to setup a "fast lane" release process next to your ordinary release process, see discussion.

Custom fixes are deployed for all environments, including development. The limitation here is: Developers will not compile against patched classes, but they will test against them at runtime.

Deploy the Changed Assembly

To deploy the changed assembly including the custom fix on any (non-developer) system:

1. Change the version number of your assembly in settings.gradle at the top of the assembly DSL block to the just released version.
2. Re-run the deployment. You may either rerun the whole deployment or make a partial redeployment, see discussion.

Depending on how you organized your production deployment, this may involved triggering Configuration Management Tools, using a VCS or custom scripts or manual actions.

When deploying to a cluster, trigger re-deployment on all cluster nodes with host-types appserver and share.

Discussion

Fast-Lane Release Process

If your release process takes too long, because it involves too many steps like rebuilding the component set and creating a database dump, you may want to create a "fast lane" process for releasing the assembly with custom fixes. In this process (e.g., a separate build plan):

• Run the assembly process with project property releaseWithDump=false. This will not execute the DBInit.
• Call the Gradle publish task of the assembly, avoid running Gradle task test.
• Give it a special version number, so developers will not try to fetch a database dump from it. To achieve this override the project property  version to something like 1.0.0.fastlane, where  1.0.0 is your regular version number.

This will only run the quick core of the assembly process, see chapter Assembly Process  in the Concept - Gradle Assembly Tools.

Even if you use this fast-lane approach the regular release will include the custom fixes as well, making sure that they will stay with the next custom fix/feature deployment.

Partial vs. Full Redeployment

You can always run the Gradle task deploy in the directory of your settings.gradle for a full deployment. This will calculate the differences between the last deployment and this one and only touch these files. (See section Modification Handling in the Concept - Gradle Deployment Tools for handling of files that were modified after deployment.)

For a faster deployment, that will only touch files coming with the custom fix, you may run a partial deployment. Run the Gradle task :<custom fix name>:deployFiles for this. (This automatically replaces files by other, non-executed deployment tasks – like from the original cartridges.)

Undeploy Custom Fix

To undeploy a custom fix:

• Remove it from the build.gradle file of the assembly.
• Republish the assembly.
• Redeploy the assembly. For partial deployment, run the tasks :<cartridge>:deployFiles for all affected cartridges. (This also checks for all other files of the cartridges whether they are up-to-date with the repository contents.)