Introduction

The locking framework provides a mechanism to prevent certain parallel processes accessing the same resources at the same time to avoid conflicts within an Intershop 7 cluster. For example, it must be impossible to import and to replicate the same products at the same time to avoid inconsistent data in the live system. The resources are entities that are stored in the database and describe what needs to be accessed (e.g., products of a repository, a file in file system, a particular product, the entire database). The according batch processes can acquire locks for these resources to avoid concurrent access. In case the lock acquisition fails, the process can wait or abort the execution.

Entities in the Locking Framework

The following entities take part in a batch process locking resources:

High Level Workflow

The high level workflow of a batch process within the application server should look like this:

1. Process creation to make it visisble in SMC
2. Process start (simple state change)
3. Acquistion of the required resources
4. Work
5. Release of the required resources
6. Process finish

References

• Cookbook - Locking Framework
• Concept - Mass Data Replication
• Concept - Impex Framework

Resources

Resources are used to describe what needs to be locked. They have a name, a description and some information about the current locking status. There are two basic types of resources: Named Resources and Instance Resources.

Instance Resources

Instance resources are used to describe a concrete persistent object instance that needs to be locked. Instance resources do not have any parent or child resources. As instance resources refer to arbitrary objects, they are created at runtime and do not need to be prepared during DBInit or DBMigrate processes.

Examples:

A back-office user locks a certain product for editing.
A back-office user locks certain CMS components for editing.

Named Resources

Named resources are used to describe an amount of objects that need to be locked. The root of all named resources is the cluster resource. More specialized resources are available, e.g., for the database, shared file system resources or schedules. If a named resource is locked, all its child resources are locked as well.

An overview of all Named Resources on your system can be found in the System Management | Monitoring | Locking | Named Resources.

Examples:

An import pipeline locks the Products resource.
A mass data replication for products locks the Products resource, too.
The context index rebuild processes lock database table resources like PRODUCT, DERIVEDPRODUCT.

Named resources have a scope that can either be a domain-specific scope or the global scope representing the resource in all domains. Normally, global named resources are created during DBInit or DBMigrate processes. The according domain-specific resources are automatically created if a named resource is acquired in a specific domain scope.

Example:

To be able to run the product import in two channels in parallel, the import processes will not lock the resource Products globally. Each process will lock the import resource Products for the particular domain, which prevents running a product import in the same channel in parallel, but still allows to run product imports in other domains in parallel.

Named resources can be placed in a hierarchy tree where a resource may have one parent resource and maybe some child resources. The following high level resources are available:

• Cluster: Represents the entire cluster.
  • Database: Represents the entire database.
    • Domains: Represents the all domain tables and their content.
    • Jobs: Represents all job tables and their content.
    • Products: Represents all job tables and their content.
      • PRODUCT: Represents the PRODUCT table.
    • ...
  • Share: Represents the shared file system.
    • Sites: Represents the sites folder in shared file system.
    • System: Represents the system folder in the shared file system.
• Import: Represents all import processes.
  • UserImport: Represents all user import processes.
  • StoreImport: Represents all store import processes.
  • ...
• Schedules: Represents all schedules.
  • CleanUp: Represents the Cleanup job.

To get the current tree from the database, use this SQL statement:

SELECT
    lpad(' ',2*(level-1)) || name || '@' || dn(domainid) || ': ' || description AS description
FROM
    isresource
CONNECT BY PRIOR
    uuid=parentresourceid;

Resource Locks

The locks of resources define a different lifecycle depending on their type. Currently, there are two lock types available: Transient Resource Lock and Persistent Resource Lock.

Persistent Resource Lock

The persistent resource lock is not lost when the application server is shut down, restarted or if it has crashed. Generally, it is used to lock Instance Resources. Persistent locks do not support a timeout-based waiting mechanism, but a lock expiration date can be defined.

Internally, the persistent locking is achieved by the following mechanism:

For each persistent object to be locked, a resource is generated in the ISRESOURCE table, and a matching row is inserted into the RESOURCELOCK table. If this row already exists, the locking fails because another process already locked this persistent object. When the resource is unlocked, the row in the RESOURCELOCK table is removed. If the server is restarted while a persistent object is locked, the object is still locked after the restart.

Transient Resource Lock

The transient resource lock is used when the acquired resources should be immediately released, in case of the processing application server is shut down, restarted or if it has crashed. Generally, it can only be applied to Named Resources. Timeouts define the maximum time the acquiring process should wait to get the lock on resources before the acquisition fails.

Internally, the transient locking is achieved by the following mechanism:

For each resource defined in the ISRESOURCE table, there is a row in the table RESOURCELOCK that refers to the resource. If the resource is to be locked, there is executed a SELECT FOR UPDATE WAIT statement on the resource's row in RESOURCELOCK. A timeout can be configured for the WAIT clause. If this timeout is reached, the locking fails because another process already has the lock. Once the row is locked, the locking thread waits until the row is unlocked again. If the server is restarted while a resource is locked, the resource will not be locked after the restart.

Processes

The processes created by the locking framework can be arranged - like the resources themselves - in a tree to describe sub processes. They define a type and name, which can be defined by the particular feature using the locking framework. Furthermore, they define a status (e.g., RUNNING, FAILED, SUCCESS), as well as a start date and an end date. The SMC provides a graphical interface to monitor them on a higher level.

Schedules

Schedules automatically create a process and a corresponding resource representing the schedule (resource tree: Cluser -> Database -> Schedules) upon execution. The created process can be used in pipelines to acquire additional resources.

A list of Schedules resources can be found in the System Management | Locking | Named Resources | Cluster | Schedules.

The Schedule resources listed here correspond to preconfigured jobs which can be found in System Management | Schedules | Scheduling.

The named resource can then be used to assign resources (database tables, files) that need to be acquired for the job execution.

Via DBInit or DBMigrate, it is possible to declare additional resources that need to be locked if the job is executed (see com.intershop.beehive.core.dbinit.preparer.locking.JobResourcePreparer). The preparer gets a mandatory parameter, which basically is a file that holds the configured job resource assignments. The syntax looks like:

<JobName>;<DomainName>;<ResourceName>

Example:

AnalyzeDatabaseSchema;root;Database

Mass Data Replication

Several processes describing the overall mass data replication are created during execution (Concept - Mass Data Replication). The mass data replication locks all resources that are assigned to staging groups via the preparer com.intershop.beehive.core.dbinit.preparer.staging.ResourceAssignmentPreparer. Normally, each staging group has its own resource. In editing systems, the assigned resources are locked on a domain-specific base, whereas in live systems globally.

Import Processes

Resources can be configured for an import. This can be done by using an accordingly configured LockImport pipelet, see Concept - Impex Framework | Locking.

In addition, it is possible to specify whether these resources are to be locked globally or on a domain-specific base. If no resources are configured for an import, the database resource is locked globally. Therefore imports and exports should always be configured to lock only the resources which are required and processes that have competing locks should be scheduled to avoid conflicts.

If an import uses the ProcessBatchJob-AnalyzeTables mechanism, PO table resources are locked globally for the time the tables are analyzed.

Back-Office Locking

Currently only product editing and content editing in the back office use the locking framework. To be able to edit a product, the user has to lock the product explicitly via a lock button. Internally, there is created an instance resource with an according persistent lock to lock the particular business object.

Locking Conflicts

Locking conflicts can occur, for example, when a product is locked (lock of an instance resource). This means that an entry is actually created in table ISRESOURCE. This will prevent any similar competing locks for these resource. The lock of a named resource may not conflict at all, e.g., if a product import process does not touch the very database fields of the locked product. If, e.g., such an update process touches fields, which are concurrently locked this will cause a locking conflict.

Locking conflicts can be monitored in the System Management | Monitoring | Locking | Locking Conflicts.

For information on unlocking resources, refer to Cookbook - Locking Framework.

API

This section describes the important API to use the locking framework in custom batch processes.

Pipelets

DBInit Preparers

Managers