Document Properties
Kbid2P3573
Last Modified04-Feb-2020
Added to KB12-Oct-2012
Public AccessEveryone
StatusOnline
Doc TypeGuidelines, Concepts & Cookbooks
Product
  • ICM 7.6
  • ICM 7.7
  • ICM 7.8
  • ICM 7.9
  • ICM 7.10

Concept - Business Object Replication

1 Introduction

This concept describes business object replication. Business object replication is intended for quick and easy transfer of business objects from the editing system to one or more live systems (see Concept - Business Object Replication#Usage).
For the transfer of large amounts of data, mass data replication has to be used.

Business object replication works by creating transfer entities called "WebObjects" from business objects by the use of configurable WebObject Providers and transferring those WebObjects to the target system(s) via a SOAP WebService. On the target system(s), the objects are validated and merged into the target system's data (see Concept - Business Object Replication#Background).

2 Glossary

Business object replication is more commonly called "object replication" and may also be referred to as "partial replication".

Term

Description

WebObject

the transfer entity containing all necessary information on the business object

WebObject Provider

or WOProvider: the class responsible for creating web objects, validating them, and merging them into the target system

Publish

Term used for object replication_ in the user interface

Referenced Object

An object that is referenced by the replicated business object, but can exist independently

Dependent Object

An object that belongs to the replicated business object, and cannot exist independently

Dependent File

A file that belongs to a replicated business object

3 Usage

Object replication is only available on systems that are marked as editing systems in a staging environment (see the administrator handbook of your Intershop 7 distribution).

3.1 End User

A user will see the business object replication interface elements on listings and detail pages of supported business objects.

On a business object's detail view, the object can be replicated using the button Publish to Live System. After selecting a target cluster on the next page, a batch process will be created performing the actual replication.

In a listing, one or more objects can be selected and replicated using the button Publish to Live System. The listing also includes a column Published, listing the date(s) and time(s) of the last successful object replication(s) to the target system(s).

After starting the replication, the user is taken to the batch process overview page.

Note

Object replication can only update and create objects, not delete them.

3.2 Developer

Information for developers (e.g. on adding object replication support to additional business objects) can be found in the Cookbook - Business Object Replication.

4 Background

4.1 Process Overview

ObjectReplication_CompleteProcess
  • ORMObject to WebObject transformation: On the editing system, given ORMObjects are transformed to unified WebObjects. That makes it easier to send and receive them with the Axis WebService. For each ORMObject, a WebObjectProvider( WOProvider) is looked up that does the collecting of dependent objects and transforms it. This procedure is repeated for each dependent object too.
  • Determine PageCache Searchwords: For each ORMObject, a SearchWordProvider is used (if existent) to determine significant search words for invalidation of the page cache. The search words will only be invalidated on affected sites (because only sites have a page cache and not all sites display the affected object). For instance, if one of the ORMObjects is a product, files of the page cache that contain the UUID, SKU or name of the ProductPO are filed for invalidation for each site that displays the product.
  • Determine affected domains for SearchIndex refresh: Using the specific ADProvider(if existent), affected domains for ORMObjects are determined. For instance, the affected domains for a product are the domain that owns the product and the domains the product is shared to.
  • Replication BatchJob: After the transformation to WebObjects, determining the affected domains and PageCache SearchWords, the editing system sends the WebObjects via an Axis WebService to the corresponding live systems. The live systems validate and merge the received WebObjects by transforming their corresponding local ORMObjects (if they already exist) to WebObjects and comparing both. In case of a failed validation, the WebObject will not be merged (and will not refresh caches for it); the other WebObjects are not affected by this. The error message (provided by the specific WebObject-Provider) is sent back to the editing system and is shown in the Batch Process Log.
  • If the merging was successful, the target system-specific LastPublishDate is written for each object. In this way, the user can see when the object was replicated (e.g. seen in Promotions).
  • After merging the changes, the ORMObjects are released from the ORMCache. On the next access of the ORMObjects the database will be queried for the new values.
  • If the replicated WebObjects have attached files, the live systems download a compressed archive of them from the editing system and extract them to the local directories.
  • After that, the caches are refreshed. The pre-calculated SearchWords and affected domains are filtered by successful WebObject validations (when a WebObject was not merged because of a failed validation, the caches will not be refreshed for it) and sent to the live systems. The live systems refresh the following caches:
    • SearchIndex: by Channel-DomainName
    • PageCache: by Channel-SiteName and their SearchWords

4.2 Process Details

4.2.1 Providers

The following diagram shows the process for an example object Sample with two referenced objects, Other and Another:

Example
ObjectReplication_Process

(1) When the publishing of a Sample is started, the replication framework first resolves all references of the Sample by means of the SampleRefOOProvider(if there is one defined). This results in a Set<Object> containing all referenced sample objects as well as the sample object itself. From the Set an Iterator is created within which, according to the PublishOrder defined in SampleRefOOProvider, the Sample is the first/last element.

Now the elements of the iterator are replicated one by one in chunks of default size 15 where for each object the corresponding <object_type>WOProvider is used.

(2) WebObjects for the referenced objects are created using the ReferencedOtherWOProvider and the ReferencedAnotherWOProvider, respectively.

(3) WebObjects for Sample and its attributes are created by means of the SampleWOProvider.

(4) The WebObjects are transferred to the live system.

(5) When receiving the objects, the validate method of the matching providers will be called, where the referenced objects of Sample are validated and merged before the Sample web object itself. The validation itself depends on (i.e. is implemented in) the WOProvider. The provider can also perform some actions depending on the outcome of the validation. If the validation fails, an error will be logged.

(6) Whenever the validation of a web object succeeds, the merge method of the matching provider is called. This method is responsible for creating/updating business objects in the live system to match the transferred object.

4.2.2 Page Cache Handling

For each ORMObject a SearchWordProvider is used (if existent) to determine significant SearchWords for invalidation of the PageCache.
The SearchWords will only be invalidated on affected sites (because only sites have a PageCache and not all sites display the affected object).
For example, if one of the ORMObjects is a product, files of the PageCache that contain the UUID, SKU or name of the ProductPO are filed for invalidation for each site that displays the product.

4.2.3 Search Index Handling

Using the specific ADProvider(if existent), affected domains for ORMObjects are determined.
For instance, the affected domains for a product are the domain that owns the product and the domains the product is shared to.

4.3 Published Objects

4.3.1 ORMObjects That Are Published During Campaign Publishing

Description

ORMObject

WoProvider

Published in 7.0

Published in 7.1

Campaign and attribute values

CampaignPO and attribute value POs

CampaignWOProvider

(tick)

(tick)

TargetGroup

Iterator<UserGroup>

CampaignWOProvider

(tick)

(tick)

TargetGroup : UserGroupRoleAssignment

Iterator<RoleAssignment>

CampaignWOProvider

(tick)

(tick)

TargetGroup : UserGroupInformation

Iterator<UserGroupInformation>

CampaignWOProvider

(tick)

(tick)

TargetGroup : UserGroup assignments

Iterator<UserGroupAssignmentPO>

CampaignWOProvider

(tick)

(tick)

UserGroupAssignment from campaign (childUserGroup) to all assigned promotions (parentUserGroup)

Iterator<UserGroupAssignmentPO>

CampaignWOProvider

(error)

(tick)

UserGroupAssignment for all assigned Pagelets

Iterator<PageletUserGroupAssignmentPO>

CampaignWOProvider

(error)

(tick)

Referenced Pagelets


ReferencedMarketingContentWOProvider each

(error)

(tick)

Referenced Promotions


ReferencedPromotionWOProvider each

(error)

(tick)

4.3.1.1 Special Cases That Require Some Post-Processing

The current campaign publishing is organized in such a way that promotions and pagelets assigned to a campaign are published before the campaign is published. So, if the final live side validation of the campaign fails, then the campaign itself is not published, whereas assigned promotions and pagelets at that moment are already updated. This requires post-processing for each assigned promotion and pagelet:

  • assigned promotions: If the campaign to be published does not yet exist in the live system, set the promotions's UseCampaignTargetGroup flag to false and disable the promotion. This procedure is complete because as soon as the promotion is published by means of the ReferencedPromotionWOProvider, its campaignUUID attribute is set to null in order to avoid the situation of the referenced promotion being assigned to another - no longer supported - campaign on live side. Now, if the campaign to be published does already exist in the live system, then two cases have to be distinguished:
    • If the promotions's UseCampaignTargetGroup flag is true and if the current campaign target group is no direct child of the promotion target group, make the current campaign's target group a direct child of the promotion's target group again.
    • If the promotion's UseCampaignTargetGroup flag is false and if the current campaign's target group is a direct child of the promotion's target group, remove this direct child relation.
  • assigned pagelets: If the campaign to be published currently does not exist in the live system or if a pagelet assigned to the campaign to be published currently has no user group assignment in the live system, then the pagelet is set "offline", i.e., "not published". In order to understand this procedure, one has to know that the assignment of a pagelet to an arbitrary campaign or promotion is represented by a special user group assignment where a campaign/promotion user group ID starts with CAMPAIGN_/ Promotion_. A pagelet that has no user group assignment must not be displayed in the storefront.

Now assume that the campaign validation is successful. If the campaign did not exist in the live system before, then the assigned and published content is fine. Otherwise there may be a difference between former and new assigned pagelets that has to be dealt with. This is done as follows:

  • All pagelets that have been assigned to the campaign before and hence to no other campaign or promotion, but will no longer be assigned to the campaign to be published, are deactivated.
  • All pagelet user group assignments to other campaigns or promotions among those pagelets that are assigned to the campaign to be published are deleted, because there must not be more than one such user group assignment for each pagelet.

The promotion post-processing in case of successful campaign validation is straightforward: No longer valid assignments to other campaigns are removed and the new assignment to the campaign to be published is confirmed.

4.3.2 ORMObjects That Are Published During Pagelet Publishing

Description

ORMObject

WoProvider

Published in 7.0

Published in 7.1

Pagelet

PageletPO

PageletWOProvider


(tick)

Parent Pagelet (not published for pagelets referenced by promotions or campaigns)

UnmodifiableIterator<Pagelet>

PageletWOProvider: BasicPageletWOProviderExtensionImpl


(tick)

Slots

Iterator<Slot>

PageletWOProvider: BasicPageletWOProviderExtensionImpl


(tick)

Placeholders

Iterator<Placeholder>

PageletWOProvider: BasicPageletWOProviderExtensionImpl


(tick)

Placeholder definitions

Iterator<PlaceholderDefinition>

PageletWOProvider: BasicPageletWOProviderExtensionImpl


(tick)

Pagelet user group assignments

Iterator<? extends PageletUserGroupAssignment>

PageletWOProvider: BasicPageletWOProviderExtensionImpl


(tick)

Assignments from pagelet to slot

Iterator<SlotPageletAssignmentPO>

PageletWOProvider: BasicPageletWOProviderExtensionImpl


(tick)

Pagelet ABTestGroup assignments

Iterator<? extends PageletAssignment<ABTestGroup>>

PageletWOProvider: MarketingContentPageletWOProviderExtensionImpl


(tick)

Pagelet Promotion assignments

Iterator<? extends PageletPromotionAssignment>

PageletWOProvider: MarketingContentPageletWOProviderExtensionImpl


(tick)

4.3.3 ORMObjects That Are Published During Promotion Publishing

Description

ORMObject

WOProvider

Published in 7.0

Published in 7.1

Promotion and attribute values

PromotionPO and attribute value POs

PromotionWOProvider

(tick)

(tick)

TargetGroup(s)

UnmodifiableIterator<UserGroup>

PromotionWOProvider

(tick)

(tick)

TargetGroup : UserGroupRoleAssignment

Iterator<RoleAssignment>

PromotionWOProvider

(tick)

(tick)

TargetGroup : UserGroupInformation

UnmodifiableIterator<UserGroupInformation>

PromotionWOProvider

(tick)

(tick)

TargetGroup : UserGroup assignments

Iterator<UserGroupAssignmentPO>

PromotionWOProvider

(tick)

(tick)

PromotionAttachments

Parameter dependendFiles in the Promotion WebObject

PromotionWOProvider

(tick)

(tick)

Rebates

Iterator<RebatePO>

PromotionWOProvider

(tick)

(tick)

Rebate: Condition "tree"

Iterator<RebateConditionPO>

PromotionWOProvider

(tick)

(tick)

Rebate: Actions

Iterator<RebateActionPO>

PromotionWOProvider

(tick)

(tick)

Rebate: RebateFilterGroup assignments

Iterator<RebateFilterGroupAssignmentPO>

PromotionWOProvider

(tick)

(tick)

Rebate: RebateFilterGroups

Iterator<RebateFilterGroupPO>

PromotionWOProvider

(tick)

(tick)

Rebate: RebateFilterGroups: assignment to RebateFilterObjects (catalogs or products)

Iterator<RebateFilterObjectAssignmentPO>

PromotionWOProvider

(tick)

(tick)

Budget

UnmodifiableIterator<Budget>

PromotionWOProvider

(tick)

(tick)

Budget Periods

Iterator<BudgetPeriod>

PromotionWOProvider

(tick)

(tick)

Budget Account

UnmodifiableIterator<Account>

PromotionWOProvider

(tick)

(tick)

Promotion Application assignments

Iterator<PromotionAppAssignmentPO>

PromotionWOProvider

(tick)

(tick)

PromotionCodeGroup assignments

Iterator<PromoCodeGroupPromoAssignmentPO>

PromotionWOProvider

(tick)

(tick)

PageletAssignments

Iterator<PageletPromotionAssignmentPO>

PromotionWOProvider

(tick)

(tick)

PageletUserGroupAssignments

Iterator<PageletUserGroupAssignmentPO>

PromotionWOProvider

(error)

(tick)

Referenced Pagelets


ReferencedMarketingContentWOProvider each

(error)

(tick)

Referenced PromotionCodeGroups


PromotionCodeGroupWOProvider each

(error)

(tick)

PromotionCodeGroup (for partial replication allowed size is configurable)

PromotionCodeGroupPO

PromotionCodeGroupWOProvider

(error)

(tick)

PromotionCodeGroup: Applications

Iterator<PromoCodeGroupAppAssignmentPO>

PromotionCodeGroupWOProvider

(error)

(tick)

PromotionCodeGroup: UserGroup/Affiliate assignments and redemption configuration

Iterator<PromoCodeGroupUGrpAssignmentPO>

PromotionCodeGroupWOProvider

(error)

(tick)

4.3.3.1 Special Cases That Require Some Post-Processing

In analogy to campaign publishing, the current promotion publishing is organized in such a way that pagelets assigned to a promotion are published before the promotion is published. So, if the final live side validation of the promotion fails, then the promotion itself is not published whereas assigned pagelets at that moment are already in the state "published". This requires a post-processing for each assigned pagelet which is done in the same way as for pagelets assigned to a campaign in the case of campaign validation failure.

The pagelet post-processing in case of successful promotion validation is the same as in case of successful campaign validation.

5 Mass Data / Performance

Business object replication is not meant for mass data. For replicating mass data, staging is used which copies content from the editing-database-scheme to the live-system database-scheme.

The replication is done by sending and receiving serialized objects via web services. If a huge object (with several megabytes) is sent, the web services could massively slow down.

If the user plans to replicate a lot of objects (e.g. 10000 products), these objects will be sent in several loops, 15 objects each loop (this value can be set by staging.objects.chunksize in <enfinity>/share/system/config/cluster/staging.properties) and the cache refresh is started when all objects have been sent and merged.

Remember that business object replication is only meant for emergency updates of a few objects.
If you want to replicate a lot of data, use the Mass Data Tasks menu.

Disclaimer

The information provided in the Knowledge Base may not be applicable to all systems and situations. Intershop Communications will not be liable to any party for any direct or indirect damages resulting from the use of the Customer Support section of the Intershop Corporate Web site, including, without limitation, any lost profits, business interruption, loss of programs or other data on your information handling system.

Customer Support
Knowledge Base
Product Resources
Support Tickets