Catalog views provide a mechanism to configure the visibility of products in a catalog for certain users. With this approach it is possible to offer a different set of products to different customers and/or customer segments.
Each user in an organization typically takes a certain role that is associated with certain access privileges. These privileges also need to be represented in the organization catalogs to enable users-specific (customer-specific) catalog browsing. Using the platform concepts, a catalog administrator can grant the respective permissions per customer segment on the product and category level. This information is used later on during catalog browsing to display exactly those categories and products a certain customer is permitted to view. The catalog view concept allows the catalog administrator to define filters (or visibility rules) that should be applied upon a catalog. The system handles the mapping of the visibility rules to the underlying catalog representation by automatically creating the appropriate permission setting on the product instance level.
This approach is based on time-consuming database operations that create entries for each product and category that is included in a view.
These operations cannot be executed on demand in the live system, because this would slow down the whole storefront application. Therefore the creation of catalog views comprises two steps.
Here are some important facts about catalog views:
A catalog view can be created manually in the Commerce Management application, imported with the standard Import/Export framework or via DBinit.
The creation of a catalog view involves the creation of the following additional objects:
In short, the filter user group is used to determine the visibility of products and categories in the filter, while the target user group is used to determine the assigned users.
The following catalog view states have been introduced:
The diagram below provides an example for a single catalog filter (view).
Catalog filters - A simple diagram
The example shows an organization with a single catalog and six offers.
The sample catalog filter contains inclusion and exclusion rules defined for the organization catalog structure.
The gray tones in the diagram are used to visualize which products are bound to which categories.
The (simplified) semantics of the sample catalog filter is the following:
Once defined, catalog filters need to be assigned to customer segments.
Using this approach, it is fairly easy to define a restricted filter for all users of an organization (i.e., a catalog filter assigned to user group "Everyone") and set up additional, less restricted features for other user groups such as managers.
The diagram provides an example for such a user - filter mapping through user groups.
Catalog filters: Mapping of user groups and filters:
In the example three catalog filters are mapped onto two users.
The two users have different roles in the organization and therefore different access privileges for the organization's catalogs.
User 1 fulfills roles 1 and 3 while user 2 fulfills roles 2, 4 and 5.
Filter 1 provides a user specific catalog view for users in roles 1 and 2.
Filter 2 provides a view for users in roles 3 and 4, whereas filter 3 is a view for users of role 5.
Based on their roles, the two users are mapped onto the following visibility rules of the organization’s catalogs:
Users always get the privileges of all filters that have been assigned to them through all of their roles.
In case of multiple filters per user (as in the example above), all inclusion and exclusion rules are interpreted as a union.
A user will therefore have access to all catalog sections provided by the union of all filters that are assigned to the roles which the user has.
In a similar vein, the user will be excluded from all catalog sections listed in the exclusion list of all catalog filters that are assigned to the roles which the user has.
The models below show the business objects relations of the catalog filter concept. They are separated in four parts:
The definition of a catalog filter contains configuration of visibility rules (inclusions and exclusions of categories/products). Exclusions can be configured only if inclusions are already configured (defined). At the time the catalog filter is in configuration phase, e.g., target group is assigned and inclusions are configured but exclusions are still not, an user assigned to this filter could see products which are going to be excluded. To avoid this scenario, an additional (preprocessing) step (before the filter is really applied to the storefront) have to be performed to ensure consistency between configured visible products and the actually visible products. This step is called Publishing. The following stages are distinguished:
According to described catalog view stages the filter definition and maintenance are separated from the actual filter execution.
Mapping of the catalog filter visibility rules to role assignments (de-normalization) performs complex mass-data operations that are not executed synchronously during filter definition but instead the publishing process is responsible for this job.
A catalog view has the following attributes:
Name of the catalog view
Online status (true or false) - Sets the status of the whole catalog view
Choose between manual publishing or a predefined time schedule.
To define the visibility of products for certain user/department roles, a catalog view should have inclusions and may have exclusions.
Inclusions might be:
The inclusion of a higher level object (e.g., catalog) implies the inclusion of all subordinated objects (e.g., categories, products).
Exclusions might be:
The inclusion/exclusion mechanism also works for dynamically assigned products. If Update Product Assignments SLDSystem job is still not executed you can only include/exclude catalogs and categories as the products are still not visible/assigned to this category. Otherwise, particular products is possible to be included/excluded.
There are different structures which store the included and excluded categories and products (during filter definition stage). These structures are called FilterProductAssignment (stores included and excluded products) and FilterObjectAssignment (stores included and excluded categories). The important properties are described in the tables below.
|Refers to catalog filter
|Refers to product SKU
Refers to how the product is related to the catalog filter
|Refers to catalog filter
|Refers to catalog category
Refers to how the category is related to the catalog filter - included in the filter or excluded from the filter.
Valid values: 1 (inclusion) and 2 (exclusion)
A product is included in a catalog view if there is an inclusion rule for this product. This means the product is explicitly selected to be included in the catalog view - not by category but the product itself.
A product is visible in a catalog view if:
If the product is assigned to more than one category and only one of them is included - it is visible only through this category.
A category is included in a catalog view if there is an inclusion rule for exactly this category. This means the category is explicitly selected to be included in the catalog view.
A category is visible in a catalog view if:
There are some rules which are applied on products/categories when include/exclude a product/category. These rules are related to forbidding the possibility to include/exclude certain products and categories when something else is already included/excluded. These rules define the following behavior:
Uninclusion of a product/category means that the inclusion rule is deleted and the corresponding category/product is no longer visible. Also, the parent/child inclusion rules defined by this category/product are removed. Mean that if the same category/product is included in another catalog filter for the same user - it will still be visible.
Unexclusion of a product/category means that the exclusion rule is deleted and the corresponding category/product is visible again. Also, the parent/child exclusion rules defined by this category/product are removed. Mean that if the same category/product is excluded from another catalog filter for the same user - it will still be visible.
There are some special cases for product bundles, retail sets and variations.
Products with type Product Bundle and Bundled Product can be treated as normal products for catalog views.
For retail sets (Type Retail Set and Part of retail set) it is different.
In the context of Intershop, the Variation Master does not exist alone - it requires at least one Variation Product assigned. That's why catalog filter inclusion/exclusion mechanism, together with the publishing process, ensures the variation master is visible only if it has at least one variation visible. This means:
For every catalog view, a set of customer segments can be assigned to define who should be able to see the products. These segments come from the customer segmentation service. When a new assignment is made a new hidden consumer and user group will be created for the assigned segment, if one doesn't already exist. The customer segment is assigned to the catalog view through that user group.
Catalog views can also be assigned directly to customers. They work in addition to the catalog views assigned to the customer segments of the customer.
As already mentioned, the publishing process is time-consuming and complex because the set of included products (explicit and implicit) has to be determined and a database entry has to be created/changed for each involved product and category. During this publishing process the visibility of the products and their categories is computed. All of these objects get an entry in their corresponding
*_RA table (e.g.,
CATALOGCATEGORY_RA) which has to be created/changed/deleted in every publishing process depending on the inclusion status.
The publishing process triggers different actions as shown on the diagram below:
On this diagram four main steps are executed with the corresponding sub-steps:
On this step the no longer valid assignments of the catalog filter are removed. These assignments include filter-product assignments, filter-category assignments and filter-group assignments (user group, customer segment). This step performs a search operation to check if a particular assigned product/category/group still exists in the system. If this product or category or user group no more exists - the corresponding assignments are removed from the catalog filter configuration.
Since IS7 supports two types of Product-Category assignments (implicit (dynamic) and explicit (manual)) there is a need to refresh especially the dynamic assignments. The categories which use dynamic product assignments are also called categories with dynamic binding. For each such a category there is a search query definition assigned which contains information how to retrieve the products to be assigned. Usually this information is a specific attribute or another (context) category. As this information is saved in a search query definition, it is ready to be used with a search server. That's why the current configuration of a dynamic bound category requires built and up-to-date product search index for the current locale. Currently all categories with dynamic binding use Solr product search index to retrieve the products (by the given attribute(s)) - the category's search query is transferred to a Solr search query, then call the search index with this query and the result is assigned to the category.
In catalog views it is possible to include/exclude a particular dynamic bound category but if the product assignments are still not initially retrieved - the category is assumed empty. That's why there is a need to extract these product assignments which will be added later (during publishing of the catalog view) to the set of visible products. For this purpose a new process is created acquiring the resource
IEProductCategoryAssignmentView which updates the corresponding database view with the retrieved product-category assignments from the search index result.
The following sub-steps describe the process of product selection and category assignments:
At this point the product-category assignments are done and ready for the publishing step.
The database table
PRODUCT_RA contains an entry for each product that is contained in the visible products set. The database table
CATALOGCATEGORY_RA contains an entry for each catalog category that is contained in the visible categories set. The visible products set is a set of products contained in the included set and not contained in the excluded set (included set minus excluded set). The visible categories set is a set of categories contained in the included set and not contained in the excluded set (included set minus excluded set).
Exclusions are used only in the context of a single catalog view. If for example you have two catalog views -
C2 - and a product
P is included (implicitly or explicitly) in both, but excluded only in
P is still in the visible product set.
The operation for marking the product and category instances as being visible in the particular view (create/change/delete an entry into/from a
*_RA table) is used later on during catalog browsing and catalog search to decide whether a product or category is supposed to be part of the result set. This means that the existing product and category role assignment tables are used to store our "visibility markers". Platform role assignments are always established between an authorization object (such as products and categories), a customer segment and a role. The role assignment connects these three entities to define "who is allowed what where". In order to integrate with this concept, an artificial user group has been created for each catalog filter and a standard role (i.e. access privilege) representing catalog visibility. The existence of a role assignment instance for the artificial user group representing the filter is interpreted as the "visibility marker" and only catalogs and products that have a role assignment to the filters of the user will be shown during catalog browsing.
There is also a post-publishing step which checks for and removes the single variation masters (masters without visible variation) and the single variation products (variations without visible master). This step is performed to ensure that there is always a visible master with at least one visible variation. This is done right after the publishing has been finished because it is no possible to detect this behavior during catalog filter configuration. One example is: someone includes a category with variations but the master is not categorized and therefore not included and not visible in the catalog view; this means the master is not visible in the storefront; if the master is not visible in the storefront - the variations are also not visible because ISH Solr integration stores the master and its variations in one single Solr document and the catalog filter attribute is applied and checked against the master only. On the other hand, the master is not needed alone since the product detail page of the master shows the data of the default variation or first visible variation (if the default is excluded/not visible). That's why catalog view publishing process ensures always a visible master with at least one visible variation by removing the corresponding single masters and variations.
Publishing or republishing is needed when:
Publishing or republishing is NOT needed when:
The last step of the publishing process is to invalidate domain specific page caches. This includes invalidation of pipeline page cache and static content cache, as well as product related ORM caches.
A catalog view "knows" about its accessibility. Use the Java extension point
com.intershop.component.mvc.capi.filter.CatalogFilterBOAccessibilityHandler-isAccessible to implement your own extensions specifying rules by which a catalog view is accessible. For the catalog view to be considered accessible, there should be no extensions available or at least one extension has to return that the catalog view is accessible.
In the standard system an accessibility extension exists that checks whether the catalog view is online, has been published and whether the catalog view is active according to the assigned target group.
This is an example export of catalog view with the latest XML schema definition.
<?xml version="1.0" encoding="UTF-8"?> <enfinity xsi:schemaLocation="http://www.intershop.com/xml/ns/enfinity/7.0/bc_mvc/impex bc_mvc.xsd" xmlns="http://www.intershop.com/xml/ns/enfinity/7.0/bc_mvc/impex" xmlns:xml="http://www.w3.org/XML/1998/namespace" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <catalog-filter id=".9UKDABnCPMAAAFFyn4DqEH2" state="1" update-interval="0"> <scope id="" type=""/> <name xml:lang="en-US">MyCatalogView</name> <description xml:lang="en-US"></description> <included-objects> <categories> <category name="206" domain="PrimeTech-Computers"/> <category name="242" domain="PrimeTech-Computers"/> <category name="150" domain="PrimeTech-Computers"/> <category name="225" domain="PrimeTech-Computers"/> <category name="2" domain="PrimeTech-Computers"/> <category name="Computers" domain="PrimeTech-Computers"/> <category name="106" domain="PrimeTech-Computers"/> <category name="191" domain="PrimeTech-Computers"/> </categories> </included-objects> <excluded-objects> <categories> <category name="3001" domain="PrimeTech-Computers"/> </categories> </excluded-objects> <filter-targets> <customers> <customer id="Heimroth"/> <customer id="Clarkson"/> </customers> <customer-segments> <customer-segment id="CG_PremiumConsumers" repository-id="PrimeTech-PrimeTechSpecials-Anonymous" /> </customer-segments> </filter-targets> </catalog-filter> </enfinity>
Customer segments export
Customer segment assignments coming from currently disabled Customer Segmentation Service will NOT be exported.
Every time a product or a category is displayed in the storefront, the catalog views have to be considered. This includes but is not restricted to:
A category is visible in the storefront if all of the following conditions are met:
In short, a category is visible if there is self/child/parent category inclusion, no category exclusion in all catalog views for the current user and not all products of this category (and subcategories) are excluded from all catalog views of the current user.
Categories are not implicitly made visible if there are only products selected for inclusion, even if those products are assigned to those categories. This means, if a category is not included and none of its ancestors/descendants is included, but the products of this category are included in the catalog view, the categories from the product category path are not visible.
A product is visible in the storefront if:
If a product is visible, but none of the categories it is assigned to are visible, the product can still be found via search.
As a web developer or consultant make sure when working with a category in the storefront to check if it is accessible, otherwise it could be shown when not expected.
In many web shops there is a search server which is used for different customer tasks such as search by term, catalog browsing, paging of search results, filter of search results by different criteria and so on. The purpose of the search server is to replace the usage of database queries to retrieve the desired information. IS7 installations use Apache Solr as such a search server. There is also a Solr integration cartridge (ac_seach_solr) which contains the code to use and maintain the Solr indexes.
During catalog browsing or search, a particular customer can see only the products and categories visible in terms of a catalog view. Currently IS7 Solr integration gets all products in the given repository and adds them to the configured search index for this domain. In the Solr index each product is represented by one Solr document with different attributes corresponding to the product attributes. So, a product could be visible for one customer and invisible for another one depending on which catalog views are assigned to each of them. Since Solr contains product information, there is a need to mark which product in which catalog view is included/visible. During catalog browsing all filters of the customer are retrieved and they are added to the Solr search query to return all products accessible in these filters.
To be able to recognize which product in which filter is included, ISH Solr integration defines the search index attribute
CatalogFilterIDs (Catalog View Filter) which contains all catalog filter IDs for a particular product. This attribute is not indexed by default because a sales channel may not have catalog views defined and the usage is useless. If a sales channel defines catalog views, the attribute
CatalogFilterIDs have to be included in the indexed attributes. This requires an index rebuild to add the newly included attribute in Solr documents. Without this attribute, Solr search ignores catalog views. Also, for products which have no assignments to catalog views, the
CatalogFilterIDs attribute is missing in the corresponding Solr document.
CatalogFilterIDs Search Index Attribute
CatalogFilterIDs attribute has the following properties:
|Catalog View Filter
|Adds a filter condition attribute for defined catalog views
|CatalogFilter, provider class: com.intershop.component.mvc.internal.searchindex.dataprovider.CatalogFilterDataProvider
|Used for Spell Checking
|Used for Autosuggest
The data provider, which is used during search index rebuild to collect and save the catalog view data in the index, works on the following way:
CatalogFilterIDs attribute to the list with collected catalog view IDs
This attribute is not set per product (Solr document) if the list with catalog filters is empty. This is, in such case the product does not have assignments to any catalog filters.
After all the catalog filters are published and the
CatalogFilterIDs attribute is added to a product search index with required rebuild - the catalog view information can be observed in the Solr Console, e.g., for a product with SKU=10680607 we have the information that it is included in a catalog view with UUID=2UEKDADA9Y4AAAFT8ZcUoPC_.
<doc> <arr name="CatalogFilterIDs"> <str>2UEKDADA9Y4AAAFT8ZcUoPC_</str> </arr> <arr name="SKU"> <str>10680607</str> </arr> <double name="ProductListPrice">1337.95</double> <arr name="CategoryNames"> <str>897_Asus</str> <str>897</str> <str>Computers</str> </arr> <date name="OnlineFrom">1970-01-01T00:00:00Z</date> <int name="OnlineFlag">1</int> <double name="ProductSalePriceNet">1337.95</double> <str name="UUID">v5AKDADAaK0AAAFT3jIFzOFh</str> <str name="CatalogDomainName">inSPIRED-Computers</str> <str name="ManufacturerName">Asus</str> ... </doc>
If a product has more than one catalog view assigned, the Solr document will look like this:
<doc> <arr name="CatalogFilterIDs"> <str>px8KDADAX20AAAFTEKcUoPDF</str> <str>2UEKDADA9Y4AAAFT8ZcUoPC_</str> <str>zZ8KDADAZCwAAAFTPecUoPDH</str> </arr> <arr name="SKU"> <str>0889296696803</str> </arr> ... </doc>
Since the catalog filter publishing process refreshes the categories with dynamic binding which require a built, up-to-date search index, it is required that all search indexes are initially built. If not built, a subsequent publishing process may hang for some time (usually 10 min) waiting for an acquired resource to be released. Also, if a category with dynamic binding is included in a catalog view and the SLDSystem job Update Product Assignments still did not ran - there is no possibility to exclude particular products from this category as the product assignments are still not calculated. The catalog view publishing process also refreshes the dynamic binding, but this happens after the catalog view is configured. At the time of configuration, dynamic bound products could still not be visible.
Because of the above mentioned problems there are some steps which have to be followed for properly setting up catalog views with Solr index:
CatalogFilterIDs attribute to the Solr product search index - to index catalog filters for each product - and rebuild the index.
Republishing of a catalog view requires also search index rebuild.
Republishing of a catalog view is needed when change inclusions/exclusions, create/change/delete product-category assignments, creation and deletion of products/categories if they are considered in a catalog view, update the search query definition of a category with dynamic binding.
For a particular, storefront-browsing user the Solr search server needs to collect the visible products for this user and must return them as a result. As already mentioned, there is a search index attribute called
CatalogFilterIDs which contains all catalog filters in which the product is visible. This attribute has to be included in the indexed attributes of a search index to be able to select only products, visible in a particular catalog view. Once the index is rebuilt and contains catalog view information, it is ready for search queries including catalog filter conditions.
Search queries are performed when browsing leaf categories with products, during paging of product lists in a category, when search by term in the search field is triggered, when filtering search results by different criteria (category, brand, price, ...). In all these cases catalog filters are also considered. A search query generation process takes place to add catalog filter condition to the query. This process has two steps:
CatalogFilterIDs attribute is included in the index, otherwise no check is performed and no catalog filters are returned. Also, if the attribute is included and the current user don't have any catalog view assigned, no filters are returned.
CatalogFilterIDs attribute is included in the index and there is at least one catalog filter retrieved for the current user. Otherwise, no condition is added to the query. If retrieved user filters are more than one - they are combined with OR operator. Thus, if the current user has three catalog views assigned, the search query will contain the condition
CatalogFilterIDs=CatalogFilterUUID1_or_CatalogFilterUUID2_or_CatalogFilterUUID3. In this way only products (Solr documents) which have
CatalogFilterIDs attribute with one of the values
will be retrieved for this user.
Catalog views can also be used in a data replication (staging) environment. This allows to configure and publish a catalog view in the EDIT system before it gets replicated to the LIVE environment. There is no separate replication group, provided by IS7, for replication of catalog views. Instead, both replication groups (CONSUMER_GROUPS and PROMOTIONS) can be used to transfer the catalog view structures to the production system. These replication groups assign
MVC_CatalogFilter staging group to the current replication task which is responsible for the staging of catalog view related objects. If only catalog views need to be staged, the CONSUMER_GROUPS replication group could be used for the replication task in order to have the replication process faster.