Introduction

When a price list import is executed in the back office to update product prices, "price change events" are created in the database. After the import, a separate job ProcessProductPriceRefresh must be executed to process the created price change events and do further things like CacheRefresh etc. The job ProcessProductPriceRefresh has a job configuration named Cleanup events. Depending on that configuration (true/false), the processed price change events are deleted (Cleanup events = true) or remain in the database (Cleanup events = false/missing). The default value is true.

In order to reduce the number of events to process and also to ensure that the events are processed only once (for option Cleanup events = false), this job only processes events that are created within a certain time frame - from the last run of this job until now.

Since the job ProcessProductPriceRefresh (which consumes the events) is independent from the price list import which creates the events, it could be that both processes run simultaneously. In this case, events are created within the time frame of the current run in which they are to be processed. However, they are not because of parallel execution. This might lead to an accumulation of unprocessed price change events in the database. It could be that there are a large amount of unprocessed items over the time.

This occurred in both modes (Cleanup events = true or false).

References

• Job - ProcessProductPriceRefresh
• Guide - 7.10.22.1 Change Behavior for Job ProductPriceChangeEvents - Refresh Product Prices

Migration

ProcessProductPriceRefresh (Cleanup events = true - default)

In order to fix that issue, the behavior of the job ProcessProductPriceRefresh was changed for the option (Cleanup events = true):

The database could contain outdated price change events that are not relevant anymore in a very large number (> 1million). These events can be removed without any side affects to ensure that the job only processes relevant data.

Therefore the job was adapted and will now automatically remove all outdated events every time the job is executed.

It is configurable how old an item must be before it is considered outdated. The age is specified in days. The default value is 30 and is set as attribute AgeOfOutdatedItemsInDays after the first run. If the value is to be changed before the first run, a new attribute named AgeOfOutdatedItemsInDays and of type Integer must be created in the existing job, which contains the desired value. A value less or equal to 0 will turn off the automatic removal of outdated events.

Running the job with Cleanup events = true usually means that afterwards all processed price change events are removed and the database should only contain future events (e.g. end date of price list). So there is no need to reduce the number of events that are processed by a lower time frame. Therefore this date check (time frame start) has been removed, so that all previously existing positions should be processed. As a result, the price change events created from the price list import during a parallel execution remain in the database after the latest run of the job ProcessProductPriceRefresh. They will then be processed with the next run.

To ensure that the processing of the events does not lead to memory problems, the process is executed in sub-steps. Only a limited number of items are loaded and processed in each sub-step. Each sub-step is started immediately after the previous one is finished. The default value is 10000 and is set as attribute BatchSize after the first run. If the value is to be changed before the first run, a new attribute named BatchSize and of type Integer must be created in the existing job, which contains the desired value.

The following picture shows the newly introduced attributes which will automatically be created with default values after the first run of the job after migration.

Since the mode "Cleanup events = true" is the default option, this fix will cover most cases. There is no further migration necessary.

ProcessProductPriceRefresh (Cleanup events = false)

The behavior for the second option (Cleanup events = false) remains unchanged.

This means there could still be unprocessed price change events. Therefore, this option is not recommended.