Related Documents
Document Properties
Kbid
28635Y
Last Modified
25-May-2021
Added to KB
05-Jun-2018
Public Access
Everyone
Status
Online
Doc Type
Guidelines, Concepts & Cookbooks
Product
ICM 7.10

Guide - Transport Framework

1 Introduction

Online shops involve a large number of data import and export processes. Product data and price data has to be imported, order data has to be exported. The origins of the product and price data are often files from another server. The target destination of the order data is also often another server. The Transport Framework can transport files from one location of the network to another, where they can be processed.

1.1 References

2 Sources

The source code belongs to the cartridges bc_transport and bc_transport_orm. Both are part of the p_platform component set.

The UI source code is added to the according Commerce Management cartridge via an extension point. 

3 Functionality

The Transport Framework allows to perform file transports via SFTP, FTP, HTTP(S), EMAIL or Azure Storage Account File Shares.
For SFTP and FTP, pushing and pulling are possible. Due to the nature of HTTP and EMAIL, files can only be read from HTTP(S) and send by mail.

Note

If using SFTP with key authentication, the private key must be provided in the correct format (PEM legacy). Newer open SSH implementations generate key pairs in the RFC4716 format by default, rather than the PEM format. If such a key is used, an exception will be thrown by the transport job. To avoid that use the following command to generate the key pair: 
ssh-keygen -t rsa -m PEM (you may include certain additional parameters to define target file and/or key strength).

4 Access Privileges

To access the Transport Framework, the user must have the access privilege Transport Manager.

5 Database Tables

A configuration of the Transport Framework is stored in the database tables TRANSPORTCONFIG and TRANSPORTCONFIG_AV.

6 DBInit

The feature requires a DBinit of the cartridge bc_transport_orm from p_platform.

7 DBInit, DBMigrate

A Transport Configuration can also be created with a DBinit step or DBmigrate step.

The preparers are:

Preparers
com.intershop.component.transport.dbinit.PrepareTransportConfiguration
com.intershop.component.transport.dbmigrate.AddTransportConfiguration

Each preparer requires a property file for configuration purposes. In the property file, the keys from the class TransportConstants are read and the result is written in the database. There is no parameter validation. This way, a blueprint can be created via DBmigrate. Such a blueprint can be completed later.

Examples can be found in the source code in the folder bc_transport_orm/staticfiles/cartridge/lib/com/intershop/component/transport/dbinit.

SFTP.properties
domain 				    = inSPIRED-inTRONICS
process.type 		    = SFTP
# Common settings
process.displayname     = testSFTP
process.id 			    = testSFTP
file.include.pattern    = 
file.exclude.pattern    = 
location.local 		    = ${IS_SHARE}\\sites\\${SITE_NAME}\\units\\${UNIT_NAME}\\impex\\export
location.archive 	    = 
# remote location settings
remote.protocol         = SFTP
remote.hostname 	    = localhost
remote.port 		    = 
location.remote         = ./test
# authentication settings
authentication.method   = PASSPHRASE
authentication.username = tester
authentication.password = password
authentication.keyfilepath = 
# transfer settings
process.direction 		= PUSH
process.transferlimit   = 
process.delete     		= true


8 Usage

8.1 Transport Configuration List

  1. Log in to the Commerce Management application (organization = Operations).
  2. Ensure you have the required access privilege.
  3. Click Transport Configuration in the left navigation.

    Each configuration belongs to a domain.

  4. Select a domain and click Apply.
    All available configurations for the domain are displayed in the list.

8.2 Create a New Configuration

  1. Go to the Transport Configuration list.
  2. Select a domain and click Apply.
  3. Enter a Name.
  4. Select a Type (EMAIL, HTTP, FTP, SFTP, AZURE).
  5. Click Add.

8.3 Delete a Configuration

  1. Go to the Transport Configuration list.
  2. Select a domain and click Apply.
  3. Mark the checkbox of the desired configuration.
    You may select multiple configurations at once.
  4. Click Delete and confirm with OK.

8.4 Edit a Configuration

  1. Go to the Transport Configuration list.
  2. Select a domain and click Apply.
  3. Click the link (Name column) of the desired configuration.
    The configuration's detail page is displayed.
  4. Enter all required parameters.
    Depending on the selected Type of the configuration, different parameters have to be specified. However, each input field provides substantial help texts.
  5. Click Apply.

Note

A configuration cannot be saved until all mandatory fields are filled with valid parameters. Up to this point a message is displayed stating that the configuration is invalid.

Please note: Only basic validation is performed for the input fields, e.g., if a URL is required, the system only checks whether the string entered can be parsed as a URL, but not whether the endpoint exists.

8.4.1 SFTP Example

The following example shows settings for SFTP:

For a push or pull, the host and the remote location must be entered. The remote location is the path to the folder which is the source or target of a push- or pull-event.

8.4.2 Azure Storage Account File Share Example

The following example shows settings for Azure storage:

Account name and key can be found in the Azure portal:

For testing purposes it is recommended to use the Microsoft Azure Storage Explorer:

9 Execute a Transport Job

A transport configuration can be executed in the System Management application.

  1. Log in to the System Management application.
  2. Select the Domain (site) where you intend to create the schedule and click Apply.
  3. Enter all required parameters and click Apply.

    ParameterValue
    PipelineFileTransportJob
    StartnodeStart

  4. Switch to the Attributes tab.

  5. The job needs two parameters:

    1. The TransportProcessID = Process ID of the transport configuration.
    2. The UnitName (name of the domain).

10 Add Additional Transport Type

To add an additional transport type, a class implementing com.intershop.component.transport.capi.provider.TransportProvider has to be created. This class is responsible for creating and determining existing TransportConfiguration objects and for creating the corresponding TransportExecutor which implements the technical transport of files.

com.intershop.component.transport.capi.provider.TransportProvider
public interface TransportProvider
{

    /**
     * The type of the transport 
     */
    String getType();

    /**
     * The name under which the created Objects are put into pipeline dictionary
     */
    String getDictionaryKey();

    /**
     * get a business object for the given persistent object
     * @param anID  the process id
     * @param someContext   a business object context
     * @param transportConfiguration    the given persistent object
     * @return
     */
    TransportProcessConfigBO getTransportProcessConfigBO(String anID, BusinessObjectContext someContext,
                    TransportConfiguration transportConfiguration);

    /**
     * create a new business object for a transport configuration
     * @param displayName  the display name 
     * @param someContext   a business object context
     * @param transportConfiguration    the given persistent object
     * @return
     */
    TransportProcessConfigBO createTransportProcessConfigBOByName(String displayName, Domain domain,
                    BusinessObjectContext someContext);

    /**
     * create a new business object for a transport configuration
     * @param processID  a process id 
     * @param someContext   a business object context
     * @param transportConfiguration    the given persistent object
     * @return
     */
    TransportProcessConfigBO createTransportProcessConfigBO(String processID, Domain domain,
                    BusinessObjectContext someContext);

    /**
     * create a new transport executor to execute a file transport
     * @param processID  a process id 
     * @param someContext   a business object context
     * @param transportConfiguration    the given persistent object
     * @return
     */
    TransportExecutorBO createTransportExecutorBO(String anID, BusinessObjectContext someContext,
                    TransportProcessConfigBO aConfig);

	/**
     * update a transport configuration for the special config type during dbinit / dbmigrate 
     * @param transportConfig   the transport config to update
     * @param config            map of properties
     */
    default void updateTransportConfiguration(TransportConfiguration transportConfig, Map<String, String> config)
    {
        
    }
}

Afterwards this class can be bound to the object graph:

AzureTransportModule
public class AzureTransportModule extends AbstractNamingModule
{
    @Override
    protected void configure()
    {
        MapBinder.newMapBinder(binder(), String.class, TransportProvider.class).addBinding(AzureTransportProvider.TYPE).to(AzureTransportProvider.class).in(Singleton.class);
    }
}

Disclaimer

The information provided in the Knowledge Base may not be applicable to all systems and situations. Intershop Communications will not be liable to any party for any direct or indirect damages resulting from the use of the Customer Support section of the Intershop Corporate Web site, including, without limitation, any lost profits, business interruption, loss of programs or other data on your information handling system.

Customer Support
Knowledge Base
Product Resources
Tickets