Guide - Transport Framework

1 Introduction

Regarding online shops there is a large amount of data import and export processes. Product data and price data has to be imported, order data has to be exported. The origins of the product and price data are often files from another server. The target destination of the order data is also often another server. The Transport Framework can transport files from one location of the network to another, where they can be processed.

2 Sources

The source code belongs to the cartridges bc_transport and bc_transport_orm. Both are part of the p_platform component set.

The UI source code is added to the according Commerce Management cartridge via an extension point. 

3 Functionality

The Transport Framework allows to perform file transports via SFTP, FTP, HTTP(S) or EMAIL.
For SFTP and FTP, pushing and pulling are possible. Due to the nature of HTTP and EMAIL, files can only be read from HTTP(S) and send by mail.

Note: If using SFTP with key authentication, private key needs to be in the correct format (PEM legacy). Newer openssh implementations by default generating key-pairs in RFC4716 format rather than PEM. Exception will be thrown by Transport job in that case when using such a key. To circumvent that, "ssh-keygen -t rsa -m PEM" (maybe with certain additional parameters to define target file and/or key strength) needs to be used to generate the key pair.

4 Access Privilege

To access the Transport Framework, the user must have the access privilege Transport Manager.

5 Database Tables

A configuration of the Transport Framework is stored in the database tables TRANSPORTCONFIG and TRANSPORTCONFIG_AV.

6 DBInit

The feature requires a dbinit of the cartridge bc_transport_orm from p_platform.

7 DBInit, DBMigrate

A Transport Configuration can also be created with a dbinit- or dbmigrate-step.

The preparers are:

Preparers
com.intershop.component.transport.dbinit.PrepareTransportConfiguration
com.intershop.component.transport.dbmigrate.AddTransportConfiguration

Each preparer requires a property file for configuration purposes. In the property file, the keys from the class TransportConstants are read and the result is written in the DB. There is no parameter validation. This way, a blueprint can be created via dbmigrate. Such a blueprint can be completed later.

Examples can be found in the source code in the folder bc_transport_orm/staticfiles/cartridge/lib/com/intershop/component/transport/dbinit.

SFTP.properties
domain 				    = inSPIRED-inTRONICS
process.type 		    = SFTP
# Common settings
process.displayname     = testSFTP
process.id 			    = testSFTP
file.include.pattern    = 
file.exclude.pattern    = 
location.local 		    = ${IS_SHARE}\\sites\\${SITE_NAME}\\units\\${UNIT_NAME}\\impex\\export
location.archive 	    = 
# remote location settings
remote.protocol         = SFTP
remote.hostname 	    = localhost
remote.port 		    = 
location.remote         = ./test
# authentication settings
authentication.method   = PASSPHRASE
authentication.username = tester
authentication.password = password
authentication.keyfilepath = 
# transfer settings
process.direction 		= PUSH
process.transferlimit   = 
process.delete     		= true

8 Usage

8.1 Transport Configuration List

  1. Log in to the Commerce Management application (organization = Operations).
  2. Ensure you have the required access privilege.
  3. Click Transport Configuration in the left navigation.

    Each configuration belongs to a domain.

  4. Select a domain and click Apply.
    All available configurations for the domain are displayed in the list.

8.2 Create a New Configuration

  1. Go to the Transport Configuration list.
  2. Select a domain and click Apply.
  3. Enter a Name.
  4. Select a Type (EMAIL, HTTP, FTP, SFTP).
  5. Click Add.

8.3 Delete a Configuration

  1. Go to the Transport Configuration list.
  2. Select a domain and click Apply.
  3. Mark the checkbox of the desired configuration.
    You may select multiple configurations at once.
  4. Click Delete and confirm with OK.

8.4 Edit a Configuration

  1. Go to the Transport Configuration list.
  2. Select a domain and click Apply.
  3. Click the link (Name column) of the desired configuration.
    The configuration's detail page is displayed.
  4. Enter all required parameters.
    Depending on the selected Type of the configuration, different parameters have to be specified. However, each input field provides substantial help texts.
  5. Click Apply.

Note

A configuration cannot be saved until all mandatory fields are filled with valid parameters. Up to this point a message is displayed stating that the configuration is invalid.

Please note that the for the input fields only a basic validation is performed, e.g., if a URL is required, the system only checks whether the string entered can be parsed as a URL, but not whether the endpoint exists.

8.4.1 SFTP Example

For a push or pull, the host and the remote location must be entered. The remote location is the path to the folder which is the source or target of a push- or pull-event.

9 Execute a Transport Job

A transport configuration can be executed in the System Management application.

  1. Log in to the System Management application.
  2. Select the Domain (site) where you intend to create the schedule and click Apply.
  3. Enter all required parameters and click Apply.

    ParameterValue
    PipelineFileTransportJob
    StartnodeStart

  4. Switch to the Attributes tab.

  5. The job needs two parameters:

    1. The TransportProcessID = Process ID of the transport configuration.
    2. The UnitName (name of the domain).

10 Add Additional Transport Type

To add an additional transport type, a class implementing com.intershop.component.transport.capi.provider.TransportProvider needs to be created. This class is responsible for creating and determining existing TransportConfiguration objects and creating the corresponding TransportExecutor which will implement the technical transport of files.

com.intershop.component.transport.capi.provider.TransportProvider
public interface TransportProvider
{

    /**
     * The type of the transport 
     */
    String getType();

    /**
     * The name under which the created Objects are put into pipeline dictionary
     */
    String getDictionaryKey();

    /**
     * get a business object for the given persistent object
     * @param anID  the process id
     * @param someContext   a business object context
     * @param transportConfiguration    the given persistent object
     * @return
     */
    TransportProcessConfigBO getTransportProcessConfigBO(String anID, BusinessObjectContext someContext,
                    TransportConfiguration transportConfiguration);

    /**
     * create a new business object for a transport configuration
     * @param displayName  the display name 
     * @param someContext   a business object context
     * @param transportConfiguration    the given persistent object
     * @return
     */
    TransportProcessConfigBO createTransportProcessConfigBOByName(String displayName, Domain domain,
                    BusinessObjectContext someContext);

    /**
     * create a new business object for a transport configuration
     * @param processID  a process id 
     * @param someContext   a business object context
     * @param transportConfiguration    the given persistent object
     * @return
     */
    TransportProcessConfigBO createTransportProcessConfigBO(String processID, Domain domain,
                    BusinessObjectContext someContext);

    /**
     * create a new transport executor to execute a file transport
     * @param processID  a process id 
     * @param someContext   a business object context
     * @param transportConfiguration    the given persistent object
     * @return
     */
    TransportExecutorBO createTransportExecutorBO(String anID, BusinessObjectContext someContext,
                    TransportProcessConfigBO aConfig);

	/**
     * update a transport configuration for the special config type during dbinit / dbmigrate 
     * @param transportConfig   the transport config to update
     * @param config            map of properties
     */
    default void updateTransportConfiguration(TransportConfiguration transportConfig, Map<String, String> config)
    {
        
    }
}

Afterwards this class can be bound to the object graph.

AzureTransportModule
public class AzureTransportModule extends AbstractNamingModule
{
    @Override
    protected void configure()
    {
        MapBinder.newMapBinder(binder(), String.class, TransportProvider.class).addBinding(AzureTransportProvider.TYPE).to(AzureTransportProvider.class).in(Singleton.class);
    }
}

Disclaimer

The information provided in the Knowledge Base may not be applicable to all systems and situations. Intershop Communications will not be liable to any party for any direct or indirect damages resulting from the use of the Customer Support section of the Intershop Corporate Web site, including, without limitation, any lost profits, business interruption, loss of programs or other data on your information handling system.

Customer Support
Knowledge Base
Product Resources
Support Tickets