In this blog post, we describe a possible approach to replicate contact and sales data from SAP S/4HANA to SAP Emarsys Customer Engagement. The content described is created individually and is not part of a ready-made SAP integration.
The described integration scenarios can be used as a template to create a custom integration with SAP S/4HANA and SAP Emarsys Customer Engagement. SAP Cloud Integration is used for message transformation for API-based data imports to Emarsys.
The following diagram depicts the high-level setup of an example integration between SAP S/4HANA and SAP Emarsys Customer Engagement using SAP Cloud Integration.
This blog post does not cover importing product data into Emarsys.
For information on uploading product data to Emarsys, refer to the Emarsys help documentation.
- Prepare your product data file
https://help.emarsys.com/hc/en-us/articles/214245045-Preparing-your-product-data-file - Loading your product data
https://help.emarsys.com/hc/en-us/articles/214245085-Uploading-your-product-data
Products/solutions/services used:
- SAP S/4HANA in the cloud or on premises
- SAP Cloud-Integration (SAP Cloud Integration Suite)
- SAP Emarsys Customer Loyalty
- SFTP server for product data (not covered in this blog post)
integration overview
In this blog post, we will focus on replicating B2C Sales Orders/Contacts and B2C Trading Partners.
As of now (Q2 2022), SAPEmarsysCustomer Engagement does not offer a public API to import product data.
Before loading data, familiarize yourself with the available data objects and their corresponding fields and data types.
- Emarsys Data Integration:https://help.emarsys.com/hc/en-us/articles/214230905-Data-onboarding-Overview
- Emarsys API documentation:https://dev.emarsys.com/docs/emarsys-api
Trading partner replication
- Emarsys Documentation - Contact Data Fields Overview
https://help.emarsys.com/hc/en-us/articles/115004637605-Overview-Contact-data-fields-Overview - Emarsys Documentation - Upload your contact details
https://help.emarsys.com/hc/en-us/articles/213706389-Uploading-your-contact-data - SAP S/4HANA Outbound Service: Trading Partner Replication via SOAP Web Service
- It is recommended to collect and send messages in batches
- Only relevant trading partners should be replicated to Emarsys
- Emarsys-API: Kontakt-API
- IFlow: contact data replication
- Process SOAP message received from SAP S/4HANA
- Converts the incoming SOAP message request into an Emarsys contact API request
- Message processing errors send notification to the defined tenant admin via aEmarsys Broadcast-Event.
Exception handling is described in this blog post "IFlow error notification" - Messages received with errors are written to the SAP Cloud Integration data store for reprocessing.
- IFlow: Contact Data Processing 1
- The failed SOAP message was reprocessed and the failed message was resubmitted to the contact data replication IFlow.
- If the message fails again, the entry is removed from the datastore and moved to another datastore for a second reprocess.
- IFlow: Contact Data Processing 2
- The failed SOAP message was reprocessed and the failed message was resubmitted to the contact data replication IFlow.
- If the message fails again, the entry is removed from the datastore and moved to another datastore for verification.
- Entries in the datastore for verification are not processed again.
customer order replication
- Emarsys Documentation: Prepare Your Sales Data File
https://help.emarsys.com/hc/en-us/articles/360003070654-Preparing-your-sales-data-file - Emarsys Documentation: Upload Your Sales Data
https://help.emarsys.com/hc/en-us/articles/213706429-Uploading-your-sales-data - SAP S/4HANA Outbound Service: Customer Order Replication via SOAP Web Service
- It is recommended to collect and send messages in batches
- Only relevant customer orders need to be replicated in SAP Emarsys Customer Engagement.
Custom filters can be set up in SAP S/4HANA to control sales orders replicated to receiving systems.
- IFlow: replication of customer order data
- Process SOAP message received from SAP S/4HANA
- Converts the incoming SOAP message request to an Emarsys Sales Data API request
- Message processing errors send an Emarsys broadcast event notification to the defined tenant admin
Product data replication
- A file-based approach is used.
- Product replication is not covered in this blog post.
- Prepare your product data file
https://help.emarsys.com/hc/en-us/articles/214245045-Preparing-your-product-data-file - Loading your product data
https://help.emarsys.com/hc/en-us/articles/214245085-Uploading-your-product-data
A secure connection is established between SAP S/4HANA and SAP Cloud Integration.
Follow the SAP S/4HANA setup guide to integrate SAP S/4HANA with SAP Marketing Cloud as the same outbound services are used for this integration.
It is recommended to apply custom filters in SAP S/4HANA to replicate only relevant data, this is especially important for sales data replication.
Also, you should consider batch data replication for better performance.
Configuration Guides for SAP S/4HANA Integration
- SAP S/4HANA Integration with SAP Marketing Cloud:https://api.sap.com/package/SAPS4HANAEnterpriseManagementOnPremiseIntegrationwithSAPHybrisMarketingCloud/overview
- SAP S/4HANA Cloud Integration with SAP Marketing Cloud:https://api.sap.com/package/SAPHybrisMarketingCloudSAPS4HANAEnterpriseCloudIntegration/integrationflow
SAP S4/4HANA Setup Overview
Summary of configuration steps.
- Configure a secure connection between systems
- Configuration in SAP Cloud Integration
- The endpoint created and the communication used must be maintained in SAP S/4HANA
- Configuration in SAP S4/4HANA
- Scenarios to configure:
- Trading partner replication
- Endless Purpose (EOP) for Business Partners
- No business partner relationship
- customer order replication
- No product replication directly from S4HANA. SOAP WS sends partial product catalog updates instead of the entire catalog as a full extract
- Initial data load from SAP S/4HANA
- First data upload for trading partners
- Entry fee for customer orders
SAP Cloud Integration helps you connect on-premises and cloud applications with other SAP and non-SAP on-premises and cloud applications. This service can handle messages in real-time scenarios that span different companies, organizations, or departments within an organization.
The SAP Integration Suite combines the integration capabilities of Process Integration, API Management, Integration Advisor, and Open Connectors into a cohesive, streamlined set of tools for enterprise integrations. To provide a complete integration experience, these services are not available separately, only as part of the Integration Suite service plan. For more information about the different service plans, seeIntegration packservice catalogue.
- SAP cloud integration help documentation:https://help.sap.com/viewer/product/CLOUD_INTEGRATION/LATEST/en-US
integration pack
With SAP Cloud Integration, you can package and publish integration content so that integration developers can use these packages in their integration scenarios.
As an integration developer, you can create integration packages for your specific domain or organization. You can also view various packages released by other integration developers and use them for your integration purposes. You can modify these packages as per your needs and load them through the web application.
- Integrationsinhaltspaket no SAP Cloud Integration:https://help.sap.com/viewer/368c481cd6954bdfa5d0435479fd4eaf/LATEST/en-US/89da0a21aa244dd0b861c7ab6287983b.html
integration flows
An integration flow lets you specify how a message is processed within a tenant.
You can use integration flows to specify specific integration patterns, such as mapping or routing.
A graphical editor allows you, as an integration developer, to model the message processing steps and specify in detail what happens to the message during processing.
- Integration content development with SAP Cloud Integration:https://help.sap.com/viewer/368c481cd6954bdfa5d0435479fd4eaf/LATEST/en-US/e6b43b4c5a5042fda30a9dfdab97eff3.html
- Elements of an integration flow:https://help.sap.com/viewer/368c481cd6954bdfa5d0435479fd4eaf/LATEST/en-US/e49dbee00fa549f78b863b9f0ab736d4.html
security artifacts
Secure user credentials and parameters
User credentials and required parameters are created and stored in the Manage Security Material component in SAP CPI.
- Management of security material:https://help.sap.com/viewer/368c481cd6954bdfa5d0435479fd4eaf/LATEST/en-US/b8ccb53b9ec44652b885476f02184a0f.html
- Implementing a user credential artifact:https://help.sap.com/viewer/368c481cd6954bdfa5d0435479fd4eaf/LATEST/en-US/6912d63bbbc64aee8bbd4ff10314c60c.html
- Implementing a safe parameter artifact:https://help.sap.com/viewer/368c481cd6954bdfa5d0435479fd4eaf/LATEST/en-US/4641d6c531d34cf7aef94ba5a873bf6e.html
key shop
All keys, key pairs, and certificates for communicating with SAP CPI are stored in the SAP CPI Keystore.
To enable a successful SSL handshake, the root certificates of connected systems must be added to the SAP CPI keystore.
- Keystore entry management:https://help.sap.com/viewer/368c481cd6954bdfa5d0435479fd4eaf/LATEST/en-US/2dc8942e02de4be59bd2afaa3bfdc591.html
SAP Emarsys Customer Engagement Value Mapping
Creates a value mapping artifact that acts as a bidirectional lookup table. Value mapping has the distinct advantage of providing bidirectional lookup capabilities that are commonly used in productive mapping scenarios.
Value mapping can be accessed from message mapping using the mapValue function.
- Configure value mappings:https://help.sap.com/viewer/368c481cd6954bdfa5d0435479fd4eaf/LATEST/en-US/6c8847fe8dcc459580a63194fc55d5b3.html
When creating a custom integration, it's a good idea to define an external parameter to make setup easier. External parameters allow you to configure the IFlow without having to edit the IFlow.
Editing the IFlow is only necessary if you need to change the transformation and processing of the message, for example B. when adding and mapping a new custom field.
For more information on defining and configuring external parameters, refer to the SAP help documentation.
- Externalize parameters of an integration flow:https://help.sap.com/viewer/368c481cd6954bdfa5d0435479fd4eaf/LATEST/en-US/45b2a0772db94bd9b0e57bc82d8d3797.html
To configure IFlow, open the integration pack and select the "Configure" option from the drop-down menu.
- Sender: Set receiver adapter for incoming messages.
- Recipient: Set the sender adapter for outgoing messages.
- If multiple sender adapters are configured, you can change the sender adapter from the drop-down list.
- More: All external IFlow properties. The default view, Views. To view the externalized properties of a specific step in the message transformation, select the step from the drop-down list.
i flow:Business Partner Replication from SAP S4 HANA to SAP Emarsys Customer Engagement
This IFlow takes incoming messages from SAP S/4HANA and turns the incoming message into a well-formatted Emarsys contact API request.
Step | Description |
Station: S4OnPrem Type: soap | SOAP sender is configured
|
content modifier Set properties and headers | Message headers and properties are defined. Exchange properties:
|
wonderful script log incoming message |
|
XSLT Mapping Filter address information |
|
message mapping Emarsys contact mapping |
To change the message schema, for example when adding new custom fields, change the schema file in the IFlow resources. The message mapping schema is updated with the message schema and the additional field can be mapped. Use graphical message mapping to fine-tune message transformation logic. Example of an Emarsys XML Schema with Emarsys Field ID: When using the Emarsys Fields field ID, XML element names cannot be numeric. Add a prefix to all field names and remove the prefix later. The XML format does not allow definitions of elements such as <1>value</1>. <?xml version="1.0" encoding="UTF-8"?> <xs:schema attributeFormDefault="not qualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema"> <xs:element name="root"> <xs:tipocomplejo> <xs:sequence> <xs:element type="xs:string" name="contact_list_id"/> <xs:element type="xs:string" name="key_id"/> <xs:element name="Contacts" maxOccurs="unlimited" minOccurs="0"> <xs:tipocomplejo> <xs:sequence> <xs:element type="xs:string" name="tmp_1" maxOccurs="1" minOccurs="0"/> <xs:element type="xs:string" name="tmp_2" maxOccurs="1" minOccurs="0"/> <xs:element type="xs:string" name="tmp_3" maxOccurs="1" minOccurs="0"/> </xs:stream> </xs:tipocomplejo> </xs:element> </xs:stream> </xs:tipocomplejo> </xs:element> </xs:schema> Example of an Emarsys XML Schema with an Emarsys String Field ID: <?xml version="1.0" encoding="UTF-8"?> <xs:schema attributeFormDefault="not qualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema"> <xs:element name="root"> <xs:tipocomplejo> <xs:sequence> <xs:element type="xs:byte" name="key_id"/> <xs:element name="Contacts" maxOccurs="unlimited" minOccurs="0"> <xs:tipocomplejo> <xs:sequence> <xs:element type="xs:string" name="Vorname" maxOccurs="1" minOccurs="0"/> <xs:element type="xs:string" firstname="lastname" maxOccurs="1" minOccurs="0"/> <xs:element type="xs:string" name="email" maxOccurs="1" minOccurs="0"/> </xs:stream> </xs:tipocomplejo> </xs:element> </xs:stream> </xs:tipocomplejo> </xs:element> </xs:schema> |
router Validate PA |
|
router remove empty |
If empty elements are removed, a delta update is sent to Emarsys without overwriting existing values. |
XSLT Mapping remove empty nodes |
This process step is only performed if the Delete empty branch option is enabled. |
wonderful script Create WSSE Header |
For more information on how to generate the WSSE header, see the Emarsys developer documentation. |
common divisor Divisor-XML | The XML splitter splits the incoming message payload into smaller packets. The packet size can be configured in the General Splitter packet parameter. The package can also be controlled by SAP S4HANA. |
XML to JSON converter | Converts the XML message to a JSON message |
wonderful script remover tmp_ | custom function called: replaceTmp Replaces all tmp_ values for field IDs to match Emarsys field IDs. |
wonderful script registration announcement | See Groovy script "Log incoming message" |
content modifier Remove Header URI |
|
Request answer send contact |
|
wonderful script validate answer |
|
router |
|
subprocesso local error handling |
|
subprocesso local error handling wonderful script LogError |
|
subprocesso local error handling Router 1 |
|
subprocesso local error handling content modifier Reset message payload |
|
subprocesso local error handling write data storage Save the message to edit it again |
|
subprocesso local error handling router |
|
subprocesso local error handling wonderful script definir host tmn |
|
subprocesso local error handling content modifier define external event |
The following external event message text is sent to Emarsys. { "email": "${property.adminEmail}", "Data": { "global": { "adminEmail": "${propiedad.adminEmail}", "timestamp": "${property.CamelCreatedTimestamp}", "CPIHost": "${header.CPITmnHost}", "CPIPath": "${header.CamelHttpUri}", "Caught exception": "${property.CamelExceptionCaught}", "expectionMessage": "${Exception.Message}", "messageProcessingLogID": "${propiedad.SAP_MessageProcessingLogID}" } } } |
subprocesso local error handling content modifier |
|
subprocesso local error handling wonderful script Create WSSE Header |
|
subprocesso local error handling Rezeptor HTTP |
|
IFlow: Trading Partner Replication from SAP S4 HANA to SAP Emarsys Customer Engagement Data Reprocessing
This IFlow reprocesses failed messages from the Business Processing IFlow (IFlow:Business Partner Replication from SAP S4 HANA to SAPEmarsys Customer Engagement). Failed trading partners are stored in SAP CPI and reprocessed once a day. A time window outside of business hours is selected for reprocessing.
Most errors are failed authentication requests when Emarsys receives too many requests at the same time. Resending the same request should fix this issue.
Each failed message can be reprocessed up to two times before being saved for review.
Step | Description |
time the start |
|
data storage selection Choose Contacts |
|
Dissident split single message |
|
Filter |
|
content modifier |
|
exception process Reset message text |
|
write data storage Write data Check data storage |
|
IFlow: SAP S4 HANA Trading Partner Replication for SAPEmarsys Customer Engagement Data Reprocessing 2
This IFlow is a copy of the first reprocess IFlow, except that the datastore write operation writes to a datastore for data validation, where the data is not encoded.
i flow:Replication of customer orders from SAP S4HANA to SAP Emarsys Customer Engagement
This IFlow takes incoming messages from SAP S/4HANA and transforms the incoming message into a well-formatted Emarsys Sales API request.
To avoid creating duplicate sales order data, sales order lines are only created when certain conditions are met.
Conditions when a sales order is considered completed may differ depending on how sales orders are processed in SAP S/4HANA.
Note that sales order imports are incremental and always add to existing sales data.
Upload your sales data:https://help.emarsys.com/hc/en-us/articles/213706429-Uploading-your-sales-data
It is recommended to control the release of sales data to Emarsys in SAP S/4HANA by defining certain conditions in SAP S/4HANA. Additional terms can be defined in SAP Cloud Integration, if required.
The described conditions are only an example and may differ for other defined systems and processes.
Data fields for installation conditions in SAP Cloud Integration can be at the order header and/or line item level.
- The order is completed and shipped
- The order will be canceled
- Order is in progress
- The item is complete and shipped.
- The item has been canceled
- The article is in progress
This is an example of how to define conditions to be implemented in SAP Cloud Integration.
The order is completed and shipped | The order will be canceled | Order is in progress | |
The item is complete and shipped. | The order is created article is created | The order is created the item has been removed | Request is ignored |
The item has been canceled | The order is created the item has been removed | The order is created The element is created with negative values. | Request is ignored |
The article is in progress | article is ignored | article is ignored | Request is ignored |
Step | Description |
Station: S4OnPrem Type: soap |
|
content modifier Set properties and headers |
|
wonderful script Register input payload |
|
common divisor Divisor-XML |
|
message mapping Emarsys contact mapping |
To change the message schema, for example when adding new custom fields, change the schema file in the IFlow resources. The message mapping schema is updated with the message schema and the additional field can be mapped. Use graphical message mapping to fine-tune message transformation logic. Emarsys Sales Data XML Schema Example <?xml version="1.0" encoding="UTF-8"?> <xs:schema attributeFormDefault="not qualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema"> <!-- Documentation:https://help.emarsys.com/hc/en-us/articles/360003070654-Preparing-your-sales-data-file#required-fields--> <xs:elementname="data"> <xs:tipocomplejo> <xs:sequence> <xs:item name="Sell Orders" minOccurs="0" maxOccurs="unlimited"> <xs:tipocomplejo> <xs:sequence> <xs:element name="order" minOccurs="0" maxOccurs="unlimited"> <xs:tipocomplejo> <xs:sequence> <xs:element type="xs:string" minOccurs="0" name="item"/> <xs:element type="xs:string" minOccurs="0" name="preço"/> <xs:element type="xs:string" minOccurs="0" name="order"/> <xs:element type="xs:string" minOccurs="0" name="timestamp"/> <xs:element type="xs:string" minOccurs="0" name="cliente"/> <xs:element type="xs:string" minOccurs="0" name="email"/> <xs:element type="xs:string" minOccurs="0" name="quantity"/> </xs:stream> </xs:tipocomplejo> </xs:element> </xs:stream> </xs:tipocomplejo> </xs:element> </xs:stream> </xs:tipocomplejo> </xs:element> </xs:schema> |
XSLT Mapping delete items |
|
To bring together |
|
router blank check |
|
Filter |
|
content modifier add root |
|
XML to CSV Converter |
|
wonderful script outgoing load log |
|
content modifier Emarsys Sales-API |
|
wonderful script set permission |
|
Rezeptor HTTP Send sales data |
|
router |
|
wonderful script logHTTPError |
|
subprocesso local error handling |
|
IFlow error notification
SAP Cloud Integration provides an SMTP adapter to send emails directly from SAP Cloud Integration Email, but no email editor or other features to easily define and design an email. As we are already building an integration with SAPEmarsys Customer Engagement, the triggered email capabilities of the solution can be leveraged to send email notifications to a recipient.
For this specific use case, using broadcast events to trigger an email may be more suitable than external events, as emails with broadcast events do not require the contact to be in your contact database and the Email transmissions are not tracked by transmission events. Broadcast emails are still considered external event emails for billing purposes.
Once created and activated, you can allow error notifications to be sent from different IFlows using the same process.
In this example we define the recipient who should receive the error message in IFlow. Additional attributes are defined and passed in the request body, which can then be used for email personalization in Emarsys. Changes to the email can be made without changing the IFlow.
For more information, see the Emarsys email-enabled documentation.
Triggered Email – End User Guide:https://help.emarsys.com/hc/en-us/articles/360016376278-End-user-guides-Triggered-Email-End-user-guide
SAP CPI Configuration
- Enable email notification by setting the MailNotification exchange property to true.
- We added this to allow the CPI manager to decide whether or not to enable error notifications.
- Add an email address for the SAP CPI administrator.
- In our example, we set the email notification recipient in IFlow as an external property.
The recipient can also be passed from the incoming request and set dynamically in the CPI.
- In our example, we set the email notification recipient in IFlow as an external property.
- In the HTTP receiver channel for the email notification, add the streaming event ID generated by Emarsys.
Emarsys Configuration
- Create new email enabled
- Select Broadcast Event as the trigger.
A broadcast event ID is provided. - Define an email template.
- Enable triggered email flow
Example of a broadcast event defined in SAP CPI.
{ "email": "${property.adminEmail}", "Data": { "global": { "adminEmail": "${propiedad.adminEmail}", "timestamp": "${property.CamelCreatedTimestamp}", "CPIHost": "${header.CPITmnHost}", "CPIPath": "${header.CamelHttpUri}", "Caught exception": "${property.CamelExceptionCaught}", "expectionMessage": "${Exception.Message}", "messageProcessingLogID": "${propiedad.SAP_MessageProcessingLogID}" } } } |
Step | Description | |
wonderful script Registry Error |
| |
Router 1 |
| |
Message reprocessing branch |
| |
router |
| |
wonderful script Definir host tmn |
| |
content modifier define external event |
| |
content modifier Remove Header URI |
| |
wonderful script |
| |
request-response send error message |
|
SAP Emarsys Customer Engagement Setup
api users
Create an API user in SAPEmarsys Customer Engagement and create a security artifact with the Emarsys API user credentials in SAP Cloud Integration. The security artifact can be accessed via IFlow.
In Emarsys, add the necessary permissions for the external contacts and events API to the API user.
- API-Benutzer:https://help.emarsys.com/hc/en-us/articles/115004740329-your-account-security-settings#api-users
Sales Data API
The Sales Data API endpoint and token can be obtained from Emarsys.
Open Forecast Data Sources and go to Manage Your Sales Data to find the Sales Data API upload details.
- Sales Data API specs:https://help.emarsys.com/hc/en-us/articles/213706429-Uploading-your-sales-data#endpoint
E-mail triggered for SAP Cloud Integration error notifications
A triggered email is created with a triggering broadcast event.
For more information on triggered emails, see the Emarsys help documentation.
- Enabled End User Guide Emails:https://help.emarsys.com/hc/en-us/articles/360016376278-End-user-guides-Triggered-Email-End-user-guide
- Launch an email campaign for virtual contacts (broadcast):https://dev.emarsys.com/docs/emarsys-api/b3A6MjQ4OTk4MzY-launch-an-email-campaign-to-virtual-contacts-broadcast
Step | Description |
---|---|
Email enabled by Emarsys | Channels > Enabled Emails > Create Message Flow |
email settings | Complete the email settings and select "Broadcast event" in the activation settings. Note the broadcast event ID
|
content creation | Set the email basics and set the error message email to be sent. |