• Skip to main content

HXM Right Now

Powered by SAP SuccessFactors

  • Features
  • Contact
1300 664 903

SAP SuccessFactors

Aug 13 2020

SAP CPI and SAP SuccessFactors Monitoring EP02: The one where CPI writes logs to Execution Manager

This blog post originally appeared in the SAP Community on 12/08/2020.

Discovery Consulting was recently involved in the global SAP SuccessFactors Implementation Design Principles (IDP) workshop on SAP SuccessFactors Integration patterns. Our practice of logging the results of SAP CPI iFlows into SAP SuccessFactors Execution Manager was picked up by the group as a potential SAP recommended best practice! Hopefully, we will see something come out from SAP in the future around this. However, whilst we wait for that to happen – here’s some details about how we at Discovery use this integration best practice to help our clients support their SAP SuccessFactors instances and be better equipped to deal with issues if and when they occur.

This post is targeted at an audience of SAP Cloud Integration developers and the SAP SuccessFactors administrator that they work with. It is a quite technical post and gives some great detail on the solutions that we use with our customers.

Firstly, a little background. Too often in the past I have seen support calls from customers about SAP SuccessFactors to/from SAP ERP (and other systems) integrations and a request to “find out what went wrong.” Many/most of these issues could be solved if only the SAP SuccessFactors administrators had access to any error/success messages that the integrations were producing. The aim of these blogs is to show how easy it is to incorporate logging to SAP SuccessFactors Execution Manager into all your SAP CPI integrations so that admins and developers have a simple way to monitor all SAP SuccessFactors integrations.

This post is the second in a series about this logging – you can read the first post which gives some details about the Execution Manager and using OData APIs to update it here:

SAP CPI and Successfactors Monitoring EP01: The one with the Execution Manager Dashboard


Enough background – on with the show!

From my first post we, hopefully, now have an idea of how easy it is to log information into the Execution Manager Dashboard in SAP SuccessFactors. It is now time to put our knowledge to use. As Dale Carnegie once said,

“Knowledge isn’t power until it’s applied.“,

It’s time for us to finally go ahead and see a bigger picture of how exactly we can harness the Execution Manager to our advantage. In this post, I’ll be focusing on how I implemented the Execution Manager in SAP Cloud Platform Integration iFlow.


Bird’s eye view of the implementation

This implementation consists of an iFlow and a simple reusable groovy script to generate EMEvent payload groovy script. Just to make things a bit modular here, I broke the iFlow into the following integration processes:

  1. Main integration process,
  2. Few local integration processes for the following:
    2.1 populating demo data(in external parameters and body) to imitate the process of generating data to be posted to Execution Manager Dashboard, and
    2.2 sending data to Execution Manager Dashboard using HTTP adapter
  3. a separate local integration process for exception handling, and
  4. a groovy script to generate EMEvent payload to be sent to Execution Manager Dashboard in SAP SuccessFactors. 

Now before I get into the details, I would like to quote an expression by Tess Flanders in 1911 that fits here perfectly,

“Use a picture. It’s worth a thousand words“.

So here, I share with you how exactly the implementation looks like when it is ready to log everything we need in the Execution Manager Dashboard **drumrolls**

Image 14

Fig.1: iFlow to log data in Execution Manager Dashboard


Digging into the iFlow

Now let’s see what’s really happening here in the iFlow above (I’ll try explaining things in the order I’ve mentioned above).

1. Main Integration Process

Image 3

Fig.2: Main integration process

a) In this example, every execution of the main integration process logs 3 separate events into the execution manager dashboard, i.e. START – data/error/warning/failure (you can log as many events representing these between a START and END event) – END :Initializing the external properties for EMEvent payload. Here’s what the ‘initialize properties’ step looks like. I have included a sample payload to log a single event from my previous post and how it maps to the properties initialized.

Image 8

Fig.3: Properties defined for EMEvent and EMMonitoredProcess payload

note: The EMMonitoredProcess payload needs to remain the same, as a single execution of the iFlow represents a process in Execution manager. All the events logged under a single process are clubbed together enclosed within a ‘START’ and an ‘END’ event.


b) Once the properties are initialized, the properties for the EMEvent payload needs to be changed for every event we wish to log. The most important one here is the ‘event_type’ property as it decides the type of event logged into the execution manager. So, the next step here is to change the event_type in a content modifier to ‘START’ as we initially need to log a START event.

c) The next step in line is to log the data that matters in the form of multiple events showing data that is required to be presented in the Execution Manager Dashboard. That is where the ‘log data’ step comes in, which triggers the Log data integration process mentioned in the next section. This integration process can be used to fire multiple events as the iFlow carries on its execution. In this case, the iFlow is designed to log a single event of the type INFO to represent a portion of information.

d) Once, all the data/error/warnings have been logged during the execution, there is another enclosing event that needs to be logged. You guessed it right, END event it is 🙂


If we fail to log the END event for a process, the process in all would have an ‘IN_PROGRESS’ state rather than the expected ‘COMPLETED’ state (please refer to my previous blog if you need a reference to different scenarios and the final process state for each).

2. Local Integration Processes

2.1 Log Data integration process

Image 9

Fig.4: Generating data to log

This local integration flow is to represent a process/set of processes that execute during the lifecycle of an integration flow where the iFlow does what it is actually designed to do (pick data from source A, transform, write into destination B). This could be a single local integration flow, or a set of flows to get your job done. This is where we log the events depending on what data is really required by the business.

2.2 Execution Manager Logger integration process

Image 10

Fig.5: Generating and logging payload to Execution Manager Dashboard

This local integration flow is principally responsible for 2 tasks:

  • Executing a script which generates a payload to log an event by using the external properties and body populated by components before this integration process is called.
  • Firing a HTTP call to SAP SuccessFactors Execution Manager Dashboard to log the event.

3. Global Exception Handler

Image 11

Fig.6: Handling errors and exceptions

While working on the solution, there were times when the iFlow during its execution ran into errors such as errors with the script, HTTP connection issues, etc. that caused the iFlow to fail. But how does that impact our logging and why should we care?
As mentioned before, it is important to log an ‘END’ event for every execution to ensure the EMMonitoredProcess does not end up in a ‘IN_PROGRESS’ state even when the actual execution of the iFlow has stopped/failed.

Therefore, this local integration process is to take care of all the errors/exceptions/failures that occur during the execution and ensuring that proper error messages are logged in to the Execution Manager, followed by an ‘END’ event.

4. EMEvent payload generating groovy script

As simple as it should be, this script picks up data from the external properties and populates them into one single JSON payload consisting of an EMEvent and an EMMonitoringProcess.

Image 19

Script 1: Generating EMEvent payload

*disclaimer: Script provided for discussion purposes only, any attempt to use this script or a variation of it in your own integrations is at your own risk. No fitness for use in any particular purpose is, or should be, implied.**

**this said, some of our experienced technical team would be more than happy to help you work with this framework to build out the best solutions for your SAP SuccessFactors instances if you’d like us work with you!


Reaping the results

Just for the sake of demonstration, I have considered executing the iFlow 4 times, once for each ‘event_type’ and its impact on the ‘final process state’ of the EMMonitoredProcess/execution as mentioned below.

Use Case Scenariofinal Process State
Use Case A:
event 1- START
event 2- INFO
event 3- END
COMPLETED_SUCCESSFULLY
Use Case B:
event 1- START
event 2- ERROR
event 3- END
COMPLETED_WITH_ERRORS
Use Case C:
event 1- START
event 2- WARNING
event 3- END
COMPLETED_WITH_WARNINGS
Use Case D:
event 1- START
event 2- INFO
event 3- FAILED
FAILED

Table 1: EMEvent use case scenarios and their impact on final process state

And this is how the scenarios look like when executed one after the other:

Image 12

Fig.7: Execution Manager Dashboard after logging the events


But how do I debug this? How do I trace a Process/execution back to the message in CPI?

If you scroll back up to Fig.3 and look at the EMMonitoredProcess payload, you will notice an attribute called ‘processInstanceId’. Process Instance ID as the name suggests, is to uniquely identify a single execution/process which clubs together several events under it.
But don’t we have something similar in SAP CPI to identify a particular execution of an artifact/iFlow?

As mentioned in the Developer’s Guide (2020), every message that travels across in an iFlow has a few header fields by default which has information about the iFlow/artifact they belong to. One such header field is ‘SAP_MessageProcessingLogID’ which uniquely identifies a particular message/execution of the iFlow and can be found under the Label MessageID in CPI’s ‘Monitor Message Processing’ page.

Let me show you below, how the ‘processInstanceId’ external property that we populated with the field ‘SAP_MessageProcessingLogID’ in CPI can be correlated with Execution Manager Dashboard’s ‘Process Instance ID’.

Image 13

Fig.8: Mapping an EMMonitoredProcess to an execution in SAP CPI


Conclusion

With everything in place, I hope you can see how this is an effective and relatively simple way to standardize the monitoring of integration flows in SAP CPI for an SAP SuccessFactors admin. It does this in a way that collates all the integrations associated with your SAP SuccessFactors instance in a single place to enable the developers and admins to monitor everything from a single dashboard.

This not only helps the developers but the business users who need to check the overall performance of the Integrations without having to access SAP CPI and learn how to navigate that platform as well as SAP SuccessFactors.


That’s all Folks!

I hope that you enjoyed these posts. I’d like to thank my colleagues and their collective efforts for helping me pull these together. It’s great to work with a team that continues to innovate, extend and make SuccessFactors integration as easy to use as possible for all our customers.

Thanks for your time!


References

Developer’s Guide, 2020. Developer’s Guide: Managing Integration Content. [Online]
Available at: help.sap.com
[Accessed 11-08-2020].

Written by Gurdev Singh · Categorized: Blog, SAP Development, SuccessFactors · Tagged: Execution Manager Dashboard, SAP Cloud Platform Integration, SAP Cloud Platform Integration for Process Services, SAP SuccessFactors

Feb 14 2020

SAP CPI and Successfactors Monitoring EP01: The one with the Execution Manager Dashboard

This blog post originally appeared in the SAP Community on 11/02/2020.

Note: This post is specifically targeted to my fellow SAP CPI developers who design SuccessFactors integration solutions and need a one-stop-shop for monitoring iflows (for developers/admins/managers) without having to go through SAP CPI logs and traces*


Background

Being fairly new to the SAP CPI space, I usually am a bit more careful just to ensure our clients get not only the solutions they want but the solutions they need (don’t we all?). In which case it would be appropriate to mention how exasperating it has been on my end to provide them with an integration solution which shows them what they really need to see (the logs – appropriate success, failure and exception messages).

There have been a few amazing blog posts earlier on how to effectively log messages, like the one here by Sven Müller where he harnesses the power of groovy scripts to append messages to a log file during runtime and then adding the message as an attachment once an iflow finishes the job. I found it to be the most reliable and easiest way an integration developer could trace what exactly happened for iflows that could execute for as long as 10 minutes (the trace functionality stops tracing data after first 10 minutes) or more.

On one hand, where I found this approach to be the most suitable for me in most of the cases, I always wondered if we could provide our clients a one-stop-shop for monitoring their standard SuccessFactors integrations along with custom SAP CPI based integrations without even feeling the need to dive deep into the dark side of SAP CPI, the ‘Monitor Message Processing’ page.

While working on a project for one of our clients, I was suggested by my mentor to look into the Execution Manager framework and figure out if we could go that extra mile to give our customers what they exactly need to monitor the iflow execution, without having to go into SAP CPI to figure out where things went wrong.

Therefore, in this article I would first like to share my findings on Execution Manager and if it’s possible to harness its superpowers in SAP SuccessFactors to further be able to use it in our SAP CPI iflows and make monitoring with SuccessFactors a lot simpler and tidier.


Execution Manager and Custom Integration

The fact that SuccessFactors welcomes custom integrations with wide-open arms, opens a whole new world of possibilities for monitoring the behaviour of our custom solutions from within SuccessFactors instances itself. The Execution Manager in SuccessFactors is the one-stop-shop which provides a dashboard to give you an overview of all the custom and standard integration processes communicating with your SuccessFactors instance.

To navigate to Execution Manager Dashboard, simply type ‘Execution Manager’ in the search box on your SuccessFactors HomePage –> select ‘Execution Manager Dashboard‘ from the options list and you’ll be welcomed by a page similar to one shown below.

SuccessFactors Execution Manager Dashboard

Fig.1 SuccessFactors Execution Manager Dashboard

  1. The navigation panel on the left-hand side allows you to choose from standard (Integration Center), scheduled and custom integrations (Middleware Integrations).
  2. The main section of the dashboard shows all the ‘Processes’ being/already executed with their Process States, Time Stamps and other details related to the processes.

The beauty of the Execution Manager is that SuccessFactors provides OData services for writing (logging) into the Execution Manager from an external middleware and takes care of how to display the data **bliss**.

Talking further about these processes, it is important to be able to log multiple messages during its execution. Therefore, if you click on the arrow displayed at the right-hand side of each process, it further leads you to a more detailed view of the logs and activities (called ‘Events’) that have occurred during the execution, as shown below.

 SuccessFactors - Events Logged in a Process

Fig.2 Events Logged in a Process


The relation between Events (EMEvent) and Processes (EMMonitoredProcess)

Each message logged as a part of a process is actually a ‘POST’ call to an entity called EMEvent (Entity Manager Event). Several such EMEvents carry a reference to a process (a single execution of a particular process) which resides in SuccessFactors as an instance of EMMonitoredProcess entity.

To create an EMMonitoredProcess, you need to create an EMEvent payload with the details of the EMMonitoredProcess you wish to create. Once a process is created, you can log several EMEvents for that process by including the details to the process that you created. For every EMEvent you create, SuccessFactors would group it within the process you’ve pointed to in the payload (further down the line in this series of articles, each such process would be created by an iflow execution in SAP CPI).

To make it easier, here’s an Entity Relationship Diagram for your reference. 

2020 02 13 14 51 32

Fig.3 EMMonitoredProcess and related entities

Furthermore, for giving users more information about a particular message/event you can create EMEventAttribute(s) for every EMEvent.


Trying it out!

Here’s a sample payload to create an instance of an an EMMonitoredProcess, which has an Event (message) logged with an attribute.

{
“eventName”: “Begin iflow”,
“eventType”: “START”,
“eventDescription”: “My iflow just started”,
“eventTime”: “/Date(1581046477000+0000)/”,
“eventAttributes”: {
“__metadata”:{
“uri”: “EMEventAttribute”
},
“name”:”details”,
“value”:”log a message and provide details”
},
“process”:{
“processDefinitionId”: “DiscoveryConsultingDemo”,
“processInstanceId”: “6ggfd9C6dbcf47648eefe633487af282”,
“processType”: “INTEGRATION”,
“processDefinitionName”: “DemoA”,
“processInstanceName”: “ProcessName”
}
}

Now let’s try creating a process and add an event to it using postman first.

SuccesFactors - Events Logged in a Process

Fig.4 Events being logged in a Process

Great Success!

Now let’s check out what our Execution Manager Dashboard looks like:

SucceesFcators - Logged events for a process

     Fig.5 Logged events for a process

A few things to take care of:

  • Make sure your ProcessInstanceId remains consistent for a group of EMEvents you wish to log under a single EMMonitoredProcess.
  • The final value of the Process State is determined when event is either of type START, END or FAILED (see table below for different possible values of EMEvent and the corresponding Process State value).
Image 20

    Table 1: EMEvent use-cases


Conclusion

Now, after trying out the Execution Manager framework it is quite certain that logging from SAP CPI is just a matter of making a few calls from within your iflow to Execution Manager and you can log all you want for the developer (like yourself), administrators and managers, all at one place.

Further Steps

If this intrigued you and you wonder how can this be implemented within SAP CPI, checkout the 2nd episode of this series where I have shared my approach of using this framework along with Exception Sub-processes to ensure the messages from SAP CPI are logged in the Execution Manager without making the iFlows messy and scary to debug

Written by Gurdev Singh · Categorized: Blog, SAP Development, SuccessFactors · Tagged: Execution Manager Dashboard, SAP Cloud Platform Integration, SAP Cloud Platform Integration for Process Services, SAP SuccessFactors

Find out more

Get in touch with the team at Discovery Consulting today:

e: information@discoveryconsulting.com.au
w: discoveryconsulting.com.au
p: +61 1300 664 903

1. HCA Mag, Four in five employees don’t feel heard, [online], https://www.hcamag.com/au/specialisation/hr-technology/four-in-five-employees-dont-feel-heard-heres-why/259501
2. ELMO Software, Whitepaper: 8 Benefits of HR Process Automation [online] Whitepaper: 8 Benefits of HR Process Automation - ELMO Software AU
3. Microsoft, The Next Great Disruption is Hybrid Work – Are we Ready? [online], https://www.microsoft.com/en-us/worklab/work-trend-index/hybrid-work

©2021 · Discovery Consulting

Website Design & Website SEO by Duelling Pixels