Real-time CDP data from Lytics to Oracle Responsys
Do you also want to categorize your customers based on a broad range of characteristics,
to customize their experience through personalized emails, special banners and pop-ups on your website?
In today's marketing efforts it is vital to understand your customers behavior.
You need to be where they are going and where they have been.
One of the biggest problems in modern marketing is connecting the dots when the customers have so many channels where they can interact with your content.
This can be a pain for siloed organization and result in ineffective marketing, due to separation of the different channels.
On your journey toward real-time personalized marketing, you need the ability to scale and connect new engagements across different channel devices.
All of this can be achieved by combining Lytics and Oracle Responsys tools.
What is CDP?
Organizing and using customer data can be difficult on your own, however, you don't want to lose out on the benefits of joining data from multiple sources and creating optimized groups of customers.
A customer data platform (CDP) can help with this through creating audiences in real-time and making sure they see what fits them.
CDP is not an alternative to traditional cloud-based data warehouses, instead they should be seen as a supplement that can help connect different data sources through queries and provide real-time behavioral insights.
Lytics explained
Lytics is a platform which has all the benefits of CDP, together with tools that utilizes machine learning to give insights on customer's needs, and helps communicate these insights across multiple channels.
Lytics use tools from data science to help marketers give their customers a personalized experience and create precise targeted campaigns.
By using Lytics you can input large amount of customer data, track the customers based on their behavior and output email target groups for different campaigns in real-time, all by using Lytics.
Responsys explained
Oracle Responsys provides tools for creating personalized marketing at enterprise scale. The oracle Responsys Campaign Management works as a cross-channel platform which allows you to create
personalized messages due to their advanced intelligence.
You can create personalized and relevant messages, based on customer's interests and preferences, and build digital marketing campaigns
for all customers and prospects. With a wide and comprehensive REST API, its also possible to apply external real-time events from other tools, like Lytics CDP.
You can read more about Responsys here.
Building a real-time connection
The goal is to make email campaigns that are personalized and makes the users engage.
To do this Lytics send real-time audience membership updates to Responsys, such that it can be used for making personalized campaigns.
All of this is done using AWS tools, Lytics own REST API, Oracle Responsys and webhooks.
Throughout the whole pipeline everything is reported to ElasticSearch, such that the process is monitored and if an error occurs, the error can easily be discovered and fixed.
Building a real-time webhook
AWS lambda functions are used to connect Lytics and Responsys. The activity in the audiences is tracked via webhooks with an endpoint in AWS. Then the data can be transferred to Responsys,
such that when a person enters or exits an Lytics audience, they will also enter or exit in Responsys. The audiences that need to be transferred to Responsys is distinguished from the rest,
because their audience-slug starts with 'email'.
In order to do all of this, three lambda functions are set up. One creating and deleting webhooks for every audience starting with the keyword 'email',
and two functions that in combination can receive information from the webhooks and send it to Responsys.
Setup webhook in Lytics
In Lytics it is possible to connect webhooks to audiences in order to monitor which users are currently in the audience and which ones have left.
These webhooks can also be set to include backfill, which means that when the webhook connects it starts with sending all the current users in the audience to the end-point,
before tracking exits and enters. These webhooks can be set up manually on the Lytics platform or they can be added through the REST API.
| Lytics API --> get webhooks | | GET https://api.lytics.io/api/subscription |
|---|---|
| Lytics API --> get audiences | | GET https://api.lytics.io/api/segment |
Building a real-time webhook
In order to automate the process of adding and deleting webhooks an AWS lambda function can be set up. It uses API calls to lytics to get a list of all webhooks and all audiences. Then it checks if all the targeted audiences have a webhook, if not it adds the missing webhooks automatically. The lambda function also automatically checks for obsolete webhooks and removes them. In order to make the lambda check the webhooks regularly, an AWS event bridge can be added. This allows for setting up a trigger rule that makes the lambda function run every 5 minutes, checking for webhooks that are obsolete or needs to be added.
Create AWS endpoint
The second lambda function receives data from the webhooks through an API Gateway endpoint. It then sends the data to AWS SQS. This service is used because SQS can tolerate large amounts of messages being transferred. The queued messages wait to be pulled by Responsys and therefore minimize the workload and does not overload Responsys, thereby avoiding backpressure.
Connecting to Responsys
The third lambda function tackles all the messages, it reads batches of maximum 100 messages at a time from SQS. It then inserts the messages directly into a table in Responsys via their REST API. Based on the environment (prod or dev), the data is inserted into different tables giving the opportunity to test without disturbing the send-out data.
Setup Responsys program
The email program in Responsys is simply connected to an event, which is activated by the webhook. It enters customers into the email flow as soon as they meet the segmentation criteria.
Last thoughts
We have benefited greatly from integrating CDP tools in email marketing, it has made sending campaign emails much easier,
especially because an automated pipeline is made to transfer the real-time data.
Oracle Responsys is a great platform for customer-based marketing, however, a tool such as Lytics is needed to get real-time campaign audiences.
Lytics lets you combine multiple data streams transferring data in real time, which can easily be used in many cases of personalized marketing.
In this pipeline, Lytics is used to create audiences that can send the updates to Responsys in real time.
To manage this transfer of data, AWS lambda functions are used to move the data. AWS SQS is added to the pipeline to handle the large amount of data and avoid backpressure.
If you want to learn more about Lytics audiences, you can get more information here.
