Recognizing and Building the Optimal Customer Experience

Over the past several years, more and more teams in Enterprise Data Management have been tasked with a succinct need from the C-Suite:We need to clean up our data and utilize it as much as possible to ensure better business outcomes.On the surface this seems like a no-brainer, but leads to two questions:Why is this so important all of the sudden?How do we go about this?What follows is an attempt to answer the industry trends driving this demand, and provide a high level overview of the concepts and best practices involved.

Why now?

Regardless of industry, the dream of any enterprise collecting data around their engagements with customers and prospects has been one thing:To set our business apart, I want to understand every way in which we engage with customers/prospects, and use that information to give them the best experience possible. As a consumer this seems rather intuitive. I’m most loyal to brands that give me the most value with their products or services, and provide the least friction in the process. Airlines make the travel process easiest (and the most enjoyable) for their most loyal customers. Retailers offer economic benefits or exclusive products/services if they feel I’ll be a repeat buyer, or in order to get me in the door for the first time.But from a business perspective, delivering the best experience to customers has only gotten more and more difficult. Sure, if I grab coffee at the same local cafe most mornings, the barista may get my order started before I reach the counter, throw in a pastry for free, or even have a more cordial conversation with me than less frequent visitors. It’s relatively easy for them to make my mornings pleasant for one key reason: they recognize me and likely only interact with me in the one medium where they are familiar with me – in-store. Personally I think this is one of several reasons why most people prefer spending their money at local businesses.Comparing this to a global enterprise, the variables in customer experience expand drastically. A global retailer interacts with exponentially more customers, via far more employees, and across a multitude of mediums, with all of this producing a lot more data against these interactions. This makes personalization all the more difficult.Today’s technology landscape presents a massive opportunity to engage with customers like never before. Studies by BCG and Google in 2023 show that companies provide anywhere from 20 to 500 customer interactions in the buying journey, a drastic increase from an average of nine in 2014*. With that opportunity comes an even bigger challenge: using data to make sure it is done right. The data behind these interactions comes from a countless number of sources, very few of which tell you the story in a consistent manner. Through innovations in cloud computing and software applications, and with the evolving practice of enterprise data management, many companies have taken the first step towards optimizing customer experience, which is getting all available data centralized in one place. The bigger step is what lies ahead: making those sources of data effectively talk to each other, and using what you hear to make that next interaction as relevant and meaningful as possible. Before we tackle how this can be done, let’s take time to understand the two primary channel categories by which companies recognize and interact with consumers and how the nuances of how that data is captured in each channel.

Marketing and customer success: Owned and operated channel engagements

The majority of effective customer interactions most often lie within a company’s own properties, both physical and digital. Marketing and customer service within your owned channels are arguably the best places to cater the customer experience. If I host a party, there’s a lot more control and ability to cater the experience in my own home than if I chose an external location.Owned channels have evolved drastically over the past decade. Through evolving technologies, this has moved from a handful of mediums such as:

• Brick and Mortar locations (i.e. in-person)
• Website
• Call Center/Customer Service
• Email
• Direct Mail
• SMS
• In-App

To a much larger swath of channels, including:

• Mobile website/applications
• Chatbots
• In-store digital experiences
• IoT Applications
• Increased and more personalized Newsletter Programs
• QR Codes
• Microsites and ancillary digital properties

All of these channels provide some sort of data back to enterprises on how the interaction went, but as mentioned above those data sets all come in their own unique flavors (ex. event logs, website/app analytics, customer satisfaction surveys, etc.). Gaining a comprehensive view of all interactions with a single customer through these sources, often referred to as “Customer 360,” is a key objective of most enterprises today. In 2022, 82% of respondents in a Gartner survey listed this as a primary goal for their business, however only 14% of respondents felt they’d actually achieved this full view. A proliferation of technology solutions have emerged to solve this very problem (more on this later), but challenges still exist in effectively recognizing the right customers across these channels.

Advertising: Engaging with customers and prospects outside of your four walls

The second category of interaction channels is where things can get complex. Advertising continues to evolve at a rapid pace, primarily due to the boom of digital channels that have emerged and quickly dominated consumer share of attention over recent years. A media or advertising plan in the early 2000’s would likely contain a mix of channels such as:

• Out of Home
• Print (Newspaper and Magazine)
• TV
• Digital

But with the rise of smartphones and internet connected devices, the media mix has come to include far more categories such as:

• Premium Digital (ad buys directly with household name web publishers)
• Programmatic (wide-spread digital buys across any and all relevant web inventory)
• Mobile web and Mobile Apps
• Connected Television

The rise of digital inventory came with a variety of solutions to ensure messages to customers were relevant and encouraged another interaction/purchase with your company. Tools like retargeting pixels and Data Management Platforms (DMP) enabled businesses to track and categorize their digital interactions with customers, and re-engage them with the most relevant offerings moving forward. Digital also opened a door for other parties to provide businesses information on consumer interests and behaviors outside of what they may have gained in their interactions, primarily through third-party data offerings, which prompted more consumer privacy regulations.

Recognition and digital privacy

Privacy legislation is not a new concept, with even the US Constitution alluding to the right to privacy within the First, Third, Fourth, and Fifth Amendments. More recent history has brought legislation focused on individual rights as it relates to healthcare (HIPPA in 1996), minors (COPPA in 1998), and financial data (Gramm Leach Bliley in 1999). The primary focus of current legislation is to inform consumers of their privacy rights in the digital space, namely the General Data Protection Regulation (GDPR) which went into effect in the EU in 2018, as well as various state-specific privacy laws in the US, starting with the California Consumer Privacy Act (CCPA) in 2020.Third-party cookies have historically been fuel for customer insights and recognition in the digital space; but today half of the open web is cookieless and companies are driving better business results without them. For example, hospitality leader Omni Hotels & Resorts saw four times more advertising effectiveness by leveraging signal-less solutions from Google’s Display & Video 360 and LiveRamp to build deeper relationships with their customers.

Recognition in a new era

As channels for interactions have increased, and data collection practices from these channels have shifted, new categories of solutions have risen to the challenge. What follows is an overview of these solutions and thoughts on best practices.

Identity and entity resolution

The process of “cleaning up data” and building a “Customer 360” view can be best summed up with the concepts of identity and entity resolution. These two concepts are often confused, and I find it best to think of them in a square-rectangle fashion. More specifically:Entity resolution (rectangle) is the overarching concept in the practice of data management, whereby a business attempts to unify all of their disparate records (ie customer interactions), within a single channel or across multiple, and tie them to a single entity. An “entity” could mean many things, such as a business/account, household, or other depending on the business’ objectives. Entity resolution can help a business answer questions such as:

• Where do we see multiple customers in the same household? Should we adjust our messaging based on this, or limit our engagements to the key buyer in the house to avoid waste/over inundation?
• For B2B businesses: What accounts are most or least engaged, and how can we cater our messages to fit the specific needs of those companies?

Identity resolution (square) is the specific type of entity resolution by which a business attempts to tie disparate records to a single individual or household.The latter is the most common (and difficult) goal in enterprise data management today. In an ideal state, this is how I could recognize one customer across all channels, and thus have the most relevant interactions possible. If I work in a call center and my company is practicing identity resolution, they could look up the phone number in an inbound call against our database and provide me with a profile of everything we know about that person before I even pick up the phone. That way I may know the specific order they’re checking in on, or the product they’ve purchased and are having an issue with, without the customer having to tell me.If I work in marketing and my company practices identity resolution, I could customize the email I send customers to exclude products they already have, or offer promotions/discounts to ones they’ve specifically shown interest in in the past.If I work in business intelligence, identity resolution could help me better analyze and showcase the optimal customer journey, meaning what number of customer interactions, across which channels, and in what order produce the most sales or conversions for my company.Both identity and entity resolution open up a wealth of opportunities for businesses, agnostic of business function or department. The underlying processes can be incredibly complex and should very much be customized to a businesses’ needs, but the desired outcomes are relatively consistent: a single profile of an individual or entity, which encapsulates all engagements they’ve had with your business, tied to a single ID (Customer/Account ID) in your database.

Resolution tools

To accomplish a full view of the customer through identity resolution practices, three primary categories of tools have emerged in the market.

Customer data platforms (CDPs)

CDPs come in a variety of shapes and sizes, but their providers promote a few primary features:

• Data unification and profile creation: This is where entity and identity resolution play a key role. CDPs offer features that give users the ability to bring in all relevant datasets, build relationships between them, and use matching rules to create unified profiles consisting of all interactions with a single customer or overarching entity.
• Profile segmentation, analysis, and journey orchestration: Following the above work, CDPs then provide users with tools which allow them to analyze those profiles to determine where the best interactions occur, and how a business can drive more of them to put as many customers in the “optimal journey” with their business as possible. They also offer the ability to bucket or segment users based on similar behaviors/interactions.

• Actioning on insights: Lastly, CDPs provide users with the ability to take these profiles or “Golden Records” and the associated insights against them, and actually take action on your learnings. This most often manifests as built in messaging tools or integrations to external partners across the marketing (ex email/SMS platforms) and advertising (ex social, programmatic, CTV) ecosystems.

Standalone identity resolution/graph builder tools

Outside of CDPs, there also exist many providers who offer solutions needed to build unified profiles as a standalone. These solutions can be thought of as a “rules engine”, whereby users can map in all of the relevant data sets for their business into a single environment, then build rules to unify and/or dedupe profiles based on what common keys sit within each dataset. These “solutions” may come as a simple UI or API that can sit atop a business’ own data warehouse or data infrastructure, or as a more managed service offering where a provider brings along a team of experts to help build to unified profiles (either in the provider’s environment or your own).

Enrichment beyond first-party data

While collecting and unifying all of your interactions with customers into a single view is the ideal state for most businesses today, there exists within this process two common limitations often faced.

Issue #1: Inability to unify data sets

Entity and identity resolution tools rely on one critical piece in the process of unifying data sets: A common key that can be used to link the data. This is done via strict (exact) matches across the keys, as well as fuzzy matching which uses a more probabilistic method (ex. A record with “Benjamin Healy” could use a fuzzy match to link to another record with “Ben Healy”). The big question here is: what happens if there’s no common key between data sets, or I’m not confident in a more probabilistic approach? Let’s use an example to illustrate:Company A is an e-commerce retailer. They’re using an identity resolution tool to clean up their data across sources, namely their shipping/transactions and registered users.One record from their shipping/transactions includes the below information from a 2020 purchase: FirstName Last Name Address Email Product Date Benjamin Healy 123 Main St. ben@yahoo.com Shoes March 1, 2020 Another record from their registration database includes the below information from a recent signup:First Name Last Name Email Date Ben Healy ben@gmail.com February 3rd, 2024Company A wants to tie any purchases a customer made prior to signing up to their profile, but in the above example there’s no real common key across both records. Sure, they could use fuzzy matching to tie the name fields together, but is that the best approach? After all, there are a lot of Ben Healy’s out there (I frequently trade emails with one who has a similar Gmail address).

Referential graphs: creating a bridge between interactions

Without a common key across your data sets/customer interactions, a business might throw their hands up in defeat during an identity resolution project, realizing they haven’t requested enough information from customers in those interactions to tie them to a single profile. Luckily, the market has recognized this and there now exists solutions to help build this bridge, namely referential graphs. At a high level, a referential graph is a third-party or public graph of consumer information that aids in the process of unifying data by providing linkages outside of what you’ve collected in your interactions as a business. Many entity and identity resolution providers offer referential graphs within their toolsets, such as Acxiom, Merkle, and LiveRamp. Revisiting the above example:Company A appends signals from a referential graph to the same records for further validation First Name Last Name Address Email Product Dateref_id Benjamin Healy 123 Main St. ben@yahoo.com Shoes March 1, 2020 abc123 First Name Last Name Email Dateref_id Ben Healy ben@gmail.com February 3rd, 2024 abc123With a reference graph’s ID appended, Company A now knows that by that provider’s standards, they deem Ben and Benjamin the same person. They can now choose to unify these records based on the common key and the provider’s graph logic.At surface level this may seem like an edge case, but in reality a reference graph can become more and more critical as businesses utilize more channels for customer interactions, and as they continually interact with customers over long periods of time. We all present ourselves differently depending on the context. I may use my full name for more “official” scenarios and my nickname for more casual interactions. Consumer behaviors also evolve over time. I may move to another apartment or change which emails I use most often as time passes. Referential graphs help businesses continue to effectively recognize customers over time, and consciously monitor those shifts in behavior to enrich their profile of a customer and eliminate waste in their business efforts (ex. Sending a mailer to my old address; emailing a personalized offer to an inbox I rarely check anymore).

Issue #2: Lack of first-party data to build a better experience

I often recognize my neighbors around New York, but without a meaningful interaction between us those moments likely won’t go past a simple hello or awkward half-smile. I could certainly stop them on the street and ask them 21 questions, but if you know anything about New York culture they’d probably only remember me as the overly-friendly guy or the one who made them late to work. Oftentimes it takes a shared contact to break the ice in these situations or learn more about them. My wife may have told me about a brief conversation with them in our communal laundry room, or my building super mentioned they’re renovating their kitchen. Small insights like this can enable a more meaningful interaction in the future, without someone feeling like their privacy has been breached or as if I’m conducting a survey to find common ground. This same concept applies in business and customer interactions. At the end of the day we’re all interacting with businesses for a specific purpose, and anything outside of this can feel like oversharing or simply a waste of time. We’ve all probably been on a customer service call where the agent may be a little too chatty, and it feels like the conversation is getting in the way of the answer we came for. While a referential graph helps to better unify and consolidate interactions, if the content of those interactions is limited, a unified profile doesn’t provide you much additional value. Referential graphs are often non-discoverable, meaning a provider will not simply give you any and all data within these graphs for you to incorporate into your own view of the customer. They supply the common key to help make your CRM or data warehouse shorter (ie less disparate records/interactions). The crux of the issue at hand is how a business can make their CRM/DWH wider, meaning how do I gain more insights or data on an individual to more effectively reach them for interactions, while also making those interactions more relevant or personalized? The following solutions aim to solve this issue.

Enrichment with contact and attribute data

Separate, but often related to referential graphs there is also the concept of profile enrichment. Contact enrichment providers offer more PII touchpoints which can be applied to your profiles. Let’s say my company does not often require emails to be provided in customer interactions, but we want to start doing more email marketing to promote personalized offers. Aside from changing how we interact with customers to require they give us their email, we could work with an enrichment provider to connect emails in their data sources to our existing profiles. Attribute enrichment providers offer more details on consumer demographics, interests, or behaviors to be applied to your profiles. A business may feel they have a comprehensive view of their interactions with a single customer, however this is far from a full view of that person. Let’s say my company sells moderately-priced clothing, but wants to branch into more luxury/high-end apparel as well. By working with an enrichment provider, my company could learn more about our existing customers’ outside of how we’ve interacted with them to date. A provider may be able to tell me more about their financial status (ex: credit score or household income), or perhaps their interest/purchase behaviors with other brands in the luxury space (ex: Often visits luxury boutiques or frequent buyers in the luxury apparel category). Once appended to my existing profiles, I can use these additional attributes to determine which customers might be best to introduce to this new offering as we test the waters.When evaluating enrichment providers it is important to understand how they source the data they offer, and more specifically how they have permissible rights to offer this data to their partners and ensure end consumers have the ability to opt out of these databases as needed.

Second-party data and data clean rooms

In the past few years, marketers have seen the overwhelming benefits of data collaboration. I feel this has surfaced as a byproduct of a series of tailwinds and headwinds:Headwinds

• Unified customer views, with limited profile content: Businesses are using EDM practices and technology solutions to build a better view of customer interactions. Along this road they’re wondering “how do I get more information on consumers outside of what we gain from our interactions?” Rather than working with an enrichment partner, they’re looking for ways to work with new or existing business partners to learn more about their customers. For example, an airline may want to expand their partnership with a credit card provider for better co-marketing efforts. A high-end automotive manufacturer may want to work with a luxury retailer to learn more about mutual customers, using real data to go beyond their anecdotal assumption that their brands have strong synergies. The possibilities are endless.
• New revenue sources through data as a service (DaaS): Many businesses are realizing in their EDM journeys that they have a wealth of data, and are exploring ways they can offer the data itself as a product/service to their partners. A grocer may want to offer a CPG brand deeper insights into what kinds of purchases are made in their stores. A movie theater may want to offer production studios deeper insights around ticket purchases. The CPG/retail industries were the first to realize the potential value data sharing could unlock, and thus a proliferation of retail media networks have surfaced. However many other industries are realizing they can utilize a similar model of data sharing to unlock mutual value within their networks.

Tailwinds

• Signal loss in advertising: The reality is an information gap exists when it comes to prospecting or gaining new customers. This is where second-party relationships will play a key role in data collaboration. Rather than simply selecting an off-the-shelf audience of third-party data, businesses are taking a more concentrated approach by exploring with other businesses how they can securely share data on potential prospects. If two companies feel they have synergies across their brands, second-party d