The 12 Days of Shopping

According to IBM’s real-time reporting unit, Black Friday online sales were up close to 20% this year over the same period in 2012.  As for Cyber Monday, sales increased 30.3% in 2012 compared to the previous year and is expected to grow another 15% in 2013. Mobile transactions are at an all time high and combined with in store sales, The National Retail Federation expects retail sales to pass the $600 billion mark during the last two months of the year alone. While that might sound like music to a retailer’s ears, as the holiday shopping season goes into full swing on this Cyber Monday, the pressure to handle the astronomical influx of data collected at dozens of possible transaction points is mounting. From websites and storefronts to kiosks and catalogues, every scarf or video game purchased this season brings with it a variety of data points that must be appropriately stored, linked, referenced and hopefully leveraged. Add to that a blinding amount of big data now being collected (such as social media activity or mobile tracking), and it all amounts to a holiday nightmare for the IT and data analysis teams. So how much data are we talking and how does it actually manifest itself? In the spirit of keeping things light, we offer you, The 12 Days of Shopping…

On the first day of shopping my data gave to me,
1 million duplicate names.

On the second day of shopping my data gave to me,
2 million transactions, and
1 million duplicate names.

On the third day of shopping my data gave to me,
30,000 credit apps,
2 million transactions, and
1 million duplicate names.

On the fourth day of shopping my data gave to me,
40 returned shipments,
30,000 credit apps,
2 million transactions, and
1 million duplicate names.

On the fifth day of shopping my data gave to me,
5 new marketing lists,
40 returned shipments,
30,000 credit apps,
2 million transactions, and
1 million duplicate names.

On the sixth day of shopping my data gave to me,
6,000 bad addresses,
5 new marketing lists,
40 returned shipments,
30,000 credit apps,
2 million transactions, and
1 million duplicate names.

On the seventh day of shopping my data gave to me,
7,000 refunds,
6,000 bad addresses,
5 new marketing lists,
40 returned shipments,
30,000 credit apps,
2 million transactions, and
1 million duplicate names.

On the eighth day of shopping my data gave to me,
8,000 new logins,
7,000 refunds,
6,000 bad addresses,
5 new marketing lists,
40 returned shipments,
30,000 credit apps,
2 million transactions, and
1 million duplicate names.

On the ninth day of shopping my data gave to me,
90,000 emails,
8,000 new logins,
7,000 refunds,
6,000 bad addresses,
5 new marketing lists,
40 returned shipments,
30,000 credit apps,
2 million transactions, and
1 million duplicate names.

On the tenth day of shopping my data gave to me,
10,000 tweets,
90,000 emails,
8,000 new logins,
7,000 refunds,
6,000 bad addresses,
5 new marketing lists,
40 returned shipments,
30,000 credit apps,
2 million transactions, and
1 million duplicate names.

On the eleventh day of shopping my data gave to me,
11 new campaigns,
10,000 tweets,
90,000 emails,
8,000 new logins,
7,000 refunds,
6,000 bad addresses,
5 new marketing lists,
40 returned shipments,
30,000 credit apps,
2 million transactions, and
1 million duplicate names.

On the twelfth day of shopping my data gave to me,
12 fraud alerts,
11 new campaigns,
10,000 tweets,
90,000 emails,
8,000 new logins,
7,000 refunds,
6,000 bad addresses,
5 new marketing lists,
40 returned shipments,
30,000 credit apps,
2 million transactions, and
1 million duplicate names.

While we joke about the enormity of it all, if you are a retailer stumbling under the weight of all this data, there is hope and over the next few weeks we’ll dive a bit deeper into these figures to showcase how you can get control of the incoming data and most importantly, leverage it in a meaningful way.

Sources:
http://techcrunch.com/2013/11/29/black-friday-online-sales-up-7-percent-mobile-is-37-percent-of-all-traffic-and-21-5-percent-of-all-purchases/

http://www.pfsweb.com/blog/cyber-monday-2012-the-results/

http://www.foxnews.com/us/2013/11/29/retailers-usher-in-holiday-shopping-season-as-black-friday-morphs-into/

When Direct Marketing Fails…

As a Direct Marketer in a tough economy, you are constantly being asked to do more with less.
Generate more leads. Improve campaign results. Increase brand awareness. Streamline operational efficiency. Unfortunately, poor quality customer and prospect data can sabotage your direct marketing efforts and your reputation in the process.

Want to get a handle on the ways that data could be tripping you up? Here’s a quick look at the top 5 Ways Bad Data is Hurting Your Direct Marketing Efforts AND how you can start to turn that trend around…

1. BAD DATA Wastes Money

In this most obvious example, companies with large, unkempt databases ‘pay’ for bad data, directly and indirectly. How much? Duplicate records mean duplicate print costs and double the postage but half the potential return while invalid or outdated addresses and garbage data is like throwing money out the window. Even in the digital world, bigger databases cost more money. Email vendors, marketing automation vendors and database marketing companies typically charge by volume of records – so larger, unclean databases will cost you more regardless of your marketing strategies.

2. BAD DATA Reduces Response Rate

While bad data has the potential to waste significant dollars per campaign, the flip side of that coin means your campaign response rates are compromised. In other words, when you send that direct mail campaign out, even if you get tons of responses – they are always divided by the ‘potential’ of your entire database (response/total sent). As a direct marketer (whether you use email, direct mail or telemarketing) the only way to demonstrate the maximum return on investment is to remove duplicate records, update old addresses and eliminate anyone who shouldn’t receive your marketing (which could include unsubscribers, gone aways or deceased records).  Now that return rate is an accurate reflection of your skill as a marketer and you wasted a lot less money in getting there.

3. BAD DATA Means Bad Targeting

How well do you know your customers and prospects? How many unique customers are in your CRM? Do you know how much they spend? What they like to buy? Where they shop? If you don’t – it means your direct marketing initiatives are being held back by a lack of information.  The remedy? Put all your customer data in one place, accurately entered and complete. This might mean merging contact data with household income. It might mean linking all their product purchases so you can determine buying habits. It might mean knowing ALL their addresses but choosing a master file so you know how and where to target them appropriately. For retailers this is known as a Single Customer View and it allows you to simply know more about your customer so you can use it to maximize your direct marketing efforts.

4. BAD DATA Hurts Your Reputation

Companies that market well have a better reputation. This needs no data to support it – simply check your kitchen counter for the catalogs and marketing pieces you’ve ‘saved’. What do they have in common? They were all mailed accurately (they made it to your mailbox) and they promote products that interest you (as compared to that motorcycle catalogue you may have tossed). And they showed up at the right time (like that home improvement coupon that arrived just after you purchased your new house). So to protect your marketing reputation you want to suppress and organize your data so that you don’t mail to the wrong people at the wrong time with the wrong message. Remove gone aways, deceased and anyone who does not want to get your mail. Your customers will thank you in the long run by continuing to purchase from your business (and not writing nasty things about you on Twitter). In addition, your ‘green’ credentials will soar by taking the initiative to reduce wasted mail.

5. BAD DATA Risks Opportunities

Almost everyone knows that Marketing and Sales go hand in hand and it’s all related to good data. When sales reps and account people have the correct information on the screen, they can go the distance in selling and up selling to prospects and customers. They can waste less time calling people that are not relevant and spend more time focusing on the ones who are. They can spend less time on each call searching for data or recollecting it and instead, wow the customer with how organized and efficient their interaction is. Accurate and complete data improves sales and wastes far less opportunities.

Mounting evidence suggests that cleaner, more accurate data has a direct impact on your direct marketing success and each year, more and more companies make data quality a key focus in both their marketing and technology departments.  Removing duplicates, updating addresses, gone away/deceased suppression and generating that all-important single customer view will allow you to

So what’s a Direct Marketer to do? Make every effort to regularly dedupe your database or CRM to keep the size reasonable and reduce waste across all your marketing initiatives. It’s also advisable to validate addresses and suppress gone aways and deceased records before a major mail campaign to reduce the waste associated with excess mailers and postage.  To take your Direct Marketing to the next level, work with your technology or database team to establish a comprehensive Single Customer View. This will allow you to use your existing customer data to drive better and more targeted Direct Marketing campaigns. You can also supplement existing data with a wide range of additional datasets. Lastly, keep good track of customers and prospects who unsubscribe or who don’t respond to specific types of mailings so you can market to only those receptive to your offer.

If you are interested in learning about how helpIT systems data quality toolset can address some of your Direct Marketing challenges, please feel free to contact us for a Free Consultation and Trial of our matchIT data cleansing software.

The Cost of Addressing Data

In a recent article in Database Marketing, James Lawson discussed the ongoing debate of whether address data should be free. Back in November 2012, The Open Data User Group (ODUG) presented a paper that suggested an Open National Address Dataset (ONAD), built from three sources: Royal Mail’s Postcode Address File (PAF), Geoplace’s National Address Gazetteer (NAG) and OS’s AddressBase Plus. All three files would be made available under the Open Government Licence as free data, just like OS Code-Point Open.

As we all know, the Royal Mail’s PAF database is the most widely adopted reference file for UK addresses. It has been since its inception and I believe will remain so for many years to come, free or not.  In the more recent years we’ve had various rumours and promises of alternative and complimentary solutions from the LLPG & NLPG, Address  Layer 2 to the more recent NAG and subsequent AddressBase offerings but they’ve all had their challenges.  As with any accurate or usable database it takes a considerable amount of time and effort to compile and somebody needs to take this burden on.  I agree with the current concept that any small to low volume usage (as delivered by the Royal Mail’s postcode finder solution) should be free but the argument of savings and the implied benefits for public bodies is currently being debated for the recently released AddressBase solution.  The questions of industry benefit really comes from the high volume corporate users and wider DM industry.  The impact to pre-sortation requirements, which the Royal Mail enforce on their direct business clients as well as the rules they impose on the wholesale Down Stream Access partners, would no doubt be part of any discussion.  Whilst it would be Holy Grail to ensure all addresses entering the Royal Mail’s network were perfectly addressed, I think any serious conversation on the topic will always be deferred while a potential sale of the Royal Mail is in the wings.

 

 

Data Quality Makes the Best Neighbor

So this week’s #dataqualityblunder is brought to you by the insurance industry and demonstrates that data quality issues can manifest themselves in a variety of ways and have unexpected impacts on the business entity.

Case in point – State Farm. Big company. Tons of agents. Working hard at a new, bold advertising campaign. It’s kind of common knowledge that they have regional agents (you see the billboards throughout the NY Tri-State area) and it’s common to get repeated promotional materials from your regional agent.

But, what happens when agents start competing for the same territory? That appears to be the situation for a recent set of mailings I received. On the same day, I got the same letter from two different agents in neighboring regions.

Same offer. Same address. So, who do I call? And how long will it take for me to get annoyed by getting two sets of the same marketing material? Although it may be obvious, there are a few impacts from this kind of blunder:

  • First of all – wasted dollars. Not sure who foots the bills here – State Farm or the agents themselves, but either way, someone is spending more money than they need to.
  • Brand equity suffers. When one local agent promotes themselves to me, I get a warm fuzzy feeling that he is somehow reaching out to his ‘neighbor’. He lives in this community and will understand my concerns and needs. This is his livelihood and it matters to him. But, when I get the same exact mailing from two agents in different offices, I realize there is a machine behind this initiative. Warm feelings gone and the brand State Farm has worked so hard to develop, loses its luster.
  • Painful inefficiency.  I am just one person that got stuck on two mailing lists. How many more are there? And how much more successful would each agent be if they focused their time, money and energy on a unique territory, instead of overlapping ones.

There are lots of lessons in this one and there are a variety of possible reasons for this kind of blunder.  A quick call to one of the agents and I learned that most of the lists come from the parent organization but some agents do supplement with additional lists but they assured me, this kind of overlap was not expected or planned. That means there is a step (or tool) in the process that is missing. It could require a change in business rules for agent marketing. It’s possible they have the rules in place but requires greater enforcement. It could just be a matter of implementing the right deduplication tools across their multiple data sources. There are plenty of ways to insure against this kind of #dataqualityblunder once the issue is highlighted and data quality becomes a priority.

The Retail Single Customer View

One of the more interesting aspects of working for a data quality company is the challenge associated with solving real world business issues through effective data management. In order to provide an end-to-end solution, several moving parts must be taken into consideration, data quality software being just one of them.

A few months back, helpIT was given the challenge of working with a multi-channel retail organization seeking a Single Customer View to improve the effectiveness of their marketing efforts. They received customer data from several channels including: Point of Sale, Website, and Call Center. Their hope was to link up every transaction over the past 5 years to a single instance of the right customer.

In a vacuum, this is a pretty straightforward job:

  1. Standardize and clean all of the data
  2. Identify the priority of transaction data sources and merge all sources together based on individual contact information. Develop a single list or table of all unique customers and assign a unique ID
  3. Take all transaction data sources and then match against the unique customers and assign the ID of the unique customer to the transaction.

With live business data it’s rarely if ever that simple. Here are some of the issues we had to circumvent:

  • Very high rate of repeat customers across multiple channels
  • Different data governance rules and standards at all points of capture
    Point of Sale – Only name and email were required. When no address was provided – store address was used. “Not Provided” also acceptable causing other incomplete data Website – Entered as preferred by customer Call Center – Typed into CRM application, no customer lookup process
  • No “My Account” section on the website, which means that all orders are treated as new customers
  • Archaic Point-of-Sale application that was too expensive to replace for this project
  • Newly created SQL Server environment that acts as a central data repository but had no starting point for unique customers

To come up with a solution that could enable the customer to develop and also maintain a Single Customer View we proposed the following workflow that could be used for both.

Website

This was immediately identified as the best source of information because the customers are entering it themselves and have the genuine desire to receive delivery or be contacted if there is an issue with the orders. The process was started with an initial batch clean up of all historical web orders as follows:

  1. Run all orders through Address Validation and National Change of Address (NCOA) to get the most up to date information on the contacts
  2. Standardize all data points using the matchIT SQL casing and parsing engine
  3. Performe contact level deduplication with matchIT SQL using combination of exact and fuzzy matching routines and included confidence scores for all matches.
  4. Our client identified the confidence thresholds that were either confident matches to commit, matches that required manual review, and unique records. They completed the manual review and incorporated some further logic based on these matches to prevent future review for the same issues. The final step in their process was to identify a future score threshold to commit transactions from other sources to the customer record.
  5. The deduplication process was finalized and a new SQL Server table was created with standardized, accurate, and unique customer data that would act as the “golden record”.
  6. All transactions from the Web history file were updated with a new column containing the unique customer ID from the “golden record” table.

Call Center

This was the next area to undertake and was essentially a repetition of the process used on the Website data with the exception of the final steps of matching the cleaned version of the Call Center data with the “golden record” table.

After performing the overlap, any unique customers from the call center were then added to the “golden record” table and assigned an ID. All the overlapping Call Center customers received the overlapped ID from the “golden record” table which was then appended to the related Call Center transactions.

Store

This was the tricky part!

Some of the data contained full and accurate customer information, but nearly 30% of the customer transaction data captured at the store level contained the store address information.

So how did we overcome this?

  1. We created a suppression table that contained only their store addresses
  2. All store transactions with the minimum information for capture (at least a name and address) were standardized and then matched against the store suppressions file yielding a list of matches (customers with store info as their address) and non matches (customers that provided their own address information)
  3. For the customers that provided their address the process then went back to the same procedure run on the call center data
  4. For the customers with store information we had to use a different set of matching logic that ignored the address information and instead looked to the other data elements like name, email, phone, credit card number and date of birth. Several matchkeys were required because of the inconsistency in what matches would be found.
  5. The client then decided for the remaining portion of customers in the Store file (3%) to put those customers in a hold table until some other piece of information popped up that would allow for a bridging of the transaction to a new transaction.

A workflow diagram of the store process can be found to the left:

The key to the whole strategy was to identify a way to isolate records with alternative matching requirements on an automated basis. Once we separated the records with store addresses we were free to develop the ideal logic for each scenario, providing an end to end solution for a very complicated but frequently occurring data issue.

If you are attempting to establish the ever-elusive single customer view, remember that there are several moving parts to the process that go well beyond the implementation of a data quality software. The solution may well require a brand new workflow to reach the desired objective.

 

For more information about establishing a Single Customer View or to sign up for a Free CRM Data Analysis, click here.