EU GDPR Headquarters

EU GDPR now just 12 months to go – where do you start?

As you should know, the EU General Data Protection Regulation (GDPR) comes into force one year from today, 25th May 2018. As we will still be in the EU then, whatever kind of Brexit we are in for, you only have 12 months to make sure that all your systems support compliance. If you need any incentive to start taking this seriously, you only have to consider that the maximum fine for breach of data protection regulations is increasing from £500,000 to €10 million or 2% of global gross revenue (whichever is higher) – that’s just for a level 1 breach, with double these amounts for a level 2 breach!

To help you on your journey to GDPR compliance, we will be publishing a series of posts about aspects of GDPR over the next few weeks. Initially, we will cover:

After that, we will look at how matchIT Data Quality Solutions can help you be compliant and avoid breaches of the new law – especially how a genuine Single Customer View helps to keep data accurate and up to date and ensures that you can respond to Subject Access Requests promptly, fully and efficiently.

The implications of GDPR are far reaching and HMG guidance is still being developed by the Department for Culture, Media and Sport, in consultation with industry bodies such as TechUK. Some companies may find that with only one year to go, they may not be able to become completely compliant by then – in which case, it is vital to mitigate potential costs of non-compliance by demonstrating effective progress, with a realistic timetable for full compliance. Look out for our next few posts to help you navigate towards that goal!

The doctor won’t see you now – NHS data needs a health check!

On BBC Radio 4 the other day, I heard that people who have not been to see their local GP in the last 5 years could face being ‘struck-off’ from the register and denied access until they re-register – the story is also covered in most of the national press, including The Guardian. It’s an effort to save money on NHS England’s £9bn annual expenditure on GP practices, but is it the most cost-effective and patient-friendly approach for updating NHS records?

Under the contract, an NHS supplier (Capita) will write every year to all patients who have not been in to see their local doctor or practice nurse in the last five years. This is aimed at removing those who have moved away or died – every name on the register costs the NHS on average around £136 (as at 2013/14) in payments to the GP. After Capita receives the list of names from the GP practice, they’ll send out two letters, the first within ten working days and the next within six months. If they get no reply, the person will be removed from the list. Of course, as well as those who have moved away or died, this will end up removing healthy people who have not seen the GP and don’t respond to either letter. An investigation in 2013 by Pulse, the magazine for GP’s, revealed that “over half of patients removed from practice lists in trials in some areas have been forced to re-register with their practice, with GP’s often blamed for the administrative error. PCTs (Primary Care Trusts) are scrambling to hit the Government’s target of removing 2.5 million patients from practice lists, often targeting the most vulnerable patients, including those with learning disabilities, the very elderly and children.” According to Pulse, the average proportion that were forced to re-register was 9.8%.

This problem of so-called ‘ghost patients’ falsely inflating GP patient lists, and therefore practice incomes, has been an issue for NHS primary care management since at least the 1990’s, and probably long before that. What has almost certainly increased over the last twenty years is the number of temporary residents (e.g. from the rest of the EU) who are very difficult to track.

A spokesperson for the BMA on the radio was quite eloquent on why the NHS scheme was badly flawed, but had no effective answer when the interviewer asked what alternatives there were – that’s what I want to examine here, an analytical approach to a typical Data Quality challenge.

First, what do we know about the current systems? There is a single UK NHS number database, against which all GP practice database registers are automatically reconciled on a regular basis, so that transfers when people move and register with a new GP are well handled. Registered deaths, people imprisoned and those enlisting in the armed forces are also regularly reconciled. Extensive efforts are made to manage common issues such as naming conventions in different cultures, misspelling, etc. but it’s not clear how effective these are.

But if the GP databases are reconciled against the national NHS number database regularly, how is it that according to the Daily Mail “latest figures from the Health and Social Care Information Centre show there are 57.6 million patients registered with a GP in England compared to a population of 55.1 million”? There will be a small proportion of this excess due to inadequacies in matching algorithms or incorrect data being provided, but given that registering a death and registering at a new GP both require provision of the NHS number, any inadequacies here aren’t likely to cause many of the excess registrations. It seems likely that the two major causes are:

  • People who have moved out of the area and not yet registered with a new practice.
  • As mentioned above, temporary residents with NHS numbers that have left the country.

To Data Quality professionals, the obvious solution for the first cause is to use specialist list cleansing software and services to identify people who are known to have moved, using readily available data from Royal Mail, Equifax and other companies. This is how many commercial organisations keep their databases up to date and it is far more targeted than writing to every “ghost patient” at their registered address and relying on them to reply. New addresses can be provided for a large proportion of movers so their letters can be addressed accordingly – if they have moved within the local area, their address should be updated rather than the patient be removed. Using the same methods, Capita can also screen for deaths against third party deceased lists, which will probably pick up more deceased names than the NHS system – simple trials will establish what proportion of patients are tracked to a new address, have moved without the new address being known, or have died.

Next, Capita could target the other category, the potential temporary residents from abroad, by writing to adults whose NHS number was issued in the last (say) 10 years.

The remainder of the list can be further segmented, using the targeted approach that the NHS already uses for screening or immunisation requests: for example, elderly people may have gone to live with other family members or moved into a care home, and young people may be registered at university or be sharing accommodation with friends – letters and other communications can be tailored accordingly to solicit the best response.

What remains after sending targeted letters in each category above probably represents people in a demographic that should still be registered with the practice. Further trials would establish the best approach (in terms of cost and accuracy) for this group: maybe it is cost-effective to write to them and remove non-responders, but if this resulted in only removing a small number, some of these wrongly, maybe it is not worth mailing them.

The bottom line is that well-established Data Quality practices of automatic suppression and change of address, allied with smart targeting, can reduce the costs of the exercise and will make sure that the NHS doesn’t penalise healthy people simply for… being healthy!

Driving Business with a Real-Time Single Customer View

Since we blogged about the challenges we overcame to deliver a Single Customer View for a major retailer a few years ago, we’ve found a lot of the same challenges repeated across other industry sectors such as non-profit, financial services and education, as well as marketing services providers managing marketing databases for companies in many different sectors. So if that’s more of the same, what’s different? In a word, time. It’s no longer good enough to have a Single Customer View that is only up to date every night, it should be up to date as quickly as your customer switches from one device to another – that is, in real time.

What are the benefits of a real-time Single Customer View?

 

Let’s stick with the multi-channel retail example both for continuity and because increasingly any product can be viewed through the eyes of a shopper, whether it is a scarf, a phone, a take-out meal, an insurance policy or a credit card account. It is widely recognized that the key to success in retail is a positive customer experience, so let’s look at some research findings:

To illustrate, if a customer orders online using their home computer for collection in store, and then after leaving home they want to change the order (using the browser on their phone or by calling the central customer service line), they expect the vendor to have the latest information about the order immediately available – otherwise, the potential for customer disenchantment is spelt out in the JDA research quoted above. If the info is all up to date, the new visit/call from the customer is an opportunity for the vendor to pitch an additional purchase, based on a 360° view of the customer’s account.

So how can you deliver a real time Single Customer View?

 

To answer this question, we first need to review where the moving data that we discussed before is coming from: keeping with the multi-channel retail example, it’s from Point-of-Sale systems in store, customers entering orders on the web site and call center operatives entering and looking up orders. These may be feeding into multiple corporate databases (ERP, Accounts, different subsidiary businesses etc.)  The challenge is: how do we perform the standardization, verification and matching that is required, classify misleading data etc. all on the fly, given that there can be as many as a dozen different data flows to handle? And how do we do all this quickly enough to ensure that the operator always has a current and complete view of the customer?

The key to meeting the challenge posed by the need for a real time Single Customer View is to accept that traditional disk-based database technology is too slow – we can’t afford the time to write a dozen or more transactions to disk, standardize and link all these by writing more records to disk and then read it all back from various disks to give the information to the operator – we can’t expect them to have a coffee break between every transaction!

To us the answer was obvious – all the data needs to be kept in computer memory, updated in memory and read back from memory, so getting away from the limitations placed by conventional hard disks and even solid state disks. But, you may say, that’s fine for small volumes of data but what if we’re streaming thousands of transactions a minute into databases with tens (or even hundreds) of millions of customers? The good news is that computer memory is so cheap these days that it’s extremely cost-effective to provision enough memory to handle even a billion customer accounts, with failover to a mirror of the data in the event of a problem.

Now it’s all very well to say “just use a lot of memory”, but can you find software that will run on all the different varieties of hardware, server technology and database systems that make up the corporate data sets? And will this software allow for the different kinds of error and discrepancy that arise when people enter name, company name, mailing address, email and multiple phone numbers? Even more challenging, will it allow for misleading data such as in store purchases being entered using a store address as the customer address, or a customer giving their partner’s phone number along with their own name and email address?

Once you’ve successfully managed to process the data real-time, you can begin to organize, understand and make use of it in real-time. To use the retail example one final time, now you can take the call from the customer on their way to collect their order and (by finding the order linked to their mobile number) enable them easily to add an item they’ve forgotten plus another item prompted by their purchase history. If the branch near home doesn’t have all the items in stock, you can direct them to the branch which does have the stock near their office – based on an up to date work address linked to the customer. With a real-time, 360° Single Customer View, it’s easy!

12 Days of Data Quality

12 Days of Data Quality

The holidays are finally here.  They always seem so far away and then, as the days grow short and temperatures fall, they tend to jump out at us in a surprise attack like a kid in a spooky costume on Halloween.  And once they are here, if you blink, they are over.  The anticipated smells of gingerbread baking in the oven, the joy of seeing a loved one open a carefully selected present, the glow of thousands of twinkling Christmas lights… all over before we were able to slow down and truly appreciate the holiday season.

So before December disappears under a pile of wrapping paper, we are inviting you to take the time to be merry, revel in the holidays, and perhaps still get a bit of work done.

Welcome to helpIT system’s 12 Days of Data Quality:

On the first Day of Data Quality, helpIT gave to me:

A Single Customer View (In a Pear Tree)

The first gift in this classic holiday carol is a Partridge in a Pear Tree.  The partridge sits alone high above the rest of the world.  Regally.  Eating pears (I imagine it eating pears) while looking down on all the lesser beings that have to see the world from ground level.

Your organization can be that partridge, sitting high above the rest.  Except in the world of database management, we are seeking a truly accurate Single Customer View, rather than a belly full of pears.  We all want the ability to look down on one contact record and obtain accurate, up-to-date information, each and every time.  Having one complete record for each customer ensures that they will receive the correct marketing materials at the correct address.  Salespeople will know a customer’s complete purchase history to analyze likely future purchases.  Customer service reps will be aware of address changes, name changes, as well as any other personal details in order to make the customer feel like they matter.  Which they do.  A lot.

Each customer in your contact database makes up a limb of your “pear tree”.  In the song, no matter how many gifts of drummers or pipers or ladies milking cows are given, it always comes back to the pear tree.  The tree is the center of everything, holding up even the partridge.  Just as your customers hold up your organization.  Make your customers feel this importance by respecting them as individuals, and as the base of your success, rather than lumping them in with the rest in your database.  The first step in doing this is by having a strong data quality solution and system in place.

On the second Day of Data Quality, helpIT gave to me:

2 Matched Records

On the second day of Christmas, my true love gave to me two turtle doves.  Which was great, in medieval times, when the doves symbolized true love’s endurance, mainly because they mated for life.  Everyone from the Bible to Shakespeare has made mention of them.

This December, give yourself another sort of true match.  Matching contact records in your database is the first step to cleaning up dirty data and obtaining a Single Customer View.  And there is no one-size-fits-all solution.

The important thing to consider when matching and deduping your database is the methodology used in the process.  Some software only matches exact records, so ‘John Smith’ and ‘John Smith’ would show up as a duplicate.  However, ‘John Smith’ and ‘Jon Smith’ would not.  So if you want a truly accurate database, you have to employ a more sophisticated method.

helpIT system’s matching software compares all the datasets in one contact record against the rest of your database.  John Smith’s address, birthday, phone number, email, or whatever other datapoints you use are all taken into consideration when pinpointing matches.  This process often picks up 20-80 percent more matches than other software.  When you multiply that by 200 million records, that’s a lot of matches.

The biggest mistake organizations make when matching records is to view it as a “one and done” solution.  Data matching, like any long-term relationship, is something that must be constantly tweaked, adapted, and carried out on a regular basis.  Although as the turtle doves can attest, this type of devotion does come with big rewards.

On the third Day of Data Quality, helpIT gave to me:

 

3 Frenchmen

 

Rather than the French Hens in the traditional song, let us meet a Frenchman whose name is Dr. Mathieu Arment. He loves to purchase designer scarfs from your company, Parisian Scarves. During his first online purchase, he entered his information as follows:

Matheiu Arment
27 Rue Pasteur
14390 Cabourg
FRANCE

The Parisian Scarves marketing department then sends him a catalogue for the Spring Collection. He flips through it while sitting at a local café sipping a latte and finds a handsome purple plaid scarf that he absolutely must have, but he has forgotten his laptop. So he calls in the order. The Parisian Scarves customer service rep does not see an account under the name she types in, Mattheiu Armond, so she creates a new account record and places his order.

Later, a second customer service rep is handling an issue with his order and decides to send a coupon as a gift for all of his trouble. The coupon is sent to Mathis Amiot. And the rep slightly misspelled his address on Rue Pasteur. Upon receiving the misguided coupon as well as two of the same catalogue addressed to slight variations of his name, Dr. Arment realizes that he is not just one Frenchman, but rather 3 separate Frenchmen in the eyes of Parisian Scarves. Feeling annoyed and undervalued that his favorite scarf company cannot even spell his name correctly, not to mention they also forgot his birthday, Dr. Arment takes his scarf shopping to another business who appreciates him as an individual.

Not all data matching software is created equal. While some compares only exact matches, helpIT system’s unique phonetics matching system will pull out similar sounding pieces of data as well as similar spellings. This will create a higher match rate, allowing for less duplicates to slip through into your database.

In this instance, the Parisian Scarves customer service rep would have typed in Mattheiu Armond, only to have the record Matheiu Arment show up as a possible match. She would have noted the similar addresses and concurred, correctly, that these were the same customer.

Accurate data matching creates a data quality firewall, preventing bad data from entering the system at point-of-entry, as well as filtering it out on a regularly scheduled check-up. So Dr. Arment can stay one Frenchman, and more importantly, he will stay a customer of Parisian Scarves.

On the fourth Day of Data Quality, helpIT gave to me:

 

4 Calling Salespeople

 

Sales is a unique industry in which every minute can translate into profits, if that minute is spent efficiently and effectively. Salespeople are constantly seeking better ways of doing things in order to increase your company’s profits, as well as their commissions. Which means every minute wasted clicking through the CRM, either in search of leads or trying to obtain accurate client data, is valuable time lost. Every phone call they make is either costing you money or making you money. What decides whether a sales team is a drain or an asset? The quality of the leads they are contacting.

A CRM that has effective data quality measures in place is filled with accurate contact records. These records can be analyzed to obtain valuable information by all arms of your organization, especially the sales department.

A good salesperson can use CRM data to offer the right products to the right potential buyers, as well as dedicate more time to leads that are statistically more likely to turn into sales. They will be able to quickly obtain the correct point-of-contact and contact information without fishing through multiple records for the same lead. A salesperson will also look more knowledgeable as they are able to talk easily with a client about their business needs.

Give your salespeople the resources they need to be a profitable addition to your company by having an accurate, up-to-date CRM.

On the fifth Day of Data Quality, helpIT gave to me:

 

5 Golden Reasons to Trial matchIT SQL

 

helpIT’s ‘12 Days of Data Quality’ continues with 5 Golden Reasons to Trial matchIT SQL. Perhaps not the golden rings the lady received in the song, but really, who needs five golden rings? Sounds like a pickpocket’s dream come true. So instead, we here at helpIT are presenting you with five reasons to try our matching system.

We hope by now that you are starting to understand how important a strong data quality management system is to the success of your organization. It can increase profits and productivity in all arms of your organization. Yet sometimes it is hard to get the ball rolling, especially if you have a lot of chiefs who are part of this decision. So consider these five reasons why a helpIT systems trial is a good place to start:

1. Quick Installation. Be processing data in less than an hour!
2. Run data cleansing processes on your own data in your own environment (even address validation).
3. Customize the matching process and fine-tune your results with dedicated Trial Customer Support.
4. Run large volumes of data to see real performance results.
5. Get the real-world examples you need to justify your business case for SQL data.

This holiday season, try matchIT SQL for 30 days for absolutely nothing! We know you will love it, but if you don’t, we will give you 5 golden rings. Or maybe just one. Or a thank-you email. Yes, if you don’t love it, we will send you an email thanking you for your time. Happy trialing!

On the sixth Day of Data Quality, helpIT gave to me:

 

6 Companies a-Laying

 

Our countdown to Christmas and better data quality measures continues! In the song, his sweetheart received 6 geese laying eggs. Which might get some odd looks around the office. Instead, consider the importance of laying a strong foundation when beginning your quest for clean data.

All geese lay eggs. But the goose that laid the golden egg got a lot more attention than the rest. Like that golden-egg laying goose, the company that lays the strongest foundation in regards to data quality will garner the most attention and achieve the best results.

Most organizations think of clearing out dirty data as something to be dealt with when absolutely necessary. When in reality, database maintenance is a process that should be consistently tweaked, monitored, and exercised. Contact data is constantly entering your system. Contacts are frequently relocating, changing names, or passing away. Which means a good database administrator is diligent in tracking these changes.

Laying the foundation for strong data quality measures is often labeled too time consuming to be dealt with. But the time invested originally will pay off in piles of golden eggs in the future.

On the seventh Day of Data Quality, helpIT gave to me:

 

7 Sales a-Swimming

 

Or rather, floundering. Whether you want to admit it or not, the odds are you are floundering in bad data, working hard just to stay afloat of the changes that occur in your contact database on a daily basis. Each sale relies on every member of your team being able to swim seamlessly through the CRM to obtain the information they need to make a client feel valued and understood.

Companies today report data analysis as one of the most effective tools for developing marketing campaigns and targeting sales leads. Many organizations use data analysis on a daily basis. However, if they are analyzing inaccurate or out-of-date data, the analysis is all but pointless. A database that does not have systems in place for catching bad data at point-of-entry, as well as a regular cleansing schedule, is a hindrance rather than a help in regards to data analysis.

This holiday season, give your data analysis the gift of a life raft. Make sure your team is swimming, rather than floundering, in the sea of contact data. Accurate data analysis will increase marketing effectiveness, reduce marketing spend, and increase productivity in all aspects of your business that work in the CRM.

On the eighth Day of Data Quality, helpIT gave to me:

 

8 Maids E-Mailing

 

While your business is probably not made up of maids, it does most likely contain many people that rely on email communications on a day-to-day basis. Email is an important means to reach prospects, current customers, and vendors. How these messages are delivered, as well as the content in them, is a strong reflection on the quality of your business model.

Do the emails look polished and professional? Or lazy and sloppy? Most organizations unintentionally accomplish the latter. A lack of data quality management systems has caused incorrect contact information to reside in their database. So Joe Smith gets an email addressed to Jo Smith. Or Jo Smith becomes a Mrs. instead of a Mr. And that’s all assuming that the email is even delivered.

Email deliverability is a key concern to many businesses, especially in regards to marketing. A great marketing campaign is irrelevant if the message is not received by the intended recipient. New email addresses are often mistyped. Another possibility is that a wrong address was given intentionally. Either way, the organization has lost a sales lead because the incorrect address is not reachable.

Email validation is a valuable and effective piece of the data quality puzzle. It will greatly increase the number of leads passed onwards to your sales team as well as ensure that marketing communications arrive to the person they were intended for. It is easy to implement, and the rewards far outweigh the costs.

On the ninth Day of Data Quality, helpIT gave to me:

 

9 Ladies Dating

 

One of the biggest challenges in your database can come from name changes. Sometimes it is from 9 ladies dating and then deciding to tie the knot. And while marriage is normally considered a wonderfully celebrated occasion, to the database administrator it means the possibility of error. Because it is almost a certainty that Ms. Smith is not calling her 17 magazine subscriptions from her honeymoon to let them know she married Mr. Clark and moved into his duplex in the Heights.

The new Mrs. Clark is a valued customer. So treat her as such by recognizing these changes as quickly as possible. Name changes and new addresses are easily dealt with when you have a proper data quality system in place. Stay tuned for tomorrow’s blog for some tips on keeping up with Mrs. Clark.

On the tenth Day of Data Quality, helpIT gave to me:

 

10 Lords a-Moving

 

The original Lords from the song might be a-leaping, but most of your customers getting around via UHaul trucks and airplanes. They are leaping across town, across the state, and sometimes, across the world. In an average year over 40 million people move. Keeping up with them can seem even harder than remembering the words to the 12 Days of Christmas.

Keep your contact database accurate and up-to-date with National Change of Address (NCOA). In one easy process your current contact address data is compared to USPS CASS and DPV certified data, correcting any typing errors and appending additional information.

On the eleventh Day of Data Quality, helpIT gave to me:

 

11 Pipers Piping

 

The Pied Piper was a character in German folklore who tried to sway a town to pay him to rid their village of rats. His pipe music would lure the rats out of hiding and they would follow him out of town. When the villagers refused to pay for this service, he piped away their children instead. Not the noblest use of his talents, but the ability to lead others is a powerful trait nonetheless.

Be the pied piper at your organization, only use your powers for good instead of evil. Make 2016 the year your organization makes data quality solutions a priority and others will be glad they followed you. Often the only thing holding a company back from reducing the costs of bad data is the knowledge and the leadership to move forwards. helpIT systems offers a full range of customer support solutions so that you and your company can feel confident about your next move.

On the twelfth Day of Data Quality, helpIT gave to me:

 

12 DBAs Drumming

 

More often than not, the squeaky wheel gets the grease. The loudest drummers in your office this season should be those making noise about the importance of data deduplication. Having an improperly deduped database can create upwards of 60 percent of dirty data in your contact database.

Those incorrect contacts are receiving marketing materials (which cost money), taking up manpower to organize and sift through in the CRM (which costs time), and getting calls from your sales people (which cost money and time).

This month alone I have received mail for 4 different past residents of my current apartment. You know what I do with it? I throw it away. So Horace will never get that credit card offer. Zachary will not be donating to the Salesian Missions. And Monique will not be showing up in court for her fifth and final notice to appear. (Feeling a little guilty about that last one.)

We hope you enjoyed our unique spin on the traditional “12 Days of Christmas”. While the holidays are nearing an end, helpIT systems is here to answer your data quality questions 365 days a year. We hope to make 2016 your best data quality year ever. Give us a call or visit our website at www.helpit.com to find out more information.

Weighing up the Cost of Bad Data

Weighing up the Cost of Bad Data

In a recent survey conducted by helpIT systems, almost 25 percent of respondents cited finances as the biggest hindrance to maintaining superior contact databases.  We get it.  Data quality solutions can carry what may seem to be a hefty pricetag, and they won’t show up two days later in a nicely wrapped package like an Amazon Prime purchase.  As such, like any other expensive and complicated decision, data quality may well get pushed to the bottom of the pile.

Then again, just like going to the gym or eating salad instead of steak, the toughest behaviors to adapt are usually the most beneficial.  Because even though database management may be something we’d rather forget about, 40 percent of those same respondents stated that their companies were losing tens of thousands of dollars each year due to poor contact data quality.  So while the solution may not be cheap and easy, the cost of living without it does not appear to be either.  Data Warehousing Institute found that the cost of bad data to US businesses is more than $600 billion each year.  Is that a number your company can afford to ignore?

Many businesses do notice these dollars disappearing and choose to do something about it.  Unfortunately however, this is often simply a “quick fix”.  They look at their messy databases, pay someone to “clean them up”, and then everyone gets a pat on the back for a job well done.  And it is.  Until someone enters a new record in the CRM, a customer moves, or perhaps even dares to get a new phone number.  And I will shock everyone by reporting that this happens all the time.  Studies indicate up to a 2 percent degradation each month…even in a perfect database.

Right now you’re probably picking up on the fact that maintaining good data is going to cost money.  You’re right.  But the fact is, avoiding that cost is only going to cost more in the long run.  Just like having a well-trained sales team, a finely-targeted marketing plan, or a boss with years of experience…great results are an investment of time and resources rather than a happy accident.

Companies that choose to invest in good data quality, as well as to view it as an ongoing process rather than a simple one-time fix, are finding that the benefits by far outweigh the initial costs.  Advertising dollars are reaching their intended audiences and sales calls are reaching the right recipient, with customer satisfaction going through the roof.  Today’s consumer expects the personal touches that can only come from having an accurate and up-to-date Single Customer View, and it is good data quality solutions that will achieve them.

How Ashley Madison Can Inspire Your Business

As each new name and every illicit detail is revealed, the 37 million members of Ashley Madison, a website promoting extramarital affairs, are scrambling to save their marriages, careers, and reputations.  This list, which is now available to anyone aware ofthe existence of Google, reportedly includes the names and sexual fantasies of members of the armed services, United Nations, and even the Vatican.  Looks like someone’s prayers weren’t heard this week.

As the extent of the contact information becomes more easily accessible, a new breed of data analyst is emerging.  Creative thinkers are using the information to win custody battles, deduce which cities have the most cheaters, and even get a leg up over another candidate for a job promotion.

If everyone from neglected housewives to tawdry tabloid writers is capable of using data to form opinions and make well-informed decisions, the question is… why aren’t you?

Now I’m not talking about crawling through Ashley Madison’s troves of cheaters, I’m talking about your company.  Your data.  Demographics, geographic locations, purchasing behavior… your contact records say a million things about your customers.  A million patterns are lying in wait, holding the key to better marketing, better operations, and better business decisions.  Whereas for Ashley Madison data spelled disaster, for you it should spell potential.

Customer data, when compromised, can be a company’s worst nightmare.  When used intelligently, customer data can increase profits and reduce the guessing game so many businesses play on a day-to-day basis.

In order to use your data intelligently, you must be confident that it is accurate and up-to-date.  If your records indicate you have 14 Jeremiah Whittinglys living in Chicago, you can either double your production of Jeremiah Whittingly personalized baseball caps, or perhaps take a closer look at how clean your data is.  I’m personally leaning towards the second option.

However, beefing up marketing efforts in Juneau, where your database says 10 percent of your client base is located, is a smart idea.  Unless your data entry employee didn’t realize ‘AK’ was the postal code abbreviation for Alaska rather than Arkansas.  In which case, polar bears stand a better chance of appreciating your new billboard than your target market.

Ridding your database of duplicate, incorrect, or incomplete records is the first step in recognizing the power of customer data.  The next step is figuring out what this data means for you and your company, and if every talk show host and dark web hacker can do it with the right tools, so can you.

Why Customers Must Be More Than Numbers

I read with some amazement a story in the London Daily Telegraph this week about a customer of NatWest Bank who sent £11,200 last month via online banking to an unknown company instead of his wife. Although Paul Sampson had correctly entered his wife’s name, sort code and account number when he first made an online payment to her HSBC account, he wasn’t aware that she had subsequently closed the account.

Mr Sampson thought he was transferring £11,200 to his wife: he clicked Margaret’s name among a list of payees saved in his NatWest banking profile and confirmed the transaction, but the payment went to a business in Leeds. Mr Sampson believes that HSBC had reissued his wife’s old account number to someone else, a company whose name they refused to tell him. NatWest told Mr Sampson it was powerless to claw the money back.

HSBC said it had contacted its customer, but it had no obligation regarding the money. HSBC insisted that the account number in question was not “recycled”, saying Mr Sampson must have made a typing error when he first saved the details, which he disputes. Although the money was in fact returned after the newspaper contacted HSBC, a very large issue has not been resolved.

Although news to most of us, it is apparently a common practice among banks in the UK to recycle account numbers, presumably because banking systems are so entrenched around 8 or 9 digit account numbers that they are concerned about running out of numbers. Apparently a recent code of practice suggests that banks should warn the customer making the payment if they haven’t sent money to this payee for 13 months, but according to the Daily Telegraph “No major high street bank could confirm that it followed this part of the code”.

The Daily Telegraph goes on to state that the recipients of electronic payments are identified by account numbers only. The names are not checked in the process, so even if they do not match, the transaction can proceed. “This is now a major issue when you can use something as basic as a mobile phone number to transfer money,” said Mike Pemberton, of solicitors Stephensons. “If you get one digit wrong there’s no other backup check, like a person’s name – once it’s gone it’s gone.” If you misdirect an online payment, your bank should contact the other bank within two working days of your having informed them of the error, but they have no legal obligation to help.

Mr Sampson obviously expected that the bank’s software would check that the account number belonged to the account name he had stored in his online payee list, but apparently UK banking software doesn’t do this. Why on earth not? Surely it’s not unreasonable for banks with all the money they spend on computer systems to perform this safety check? It’s not good enough to point to the problems that can arise when a name is entered in different ways such as Sheila Jones, Mrs S Jones, Sheila M Jones, SM Jones, Mrs S M Jones, Mrs Sheila Mary Jones etc.

These are all elementary examples for intelligent name matching software.  More challenging are typos, nicknames and other inconsistencies such as those caused by poor handwriting, which would all occur regularly should banks check the name belonging to the account number. But software such as matchIT Hub is easily available to cope with these challenges too, as well as the even more challenging job of matching joint names and business names.

There are also issues in the USA with banking software matching names – I remember when I first wanted to transfer money from my Chase account to my Citibank account, I could only do so if the two accounts had exactly the same name – these were joint accounts and the names had to match exactly letter for letter, so I had to either change the name on one of the accounts or open a new one! Having been an enthusiastic user of the system in the USA for sending money to someone electronically using just their email address, I’m now starting to worry about the wisdom of this…

We banking customers should perhaps question our banks more closely about the checks that they employ when we make online payments!

The 12 Days of Shopping

According to IBM’s real-time reporting unit, Black Friday online sales were up close to 20% this year over the same period in 2012.  As for Cyber Monday, sales increased 30.3% in 2012 compared to the previous year and is expected to grow another 15% in 2013. Mobile transactions are at an all time high and combined with in store sales, The National Retail Federation expects retail sales to pass the $600 billion mark during the last two months of the year alone. While that might sound like music to a retailer’s ears, as the holiday shopping season goes into full swing on this Cyber Monday, the pressure to handle the astronomical influx of data collected at dozens of possible transaction points is mounting. From websites and storefronts to kiosks and catalogues, every scarf or video game purchased this season brings with it a variety of data points that must be appropriately stored, linked, referenced and hopefully leveraged. Add to that a blinding amount of big data now being collected (such as social media activity or mobile tracking), and it all amounts to a holiday nightmare for the IT and data analysis teams. So how much data are we talking and how does it actually manifest itself? In the spirit of keeping things light, we offer you, The 12 Days of Shopping…

On the first day of shopping my data gave to me,
1 million duplicate names.

On the second day of shopping my data gave to me,
2 million transactions, and
1 million duplicate names.

On the third day of shopping my data gave to me,
30,000 credit apps,
2 million transactions, and
1 million duplicate names.

On the fourth day of shopping my data gave to me,
40 returned shipments,
30,000 credit apps,
2 million transactions, and
1 million duplicate names.

On the fifth day of shopping my data gave to me,
5 new marketing lists,
40 returned shipments,
30,000 credit apps,
2 million transactions, and
1 million duplicate names.

On the sixth day of shopping my data gave to me,
6,000 bad addresses,
5 new marketing lists,
40 returned shipments,
30,000 credit apps,
2 million transactions, and
1 million duplicate names.

On the seventh day of shopping my data gave to me,
7,000 refunds,
6,000 bad addresses,
5 new marketing lists,
40 returned shipments,
30,000 credit apps,
2 million transactions, and
1 million duplicate names.

On the eighth day of shopping my data gave to me,
8,000 new logins,
7,000 refunds,
6,000 bad addresses,
5 new marketing lists,
40 returned shipments,
30,000 credit apps,
2 million transactions, and
1 million duplicate names.

On the ninth day of shopping my data gave to me,
90,000 emails,
8,000 new logins,
7,000 refunds,
6,000 bad addresses,
5 new marketing lists,
40 returned shipments,
30,000 credit apps,
2 million transactions, and
1 million duplicate names.

On the tenth day of shopping my data gave to me,
10,000 tweets,
90,000 emails,
8,000 new logins,
7,000 refunds,
6,000 bad addresses,
5 new marketing lists,
40 returned shipments,
30,000 credit apps,
2 million transactions, and
1 million duplicate names.

On the eleventh day of shopping my data gave to me,
11 new campaigns,
10,000 tweets,
90,000 emails,
8,000 new logins,
7,000 refunds,
6,000 bad addresses,
5 new marketing lists,
40 returned shipments,
30,000 credit apps,
2 million transactions, and
1 million duplicate names.

On the twelfth day of shopping my data gave to me,
12 fraud alerts,
11 new campaigns,
10,000 tweets,
90,000 emails,
8,000 new logins,
7,000 refunds,
6,000 bad addresses,
5 new marketing lists,
40 returned shipments,
30,000 credit apps,
2 million transactions, and
1 million duplicate names.

While we joke about the enormity of it all, if you are a retailer stumbling under the weight of all this data, there is hope and over the next few weeks we’ll dive a bit deeper into these figures to showcase how you can get control of the incoming data and most importantly, leverage it in a meaningful way.

Sources:
http://techcrunch.com/2013/11/29/black-friday-online-sales-up-7-percent-mobile-is-37-percent-of-all-traffic-and-21-5-percent-of-all-purchases/

http://www.pfsweb.com/blog/cyber-monday-2012-the-results/

http://www.foxnews.com/us/2013/11/29/retailers-usher-in-holiday-shopping-season-as-black-friday-morphs-into/

The New Paradigm in Healthcare Data Quality

There is no higher importance in managing customer information than when making decisions on health care. While most markets are busy striving for a ‘single customer view’ to improve customer service KPIs or marketing campaign results, healthcare organizations must focus on establishing  a ‘single patient view’, making sure a full patient history is attached to a single, correct contact.  Unlike in traditional CRM solutions, healthcare data is inherently disparate
and is managed by a wide variety of patient systems that, in addition to collecting and managing contact data, also tracks thousands of patient data points including electronic health records, insurance coverage, provider names,  prescriptions and more. Needless to say, establishing the relationships between patients and their healthcare providers, insurers, brokers, pharmacies and the like or even grouping families and couples together, is a significant challenge. Among them are issues with maiden/married last names, migration of individuals between family units and insurance plans, keying errors at point of entry or even deliberate attempts by consumers to defraud the healthcare system.

In many cases, the single patient view can be handled through unique identifiers , such as those for group health plans or for individuals within their provider network. This was an accepted practice at a recent Kaiser Permanente location I visited, where a gentleman went to the counter and reeled off his nine digit patient number before saying “hello”. But while patient ID numbers are standard identifiers, they will differ between suppliers and patients can’t be relied on to use it as their first method of identification. This is where accuracy and access to other collected data points (I.e. SSN, DOB and current address) becomes critical.

While healthcare organizations have done a decent job so far of attempting to establish and utilize this ‘single patient view’, the healthcare data quality paradigm is shifting once again. For example, The Patient Protection and Affordable Care Act (PPACA) means that healthcare organizations will now have to deal with more data, from more sources and face tougher regulations on how to manage and maintain that data.  The ObamaCare Health Insurance Exchange Pool means that more Americans can potentially benefit from health insurance coverage, increasing the number with coverage by around 30 million. Through these new initiatives, consumers will also have greater choice for both coverage and services  – all further distributing the data that desperately needs to be linked.

With such inherent change – how do you effectively service patients at the point-of-care? And, do you want your trained medics and patient management team to be responsible for the data quality audit before such care can even begin?

So what are the new dynamics that healthcare companies need to plan for?

  • Addition of new patients into a system without prior medical coverage or records
  • Frequent movement of consumers between healthcare plans under the choice offered by the affordable care scheme
  • Increased mobility of individuals through healthcare systems as they consume different vendors and services

This increased transactional activity means healthcare data managers must go beyond the existing efforts of linking internal data and start to look at how to share data across systems (both internal and external) and invest in technology that will facilitate this critical information exchange. Granted, this will be a significant challenge given the fact that many organizations have several proprietary systems, contract requirements and privacy concerns but oddly enough, this begins with best practices in managing contact data effectively.

Over the last year, I’ve worked with an increasing number of customers on the issue of managing the introduction of new data into healthcare databases.  Just like healthcare, data quality is both preventative and curative. Curative measures include triage on existing poor quality data, and investigating the latent symptoms of unidentified relationships in the data. The preventative measures are to introduce a regimen of using DQ tools to accurately capture new information at
point of entry efficiently, and to help identify existing customers quickly and accurately.

For healthcare customers, we’ve managed to do just this by implementing helpIT systems’ technology, matchIT SQL to deal with the backend data matching, validation and merging and findIT S2 to empower users to quickly and accurately identify existing patients or validate new patient details with the minimum of keystrokes. This complementary approach gives a huge return on investment allowing clinical end-users to focus on the task at hand, rather than repeatedly dealing with data issues.

Whenever there is movement in data or new sources of information, data quality issues will arise. But when it comes to healthcare data quality, I’m sure healthcare DBA’s and other administrators are fully aware of the stakes at hand. Improving and streamlining data capture plus tapping into the various technology connectors that will give physicians and service providers access to all patient data will have a profound effect on patient care, healthcare costs, physician workloads and access to relevant treatment. Ultimately, this is the desired outcome.

I’m delighted to be engaged further on this subject so if you have more insight to share, please comment on this or drop me a line.


Click & Collect – How To Do It Successfully?

In the UK this Christmas, the most successful retailers have been those that sell online but allow collection by the shopper – in fact, these companies have represented a large proportion of the retailers that had a good festive season. One innovation has been the rise of online retailers paying convenience stores to take delivery and provide a convenient collection point for the shopper, but two of the country’s biggest retailers, John Lewis and Next, reckon that click and collect has been the key to their Christmas sales figures – and of course they both have high volume e-commerce sites as well as many bricks and mortar stores.

The article here by the Daily Telegraph explains why “click and collect” is proving so popular, especially in a holiday period. The opportunities for major retailers are  obvious, especially as they search for ways to respond to the Amazon threat – but how do they encourage their customers to shop online and also promote in store shopping? The key is successful data-driven marketing: know your customer, incentivize them to use loyalty programs and target them with relevant offers. However, this also presents a big challenge – the disparity and inconsistency in the data that the customer provides when they shop in these different places.

In store, they may not provide any information, or they may provide name and phone number, or they may use a credit card and/or their loyalty card. Online they’ll provide name, email address and (if the item is being delivered), credit card details and their address. If they are collecting in store, they may just provide name and email address and pay on collection – and hopefully they’ll enter their loyalty card number, if they have one. To complicate matters further, people typically have multiple phone numbers (home, office, mobile), multiple addresses (home and office, especially if they have items delivered to their office) and even multiple email addresses. This can be a nightmare for the marketing and IT departments in successfully matching this disparate customer data in order to establish a Single Customer View. To do this, they need software that can fulfill multiple sophisticated requirements, including:

  • Effective matching of customer records without being thrown off by data that is different or missing.
  • Sophisticated fuzzy matching to allow for keying mistakes and inconsistencies between data input by sales representatives in store and in call centers, and customers online.
  • The ability to recognize data that should be ignored – for example, the in-store purchase records where the rep keyed in the address of the store because the system demanded an address and they didn’t have time to ask for the customer’s address, or the customer didn’t want to provide it.
  • Address verification using postal address files to ensure that when the customer does request delivery, the delivery address is valid – and even when they don’t request delivery, to assist the matching process by standardizing the address.
  • The ability to match records (i) in real-time, in store or on the website (ii) off-line, record by record as orders are fed though for fulfillment and (iii) as a batch process, typically overnight as data from branches is fed through. The important point to note here is that the retailer needs to be able to use the same matching engine in all three matching modes, to ensure that inconsistencies in matching results don’t compromise the effectiveness of the processing.
  • Effective grading of matches so that batch and off-line matching can be fully automated without missing lots of good matches or mismatching records. With effective grading of matching records, the business can choose to flag matches that aren’t good enough for automatic processing so they can be reviewed by users later.
  • Recognition of garbage data, particularly data collected from the web site, to avoid it entering the marketing database and compromising its effectiveness.
  • Often, multiple systems are used to handle the different types of purchase and fulfillment. The software must be able to connect to multiple databases storing customer data in different formats for the different systems

With a wide range of data quality solutions on the market, it’s often difficult to find a company that can check all of these boxes. That’s where helpIT systems comes in. If you are a multi-channel retailer currently facing these challenges, contact helpIT systems for a Free Data Analysis and an in depth look at how you can achieve a Single Customer View.