Bridging the gap

Bridging the skills gap

TechMarketView’s UKHotViews© post Are you hiding from your skills crisis? last week really struck a chord. Kate Hanaghan gave some interesting feedback about Microsoft’s Cloud Skills Report (which surveyed 250 mid-sized to large UK organisations) but in our experience, many of the same issues apply to moving from proprietary in house systems or legacy packaged software to industry-standard data platforms such as SQL Server.

According to Kate, “individuals themselves are not always keen to move away from the technologies they have spent years working with” and suppliers need to “convince technologists (who tend to be middle aged and highly experienced) they must re-skill in certain areas to support the business as it attempts to grow in digital areas”.

Although as Kate says, many legacy technologies will be around for many years to come, I think that with the increasing pace of technological change, individuals are unwise if they ignore opportunities to embrace new technologies. Movement to the cloud is now so rapid that cost and competitive pressures will force many organisations that are currently steadfastly “on premise” to start moving across sooner rather than later – particularly marketing services companies where demand is elastic. Companies and individuals who try and move from 20 year old, non-standard technology straight to the cloud will struggle, whereas companies with more modern infrastructure and techies with more modern skills will have more of an evolutionary, beaten path .

Apart from competitive pressures, there are many other sound reasons for moving from such aging systems to industry-standard data platforms, as we wrote in Data cleansing – inside the database or on your desktop? One of the key reasons is that using a platform like SQL Server is much more extensible – for example, in the marketing services sector, our matchIT SQL package can connect directly with MIS products upstream and document composition products downstream using shared data, so all the data is processed within SQL Server. For the company, data is more secure and both errors and turnaround time are greatly reduced. For IT staff, it means they can enhance their CV’s with sought-after skills and be ready to embrace the next opportunity a rapidly changing world gives them – such as using Microsoft Azure or Apache Spark for cloud deployment.

I’ll leave the last word to Kate, who wrote to me about her post: “In some ways I just find it so hard to understand. Who wouldn’t want to future-proof their career?! I mean, we’re going to be working till we’re 80!!”

 

matchIT SQL – finalist at the DBM Awards 2016

matchIT SQL – finalist at the DBM Awards 2016

We were thrilled to be selected as finalists for this prestigious industry award, and to join our peers, which included some of the industry’s finest at the 2016 DBM awards. The event was hosted in an impressive and historic location in London, The Brewery.

Whilst helpIT systems offers a range of deployments for its class leading matchIT Data Quality Solutions, it was our  SQL Server integrated solution matchIT SQL that was recognised as standing out from the crowd, in the fiercely competitive “Innovation in database marketing-related software” category.

matchIT SQL’s shortlisting was in part due to our innovative and seamless integration with Microsoft SQL Server, and it’s unique ability to bring together a comprehensive array of contact data quality functions, all available and accessed natively within Microsoft SQL Server. Whether it be highly accurate and intelligent data matching, UK or international address validation, suppression screening and integrated email validation, through to its Power BI reporting or blend of local or Microsoft Azure and/or web service deployment, matchIT SQL is obviously hitting the mark.

Perhaps it is because marketing agencies and data professionals alike have realised how matchIT SQL helps them build and maintain highly accurate single customer views and comply with the new GDPR. Or that it enables the seamless preparation of targeted and accurate marketing and campaign data, as well as intelligently manage and monitor data workflows and data feeds coming into a business.

Whatever it was that resulted in matchIT SQL being shortlisted, we are very proud of our team here at helpIT systems who are behind this innovative technology, and who have been providing such class leading software in the Contact Data Quality space for over 25 years.

On the night, the award went to Purple Agency, who were behind a bespoke solution created for the Financial Times, which involved a system to capture and hold every nugget of customer data to help achieve complex goals. While matchIT SQL may not have been chosen as the winner on this occasion, we’re very proud of the selection as finalists and the recognition that helpIT systems is still punching well above its weight and is a pioneer in developing innovative Database Marketing related solutions.

Thank you to the team at the DBM Awards, and in particular to our fantastic hosts Stuart Cal, Anthony Begley and the team, who made the evening the great success it was.

Data cleansing – inside the database or on your desktop?

Data cleansing – inside the database or on your desktop?

We’ve been gathering feedback from some of our service provider clients recently about the impact on them of change: there’s a lot of things that they mentioned to us: organisational change and new demands due to new customers come up as often as they always have done, but what about enforced change from outside the organisation?

Two aspects that clients frequently mentioned are the new EU General Data Protection Regulations (GDPR) and the competitive need to keep up with advances in technology.  That’s why we were particularly pleased to get some great feedback from one of our clients who recently switched from a desktop data cleansing/mailing package to matchIT SQL.  They wanted to switch to matchIT SQL because they already use Microsoft SQL Server for all storage and manipulation of their customer data, and saw a lot of potential for increased automation with matchIT SQL – leveraging their familiarity with SQL Server Integration Services (SSIS) and Stored Procedures to achieve this.  However, the benefits they gained were far more fundamental and far-reaching than they envisaged…

Reducing the potential for errors

Having to export data from one system and import it into another (perhaps multiple times) introduces opportunity for mistakes: keeping record layouts in line, splitting and combining files etc.  By processing data in one system instead of several, the potential for these kinds of mistakes is eliminated and the corresponding time taken for Q/A (and sometimes repetition of steps) is greatly reduced.  Automating regular jobs also had a major impact on reducing errors and reruns.

Reducing data processing time

By fully utilising the power of their hardware, automatically spreading processing over all the machine cores available, they increased throughput dramatically.  Automation also allowed them to introduce more performance tuning – set up once and then repeated easily as often as required, without any additional user effort.  All told, our client is now able to turn round large jobs involving millions of records 4 times faster than they were able to with the old desktop system.  As they are finding that the data they get from their customers tends to be less consistent and more fragmented these days, this saving on turnaround time allows them to stick to deadlines which used to be in great danger of slipping.

Protecting data by processing it within a secure database

The increasing concern these days around data security, and the increased burden of the EU GDPR regarding data accuracy and the right of consumers to get a copy of their data, can cause many a sleepless night.  Two of the big benefits of processing all data within SQL Server are the access control that it provides and the auditability.  When you are exporting data from one system to another, perhaps via a flat file, it is very difficult to enforce such strict security while the data is in flight – and any security system is only as good as its weakest link.

Extensibility

Using Microsoft SQL Server together with all the off the shelf tools available means future enhancements can be made much more easily. Additionally as hardware performance and technology improves further in the future, SQL Server as a platform will keep up with those advances.

Bottom line

Working with a single solution inside a secure, high performance database system can have a huge impact on your business.  Whether it’s SQL Server or another database system that you use, you owe it to yourself to find out what the options are for switching to a data cleansing solution that lives inside your database.  Your IT staff will be happy to work with new server-based technology, they will get less out-of-hours calls, it will be easier for you to hire new staff with relevant skills – and you too will sleep more soundly!

Ready to find out how you can start benefiting too? Arrange a free 30 Day Trial of matchIT SQL – simply click here.

 

The doctor won’t see you now – NHS data needs a health check!

On BBC Radio 4 the other day, I heard that people who have not been to see their local GP in the last 5 years could face being ‘struck-off’ from the register and denied access until they re-register – the story is also covered in most of the national press, including The Guardian. It’s an effort to save money on NHS England’s £9bn annual expenditure on GP practices, but is it the most cost-effective and patient-friendly approach for updating NHS records?

nhs-logoUnder the contract, an NHS supplier (Capita) will write every year to all patients who have not been in to see their local doctor or practice nurse in the last five years. This is aimed at removing those who have moved away or died – every name on the register costs the NHS on average around £136 (as at 2013/14) in payments to the GP. After Capita receives the list of names from the GP practice, they’ll send out two letters, the first within ten working days and the next within six months. If they get no reply, the person will be removed from the list. Of course, as well as those who have moved away or died, this will end up removing healthy people who have not seen the GP and don’t respond to either letter. An investigation in 2013 by Pulse, the magazine for GP’s, revealed that “over half of patients removed from practice lists in trials in some areas have been forced to re-register with their practice, with GP’s often blamed for the administrative error. PCTs (Primary Care Trusts) are scrambling to hit the Government’s target of removing 2.5 million patients from practice lists, often targeting the most vulnerable patients, including those with learning disabilities, the very elderly and children.” According to Pulse, the average proportion that were forced to re-register was 9.8%.

This problem of so-called ‘ghost patients’ falsely inflating GP patient lists, and therefore practice incomes, has been an issue for NHS primary care management since at least the 1990’s, and probably long before that. What has almost certainly increased over the last twenty years is the number of temporary residents (e.g. from the rest of the EU) who are very difficult to track.

A spokesperson for the BMA on the radio was quite eloquent on why the NHS scheme was badly flawed, but had no effective answer when the interviewer asked what alternatives there were – that’s what I want to examine here, an analytical approach to a typical Data Quality challenge.

First, what do we know about the current systems? There is a single UK NHS number database, against which all GP practice database registers are automatically reconciled on a regular basis, so that transfers when people move and register with a new GP are well handled. Registered deaths, people imprisoned and those enlisting in the armed forces are also regularly reconciled. Extensive efforts are made to manage common issues such as naming conventions in different cultures, misspelling, etc. but it’s not clear how effective these are.

But if the GP databases are reconciled against the national NHS number database regularly, how is it that according to the Daily Mail “latest figures from the Health and Social Care Information Centre show there are 57.6 million patients registered with a GP in England compared to a population of 55.1 million”? There will be a small proportion of this excess due to inadequacies in matching algorithms or incorrect data being provided, but given that registering a death and registering at a new GP both require provision of the NHS number, any inadequacies here aren’t likely to cause many of the excess registrations. It seems likely that the two major causes are:

  • People who have moved out of the area and not yet registered with a new practice.
  • As mentioned above, temporary residents with NHS numbers that have left the country.

To Data Quality professionals, the obvious solution for the first cause is to use specialist list cleansing software and services to identify people who are known to have moved, using readily available data from Royal Mail, Equifax and other companies. This is how many commercial organisations keep their databases up to date and it is far more targeted than writing to every “ghost patient” at their registered address and relying on them to reply. New addresses can be provided for a large proportion of movers so their letters can be addressed accordingly – if they have moved within the local area, their address should be updated rather than the patient be removed. Using the same methods, Capita can also screen for deaths against third party deceased lists, which will probably pick up more deceased names than the NHS system – simple trials will establish what proportion of patients are tracked to a new address, have moved without the new address being known, or have died.

Next, Capita could target the other category, the potential temporary residents from abroad, by writing to adults whose NHS number was issued in the last (say) 10 years.

The remainder of the list can be further segmented, using the targeted approach that the NHS already uses for screening or immunisation requests: for example, elderly people may have gone to live with other family members or moved into a care home, and young people may be registered at university or be sharing accommodation with friends – letters and other communications can be tailored accordingly to solicit the best response.

What remains after sending targeted letters in each category above probably represents people in a demographic that should still be registered with the practice. Further trials would establish the best approach (in terms of cost and accuracy) for this group: maybe it is cost-effective to write to them and remove non-responders, but if this resulted in only removing a small number, some of these wrongly, maybe it is not worth mailing them.

The bottom line is that well-established Data Quality practices of automatic suppression and change of address, allied with smart targeting, can reduce the costs of the exercise and will make sure that the NHS doesn’t penalise healthy people simply for… being healthy!

Driving Business with a Real-Time Single Customer View

Since we blogged about the challenges we overcame to deliver a Single Customer View for a major retailer a few years ago, we’ve found a lot of the same challenges repeated across other industry sectors such as non-profit, financial services and education, as well as marketing services providers managing marketing databases for companies in many different sectors. So if that’s more of the same, what’s different? In a word, time. It’s no longer good enough to have a Single Customer View that is only up to date every night, it should be up to date as quickly as your customer switches from one device to another – that is, in real time.

What are the benefits of a real-time Single Customer View?

 

Let’s stick with the multi-channel retail example both for continuity and because increasingly any product can be viewed through the eyes of a shopper, whether it is a scarf, a phone, a take-out meal, an insurance policy or a credit card account. It is widely recognized that the key to success in retail is a positive customer experience, so let’s look at some research findings:

To illustrate, if a customer orders online using their home computer for collection in store, and then after leaving home they want to change the order (using the browser on their phone or by calling the central customer service line), they expect the vendor to have the latest information about the order immediately available – otherwise, the potential for customer disenchantment is spelt out in the JDA research quoted above. If the info is all up to date, the new visit/call from the customer is an opportunity for the vendor to pitch an additional purchase, based on a 360° view of the customer’s account.

So how can you deliver a real time Single Customer View?

 

To answer this question, we first need to review where the moving data that we discussed before is coming from: keeping with the multi-channel retail example, it’s from Point-of-Sale systems in store, customers entering orders on the web site and call center operatives entering and looking up orders. These may be feeding into multiple corporate databases (ERP, Accounts, different subsidiary businesses etc.)  The challenge is: how do we perform the standardization, verification and matching that is required, classify misleading data etc. all on the fly, given that there can be as many as a dozen different data flows to handle? And how do we do all this quickly enough to ensure that the operator always has a current and complete view of the customer?

The key to meeting the challenge posed by the need for a real time Single Customer View is to accept that traditional disk-based database technology is too slow – we can’t afford the time to write a dozen or more transactions to disk, standardize and link all these by writing more records to disk and then read it all back from various disks to give the information to the operator – we can’t expect them to have a coffee break between every transaction!

To us the answer was obvious – all the data needs to be kept in computer memory, updated in memory and read back from memory, so getting away from the limitations placed by conventional hard disks and even solid state disks. But, you may say, that’s fine for small volumes of data but what if we’re streaming thousands of transactions a minute into databases with tens (or even hundreds) of millions of customers? The good news is that computer memory is so cheap these days that it’s extremely cost-effective to provision enough memory to handle even a billion customer accounts, with failover to a mirror of the data in the event of a problem.

Now it’s all very well to say “just use a lot of memory”, but can you find software that will run on all the different varieties of hardware, server technology and database systems that make up the corporate data sets? And will this software allow for the different kinds of error and discrepancy that arise when people enter name, company name, mailing address, email and multiple phone numbers? Even more challenging, will it allow for misleading data such as in store purchases being entered using a store address as the customer address, or a customer giving their partner’s phone number along with their own name and email address?

Once you’ve successfully managed to process the data real-time, you can begin to organize, understand and make use of it in real-time. To use the retail example one final time, now you can take the call from the customer on their way to collect their order and (by finding the order linked to their mobile number) enable them easily to add an item they’ve forgotten plus another item prompted by their purchase history. If the branch near home doesn’t have all the items in stock, you can direct them to the branch which does have the stock near their office – based on an up to date work address linked to the customer. With a real-time, 360° Single Customer View, it’s easy!

New Years Resolutions

2016 New Year’s Resolutions – The Stats

The New Year has arrived, and with January 1st comes the obligatory New Year’s Resolutions.  Almost 50 percent of adults consistently make these resolutions, according to Statistics Brain Research Institute.  Which is surprising considering that statistically we are more likely to fail than succeed.  Yet human perseverance prevails each year as people vow to change their lives in every way from getting into shape to falling in love.

Because we love data, we were especially interested in these numbers, as well as those found in the new study released by the University of Scranton stating that only 1 in 8 people achieve their New Year’s Resolutions.  It sounds much worse than it is.  Consider some other odds:

  • Odds of being audited by the IRS:  1 in 175
  • Odds of finding a pearl in an oyster:  1 in 12,000
  • Odds of dating a supermodel:  1 in 88,000
  • Odds of becoming a billionaire:  1 in 7,000,000
  • Odds of winning $1000 in the McDonald’s Monopoly game:  1 in 36,950,005 (that’s a lot of Big Macs!)

So all odds considered, keeping your New Year’s Resolutions seems very doable.  At least we think so.  And the study goes on to say that people who make resolutions are 10 times more likely to change their lives than those who don’t.  Which means that succeed or fail, trying is half the battle.

To help you kick off the New Year, helpIT systems made a few resolutions of our own to inspire our colleagues and ourselves.  We want to prioritize database quality management in 2016 for a more profitable and productive New Year.  Statistically, 1 in 8 of you will take our challenge.  Will you be that one?

New Year’s Resolution #1:  Getting Organized

Getting Organized is the second most popular New Year’s Resolution, right after shaping back up to pre-holiday jeans size.  The average office employee spends 1.5 hours per day (6 hours a week) looking for things!  According to the authors of Book of Odds, From Lightning Strikes to Love at First Sight, men at home are constantly looking for clean socks, remote control, wedding album, car keys (guilty!), and driver’s license.  Women are always on the hunt for their favorite shoes, a child’s toy, wallet, lipstick, and the remote control.

Let contact data records be the one thing you are not looking for this year.  If we spend that many hours looking for the remote control, imagine how many hours of productivity are lost each year by employees sifting through CRMs for contact data.  “Dirty data” not only sucks hours of productivity out of your day, it will also affect the success of marketing efforts, the sales process, and the bottom line.

Getting organized in your contact database is the first step for a #CleanData2016.  Since we know database management can be a daunting task, like so many New Year’s Resolutions are, we at helpIT systems have 25 years experience and are here to help.

This month helpIT systems is offering a FREE analysis of your company’s contact database by one of our data quality experts.  The analysis will review the effectiveness of current data quality initiatives, pinpoint weaknesses, and run a free data deduplication and matching test on your own data.

Don’t miss out on your chance to kick the New Year off right, click here to claim your free analysis.

New Year’s Resolution #2:  Saving Money

Who wouldn’t like to have a few extra dollars or pounds in their pocket this year?  The second most popular New Year’s Resolution is to save money.  People go about this all sorts of ways, from cutting back on designer handbag purchases to taking public transit instead of their car.  But here are a few ways you might be wasting money without even realizing it:

  1. Small fees that add up:  Credit card interest, paying for speedy shipping, and ATM fees all add up over time.  So while it might seem worth it to have that Amazon purchase overnighted or to hit the ATM at a concert rather than skip out on purchasing a Dave Matthews Band t-shirt, remember that a few years ago ATM fees totaled $7 billion.  To put that in perspective, the average ATM fee is $3.  Let’s say you use an ATM twice a month…that’s THREE Dave Matthews t-shirts from Amazon…maybe four if you don’t pay for the expedited shipping.
  2. Bad habits:  nearly half of Americans consume soft drinks daily, and their fast food consumption totaled $117 billion last year.  That’s almost $400 per person!  Throw in the cost of alcohol, cigarettes, and that daily $5 mocha cappuccino grande at your local Starbucks, and these habits are adding up quick.
  3. Good habits that you’re not actually doing:  While we had the best of intentions when joining that gym, signing up for Spanish lessons, or purchasing the Daily Deal for unlimited monthly meditation sessions, how often have you used it?  Studies show that gyms sell memberships expecting only 18 percent of members to use their facilities on the regular.  Take a look at what you are paying for and ask, “Am I really using this?”
  4. Gambling:  With the American Powerball topping out at a record $1.3 billion this week, spending some of your hard earned cash on a lottery ticket might seem like a worthwhile investment.  The sight of all those zeros make normally sane people forget they have a better chance of being struck by lightning, becoming President of the United States, or being attacked by a shark.  Probably a better chance of all three of those happening at the same time before you hit those lucky numbers.
  5. Waste:  A mind-boggling 33 percent of the world’s food is thrown away each year.  The math works out to about $529 per person.  That’s a nice start towards a down payment on a car or a beach vacation.  Most households could also cut energy costs by a third if they followed recommended guidelines.

So as you are looking to save money this year, consider all the places your money is going, rather than just the obvious few.  Here in the data quality world, we see this happen all the time.  Companies know exactly how much money they are losing due to employee turnover or loss of market share.  However when it comes to how much they are losing due to poor data quality, most are in the dark.  Studies suggest that companies are losing billions of dollars each year from poor data quality.  Don’t hide behind 2015’s denials, whether it’s how much that cup of coffee is really costing you or the effects of dirty data on your organization.  It’s time for a #CleanData2016.

New Year’s Resolution #3:  Be Healthy

This is a big one.  Whether it is to get fit, join a gym, meditate, or eat better, many people focus their New Year’s resolutions on improving their health.  One healthy habit can unintentionally permeate into other behaviors, often changing many aspects of a person’s life for the better.

The tricky part is making these goals stick.  Studies have shown that on average, a person needs to maintain a behavior for 66 days before it becomes a habit.  Some behaviors are harder to change than others, meaning that the 66 day rule is just a guideline rather than an absolute.

Change is hard.  Anyone who has been on a diet or quit smoking knows this.  But the great part, the biggest relief, is that it gets easier.  In his book The One Thing, Gary Keller states that, “Success is actually a short race—a sprint fueled by discipline just long enough for habit to kick in and take over.”  Meaning that we don’t have to be this disciplined forever, we just have to do it long enough for it to become a habit.  Maybe that is 66 days.  Maybe it is 246 days.  But once it is a habit, the effort needed to keep eating veggies or meditating daily will decrease substantially.  After all, how much thought do you put into brushing your teeth in the morning or buckling your seatbelt?  Habits are sometimes done without us even realizing it!

Here are a few tips to help keep you building habits in 2016:

Track yourself.  Imagine a bowl of M&M’s was on your desk right now.  It is mid-afternoon, the sun is streaming in your window, and the to-do list does not seem to be getting any shorter as the minutes slowly tick towards 5:00.  Would you reward your hard efforts so far with one M&M?  Two?  Perhaps a handful.  After all, they’re small.

Now imagine that for every M&M you ate, you had to pull out a little journal and write “1 M&M – 25 Calories”.   Would you still eat a handful?  Probably not.  For whatever behavior you are trying to eliminate or add to your life, write it down.  Every minute, every calorie, every dollar spent.  Darren Hardy advocates for this method in his book The Compound Effect.  As the name implies, these little actions add up big over time.

Mix it up.  Everyone gets into a rut.  Dr. Frank Farley, a professor of psychological studies in education at Philadelphia’s Temple University, tells Wall Street Journal that making the same resolutions year after year can lead to boredom and failure as a result. Want to lose 20 pounds?  Try pledging to walk 3 miles every day instead.  Focusing on adding a healthy behavior rather than the end result can help you feel a sense of daily accomplishment.  Each day of completing your walking resolution will bring you closer to your underlying goal.

Let others help.  No one accomplishes anything alone.  The world’s most successful people had advisors, mentors, and colleagues in their corner that made their achievements possible.  God had Moses.  Barnum had Bailey.  Let others in on what you are trying to accomplish.  Even better, find someone who has the same goals as you so you can encourage each other.

New Year’s Resolution #4:  Stop Procrastinating

In the madness that ensues during the holidays, the calm of January often leaves many people confused.  Where did all this time come from?  And more importantly, what in the world do we do with it?  Several of you are already dreaming of a Star Wars movie marathon or the chance to conquer the next level of Angry Birds.

Yet many people are shrugging off those comfortable time-killers and resolving to make 2016 a productive year both personally and professionally.  This could mean finally training for that 10K run, spending more time with family and friends, or even chasing a passion like watercolors or writing a great novel.

Companies are stopping the procrastinations of 2015 and seeking a more effective data quality plan for the New Year.  While cleaning up millions of contact data records and stopping the influx of bad data can seem like a daunting task, it is one situation that will not improve by delaying the process.  For every year companies procrastinate, bad records are piling up in CRMs, and the effects are staggering.  Departments ranging from marketing to customer service are seeing money and time wasted due to poor data quality.

How do companies accomplish a task of this magnitude?  The same way you eat an elephant…one bite at a time.  Let us help by knocking out a few of those last year excuses:

  • I don’t have the time for a project like that.  Do you have 20 minutes?  Yes?  Twenty minutes will get you started on a free data quality analysis with one of our database experts.  Do you have 20 minutes tomorrow?  If you could spend 20 minutes of each day working on data quality, by the end of 2016 you will have put in 86 hours.  That’s almost FOUR full 24 hour days!  A lot can be done in 86 hours.
  • I don’t know where to start.  Start with those 20 minutes on the phone with one of our data quality experts.  They will talk about your data, your company’s goals, and solutions tailored for you.  While many companies sell a one-size-fits-all solution, there is no “one-size” company.  Let our knowledgeable staff build a solution that is best for your company individually.
  • We tried that last year and the problem just came back.  Maintaining data quality is a habit to be maintained, not a one-time accomplishment.  Just like eating jelly doughnuts will eliminate last year’s workout goals, dirty data will creep up on you if the correct systems are not in place.  helpIT systems offers our clients complete data solutions with long-term results rather than a few quick fixes.
  • I don’t have the money.  Sure you do.  Except you are throwing it away in wasted marketing spend and lost productivity each year.  We work with hundreds of companies that originally thought “we don’t have the money” who have since discovered that not only do they have the money for a data quality solution, they have much more.  The profits realized from clean contact databases enabled them to accomplish many other projects that had been on the back burner as well.

Don’t delay.  This is our last week of offering FREE Data Quality Analysis.  Request yours here.

 

 

Data, Hoverboards and Ashley Madison. 2015 Review

Data, Hoverboards and Ashley Madison. 2015 Review

Imagine if Marty McFly had gone back to 1985 telling the world what he saw in the real 2015, they would have locked him up in a loony bin. That people of today walk around with electronic devices smarter than anything or everything in the world had back then, in their pockets. And these devices can do almost everything, from hailing a cab, buying 3D printed ornaments from Venezuela, check Chinese stock markets or watch television.

The irony is that people of today call these devices “phones”, which is one of the less popular functions of them. Yet the possibilities of phones grow every day, largely due to all the data they collect. Data being generated and recorded of every person, making their lives easier, less congested and more connected. While we churn this data and create it by the petabyte everyday, we’re realizing that making sense of it is the new challenge – and securely maintaining and storing accurate data is essential.

At helpIT we’ll have been working with data for 25 years as of 2016, which makes us some of the few young adults in a world of newborn data companies. But before we turn 25, we thought it might be good to take a look back at 2015, our 24th year and reflect on how data has become rather newsworthy.

Headlines that worried us…

The most shocking of data scandals in 2015 was the breach of security at AshleyMadison.com, an online marketplace for married folk with a wandering eye.  The world watched in fascinated horror as the site’s 37 million members scrambled to save their marriages, careers, and reputations.  No one was exempt:  the compromised membership list even included members of the United Nations and the Vatican.  More worrisome to companies watching this event unfold was the suspicion that the leak came from a disgruntled employee.  Which is a good reminder to all of us that while protecting your database from outside threats is a priority, never overestimate the loyalties of those on the inside as well.

While the original cyber-attack on Sony Pictures Entertainment occurred in 2014, the aftershocks were still arriving in the new year.  Especially when Wikileaks made the decision to create a searchable data dump on more than 30,000 private documents from the breach.  Citing public interest, Wikileaks claimed that Sony’s influence on Washington made the inner workings of their company relevant to the general public.  Companies everywhere began beefing up their privacy settings.

Some good news…

As the data security attacks kept coming, businesses were forced to reanalyze the priority placed on data security as well as data’s importance to their organizations.  Quality database management not only keeps customers safe, but businesses now recognize that accurate contact data can be used to keep customers happy.  According to an Accenture poll, 89 percent of business leaders believe big data will revolutionize business operations in the same way the Internet did.

These same leaders are planning on pursuing big data projects in 2016 in order to seize a competitive edge.  To this effect, many businesses are recognizing that the first step is improving the quality of their contact databases in order to better serve their customers.  For consumers, this should bring a more personalized and effective e-commerce experience in the New Year.

What we were up to…

With database management taking center stage on the corporate agenda, we at helpIT systems have been hard at work to meet this rising demand.  While we have a wide range of unrivaled data quality software solutions, the year of 2015 began a shift away from individual products towards a more complete data quality solution providing customers with a one-stop shop for data quality, data matching, and data enrichment.  Our staff has been working diligently to ensure that we are ready to provide both our current and our new customers with stellar customer service and technical support that exceeds expectations.

To that end, helpIT systems rolled out a new website designed with the customer experience in mind.  Our goal was to provide more value through instructional tools and industry resources.  While data quality software and solutions are our primary focus, we strive to be a resource to those who visit www.helpit.com looking for answers about all aspects of data quality.

Expectations for 2016…

Perhaps the most pervasive change in the data industry during 2015 was the rapid growth in the “internet of things”.  While the technology has been around for years, we saw a large jump in the amount of devices created to monitor data in our personal lives.  Sales of everything from Fitbits, to monitor your health, to Nests, a home thermostat that adapts to your behaviors and the seasons, have boomed.  We expect to see more of this in 2016.

While this data will save electricity, ideally make us rethink our fitness goals, and improve home security, both companies and consumers are looking ahead to ensure this data is better protected than the troves that came before it.  Because while the concept of data has been around for a while, 90 percent of the world’s data was collected within the last two years.  Companies are learning how important it is to keep this information safe, and that is good news for everyone in 2016.
As all of this data is being collected, the feat of managing it looms before us like perfecting a true hover-board (which can actually just about hover over water).  A company can only hope to gain the competitive edge from data analysis if it possesses secure, accurate, and searchable contact data.  helpIT systems has been a partner to companies in this goal for the past 24 years and we will continue to grow and be there for 24 more.

Happy Holiday and a Prosperous New Year to all!

12 Days of Data Quality

12 Days of Data Quality

The holidays are finally here.  They always seem so far away and then, as the days grow short and temperatures fall, they tend to jump out at us in a surprise attack like a kid in a spooky costume on Halloween.  And once they are here, if you blink, they are over.  The anticipated smells of gingerbread baking in the oven, the joy of seeing a loved one open a carefully selected present, the glow of thousands of twinkling Christmas lights… all over before we were able to slow down and truly appreciate the holiday season.

So before December disappears under a pile of wrapping paper, we are inviting you to take the time to be merry, revel in the holidays, and perhaps still get a bit of work done.

Welcome to helpIT system’s 12 Days of Data Quality:

On the first Day of Data Quality, helpIT gave to me:

A Single Customer View (In a Pear Tree)

The first gift in this classic holiday carol is a Partridge in a Pear Tree.  The partridge sits alone high above the rest of the world.  Regally.  Eating pears (I imagine it eating pears) while looking down on all the lesser beings that have to see the world from ground level.

Your organization can be that partridge, sitting high above the rest.  Except in the world of database management, we are seeking a truly accurate Single Customer View, rather than a belly full of pears.  We all want the ability to look down on one contact record and obtain accurate, up-to-date information, each and every time.  Having one complete record for each customer ensures that they will receive the correct marketing materials at the correct address.  Salespeople will know a customer’s complete purchase history to analyze likely future purchases.  Customer service reps will be aware of address changes, name changes, as well as any other personal details in order to make the customer feel like they matter.  Which they do.  A lot.

Each customer in your contact database makes up a limb of your “pear tree”.  In the song, no matter how many gifts of drummers or pipers or ladies milking cows are given, it always comes back to the pear tree.  The tree is the center of everything, holding up even the partridge.  Just as your customers hold up your organization.  Make your customers feel this importance by respecting them as individuals, and as the base of your success, rather than lumping them in with the rest in your database.  The first step in doing this is by having a strong data quality solution and system in place.

On the second Day of Data Quality, helpIT gave to me:

2 Matched Records

On the second day of Christmas, my true love gave to me two turtle doves.  Which was great, in medieval times, when the doves symbolized true love’s endurance, mainly because they mated for life.  Everyone from the Bible to Shakespeare has made mention of them.

This December, give yourself another sort of true match.  Matching contact records in your database is the first step to cleaning up dirty data and obtaining a Single Customer View.  And there is no one-size-fits-all solution.

The important thing to consider when matching and deduping your database is the methodology used in the process.  Some software only matches exact records, so ‘John Smith’ and ‘John Smith’ would show up as a duplicate.  However, ‘John Smith’ and ‘Jon Smith’ would not.  So if you want a truly accurate database, you have to employ a more sophisticated method.

helpIT system’s matching software compares all the datasets in one contact record against the rest of your database.  John Smith’s address, birthday, phone number, email, or whatever other datapoints you use are all taken into consideration when pinpointing matches.  This process often picks up 20-80 percent more matches than other software.  When you multiply that by 200 million records, that’s a lot of matches.

The biggest mistake organizations make when matching records is to view it as a “one and done” solution.  Data matching, like any long-term relationship, is something that must be constantly tweaked, adapted, and carried out on a regular basis.  Although as the turtle doves can attest, this type of devotion does come with big rewards.

On the third Day of Data Quality, helpIT gave to me:

 

3 Frenchmen

 

Rather than the French Hens in the traditional song, let us meet a Frenchman whose name is Dr. Mathieu Arment. He loves to purchase designer scarfs from your company, Parisian Scarves. During his first online purchase, he entered his information as follows:

Matheiu Arment
27 Rue Pasteur
14390 Cabourg
FRANCE

The Parisian Scarves marketing department then sends him a catalogue for the Spring Collection. He flips through it while sitting at a local café sipping a latte and finds a handsome purple plaid scarf that he absolutely must have, but he has forgotten his laptop. So he calls in the order. The Parisian Scarves customer service rep does not see an account under the name she types in, Mattheiu Armond, so she creates a new account record and places his order.

Later, a second customer service rep is handling an issue with his order and decides to send a coupon as a gift for all of his trouble. The coupon is sent to Mathis Amiot. And the rep slightly misspelled his address on Rue Pasteur. Upon receiving the misguided coupon as well as two of the same catalogue addressed to slight variations of his name, Dr. Arment realizes that he is not just one Frenchman, but rather 3 separate Frenchmen in the eyes of Parisian Scarves. Feeling annoyed and undervalued that his favorite scarf company cannot even spell his name correctly, not to mention they also forgot his birthday, Dr. Arment takes his scarf shopping to another business who appreciates him as an individual.

Not all data matching software is created equal. While some compares only exact matches, helpIT system’s unique phonetics matching system will pull out similar sounding pieces of data as well as similar spellings. This will create a higher match rate, allowing for less duplicates to slip through into your database.

In this instance, the Parisian Scarves customer service rep would have typed in Mattheiu Armond, only to have the record Matheiu Arment show up as a possible match. She would have noted the similar addresses and concurred, correctly, that these were the same customer.

Accurate data matching creates a data quality firewall, preventing bad data from entering the system at point-of-entry, as well as filtering it out on a regularly scheduled check-up. So Dr. Arment can stay one Frenchman, and more importantly, he will stay a customer of Parisian Scarves.

On the fourth Day of Data Quality, helpIT gave to me:

 

4 Calling Salespeople

 

Sales is a unique industry in which every minute can translate into profits, if that minute is spent efficiently and effectively. Salespeople are constantly seeking better ways of doing things in order to increase your company’s profits, as well as their commissions. Which means every minute wasted clicking through the CRM, either in search of leads or trying to obtain accurate client data, is valuable time lost. Every phone call they make is either costing you money or making you money. What decides whether a sales team is a drain or an asset? The quality of the leads they are contacting.

A CRM that has effective data quality measures in place is filled with accurate contact records. These records can be analyzed to obtain valuable information by all arms of your organization, especially the sales department.

A good salesperson can use CRM data to offer the right products to the right potential buyers, as well as dedicate more time to leads that are statistically more likely to turn into sales. They will be able to quickly obtain the correct point-of-contact and contact information without fishing through multiple records for the same lead. A salesperson will also look more knowledgeable as they are able to talk easily with a client about their business needs.

Give your salespeople the resources they need to be a profitable addition to your company by having an accurate, up-to-date CRM.

On the fifth Day of Data Quality, helpIT gave to me:

 

5 Golden Reasons to Trial matchIT SQL

 

helpIT’s ‘12 Days of Data Quality’ continues with 5 Golden Reasons to Trial matchIT SQL. Perhaps not the golden rings the lady received in the song, but really, who needs five golden rings? Sounds like a pickpocket’s dream come true. So instead, we here at helpIT are presenting you with five reasons to try our matching system.

We hope by now that you are starting to understand how important a strong data quality management system is to the success of your organization. It can increase profits and productivity in all arms of your organization. Yet sometimes it is hard to get the ball rolling, especially if you have a lot of chiefs who are part of this decision. So consider these five reasons why a helpIT systems trial is a good place to start:

1. Quick Installation. Be processing data in less than an hour!
2. Run data cleansing processes on your own data in your own environment (even address validation).
3. Customize the matching process and fine-tune your results with dedicated Trial Customer Support.
4. Run large volumes of data to see real performance results.
5. Get the real-world examples you need to justify your business case for SQL data.

This holiday season, try matchIT SQL for 30 days for absolutely nothing! We know you will love it, but if you don’t, we will give you 5 golden rings. Or maybe just one. Or a thank-you email. Yes, if you don’t love it, we will send you an email thanking you for your time. Happy trialing!

On the sixth Day of Data Quality, helpIT gave to me:

 

6 Companies a-Laying

 

Our countdown to Christmas and better data quality measures continues! In the song, his sweetheart received 6 geese laying eggs. Which might get some odd looks around the office. Instead, consider the importance of laying a strong foundation when beginning your quest for clean data.

All geese lay eggs. But the goose that laid the golden egg got a lot more attention than the rest. Like that golden-egg laying goose, the company that lays the strongest foundation in regards to data quality will garner the most attention and achieve the best results.

Most organizations think of clearing out dirty data as something to be dealt with when absolutely necessary. When in reality, database maintenance is a process that should be consistently tweaked, monitored, and exercised. Contact data is constantly entering your system. Contacts are frequently relocating, changing names, or passing away. Which means a good database administrator is diligent in tracking these changes.

Laying the foundation for strong data quality measures is often labeled too time consuming to be dealt with. But the time invested originally will pay off in piles of golden eggs in the future.

On the seventh Day of Data Quality, helpIT gave to me:

 

7 Sales a-Swimming

 

Or rather, floundering. Whether you want to admit it or not, the odds are you are floundering in bad data, working hard just to stay afloat of the changes that occur in your contact database on a daily basis. Each sale relies on every member of your team being able to swim seamlessly through the CRM to obtain the information they need to make a client feel valued and understood.

Companies today report data analysis as one of the most effective tools for developing marketing campaigns and targeting sales leads. Many organizations use data analysis on a daily basis. However, if they are analyzing inaccurate or out-of-date data, the analysis is all but pointless. A database that does not have systems in place for catching bad data at point-of-entry, as well as a regular cleansing schedule, is a hindrance rather than a help in regards to data analysis.

This holiday season, give your data analysis the gift of a life raft. Make sure your team is swimming, rather than floundering, in the sea of contact data. Accurate data analysis will increase marketing effectiveness, reduce marketing spend, and increase productivity in all aspects of your business that work in the CRM.

On the eighth Day of Data Quality, helpIT gave to me:

 

8 Maids E-Mailing

 

While your business is probably not made up of maids, it does most likely contain many people that rely on email communications on a day-to-day basis. Email is an important means to reach prospects, current customers, and vendors. How these messages are delivered, as well as the content in them, is a strong reflection on the quality of your business model.

Do the emails look polished and professional? Or lazy and sloppy? Most organizations unintentionally accomplish the latter. A lack of data quality management systems has caused incorrect contact information to reside in their database. So Joe Smith gets an email addressed to Jo Smith. Or Jo Smith becomes a Mrs. instead of a Mr. And that’s all assuming that the email is even delivered.

Email deliverability is a key concern to many businesses, especially in regards to marketing. A great marketing campaign is irrelevant if the message is not received by the intended recipient. New email addresses are often mistyped. Another possibility is that a wrong address was given intentionally. Either way, the organization has lost a sales lead because the incorrect address is not reachable.

Email validation is a valuable and effective piece of the data quality puzzle. It will greatly increase the number of leads passed onwards to your sales team as well as ensure that marketing communications arrive to the person they were intended for. It is easy to implement, and the rewards far outweigh the costs.

On the ninth Day of Data Quality, helpIT gave to me:

 

9 Ladies Dating

 

One of the biggest challenges in your database can come from name changes. Sometimes it is from 9 ladies dating and then deciding to tie the knot. And while marriage is normally considered a wonderfully celebrated occasion, to the database administrator it means the possibility of error. Because it is almost a certainty that Ms. Smith is not calling her 17 magazine subscriptions from her honeymoon to let them know she married Mr. Clark and moved into his duplex in the Heights.

The new Mrs. Clark is a valued customer. So treat her as such by recognizing these changes as quickly as possible. Name changes and new addresses are easily dealt with when you have a proper data quality system in place. Stay tuned for tomorrow’s blog for some tips on keeping up with Mrs. Clark.

On the tenth Day of Data Quality, helpIT gave to me:

 

10 Lords a-Moving

 

The original Lords from the song might be a-leaping, but most of your customers getting around via UHaul trucks and airplanes. They are leaping across town, across the state, and sometimes, across the world. In an average year over 40 million people move. Keeping up with them can seem even harder than remembering the words to the 12 Days of Christmas.

Keep your contact database accurate and up-to-date with National Change of Address (NCOA). In one easy process your current contact address data is compared to USPS CASS and DPV certified data, correcting any typing errors and appending additional information.

On the eleventh Day of Data Quality, helpIT gave to me:

 

11 Pipers Piping

 

The Pied Piper was a character in German folklore who tried to sway a town to pay him to rid their village of rats. His pipe music would lure the rats out of hiding and they would follow him out of town. When the villagers refused to pay for this service, he piped away their children instead. Not the noblest use of his talents, but the ability to lead others is a powerful trait nonetheless.

Be the pied piper at your organization, only use your powers for good instead of evil. Make 2016 the year your organization makes data quality solutions a priority and others will be glad they followed you. Often the only thing holding a company back from reducing the costs of bad data is the knowledge and the leadership to move forwards. helpIT systems offers a full range of customer support solutions so that you and your company can feel confident about your next move.

On the twelfth Day of Data Quality, helpIT gave to me:

 

12 DBAs Drumming

 

More often than not, the squeaky wheel gets the grease. The loudest drummers in your office this season should be those making noise about the importance of data deduplication. Having an improperly deduped database can create upwards of 60 percent of dirty data in your contact database.

Those incorrect contacts are receiving marketing materials (which cost money), taking up manpower to organize and sift through in the CRM (which costs time), and getting calls from your sales people (which cost money and time).

This month alone I have received mail for 4 different past residents of my current apartment. You know what I do with it? I throw it away. So Horace will never get that credit card offer. Zachary will not be donating to the Salesian Missions. And Monique will not be showing up in court for her fifth and final notice to appear. (Feeling a little guilty about that last one.)

We hope you enjoyed our unique spin on the traditional “12 Days of Christmas”. While the holidays are nearing an end, helpIT systems is here to answer your data quality questions 365 days a year. We hope to make 2016 your best data quality year ever. Give us a call or visit our website at www.helpit.com to find out more information.

Who is the “Current Resident”, anyway?

Our CEO received an extremely heavy, expensive-looking catalog in the mail the other day, from an upmarket retailer and addressed to a previous occupant of his house “Or Current Resident”. When you receive catalogs in the mail that are addressed to the previous homeowner or the “current resident”, do you read them or toss them? Obviously the company hopes that anyone who receives a catalog at this address will more than likely take a gander at what’s being offered.
But is this a cost-effective supposition? When you consider the resources wasted on shipping a catalog to anyone that lives at a particular address, you have to wonder whether this is a smart strategy or just a cop out from cleansing a database.

Using address verification software which includes the National Change of Address (NOCA) service would help catalog senders increase their return on investment by updating their databases as frequently as needed. The NCOA service would ensure that databases are updated with the customer’s current address information, or warn of deceased or moved customers who did not give a forwarding address.

NCOA relies on the customers filling out a Change of Address form, and the USPS internal databases which keep track of customer information, which it then relays to the NCOA service. Rather than make use of NCOA data, many companies add “Or Current Resident” to the name from their databases, as the most timely and least expensive method of allowing that the addressee may no longer be there.

Set against the convenience of this tactic, these factors should also be considered:

  • The expense of shipping items to an old address
  • The much reduced chance of the new resident making a purchase
  • Losing track of a past customer
  • Alienating the new mail recipient

But does the NCOA process take that much time, or add to the expense of the mailing? Well, the answer is “no” on both counts! A whole spectrum of NCOA options is available, from desktop software that can be used by marketers, and software integrated into the corporate database (both contacting an NCOA service under the hood) to online bureaus who take your data and return the updated file a few hours later. The cost depends on data volumes, but even if you only have a few thousand records in your mailing file, you can always find an option that saves you money compared with print and mail costs – especially if your catalog is bulky.

Sending catalogs to the “current resident” might sound like easy advertising, but it doesn’t deliver return on investment for the costs of printing and mailing and it doesn’t help your brand. It really is easy and much smarter to keep track of customers with NCOA services, stop shipments to non-existent customers and even save money to reinvest in other positive, more effective marketing efforts.

Where Big Data, Contact Data and Data Quality come together

We’ve been working in an area of untapped potential for Big Data for the last couple of years, which can best be summed up by the phrase “Contact Big Data Quality”. It doesn’t exactly roll off the tongue, so we’ll probably have to create yet another acronym, CBDQ… What do we mean by this? Well, our thought process started when we wondered exactly what people mean when they use the phrase “Big Data” and what, if anything, companies are doing in that arena. The more we looked into it, the more we concluded that although there are many different interpretations of “Big Data”, the one thing that underpins all of them is the need for new techniques to enable enhanced knowledge and decision making. I think the challenges are best summed up by the Forrester definition:

“Big Data is the frontier of a firm’s ability to store, process, and access (SPA) all the data it needs to operate effectively, make decisions, reduce risks, and serve customers. To remember the pragmatic definition of Big Data, think SPA — the three questions of Big Data:

  • Store. Can you capture and store the data?
  • Process. Can you cleanse, enrich, and analyze the data?
  • Access. Can you retrieve, search, integrate, and visualize the data?”

http://blogs.forrester.com/mike_gualtieri/12-12-05-the_pragmatic_definition_of_big_data

As part of our research, we sponsored a study by The Information Difference (available here) which answered such questions as:

  • how many companies have actually implemented Big Data technologies, and in what areas
  • how much money  and effort are organisations investing in it
  • what areas of the business are driving investment
  • what benefits are they seeing
  • what data volumes are being handled

We concluded that plenty of technology is available to Store and Access Big Data, and many of the tools that provide Access also Analyze the data – but there is a dearth of solutions to  Cleanse and Enrich Big Data, at least in terms of contact data which is where we focus. There are two key hurdles to overcome:

  1. Understanding the contact attributes in the data i.e. being able to parse, match and link contact information. If you can do this, you can cleanse contact data (remove duplication, correct and standardize information) and enrich it by adding attributes from reference data files (e.g. voter rolls, profiling sources, business information).
  2. Being able to do this for very high volumes of data spread across multiple database platforms.

The first of these should be addressed by standard data cleansing tools, but most of these only work well on structured data, maybe even requiring data of a uniform standard – and Big Data, by definition, will contain plenty of unstructured data which is of widely varying standards and degrees of completeness. At helpIT systems, we’ve always developed software that doesn’t expect data to be well structured and doesn’t rely on data being complete before we can work with it, so we’re already in pretty good shape for clearing this hurdle – although semantic annotation of Big Data is more akin to a journey than a destination!

The second hurdle is the one that we have been focused on for the last couple of years and we believe that we’ve now got the answer – using in-memory processing for our proven parsing/matching engine, to achieve super-fast and scalable performance on data from any source. Our new product, matchIT Hub will be launching later this month, and we’re all very excited by the potential it has not just for Big Data exploitation, but also for:

  • increasing the number of matches that can safely be automated in enterprise Data Quality applications, and
  • providing matching results across the enterprise that are always available and up-to-date.

In the next post, I’ll write about the potential of in-memory matching coupled with readily available ETL tools.