Data, Hoverboards and Ashley Madison. 2015 Review

Data, Hoverboards and Ashley Madison. 2015 Review

Imagine if Marty McFly had gone back to 1985 telling the world what he saw in the real 2015, they would have locked him up in a loony bin. That people of today walk around with electronic devices smarter than anything or everything in the world had back then, in their pockets. And these devices can do almost everything, from hailing a cab, buying 3D printed ornaments from Venezuela, check Chinese stock markets or watch television.

The irony is that people of today call these devices “phones”, which is one of the less popular functions of them. Yet the possibilities of phones grow every day, largely due to all the data they collect. Data being generated and recorded of every person, making their lives easier, less congested and more connected. While we churn this data and create it by the petabyte everyday, we’re realizing that making sense of it is the new challenge – and securely maintaining and storing accurate data is essential.

At helpIT we’ll have been working with data for 25 years as of 2016, which makes us some of the few young adults in a world of newborn data companies. But before we turn 25, we thought it might be good to take a look back at 2015, our 24th year and reflect on how data has become rather newsworthy.

Headlines that worried us…

The most shocking of data scandals in 2015 was the breach of security at AshleyMadison.com, an online marketplace for married folk with a wandering eye.  The world watched in fascinated horror as the site’s 37 million members scrambled to save their marriages, careers, and reputations.  No one was exempt:  the compromised membership list even included members of the United Nations and the Vatican.  More worrisome to companies watching this event unfold was the suspicion that the leak came from a disgruntled employee.  Which is a good reminder to all of us that while protecting your database from outside threats is a priority, never overestimate the loyalties of those on the inside as well.

While the original cyber-attack on Sony Pictures Entertainment occurred in 2014, the aftershocks were still arriving in the new year.  Especially when Wikileaks made the decision to create a searchable data dump on more than 30,000 private documents from the breach.  Citing public interest, Wikileaks claimed that Sony’s influence on Washington made the inner workings of their company relevant to the general public.  Companies everywhere began beefing up their privacy settings.

Some good news…

As the data security attacks kept coming, businesses were forced to reanalyze the priority placed on data security as well as data’s importance to their organizations.  Quality database management not only keeps customers safe, but businesses now recognize that accurate contact data can be used to keep customers happy.  According to an Accenture poll, 89 percent of business leaders believe big data will revolutionize business operations in the same way the Internet did.

These same leaders are planning on pursuing big data projects in 2016 in order to seize a competitive edge.  To this effect, many businesses are recognizing that the first step is improving the quality of their contact databases in order to better serve their customers.  For consumers, this should bring a more personalized and effective e-commerce experience in the New Year.

What we were up to…

With database management taking center stage on the corporate agenda, we at helpIT systems have been hard at work to meet this rising demand.  While we have a wide range of unrivaled data quality software solutions, the year of 2015 began a shift away from individual products towards a more complete data quality solution providing customers with a one-stop shop for data quality, data matching, and data enrichment.  Our staff has been working diligently to ensure that we are ready to provide both our current and our new customers with stellar customer service and technical support that exceeds expectations.

To that end, helpIT systems rolled out a new website designed with the customer experience in mind.  Our goal was to provide more value through instructional tools and industry resources.  While data quality software and solutions are our primary focus, we strive to be a resource to those who visit www.helpit.com looking for answers about all aspects of data quality.

Expectations for 2016…

Perhaps the most pervasive change in the data industry during 2015 was the rapid growth in the “internet of things”.  While the technology has been around for years, we saw a large jump in the amount of devices created to monitor data in our personal lives.  Sales of everything from Fitbits, to monitor your health, to Nests, a home thermostat that adapts to your behaviors and the seasons, have boomed.  We expect to see more of this in 2016.

While this data will save electricity, ideally make us rethink our fitness goals, and improve home security, both companies and consumers are looking ahead to ensure this data is better protected than the troves that came before it.  Because while the concept of data has been around for a while, 90 percent of the world’s data was collected within the last two years.  Companies are learning how important it is to keep this information safe, and that is good news for everyone in 2016.
As all of this data is being collected, the feat of managing it looms before us like perfecting a true hover-board (which can actually just about hover over water).  A company can only hope to gain the competitive edge from data analysis if it possesses secure, accurate, and searchable contact data.  helpIT systems has been a partner to companies in this goal for the past 24 years and we will continue to grow and be there for 24 more.

Happy Holiday and a Prosperous New Year to all!

Weighing up the Cost of Bad Data image

Weighing up the Cost of Bad Data

In a recent survey conducted by helpIT systems, almost 25 percent of respondents cited finances as the biggest hindrance to maintaining superior contact databases.  We get it.  Data quality solutions can carry what may seem to be a hefty pricetag, and they won’t show up two days later in a nicely wrapped package like an Amazon Prime purchase.  As such, like any other expensive and complicated decision, data quality may well get pushed to the bottom of the pile.

Then again, just like going to the gym or eating salad instead of steak, the toughest behaviors to adapt are usually the most beneficial.  Because even though database management may be something we’d rather forget about, 40 percent of those same respondents stated that their companies were losing tens of thousands of dollars each year due to poor contact data quality.  So while the solution may not be cheap and easy, the cost of living without it does not appear to be either.  Data Warehousing Institute found that the cost of bad data to US businesses is more than $600 billion each year.  Is that a number your company can afford to ignore?

Many businesses do notice these dollars disappearing and choose to do something about it.  Unfortunately however, this is often simply a “quick fix”.  They look at their messy databases, pay someone to “clean them up”, and then everyone gets a pat on the back for a job well done.  And it is.  Until someone enters a new record in the CRM, a customer moves, or perhaps even dares to get a new phone number.  And I will shock everyone by reporting that this happens all the time.  Studies indicate up to a 2 percent degradation each month…even in a perfect database.

Right now you’re probably picking up on the fact that maintaining good data is going to cost money.  You’re right.  But the fact is, avoiding that cost is only going to cost more in the long run.  Just like having a well-trained sales team, a finely-targeted marketing plan, or a boss with years of experience…great results are an investment of time and resources rather than a happy accident.

Companies that choose to invest in good data quality, as well as to view it as an ongoing process rather than a simple one-time fix, are finding that the benefits by far outweigh the initial costs.  Advertising dollars are reaching their intended audiences and sales calls are reaching the right recipient, with customer satisfaction going through the roof.  Today’s consumer expects the personal touches that can only come from having an accurate and up-to-date Single Customer View, and it is good data quality solutions that will achieve them.

helpIT logo blue background header

How Ashley Madison Can Inspire Your Business

As each new name and every illicit detail is revealed, the 37 million members of Ashley Madison, a website promoting extramarital affairs, are scrambling to save their marriages, careers, and reputations.  This list, which is now available to anyone aware ofthe existence of Google, reportedly includes the names and sexual fantasies of members of the armed services, United Nations, and even the Vatican.  Looks like someone’s prayers weren’t heard this week.

As the extent of the contact information becomes more easily accessible, a new breed of data analyst is emerging.  Creative thinkers are using the information to win custody battles, deduce which cities have the most cheaters, and even get a leg up over another candidate for a job promotion.

If everyone from neglected housewives to tawdry tabloid writers is capable of using data to form opinions and make well-informed decisions, the question is… why aren’t you?

Now I’m not talking about crawling through Ashley Madison’s troves of cheaters, I’m talking about your company.  Your data.  Demographics, geographic locations, purchasing behavior… your contact records say a million things about your customers.  A million patterns are lying in wait, holding the key to better marketing, better operations, and better business decisions.  Whereas for Ashley Madison data spelled disaster, for you it should spell potential.

Customer data, when compromised, can be a company’s worst nightmare.  When used intelligently, customer data can increase profits and reduce the guessing game so many businesses play on a day-to-day basis.

In order to use your data intelligently, you must be confident that it is accurate and up-to-date.  If your records indicate you have 14 Jeremiah Whittinglys living in Chicago, you can either double your production of Jeremiah Whittingly personalized baseball caps, or perhaps take a closer look at how clean your data is.  I’m personally leaning towards the second option.

However, beefing up marketing efforts in Juneau, where your database says 10 percent of your client base is located, is a smart idea.  Unless your data entry employee didn’t realize ‘AK’ was the postal code abbreviation for Alaska rather than Arkansas.  In which case, polar bears stand a better chance of appreciating your new billboard than your target market.

Ridding your database of duplicate, incorrect, or incomplete records is the first step in recognizing the power of customer data.  The next step is figuring out what this data means for you and your company, and if every talk show host and dark web hacker can do it with the right tools, so can you.

Where Big Data, Contact Data and Data Quality come together

We’ve been working in an area of untapped potential for Big Data for the last couple of years, which can best be summed up by the phrase “Contact Big Data Quality”. It doesn’t exactly roll off the tongue, so we’ll probably have to create yet another acronym, CBDQ… What do we mean by this? Well, our thought process started when we wondered exactly what people mean when they use the phrase “Big Data” and what, if anything, companies are doing in that arena. The more we looked into it, the more we concluded that although there are many different interpretations of “Big Data”, the one thing that underpins all of them is the need for new techniques to enable enhanced knowledge and decision making. I think the challenges are best summed up by the Forrester definition:

“Big Data is the frontier of a firm’s ability to store, process, and access (SPA) all the data it needs to operate effectively, make decisions, reduce risks, and serve customers. To remember the pragmatic definition of Big Data, think SPA — the three questions of Big Data:

  • Store. Can you capture and store the data?
  • Process. Can you cleanse, enrich, and analyze the data?
  • Access. Can you retrieve, search, integrate, and visualize the data?”

http://blogs.forrester.com/mike_gualtieri/12-12-05-the_pragmatic_definition_of_big_data

As part of our research, we sponsored a study by The Information Difference (available here) which answered such questions as:

  • how many companies have actually implemented Big Data technologies, and in what areas
  • how much money  and effort are organisations investing in it
  • what areas of the business are driving investment
  • what benefits are they seeing
  • what data volumes are being handled

We concluded that plenty of technology is available to Store and Access Big Data, and many of the tools that provide Access also Analyze the data – but there is a dearth of solutions to  Cleanse and Enrich Big Data, at least in terms of contact data which is where we focus. There are two key hurdles to overcome:

  1. Understanding the contact attributes in the data i.e. being able to parse, match and link contact information. If you can do this, you can cleanse contact data (remove duplication, correct and standardize information) and enrich it by adding attributes from reference data files (e.g. voter rolls, profiling sources, business information).
  2. Being able to do this for very high volumes of data spread across multiple database platforms.

The first of these should be addressed by standard data cleansing tools, but most of these only work well on structured data, maybe even requiring data of a uniform standard – and Big Data, by definition, will contain plenty of unstructured data which is of widely varying standards and degrees of completeness. At helpIT systems, we’ve always developed software that doesn’t expect data to be well structured and doesn’t rely on data being complete before we can work with it, so we’re already in pretty good shape for clearing this hurdle – although semantic annotation of Big Data is more akin to a journey than a destination!

The second hurdle is the one that we have been focused on for the last couple of years and we believe that we’ve now got the answer – using in-memory processing for our proven parsing/matching engine, to achieve super-fast and scalable performance on data from any source. Our new product, matchIT Hub will be launching later this month, and we’re all very excited by the potential it has not just for Big Data exploitation, but also for:

  • increasing the number of matches that can safely be automated in enterprise Data Quality applications, and
  • providing matching results across the enterprise that are always available and up-to-date.

In the next post, I’ll write about the potential of in-memory matching coupled with readily available ETL tools.