Garbage in, garbage out. That’s a top ten comment from data professionals. Why bother spending time extracting data if you know it wasn’t captured well or organized correctly in the first place?
At TCB Analytics, we believe the exercise of working with data is a valuable business process even if the data itself isn’t tested for quality. Yes, you read that correctly. The data itself may be “crap”, and you may know it’s “crap” ahead of time, but if you want to figure out how to make it better, then you should take time to visualize it.
Here’s a good example of what we mean.
Recently, we were speaking with a prospect who wanted to gauge customer interest in a product portfolio. We asked for an extract from their CRM system or some web logs so we could analyze customer requests. He was reluctant to provide the data, saying it wasn’t valuable because the data was incomplete and not descriptive enough to tell us anything. Eventually he gave us a subset of the data so we could visualize and better understand the data sources.
The data was indeed incomplete. Product names were spelled incorrectly. Some products weren’t in the CRM system at all. The data was dirty. Lots of duplicates and variations of data that needed to be grouped. Normal bad data stuff. The web logs weren’t any better. Since the website wasn’t optimized, the search and user data was sparse. Product page hits didn’t give us SKU-level data, and there was no integration between the shopping cart and payment vendor, so there was no visibility into that portion of the transaction.
You could imagine when we went back to the client that they would say, “see, that’s what you get for working with bad data”. But they didn’t say that at all. They were amazed that organizing and visualizing this bad data showed how quickly and effectively they could fill technology gaps and optimize organizational processes.
Through our data work, the CRM team recommended a new universal product SKU taxonomy. The website map was cleaned up and now captured better click stream behavior. They hired an intern to properly set up Google Analytics. All these recommendations came from working with “crap” data. However in this context, “crap” was less about the data and more the attitude towards capturing and storing information. A quick data visualization project can actually turn crap into strategic business improvements.
•