Data Addiction

My name is Austin and I am an addict.

When I wake up in the morning and my mind recognizes controlled consciousness, I uncontrollably think about what answers I’ll be looking for in data today. Most of these answers don’t have a companion question to go with them yet – they spring from my current understanding of the data and a sudden realization that another nugget of insight can be gleaned from it. The question to the answer usually follows in the thought process, but not always so quickly. I wonder what additional pieces of data I’ll need to finalize the answer and make it presentable and if I have access to it. Then, I open my eyes and try to remember what day of the week it is. Finally, I get out of bed.

Through my morning routine, I try paying attention to my wife and 3 kids as we all prepare for the day. I fight the urge to continue my thought process that began the day. When I get to the office, I hope the first few hours are free from meetings so I have the time to formulate my morning thoughts. Data consumes my life - I am addicted to it.

To succeed today, a company must also be addicted to data. They need data scientists, DBAs, analysts, and other flavors of data gurus to effectively run in their given market. Unfortunately, most companies don’t know what they have and ironically, they always want more. That’s like not knowing what you have in your lunch box and going through the cafeteria line to get more food. You grab your tray and tell the gourmet chef you want more food.

What’ll you have?

Everything!

Do you want a burger? A sandwich? Soup?

I don’t know what those things are, but I’ll have all of them and everything else on the menu. Oh, and I don’t want just for me, I want ALL the food you have behind this counter!

That sounds silly in the cafeteria, but that’s exactly what companies are doing with data. “What they are doing is making themselves dysfunctional by taking all this data.” –WSJ William Binney, retired NSA.

Too much raw data deteriorates constructive analysis. Check out this article in the Huffington Post.

Usually the call to "store everything" or "we want it all" comes from a leader that doesn't really know what is available or what they actually need or what can be stored. It's their silent cry for help because they really haven't formed there thoughts yet. It's usually a call from the business leader who doesn't know yet what they need or what IT can provide or it comes from the IT leader who doesn't know what the business really needs and is worried the business will introduce scope creep later on.

Luckily for me, I'm also the DBA as well as the team lead of our small engineering team. If I don't think we need to store the data, we don't. If we haven't used data we are currently storing, we archive it. I know most aren't lucky enough to be close enough to the business to be the analyst and close enough to IT to be the DBA as well, but that's where I find myself and that's where I think most analysts should strive to be.

You see, I am a data addict, but I'm also a business analyst - I understand what the business needs before they do and I've also learned the pitfalls of gathering too much data. I think companies will be much better at managing data...or managing the data-lunch-line a lot better if their IT shops were more intimately involved with the business.

Ask yourself, are you a data analyst? Are you familiar enough with a database to write queries and manage a few tables? you may want to consider influencing your company to merge the two. You may find yourself becoming a lot more productive by cutting out the extra data noise. How do YOU cut through the noise when you don't understand both the database structure and the business needs?