Big Data is Curious

I was watching a TV show about Studs Terkel, a man who wrote about American life. His story is an interesting one. One statement I heard on the show was:

“Not everyone has a depth of curiosity.
And not everyone has depth of understanding”

I found this a interesting comment and one that I relate to in my everyday world. It is so true that data is expanding faster than our ability to analyze it. This is where Big Data comes in and together with an Agile approach to development can form and important part of a data solution. The ability to collect more structured and unstructured data gives us the ability to feed our curiosity. It is imperative that today’s analysts have the same curiosity as the ancient explorers. Curiosity is opportunity to learn; to discover, this is where real innovation happens. We must use Big Data to gather information to feed the need for understanding. We need to find meaning in this information and exploring it in a dynamic way can support this need. So we look to Big Data to provide this sandbox of data for analysis and knowledge.

There are concerns with Big Data which we also need to address. Security is one of the biggest concerns for most organization. So now we need to look at Big Data and how it is deployed. Often we look to create Clouds of data, but is this data secure? Is this data protected? The danger with collecting more information is how and what to protect? This is a new challenge and if we are to succeed in getting data more accessible and more useable we also need to ensure it is protected. We are seeing that as big data matures and Hadoop continues to become mainstream, we are seeing products needed to support these requirements.

So as the technology advances and our curiosity grows we will be able to create solutions which will provide a robust ability for analysts to gain business understandings.