Data in the Time of Change

Back at it... I have been working on various other social forums of blogging and writing but I am back at it in my original blog. Time to get data to work.

Data is what I continue to focus on today. All aspects of data and how it should be used for the greater good.

Here is something I wrote recently for the IOUG and thought you might find interesting. I am also finishing up some ideas for 2015 which should be coming soon. I resolve for 2015 to be a better blogger.

Data in the Time of Change

By Ian Abramson, EPAM Systems Inc.

. We look at changes in technology, processes changes, professional changes and others which are impacting us as Oracle and Data Professionals. I find it intriguing that in my everyday job at EPAM I am one of those who helps companies change how they treat and use their data. Big Data and advanced analytics is all about how organizations and individuals are changing to the new data paradigm and I am lucky enough to be part of it. We are empowering the individual to make choices based on an unprecedented access to information which we can now provide. 

The change has been in many ways a gradual change and in other ways very sudden. The technology in Big Data ecosystem is one which changes faster than Superman in a phone booth.  We see releases from Apache, the various Hadoop distributions along with new projects which surround the technology becoming available each and every week. In August Apache released Spark for general release. Spark allows you to now interact with Hadoop up to 100 times faster. They do this by changing how they get data from the cluster as they reduce overhead and do as much work in memory as possible. Why is this important? It is important as existing Hadoop installations will want to find better performance compared to the existing tools. So now you have a choice. Do you begin using Spark or if you are using Cloudera should you use Impala? You know that you need to change but the question is which change will bring the most benefit. Big Data requires one to embrace change.

Besides the technology evolution we are seeing more desire for businesses to change. Big Data has opened up new opportunities within business. We now are providing them with access to information which previously had been unavailable or difficult to access and analyze. So we need to be sure we control this access because with great power comes great responsibility. Embrace change but make changes to improve the experience or the organization not simply to change.

The following infographic illustrates how data and analytics is changing and how we can make a difference if we can all evolve and change: