The challenge in delivering data projects has always be wrought with dangers. Data projects tend to be large and encompass many aspects of the organization. As a result the time to build a complete data solution can take months and years. So often these projects found that at the end of a long waterfall-style project that the results were less than expected. The system may have hit many of the business requirements, but it missed on others, while new requirements have not even entered into delivery process. It is said that 50% of data warehouse fail, while other studies have shown this to be even higher. In my travels, I would say that we hit the target most of the time, but usually it takes longer to achieve then expected. So why should we then consider Agile?
Agile is the approach which is based on four basic values which were defined in the Agile Manifesto. These are:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
You should note that we value both sides of the equation, but that we value the ones to the right more. These basic values provide us with basis for working in a collaborative environment which can focus on incremental working software.
How does Agile this help us to be more successful in data projects? The project approach for data which we follow at Thoughtcorp focuses on using Agile. This results in an approach which delivers the solution incrementally. By understanding the big picture of data the need of an organization we can divide a project into iterations which build upon each other while delivering working software. This is done via a prioritization process. In the Agile world this is known as Kanban Development. In this approach we continually review our priorities and check to see if the business now has new requirements. This allows the project to alter its trajectory based on real needs which are now better understood. This does result in improved project performance and a better solution for everyone.
The basic answer is that it does help. It has shown that data projects can be delivered faster and more effectively. We have seen that our productivity is increased by 27% and that defects are reduced by 35% versus typical data projects. In addition the number of features delivered was higher than anticipated. All of this resulted in a project which included over 300 data objects and 150 reports and was delivered successfully in 8 iterations. Below is our project wall looked like as we were working on things, as we had a lot of collaboration take place:
This project was one example of how Agile made data work. We learned along the way and perfected our approach. The great thing about Agile is that you make it work for your team and your projects, but you have to invest the time and effort to make it work.
And don’t forget COLLABORATE 12 is coming up in Las Vegas is just a couple of weeks. I hope to see everyone there. I will be speaking all about Big Data and how it fits into today’s data ecosystem.
No comments:
Post a Comment