The New Data Warehouse 2.0.11

Its an interesting time for me and the rest of us at Thoughtcorp we continue to help organizations in innovative ways, using innovative techniques to deliver them. Over the past few years I have seen changes to the way systems are envisioned and how they are realized. So it also made me look at how I build data warehouse and reporting solutions today in 2011.

When I first began building reporting systems; they were not even call data warehouses yet. I build a system to do call detail analysis for a telecommunications company  using SQL*Plus as its reporting tool. Today I build systems that focus on providing business value and business insights. The key to build a flexible and reliable system which provide data in an easy to consume manner. We have converted tons of paper reporting into dashboards, Key Performance Indicators, modeling and other reporting capabilities which allow business to react to change and anticipate trends. Today organizations which do not have data warehouse technology in some form will quickly become business laggards.

So what has changed is how we deliver solutions. The key for a successful data warehouse/reporting solution is that the information provided in the required format provides the necessary information to create an action which will enhance the business. So the involvement of the stakeholders and users drive much of this. Based on information required in the business processes we design the warehouse to service these needs efficiently. However, in the “old” days we would build the entire data warehouse and expect that what we built will satisfy the requirements. Today by building using Agile techniques we build in smaller self-contained development unit which address all aspects including data, loading and reporting to provide quick wins an help the people get the product in their hands to work into a better solution.Today the biggest difference is data warehouse architects do not assume the solution, but build a framework in which a system can develop to meet needs and drive innovation and change. So out with the Big Bang data warehouse and in with the Agile Data Warehouse or DW 2.0.11.

Unexpected Learning

Last week I had the opportunity to take some time off and take a cruise. Considering how cold things have been around here lately it was a great time to get warm and enjoy some sun. As expected I did that, but unexpectedly I also learned some things along the way. This was a cruise with unexpected learning. And maybe I should have expected it. So what did I learn on my Winter vacation?

Well to start with my ship offered seminars on a variety of topics. One was an Improv class. Seemed like a better option than the Rumba class. So we attended and I learned lessons on working in a team, listening to your team and focusing on your task at hand and not lose that focus while others may struggle. The most important thing was to ensure the team continued to function. Through some exercises that I plan to bring back to my colleges we can bring these unexpected lessons home.

I also had the chance to meet someone in a similar field as us at Thoughtcorp, in that he did retail analysis. It was interesting to exchange stories on how we solved problems for our respective clients and how to better use retail data. Today’s challenges in retail are one that we focus on at Thoughtcorp and we really started to look at ways that retailers need to bring data together to enhance businesses.

Then I started reading a book, named “Start-up Nation” about how Israel has become a leader in innovation and technology. It is an interesting lesson on how the ability to be innovative and creative and not to take no for an answer when all around say you are bound to fail. I read about a company that approached PayPal about a new concept to capture fraud. The concept was that by separating people into good and bad people you can detect fraud. Good people leave a trail on the Internet; bad people don’t. Somehow a small Israeli company was able to deliver more accurate results in a exponentially short time frame compared to PayPal’s own analysts. It was impressive and was based upon the way that governments find terrorists. This was an approach that PayPal recognized as revolutionary. The stories continue and the lessons one can learn from a book like this can be translated to everyday business.

A great time on a great ship and met some great people who helped me make some unexpected discoveries.

BTW….the picture above is our ship the Celebrity Eclipse parked next the HMS Bounty….imagine the difference in technology of these two ships….hmmmm. 

New Year and New Challenges

As we begin a new year I start looking back and looking forward. How did I and how did the industry change in the past year and what should I now expect in the new year? In the past year at Thoughtcorp I look back at the solutions we created and how we are creating solutions 12 months later. I see a change coming and I look forward to what it will bring.

What I discuss is how we deliver business reporting and analytical solutions. At a basic level what organizations need today is no different than what they have always needed. Organizations need access to information on a timely basis to provide insightful information which can better enable the business to react to changing conditions. The new year also brings an approach which we have been developing over the past year around using Agile development methods in a data warehouse project. The key is that by going Agile, that your project will deliver in an expected manner with the business highly involved in the resulting solution. We have seen how information can transform a business. Agile has often been discussed in the context of BI, bur has rarely been used for an Enterprise data warehouse project. At Thoughtcorp we are now doing just that and have already seen the success and efficiencies it can bring to delivering project on time and more importantly meeting insightful business requirements.

So today I start to see that shift of using the Agile manifesto as a base for all data projects. As for reporting we start seeing organizations that previously had little integrated reporting can now start to mature and move ahead of basic reporting and truly embrace business reporting.

What is Takes to Succeed?

This morning I was driving to work and listening to my usual morning radio. I admit it I listen to Howard Stern on Sirius. It is some of the most interesting and humorous radio I ever listen to. This morning he had the second half of the Staff IQ Test. The last one ended with his engineer having a 121, and today more of the staff was tested. A score of 120 or higher is considered highly intelligent. But of course in the nature of the show it was as much a competition for bragging rights. Today it came down to two staffers, John and Will, with Will winning with a 130, considered among the top 2% of of the population. It was hilarious as Howard lost all is bets as he misevaluated everyone’s IQ and it made me wonder about what makes people successful. Is it IQ? Is it experience? Is it knowledge? Is it personality? Of course its a combination of all of these.

People think that it is merely the intelligence and knowledge that one has that drives success. Or could it be someone taking the initiative to create an idea that changes things. So do we acquire this drive from nurture or nature? When I consider people to work at Thoughtcorp, I look at what they know, but really I consider how they think. How do they answer questions? What is the though process behind solving problems. I obviously consider technical savvy, but tools and techniques cannot. So we must have that basic ability to communicate. To speak our thoughts clearly and concisely. That is the key to succeed. The intelligence we have is shown through our ability to communicate it to others. So what we learn you should share and what others share with you make you more successful. So more than simply IQ, intelligence and success come through our ability to show how we understand the world.

If you think you have this type of intelligence and want to work in that type of environment then check out some hot jobs currently open at Thoughtcorp (Click Here for Employment Opportunities).

The Oracle OpenWorld User Perspective

As we return back from Oracle OpenWorld, I thought that I would blog on some my experiences and how being active in your user community can make a difference.

I find that my experience at OpenWorld is something of a special one. An experience that is based on being involved and having the goal to help others. I attend in many ways. I attend as a leader in the user group community as a Board Member of the IOUG. I am there as a writer, representing Oracle Press and my book “Oracle Database 11g Beginner’s Guide”, there as a member of the Thoughtcorp team and as an Oracle professional and ACE trying to learn what is new and how to be a better solution provider. The value that I get from all of these things is immeasurable. I am always am surprised when I have to explain the value of being involved.

At OpenWorld I had the chance to meet Ironman (if you have seen Ironman 2. you will know that Oracle was a great supporter. This is me with the suits before I had the chance to fly around in one.

I was also invited to an event where I had the chance to meet Mark Hurd, the new President of Oracle. It was a pleasure to meet him and I look forward to seeing how myself and the user group will work with this new regime. And can you find the Oracle ACE in the sculpture gallery at SFMOMA, where the event was held.

The event was a clear event where the message was that Oracle provides us with a chance to leverage all aspects of the technology stack from one place. The announcement of the ExaLogic machine was exciting as was the continued messaging around Oracle’s BI offerings. It is an exciting time to be part of the Oracle community and in the coming weeks I will continue to discuss about what I learned and some of the new technologies which will be important to us in teh coming months and years.

Standard Configurations May be Your Solution

It’s really quite amazing to me how many organizations have database and or application performance issues and then spend incredible amounts of time in their diagnosis and treatment. We solve the issues in a number of ways; we improve SQL access to the database (the most common issue), we change database parameters and execution plans to improve memory and data usage during the execution of other queries and we change the database from both a design and deployment perspective. In the end most performance issues are solved but what happens to our learning's from these issues? Generally we make changes to the one database and then wait until the next performance crisis arises, so our new solution is often lost and the knowledge not shared.

So how can we fix these types of issues? How can one performance issue help the entire organization? If the organization adopts the use of standard approaches to data and databases a lot of these issues can be reduced if not eliminated. Consider a large company who has many databases, often 100’s of database instances. In this company they need a team of DBAs who are responsible for a subset of these databases but yet they have no synergy and the differences in these databases make each one a custom deployment, which requires significant  involvement from the various IT teams. Consider if the organization adopts the use of standard configurations. If they decide to create databases in 3 to 5 standard configurations. They may create one for small applications, another for web deployments, another for analytic and data intensive applications and another for mission critical applications. This creates a limited number of selections, but each one is optimized for the application and allows for quick and simple deployments. It also reduces the variations which increase maintenance costs. The same can go for data models. Tables can be deployed based on standard approaches to how tables, indexes, partitions and other objects get created. By creating and defining standards, you reduce the risk while maintaining the highest quality and optimal performance. The key is that when we make one change we can easy replicate and deploy it in all environments such that the entire organization can gain the benefits.

As we come up to Oracle OpenWorld and the IOUG User forum, you will begin to see this type of message for larger organizations to provide them with the ability to deploy applications faster and maintain the infrastructure more effectively and efficiently. By simplifying the choices you can improve your ability respond to issues and ultimately reduce costs

Arming Us with Knowledge

As the time for Oracle OpenWorld approaches (Sept 19 to 23rd) I started to think about knowledge and learning. This is a fundamental right and gift that I hope to have instilled in my children and what I hope we have all done for our own.

So then this morning I see a billboard that said “In the war of ideas; Are we sending out children out unarmed?”

This is a question that seemed profound to me today. We live in a world where our knowledge is one of the most important qualities we can each have. This knowledge is what makes us productive but it also makes us interesting.

So how do we address this? We send our children to schools where there are educated on the way the world works and how we can find our place in it. However is this preparing us for what we really need? Do we have the skills we really need? Are we street savvy? Today the world is a complex place. One where we must be prepared for multiple influences on our everyday lives. We need to understand the influences of activity in all corners of the Earth. We need to understand how economic factors in the Far East impact North America and vice versa as well. You need to look at the big picture to be able to take a stand on how you react to various scenarios. It is no different in the IT industry. This is a place that has so many influences that you really need to understand how each impacts an IT project. You need to be armed with knowledge.

The world today is about acquiring knowledge and using it in a way that benefits everyone around you. So it is imperative to gather knowledge and quickly turn it into value. It is never too early to start this journey, but it is one that you must start so that you can be prepared as well as creating an environment where ideas are valued.

Oracle BI 11g Launched

I realize that it has already been a few weeks since Oracle officially announced the availability of the new Oracle BI 11g, but today was the launch event in the Toronto area and was it ever well attended. This is the version of a BI suite which will allow Oracle to compete in a mature market and I was not surprised to see so much interest in it from today’s capacity audience.

We have been waiting for quite a while to see the final release of the software (which is now available from Oracle) which has been much anticipated. We in the BI and data warehousing space have seen so many products over the years, such as Cognos, MicroStrategy among others, but Oracle never really had a great answer and Answers to these vendors, that is why this release is so important.

The latest incarnation of the OBIEE stack includes many new functions and features. It has a solid approach to dashboards, scorecards and reporting and provide it all from a common interface, which is a nice change. The software will still need to prove itself but at this point things look very promising. As I begin to work with the product I will be sharing my experiences. I will also be at Oracle OpenWorld in just a few weeks and I excited to see more on BI and all of the other DW technologies, and hope to meet some of you there to exchange even greater ideas.

The IOUG Leadership

This past weekend I got to attend the annual IOUG Summer Board meeting. It is at these meetings where we plan the strategy for the user group in the near and long term. This meeting was no different and some great work happened. I believe that the IOUG is the place you need to be if you use Oracle technology; it is one place where you can create an life-long environment for professional and personal success. What was also important was that Andy Flower, led the group for the first time as our new incoming President. This was a role I had filled for the past two years and Andy now was filling and doing a great job. So I decided why not Blog about the IOUG Presidents that I knew and show how each one helped the user group at various times and remember those that helped build what we now enjoy in the changing user community.

If we look back the first President of the IOUG was Michael Corey, a big man, with a big voice and a personality that drew people in. He was one of the founding members of the community when Oracle did not understand how this group of independent user, customers and consultants could help make the product better, but that’s what he and the other Oracle users who put together a group which found Oracle’s ear and helped us start this road that is the user community

After the bravado of Mike Corey came the calm and diplomatic Merrilee Nohr, a true professional who brought a calmness and a direction which turned this group into a more professional organization. Merrilee struggled to build a solid relationship at a time when Oracle was starting to evolve into what it is today and in many ways Merrilee educated Oracle on the value that the community brought and how important it was for Oracle to support us.

After Merrilee I recall Rich Niemiec. There are few people like Rich, for those of you how know of him, know he is a great technical genius who taught us lots about performance and leadership. He led as any good Ex-Marine would and helped to motivation the Board and the community onto better things. As Rich once said to me “Leaders are chosen when they are needed and ready to take on the challenge,'” and Rich was there to help stabilize the IOUG during some rough times in working with Oracle.

After Rich, came Kim Floss, a customer of Oracle’s, which is always a good thing, as Oracle likes their customers and when one came to lead the user group like Kim, it was a great for our relations with Oracle as the relationship evolved.

Following Kim, was Ari Caplan. Another one of a kind leader. He brought a quiet calm and a strong leadership. He knew about a lot of cool stuff and was a business leader in the Oracle community. He also had a skill for baseball and analytics and today he works for the Chicago Cubs, maybe Larry can use him one day if he buys a team. And of course it was during his watch when we launched the first COLLABORATE conference and a new era in user group events was born.

Then came me. I am proud of what I did to help the user community and Oracle during my two year tenure. It was exciting and evolving as Oracle continued to expand while the user group continued to focus on the networking, advocacy and education. And together with the Board we all did a good job.

Which leads me to the current President. Andy Flower, who is a great, engaging and strong leader who is focused on continuing to grow the community. It will be under Andy’s leadership when we will welcome one of the newest communities to the IOUG; that of the MySQL community. After our meeting I am confident that Andy will help us move the community to a period growth while still keeping us focused on our core community.

Plus he looks a lot like Bobby Flay, so we can get good tables in restaurants and bars.

The IOUG has come a long way in the past years and will continue to evolve and change just as our great leadership has done over the years,

What it Means to be Part of a User Group?

As the summer continues and I continue to struggle with a golf game that continues to elude me I start to think about the user group. The IOUG is the user group that I know the most about but it follows for any user group member of an independent user group.

The summer is the quiet time for user groups. We have finished one of the greatest Oracle user events in May, when Collaborate 10 ended and now we are interacting with our members in the virtual world. The IOUG as other Oracle groups run independently, which means we receive no funding from Oracle. They provide the user community access to the resources available at Oracle, which is a great value. We also support Oracle at their events; as we will be doing during this year on Sunday Sept 19th at Oracle OpenWorld where we will be providing a great technical program.

So at this time you should consider being part of the group and include your voice in the independent vision from your user group. It’s time to get involved and time to include yourself in some information and knowledge sharing.

Analytics In Sports

Today I was reading a story about how Marian Hossa chose to play with the Chicago Black Hawks this season. He chose the team based on a statistical knowledge of what it takes to win.

">Hossa’s agent has developed a model which analyzes goals by the top 6 forwards, other measures for defensemen, and a  goalie’s save percentage all add up to success. The key measure for Hossa and Winter was the fact that a team who scores 143 goals in it’s top 6 will achieve 100 points. This along with quality of management helped him make the decision and now he is appearing in the Stanley Cup final for the 3rd consecutive year.

Analytics in sports is no stranger, but it is one which has been slow on the uptake. Baseball has always been an active statistical league, but companies like Ariball (www.ariball.com) have taken the analysis of the these numbers to a new level. He helps teams and players understand their own strengths and weaknesses as well as those of their opponent to achieve maximum performance. Consider that we can easily look at where every ball hit at Yankee Stadium landed in 2009 as you see below:

Imagine what you can do with this by adding the dimensions of hitter, pitcher, pitch, count and more….lots of power.

And considering the controversy of last night’s blown call in the Detroit-Cleveland game where Armando Gallarraga lost a perfect game….was this simply a mistake or a blown call or was it official bias? If it was Roy Halladay or some other great pitcher on the mound would the call had been “right”. The statistics definitely do show this for the NBA. When a star is in foul trouble they have fouls called about 1/2 as often a non-stars. So how would things had gone last night if we were watching a future all-star?

Sports statistics are here to stay and just as they make a company more competitive it does the same for a sports team.

The Changing Face of Oracle Technology

The coming year is one which will provide Oracle users and user groups with the chance to evolve and change as Oracle continues to change.

In the past year wehave seen significant changes in technology and the Oracle technology stack. The changes are comig and the evolution of Oracle as a company and as a technology is changing. We are beginning to see the changes in the latest products which are coming from Oracle.

The BEA acquisition is showing this as the latest release of WebCenter Suite has truly shown great changes whic allow it to better integrate wih Oracle and other products. The changes hae meant that the product and it's support for Web 2.0 initiatives is simple and easy to implement.

The Seibel acquistion changed Oracle in the most obvisious way by allowing Oracle to offer a world-class CRM solution. The acquisition also made some real changes to Oracle's BI offering. Oracle BI which grew out of acquired products have allowed Oracle to compete with the big boys of BI. OBIEE and especially the coming release of the prduct will provide users with a solid product which can be used to depoloy a solid BI solution. In addition Hyperion and Essbase complement this tool with robust forcasting and budgeting capabilities. The total package.

Other acquisition will change things even more, the Sun acquisition will be a game changer and one that will forever change the face of Oracle, but we will watch as the technology changes and improves our solutions and our businesses and see what change we embrace and others which fall to the side.

I also want to thank Patrick Fordham (www.torontophotographer.com/) who recently took some heads shots for Oracle magazine and which I am now using as my new profile picture. Thanks.