Wednesday, February 16, 2011

Data Warehousing and the DW Appliance

Today I attended a seminar presented by Oracle about the Exadata database machine. As usual it was presented as a compelling argument that a complete solution like a data warehouse appliance is one that ultimately reduces complexity and increases capacity.

During the session today, they quoted a study from TDWI which says that within 3 years 78% of respondents expect to be using a data warehouse appliance. I find this a staggering figure. Consider that today DW appliances have evolved and each of the major database vendors have their own flavour, but we still continue to see that traditional databases significantly outnumber appliances. I do admit that the argument for an appliance is one that needs to be considered. The idea that a single server or linked servers can come into a data centre and within days can be productive is incredible. The bigger question becomes which appliance do I choose? Do I buy Exadata, Teradata, Data Allegro, or Netezza? I guess that answer depends on what you want to achieve and the basic knowledge and comfort level you have with each vendor.

I find that the real differentiator is the functionality available within the appliance. As you might know Netezza uses Postgres as it’s base database. Although a great database, it is provided within only the basic SQL set of commands. To extend it out, you need to buy additional options and features. Then you look at a database like Oracle’s which has 30 years of development in it and now we are presented with a database that supports very advanced SQL. Oracle has provided an appliance that runs the database. It has not created a database which happens to run on an appliance. I often wonder how many people are aware of the advanced functions and analytics available in the Oracle database. The database has a significant number of statistical functions from the simple to the esoteric. The database contains a number of data mining functions as well. Although Oracle sells a great front-end tool in Oracle Data Mining (ODM), the functions which the tools use are available to all database users. So when you start to look at what appliance, you need to look at more than simple administration and technology, but at capability. The appliance who can do the most without any additional costs seems like a solid option. Now as to whether or not we will see a big increase  in the use of appliances will depend on how well and how cost effectively these appliances can replace our current hardware platforms.

Friday, February 11, 2011

Oracle User Group Season

As the winter starts to settle in and I ponder what I will be doing in the next few months I begin to plan my spring speaking season….and of course golf season.

We are a mere 2 1/2 months away from the the start of the COLLABORATE 11 conference. As well I am working with the Toronto Oracle User Group to help to put on another BI/DW day in March or April. So it’s time to get my materials together.

My session at COLLABORATE 11, is named Oracle Data Warehousing: Soup to Nuts. The focus is on providing a solid knowledge foundation in all of the areas which data warehouse architects need to be concerned with. I, along with my co-presenters Albert Hui and Jatinder Bhardwaj are planning to present details on how a data warehouse is designed and built. We will discuss how Agile development methodologies can be applied to data warehouse projects. This is something which organizations have not considered is if Agile can be applied to these projects. From the Thoughtcorp approach to Agile and warehousing has been combined and has now been shown in action to provide similar benefits to DW projects. In addition we will get technical and illustrate the features in Oracle which every data warehouse needs and when and why they should be used. This will be your toolkit for deployment. We will discuss ETL and how data should be loaded. This provides the attendee with knowledge on how you should use tools like OWB or ODI in your project. Finally we hope to touch on some of the benefits and successes of using the Exadata and how it can provide significant value in large data project.

I know it seems like a long way away, but in reality it could not be closer. So get on board. Get registered and come on out and see myself and others at COLLABORATE or at the next TOUG meeting and share your experiences along with ours.