Thursday, April 7, 2011

Baseball and BI

The Major League baseball season started last weekend and the teams that should be winning are panicking (consider an 0-5 start for the Red Sox), while others are looking at what might happen after a quick start. The length of the baseball schedule, tends to be a big equalizer and as the Summer progresses into the Fall, the cream of the crop tends to come up to the top. So although the Toronto Blue Jays have started 4-1 and look to have some real pitching this year; I am not ready to get overly excited… it’s early. After all there are 157 games left. The playoff race is months away, but yet we get excited none the less. It is baseball season.

It is also the time when one of the most analyzed sports is changing the way they do business. I think I mentioned in the past that baseball is a sport that collects more information, which is at a very granular level, compared to other sports. It provides so much data that the science of sport not longer refers only to medicine but to data as well. Data may even be more important than ever in baseball and growing in all sports. Today, when someone \wants to sign a free agent or trade for a veteran or prospect player; the first thing a team will look at is their data.  Men like Bill James and Ari Kaplan have made analytics part of the game but also part of the business. By tracking how a player reacts to situations you can now extrapolate how they will fit into your type of offense and see in advance the benefit they will give to the team. The concept of “gut-free” still exists in baseball but data is quickly replacing it as part of multi-million dollar decisions.

So as we start the season we will be able to see if Mariano Rivera can continue to deliver heat in the strike zone, or the edge of the strike zone. We will that Derek Jeter likes to hit fastballs, but hits sliders even more effectively, all of this will be tracked. We all need to find the edge. Businesses have understood the value of data, but today sports are embracing them and changing the way they  make game and player decisions….it’s all in the stats.

Play Ball! Or Run the Report!

Go Expos!

File:Montreal Expos logo.png

Saturday, April 2, 2011

Thoughtcorp or is it The Addams Family

This week one of my favourite people and client sent out an email about the people that work at Thoughtcorp and how we look like the Adams Family…. Now being someone who is being compared to Uncle Fester, I still think it is so funny. So let me introduce you to some of the Thoughtcorp Addams family. Thanks to John H for this fantastic comparison to the famous family.
First is me, Ian Abramson, as Uncle fester.
clip_image001 clip_image002
And next up is our Co-CEO David Bercovitch or should I say Lurch, or at least a shorter version of him.
clip_image003 clip_image004
And then is Val Hedow, of Director of HR, or as some will now refer to as Morticia…. It’s almost uncanny.
clip_image005      clip_image006
And of course, Gino Marckx, of Agile Practice Director, when it comes to Agile he is second to none, but amazing I think he will be known as Gomez Adams.

clip_image007clip_image008
Finally the Dino Chronopoulos, the head of the Finance group at Thoughtcorp, seems to look a lot like Pugsley….interesting.
clip_image009clip_image010
I love and To put a real spin on this. There are places where you have a chance to work where you can build or do great things, but you always have to remember to have some fun at work and let it loose. Thanks John for helping me understand that I now for the Addam’s Family.

Thursday, March 24, 2011

Open Season on the Database

For years we have contemplated open-source databases and how they fit into our overall database technology strategy. The complexity of what is the right choice for an organization has many answers. It is no longer a simple question of choosing between Oracle or Microsoft or even IBM; no it’s a choice that needs to include MySQL (unless that counts as Oracle?), Postgres, or now Hadoop and Cassandra. The choices are as complex as the systems we need to support.  From my perspective it is now open-season on the database. The choice is no longer straightforward.

The emergence of “The Cloud” and the need to distribute more data more effectively has added new pressures. The need to analyze massive amounts of data more efficiently is a must-have. We look to the Oracle’s of the world they respond with a system that is scalable, efficient and leverages the power of Oracle to enable the business. The Oracle database on an Exadata server is a solid enterprise-grade solution that will support this need. It is robust and powerful, but it is costly. So organizations who need to similar functionality but want to do it using open-source technology or some lower cost-model of software now also have options. At Thoughtcorp we are leading providers in many areas of technologies and our team has varied knowledge and much of it focused on today’s latest technologies including wireless, apps and of course using open-source technologies to achieve their goals. The data group has seen this shift as well and now I am working on how products like Hadoop, which can be used for massive data analysis. It is like Google file system (GFS) and allows for the collection and analysis of data. From Hadoop other solutions have emerged. Hive provides a easy to use tool to work with Hadoop to simplify analysis. Now Hadoop has been combined Cassandra and open-source database from Facebook. What does this give us? A real-time database with big data capabilities. Wow…that’s a mouthful.

Of course these open solutions come today with some operational risks due to single possible points of failure. of course if you can manage the risk. You may have an alternative to today’s  enterprise databases. These new data options provide us more with choice and options and in a world where you consider…. If it’s good enough for Facebook and Google, is it something I should consider?

Tuesday, March 8, 2011

Building Data Warehouses for the Masses

Often data warehouse designers and architects are often accused of building systems which do not always serve the enterprise but only focused portions of the organization. I have seen this occur at numerous clients who did not invest in a long-term vision and long-term data strategy.

So how do we avoid this trap and ensure that we produce a system which does not go “silo” but sets a course to meet current and future needs at the enterprise level and not departmentally. I was recently working on a presentation about data warehousing design and the question of Ralph Kimball (Dimensional) versus Bill Inmon’s (Normalized) design approach of which is better. The question is that both work, if you keep to the big picture. Design your high-level design upfront and adhere to design and development standards, It is about setting a clear course for DW design and never allowing for an independent or silo’ed solution to be developed. The key is to always look for integration opportunities. Add to your core data warehouse; don’t built a new structure which is not tied in and used conformed dimensions.

Bill Inmon, said in referring to Dimensional modeling that “A 1000 minnows do not make a whale”. Build with a great purpose in mind  that you are building an integrated, stable data repository to support business reporting and analytics.

Wednesday, February 16, 2011

Data Warehousing and the DW Appliance

Today I attended a seminar presented by Oracle about the Exadata database machine. As usual it was presented as a compelling argument that a complete solution like a data warehouse appliance is one that ultimately reduces complexity and increases capacity.

During the session today, they quoted a study from TDWI which says that within 3 years 78% of respondents expect to be using a data warehouse appliance. I find this a staggering figure. Consider that today DW appliances have evolved and each of the major database vendors have their own flavour, but we still continue to see that traditional databases significantly outnumber appliances. I do admit that the argument for an appliance is one that needs to be considered. The idea that a single server or linked servers can come into a data centre and within days can be productive is incredible. The bigger question becomes which appliance do I choose? Do I buy Exadata, Teradata, Data Allegro, or Netezza? I guess that answer depends on what you want to achieve and the basic knowledge and comfort level you have with each vendor.

I find that the real differentiator is the functionality available within the appliance. As you might know Netezza uses Postgres as it’s base database. Although a great database, it is provided within only the basic SQL set of commands. To extend it out, you need to buy additional options and features. Then you look at a database like Oracle’s which has 30 years of development in it and now we are presented with a database that supports very advanced SQL. Oracle has provided an appliance that runs the database. It has not created a database which happens to run on an appliance. I often wonder how many people are aware of the advanced functions and analytics available in the Oracle database. The database has a significant number of statistical functions from the simple to the esoteric. The database contains a number of data mining functions as well. Although Oracle sells a great front-end tool in Oracle Data Mining (ODM), the functions which the tools use are available to all database users. So when you start to look at what appliance, you need to look at more than simple administration and technology, but at capability. The appliance who can do the most without any additional costs seems like a solid option. Now as to whether or not we will see a big increase  in the use of appliances will depend on how well and how cost effectively these appliances can replace our current hardware platforms.

Friday, February 11, 2011

Oracle User Group Season

As the winter starts to settle in and I ponder what I will be doing in the next few months I begin to plan my spring speaking season….and of course golf season.

We are a mere 2 1/2 months away from the the start of the COLLABORATE 11 conference. As well I am working with the Toronto Oracle User Group to help to put on another BI/DW day in March or April. So it’s time to get my materials together.

My session at COLLABORATE 11, is named Oracle Data Warehousing: Soup to Nuts. The focus is on providing a solid knowledge foundation in all of the areas which data warehouse architects need to be concerned with. I, along with my co-presenters Albert Hui and Jatinder Bhardwaj are planning to present details on how a data warehouse is designed and built. We will discuss how Agile development methodologies can be applied to data warehouse projects. This is something which organizations have not considered is if Agile can be applied to these projects. From the Thoughtcorp approach to Agile and warehousing has been combined and has now been shown in action to provide similar benefits to DW projects. In addition we will get technical and illustrate the features in Oracle which every data warehouse needs and when and why they should be used. This will be your toolkit for deployment. We will discuss ETL and how data should be loaded. This provides the attendee with knowledge on how you should use tools like OWB or ODI in your project. Finally we hope to touch on some of the benefits and successes of using the Exadata and how it can provide significant value in large data project.

I know it seems like a long way away, but in reality it could not be closer. So get on board. Get registered and come on out and see myself and others at COLLABORATE or at the next TOUG meeting and share your experiences along with ours.

Wednesday, January 26, 2011

The New Data Warehouse 2.0.11

Its an interesting time for me and the rest of us at Thoughtcorp we continue to help organizations in innovative ways, using innovative techniques to deliver them. Over the past few years I have seen changes to the way systems are envisioned and how they are realized. So it also made me look at how I build data warehouse and reporting solutions today in 2011.

When I first began building reporting systems; they were not even call data warehouses yet. I build a system to do call detail analysis for a telecommunications company  using SQL*Plus as its reporting tool. Today I build systems that focus on providing business value and business insights. The key to build a flexible and reliable system which provide data in an easy to consume manner. We have converted tons of paper reporting into dashboards, Key Performance Indicators, modeling and other reporting capabilities which allow business to react to change and anticipate trends. Today organizations which do not have data warehouse technology in some form will quickly become business laggards.

So what has changed is how we deliver solutions. The key for a successful data warehouse/reporting solution is that the information provided in the required format provides the necessary information to create an action which will enhance the business. So the involvement of the stakeholders and users drive much of this. Based on information required in the business processes we design the warehouse to service these needs efficiently. However, in the “old” days we would build the entire data warehouse and expect that what we built will satisfy the requirements. Today by building using Agile techniques we build in smaller self-contained development unit which address all aspects including data, loading and reporting to provide quick wins an help the people get the product in their hands to work into a better solution.Today the biggest difference is data warehouse architects do not assume the solution, but build a framework in which a system can develop to meet needs and drive innovation and change. So out with the Big Bang data warehouse and in with the Agile Data Warehouse or DW 2.0.11.

Monday, January 24, 2011

Unexpected Learning

Last week I had the opportunity to take some time off and take a cruise. Considering how cold things have been around here lately it was a great time to get warm and enjoy some sun. As expected I did that, but unexpectedly I also learned some things along the way. This was a cruise with unexpected learning. And maybe I should have expected it. So what did I learn on my Winter vacation?

Well to start with my ship offered seminars on a variety of topics. One was an Improv class. Seemed like a better option than the Rumba class. So we attended and I learned lessons on working in a team, listening to your team and focusing on your task at hand and not lose that focus while others may struggle. The most important thing was to ensure the team continued to function. Through some exercises that I plan to bring back to my colleges we can bring these unexpected lessons home.

I also had the chance to meet someone in a similar field as us at Thoughtcorp, in that he did retail analysis. It was interesting to exchange stories on how we solved problems for our respective clients and how to better use retail data. Today’s challenges in retail are one that we focus on at Thoughtcorp and we really started to look at ways that retailers need to bring data together to enhance businesses.

Then I started reading a book, named “Start-up Nation” about how Israel has become a leader in innovation and technology. It is an interesting lesson on how the ability to be innovative and creative and not to take no for an answer when all around say you are bound to fail. I read about a company that approached PayPal about a new concept to capture fraud. The concept was that by separating people into good and bad people you can detect fraud. Good people leave a trail on the Internet; bad people don’t. Somehow a small Israeli company was able to deliver more accurate results in a exponentially short time frame compared to PayPal’s own analysts. It was impressive and was based upon the way that governments find terrorists. This was an approach that PayPal recognized as revolutionary. The stories continue and the lessons one can learn from a book like this can be translated to everyday business.

A great time on a great ship and met some great people who helped me make some unexpected discoveries.

Celebrity Eclipse - Eastern Carribean Cruise 2011 168BTW….the picture above is our ship the Celebrity Eclipse parked next the HMS Bounty….imagine the difference in technology of these two ships….hmmmm. 

Saturday, January 8, 2011

New Year and New Challenges

As we begin a new year I start looking back and looking forward. How did I and how did the industry change in the past year and what should I now expect in the new year? In the past year at Thoughtcorp I look back at the solutions we created and how we are creating solutions 12 months later. I see a change coming and I look forward to what it will bring.

What I discuss is how we deliver business reporting and analytical solutions. At a basic level what organizations need today is no different than what they have always needed. Organizations need access to information on a timely basis to provide insightful information which can better enable the business to react to changing conditions. The new year also brings an approach which we have been developing over the past year around using Agile development methods in a data warehouse project. The key is that by going Agile, that your project will deliver in an expected manner with the business highly involved in the resulting solution. We have seen how information can transform a business. Agile has often been discussed in the context of BI, bur has rarely been used for an Enterprise data warehouse project. At Thoughtcorp we are now doing just that and have already seen the success and efficiencies it can bring to delivering project on time and more importantly meeting insightful business requirements.

So today I start to see that shift of using the Agile manifesto as a base for all data projects. As for reporting we start seeing organizations that previously had little integrated reporting can now start to mature and move ahead of basic reporting and truly embrace business reporting.

Thursday, November 18, 2010

What is Takes to Succeed?

This morning I was driving to work and listening to my usual morning radio. I admit it I listen to Howard Stern on Sirius. It is some of the most interesting and humorous radio I ever listen to. This morning he had the second half of the Staff IQ Test. The last one ended with his engineer having a 121, and today more of the staff was tested. A score of 120 or higher is considered highly intelligent. But of course in the nature of the show it was as much a competition for bragging rights. Today it came down to two staffers, John and Will, with Will winning with a 130, considered among the top 2% of of the population. It was hilarious as Howard lost all is bets as he misevaluated everyone’s IQ and it made me wonder about what makes people successful. Is it IQ? Is it experience? Is it knowledge? Is it personality? Of course its a combination of all of these.

People think that it is merely the intelligence and knowledge that one has that drives success. Or could it be someone taking the initiative to create an idea that changes things. So do we acquire this drive from nurture or nature? When I consider people to work at Thoughtcorp, I look at what they know, but really I consider how they think. How do they answer questions? What is the though process behind solving problems. I obviously consider technical savvy, but tools and techniques cannot. So we must have that basic ability to communicate. To speak our thoughts clearly and concisely. That is the key to succeed. The intelligence we have is shown through our ability to communicate it to others. So what we learn you should share and what others share with you make you more successful. So more than simply IQ, intelligence and success come through our ability to show how we understand the world.

If you think you have this type of intelligence and want to work in that type of environment then check out some hot jobs currently open at Thoughtcorp (Click Here for Employment Opportunities).

Thursday, September 30, 2010

The Oracle OpenWorld User Perspective

As we return back from Oracle OpenWorld, I thought that I would blog on some my experiences and how being active in your user community can make a difference.

I find that my experience at OpenWorld is something of a special one. An experience that is based on being involved and having the goal to help others. I attend in many ways. I attend as a leader in the user group community as a Board Member of the IOUG. I am there as a writer, representing Oracle Press and my book “Oracle Database 11g Beginner’s Guide”, there as a member of the Thoughtcorp team and as an Oracle professional and ACE trying to learn what is new and how to be a better solution provider. The value that I get from all of these things is immeasurable. I am always am surprised when I have to explain the value of being involved.

At OpenWorld I had the chance to meet Ironman (if you have seen Ironman 2. you will know that Oracle was a great supporter. This is me with the suits before I had the chance to fly around in one.

image

I was also invited to an event where I had the chance to meet Mark Hurd, the new President of Oracle. It was a pleasure to meet him and I look forward to seeing how myself and the user group will work with this new regime. And can you find the Oracle ACE in the sculpture gallery at SFMOMA, where the event was held.

IMG00384

The event was a clear event where the message was that Oracle provides us with a chance to leverage all aspects of the technology stack from one place. The announcement of the ExaLogic machine was exciting as was the continued messaging around Oracle’s BI offerings. It is an exciting time to be part of the Oracle community and in the coming weeks I will continue to discuss about what I learned and some of the new technologies which will be important to us in teh coming months and years.

Monday, September 13, 2010

Standard Configurations May be Your Solution

It’s really quite amazing to me how many organizations have database and or application performance issues and then spend incredible amounts of time in their diagnosis and treatment. We solve the issues in a number of ways; we improve SQL access to the database (the most common issue), we change database parameters and execution plans to improve memory and data usage during the execution of other queries and we change the database from both a design and deployment perspective. In the end most performance issues are solved but what happens to our learning's from these issues? Generally we make changes to the one database and then wait until the next performance crisis arises, so our new solution is often lost and the knowledge not shared.

So how can we fix these types of issues? How can one performance issue help the entire organization? If the organization adopts the use of standard approaches to data and databases a lot of these issues can be reduced if not eliminated. Consider a large company who has many databases, often 100’s of database instances. In this company they need a team of DBAs who are responsible for a subset of these databases but yet they have no synergy and the differences in these databases make each one a custom deployment, which requires significant  involvement from the various IT teams. Consider if the organization adopts the use of standard configurations. If they decide to create databases in 3 to 5 standard configurations. They may create one for small applications, another for web deployments, another for analytic and data intensive applications and another for mission critical applications. This creates a limited number of selections, but each one is optimized for the application and allows for quick and simple deployments. It also reduces the variations which increase maintenance costs. The same can go for data models. Tables can be deployed based on standard approaches to how tables, indexes, partitions and other objects get created. By creating and defining standards, you reduce the risk while maintaining the highest quality and optimal performance. The key is that when we make one change we can easy replicate and deploy it in all environments such that the entire organization can gain the benefits.

As we come up to Oracle OpenWorld and the IOUG User forum, you will begin to see this type of message for larger organizations to provide them with the ability to deploy applications faster and maintain the infrastructure more effectively and efficiently. By simplifying the choices you can improve your ability respond to issues and ultimately reduce costs