Oracle Users In Action (A Collaborate16 Conference Review)

The recent COLLABORATE16 conference held by the 3 major Oracle user communities (IOUG, Quest and OAUG) brought together individuals focus on Oracle technology, middleware and  applications such as JDE and PeopleSoft. The event was attending by about 6,000 professionals from around the world.

The 4 day event had many themes and introduced ideas but my thoughts was focused on the messages in the following areas:

• The upcoming release of Oracle Database 13c. An updated version with expanded features. Still under NDA so we have little to discuss at the moment.
• The Cloud
• Business Intelligence (Discussion around the concept of bimodal BI)
• Security
• Big Data and Internet of Things

We all attend many sessions during a 4 day conference and people either make a great or insightful statement which we later quote or they use a quote to make a point. In one case the use of a quote made an impression on me. It was a quote by Edwards Deming in 1942 which continues to resonate today:

“Scientific data are not taken for museum purposes; they are taken as a basis for doing something. If nothing is to be done with the data, then there is no use in collecting any. The ultimate purpose of taking data is to provide a basis for action or a recommendation for action. The step intermediate between the collection of data and the action is prediction.”

-          W. Edwards Deming

o   On a Classification of the Problems of Statistical Inference, June 1942, Journal of the American Statistical Association.

This idea which Deming states is that we should not collect data without purpose, without a use case. He was working for the US Census Bureau and said that unless data is useful why collect it. We need to consider the same when we work with data today. We should not try and boil the ocean; rather we should look to use data with focus.

Core to the conference was discussion around the database. Seminars included backup, security, performance tuning and databases on appliances like Exadata. The conference also held Oracle 13c beta session about new functions and features and to prepare people for the upcoming release. It is expected that the next release of the Oracle database (13c) will be around the Fall or sooner, depending on how beta testing progresses but one can expect it will be out in time for Oracle OpenWorld at the latest. There was onsite beta testing taking place at the event for customers who are part of the beta program.  Enterprise Manager 13c which includes support for both on-premise and in-cloud databases is improved and extended.

A big topic of conversation was Cloud. This permeated across all technology and applications. Oracle is focused on becoming the biggest Cloud provider of databases, applications and other components provided through their SaaS and PaaS strategy. 

(http://si.wsj.net/public/resources/images/BN-KT065_1013_c_G_20151013173315.jpg)

All products which Oracle provides are now available via the Oracle Cloud. This strategy allows for monthly feature and fix releases which are said to not impact past functionality…… current prices for the Oracle Cloud are quite aggressive in comparison to their competitors

Consider Oracle’s new Data Visualization product. This product to compete against products like Tableau are currently inly available as a desktop single user version or as a Cloud service. (https://cloud.oracle.com/en_US/data_visualization?resolvetemplatefordevice=true&tabID=1445271963053). This is indicative of the Cloud strategy where software is developed so that the minimum viable product is made available and then development of features continue.

This leads into the new concept of Bi-Modal Business Intelligence. The concept of bi-modal is the merging of two approaches to information management. The first being the traditional approach where formal processes and policies are put in place to support well defined and mature requirements. The second mode is one which is more Agile and addresses issues using an approach of discovery. The following slide from Gartner provides some guidance on the differences between the two modes:

(http://images.indianexpress.com/2015/04/bimodal-it.jpg)

This approach was originally created to support a general change to how development works and to introduce Agile into the workplace, it has been further extended for BI. The following illustrates these modes as they relate to Business Intelligence:

(http://www.targit.com/~/media/images/targit/blog-images/post-images/bimodal.ashx?h=384&w=701&la=en)

One seminar talked about how the Oracle tool set addresses these 2 modes of operation. Oracle and a couple of the speakers address the question by suggesting that the Oracle BI suite satisfies both sides of the equation. The idea is that OBIEE is the Mode 1 approach with highly governed and structured approach to reporting. Although the product does include the ability for some individuals to create Mode 2 reports, the requirement for curated data defined by the semantic layer does go against some of the concepts within the definition of Mode 2.  Oracle is now offering Oracle’s Data Visualization (DV). This is the product Oracle has released to compete with Tableau. The advantage is that you can better govern the data access to data sources which is a step ahead of Tableau which thrives on using non-curated data. At this point this product should appeal to organizations who wish to reduce the complexity of tools which they support. DV is currently only available as a Cloud service. There was some discussion of making it available as a stand-alone version, but not as a server-based version.

The conversation around Security continues to be a big concern for most database professionals. The speakers discussed methods for securing data which includes masking and other advanced methods of encryption. When using Oracle there are enough features when used in the basic and advanced security features that no database today should be left unprotected. Generally the issues are caused by organizations not using what is available today. In the Big Data space, security continues to be an issue. The Data Lake is presenting new challenges to ensure that data in Hadoop is protected. Most organizations today merely place Data Lakes in restricted areas/servers. This is really only the first step, security is developing in the space and should be a major concern. In a recent survey, Oracle customers stated that security and performance are the two major concerns with the adoption of Big Data in their organization.

During the opening session the results of a survey of Oracle users was presented on the topics of what people are doing and planning and what are the biggest challenges they are experiencing. The key findings were around the adoption of Cloud which is advancing while Big Data is moving with more caution.  A good summary of this discussion may be found at; http://www.dbta.com/Editorial/News-Flashes/Ground-Breaking-Research-on-New-IT-Trends-Adoption-is-Presented-at-COLLABORATE-16-110361.aspx

The last area which I will focus on is that of Big Data and IoT. The conversation of the topic continues at the periphery of this group. While almost all attendees have Data Warehouses, few have active Big Data initiatives and fewer in IoT. One interesting presentation was about using R for Data Quality. I found this to be a great approach to using the new Big Data tools to ensure quality within the repository. The use of R provides many of the statistical methods which may be used to profile data. I thought this was a good approach to how one may implement a DQ profiling approach. The presentation may be seen at: http://www.slideshare.net/michellekolbe/data-profiling-with-r

I find it interesting that this user community continues to be very much focused on databases and not in the evolution of data. They are doing a good job in sharing information about the database, middleware and applications but they also are more resistant to newer technologies like the transition to Big Data. This may be the nature of this group, but considering that the people here are the technologists who manage most of the business critical applications the biggest trend for them is moving to the Cloud, and to a lesser extent the data which is being generated by these applications.

If Someone Says “I Think”; Tell Them to Prove It! – IOUG Real World Performance: A Review

Seems like an odd statement. We always hear people say “I think…” and then they go on to tell you what they have hypothesized. Now you wonder is this story factual? Does it REALLY work? It is then time that you need to ask them to prove it. We don’t often challenge our peers to “Prove It”. We accept thoughts and then experiment on our own to see if this thought proves to be correct.

This past Friday I had the chance to attend the IOUG’s Real World Performance seminar which was held in Toronto. It was at this seminar led by 3 of most knowledgeable Oracle people in the world, where the proof showed us how to change the way we think. As they say there is no reason for things to run slowly today, but that instead that people have made the choice to run this way and not take advantage of new technology and new methods.

Andrew Holdworth, Tom Kyte and Graham Wood, led this amazing day of proof.  The image below is a picture from my chair… of the 3 of them up front in the room(as they were all day)

It was a day where the 3 Oracle database performance gurus showed us a new way to look at performance. They discussed data warehouses and operational systems. So what were some of teh highlights:

• Showed today’s data warehouses have undersized CPU’s and insufficient I/O
• Improve data loading, as data size will continue to expand
• Stop using SQL*Loader, it’s time has passed. Consider external tables
• Read compressed data if possible
• Statistics are critical to success
• Make sure cardinality is accurate in table statistics
• Use SQL*Monitor to investigate performance of SQL
• Manage your database and system resources. Use Instance Caging if needed
• Used set-based operations as often as possible
• Check your SQL statements for issues

The biggest thing takeaways I came away with is that today we should try to achieve full CPU utilization, we should optimize how we transfer data into a data warehouse, we need to make sure our SQL statements work as expected and that indexes and partitions can be your friends and your enemies.

It is amazing to me to see how we things have changed over time. Today we need to manage our data better, our databases more effectively and the improve the process of loading and retrieval. The tools are there for you, we just need to use them. As Tom Kyte said, “Why would people choose to run their database slowly?” 

There are more of the IOUG Real World Performance being held this year. If you get the chance to attend, you should; as it can change your life or at least the life of your database.

Oracle Database 11g Release 2 Is Now Available

Today is a big day in the lifecycle of the Oracle database. Today Oracle released it’s first port of the 11g R2 for Linux. With the release of this version of the database Oracle has brought us a solid release of the 11g database.

This release focuses on reducing costs and Oracle has provided and improved numerous features that will provide this support. The move is to easier management and features to optimize feature management. The new release will make ASM management easier and ASM will now support all data structure storage. The database has improved more than storage but the way that data is stored. Improved compression and partitioning. Being so data warehouse focused, I always look at those aspects. There is improvements in OLAP and Analytics. You might want to find Charlie Berger of Oracle, who would will tell you the depth of analytical capabilities in the database. I am always surprised how many people buy SAS and don’t know what is available in their Oracle database. And if you have a Linux server you can download the latest release.

The IOUG was heavily involved in the beta testing of release 2 and the feedback was really good, so the anticipation has been building for all of us to get to this release out and into the public.

I am excited about this release as it should give organizations the confidence to start wide adoption of 11g. The time has come for Oracle Database 11g and release 2 will provide the confidence and stability that is needed in today’s 24/7 world.

Check out the Oracle 11g Release 2 web page for more information. The IOUG is planning some 11g R2 webcasts and will be the talk of OpenWorld.

The Data Warehouse Team

Building the right team is a subject that I have discussed here before, but I wanted to talk about how to put the right team together and what skills are needed to build a data warehouse or business intelligence solutions.

So who is on the team?

• Project Manager
• Data Warehouse Architect
• Business Analysts
• ETL Lead Developer
• BI Lead Developer
• QA Analyst
• Various other developers and analysts.

So where do we start? The key to me in any successful data warehousing project is having someone who can be the Data Warehouse Architect. This is the person who will drive the solution and ensure that the most efficient solution is implemented. This person must have the ability to understand DW/BI technology and understands how to translate business needs into a solution. This person will directly impact the project’s success so choose carefully and choose experience.

DW and BI projects need standard project management controls, as many projects are complex and have many moving parts. This person should be knowledgeable in the DW and BI arena. Just another project to manage.

The ETL and BI Leads are also critical to the ultimate success of the project. These people will create and establish the approach that will be used to deploy the system. These people are generally the most experienced technology people on the team and can provide leadership to the rest of the development team. These people may also be development leads, but generally we try to focus these roles on ensuring that development is executed as needed.

Finally you need you QA, analysts and developers who will fill the remainder of the team.

Ultimately the team will need to work well and the leads are important people who must ensure that communication occurs between various people so that everyone understands their role in the overall project and can see how it contributes to success.

So build the right team with the right people who want to build solutions that provide organizations with competitive intelligence capabilities.