A common theme heard in the IT marketplace is that innovative technologies are enabling new insights to be harvested from data.  Cloud computing provides the necessary infrastructure that allows smaller organizations to enjoy the value from these insights.  As cloud computing becomes mainstream, analytics driven by new sources are leveraged by formation of new partnerships.  GE and Google recently announced a partnership for the utility business that integrates Google map functions with GE’s geospatial analytical tools to improve visualization. Any way you look at it, integrating consumer and industrial assets with speed and ease is game changing.

Recently, I attended the Strata and IBM Pulse 2013 Conferences that were relatively different but that both incorporated common analytics themes. The Strata Conference is independently organized and focused on data analytics while IBM Pulse, run by the company’s Tivoli organization, primarily targets systems management.

Pulse 2013 focused more attention on analytics than last year’s event.  While the Strata Conference focused on start-ups that demonstrated interesting analytic products, IBM is looking to leverage analytics as a core competency by embedding those processes into current solutions.  Speaking at IBM’s recent investor meeting, CEO Ginni Rometty projected $20B in analytics and big data revenues by 2015.  To meet this goal, all IBM brands seem to be adding analytic capabilities.

One analytics area I found interesting at both Strata and IBM Pulse were solutions focused on system logs.  Computer data logging is a common practice and supports the goal of keeping a history of administrator activities and problem diagnoses.  After an outage, logs are thoroughly analyzed by technicians who scour for the root cause then take appropriate steps to fix malfunctioning components.

The number of inter-connected components in a cloud computing infrastructure significantly increases the volume and variety of log data produced, as well as its importance.  Traditional log file solutions leveraged legacy database technologies that are unable to process data being created at the variety and velocity today in a timely and cost effective manner. At the Strata Conference, Glassbeam and Splunk offered interesting solutions; and at Pulse, IBM released a new product capability in their SmartCloud portfolio that analyzes logs, events and metrics in an integrated manner. Let’s discuss each in turn.

• Glassbeam’s goal is to provide a platform to help companies get intelligence from machine data.  Their partnerships with LogiXML, OpSource (now part of Dimension Data) and Vertica (now part of HP) give them the capability to quickly assemble an innovative solution.  Capabilities that focus on customer intelligence, product engineering and service revenue will be of interest to any organization looking to leverage intelligence and value from log data.

• Splunk surprised a lot of folks when their stock price doubled on the first day of trading after they went public — offering insight to investors’ thinking regarding Splunk’s capabilities and the value of log analytics to customers. While Glassbeam is a niche and vertical solution, Splunk is a broad solution that covers the gamut of logs from call centers to click streams.

• IBM is a late entrant into developing a core product focused on log analytics leveraging new technologies. Their new product is promising as it integrates competencies from multiple software divisions as well as the recent Vivisimo acquisition.  This combination of products brings the challenge of integration and installation while allowing the new product to pull strengths of multiple best-of-breed products.  By integrating systems manuals as an additional source to identify specific problems, this new workload analytics capability will be a good addition for existing and new IBM Tivoli customers.

Application logs have offered ways to diagnose system problems for a long time. Analyzing logs from multiple operational processes is a new challenge, and new technologies like Hadoop provide an excellent way to proactively act on intelligence gained.  As technologies mature, technology giants will be forced to introduce new products to meet customer needs.

The ability to create new products rapidly is demonstrated by newcomers like Glassbeam and Splunk, changing the IT landscape.  Pulse 2013 exhibited legacy product improvements along with new products bundling multiple capabilities from a diverse portfolio to address customer demand. Existing technology vendors need to quickly build their own products or acquire companies that will allow them to fill in this void in their portfolios.  Predictive analytics from log data is a good example of how businesses can add significant value to their customers.