In the last few years the world has witnessed dramatic increase in IT Systems creating, consuming and modifying information that produces huge logs which are not being put to use, most of this data is just left unprocessed. If this is processed at the right time, it could help the business leaders take better informed decisions.
It is known fact that ~80% of data that is produced today is unstructured and that is being processed by Enterprise Content Management systems. But even the structured data increased exponentially due to the increase in adoption of sensors for business / supply chain systems to transform their business / take advantage to technology / reduce costs. IoT is the major contributor for structured logs produced much faster then ever before.
This is data "stream" flowing in real time like a stream of water and if taped and used at the right time, it could provide lot of business value.
Many organizations are just ignoring this huge stream of data, while there is a growing concern of how to consume and analyze this data in real time to help decision making. Remember much of this real time data is very valuable as soon as it arrives and it's value depletes with time. In other words value of data is inversely proportional to time. Therefore near real time analytics of this huge data / Big data gained momentum and IBM has a solution.
Some of the examples ... Sensors on hundreds of escalators carrying millions of users at a train station / airports, per a time period, could produce gigabytes of data that should be processed as soon as it arrives and provide insights to predict when to call for maintenance, before a breakdown. Another scenario could be where live bank card transactions that happen like a million per second could be processed in real time to identify fraud and decline payments. The examples runs endless and interesting as we think deeper. IBM Infosphere Streams can be leveraged to solve the problem of stream processing using SPL (Stream processing language). Infosphere streams can run both on-prem or in-cloud.
InfoSphere Streams provides an execution platform and services for user-developed applications that ingest, filter, analyze, and correlate the information in streaming data. Major components are Streams run-time engine, Steams Processing Language (SPL), Steams Studio and Steams Console.
A Streams Console is a web-based graphical user interface that is provided by the Streams Web Service (SWS), one of the runtime services. You can use the Streams Console to monitor and manage your Streams instances and applications from any computer that has HTTPS connectivity to the server that is running SWS. The Streams Console also supports data visualization in charts and tables.
To experience the streams processing power, IBM has provided the following resources free for learning
Download the QuickStart Edition for Linux or as a VMware Image: http://www-01.ibm.com/software/data/infosphere/stream-computing/trials.html
This VM Image is free to use and has all the software that just runs without any additional configuration.
Streams Quick Start Guide: https://developer.ibm.com/streamsdev/docs/streams-quick-start-guide/
Introductory lab: https://developer.ibm.com/streamsdev/docs/introductory-lab-for-streams-4-0-1/
This lab needs no SPL programming skills but gives an understanding of various concepts, helps the reader to build an application, run and test it.
Enjoy streaming!
