Considerations to implement Cloud Integration

Every company has a Cloud offering and most of the businesses have concluded or considering to move to the cloud due to its benefits.  Based on an IBM Survey, customers are 83% more likely to use a Hybrid cloud.

Hybrid Cloud is an integration of applications/resources/services that are spread across different cloud vendors or cloud applications integrated with on-prem assets.

So it is essential to understand Cloud Integration.

First, let's list out different types of cloud integration possibilities

• Integrating two applications within the same cloud
• Integrating a cloud application with on-prem resources
• Integrating two SaaS applications hosted on different cloud vendors

 On-prem and Cloud have their own benefits so the industry is migrating towards best of both worlds called Dynamic Hybrid Cloud which is a combination of both (on-prem & public cloud) to balance risk and speed of deployment. Also, the end users migration to mobile and social usage patterns is forcing traditional businesses to leverage Public clouds to deploy System of Engagement (SOE) i.e customer UI and integrate them with System of Records (SOR) resources hosted on-prem for better security.

Public Cloud Low energy Pay as you use Elastic

Private Cloud Customization flexibility Highly secure Management flexibility

Attributes to consider for Cloud integration (*refer OpenGroup)

• Scalable: Should be elastic to absorb the changing workloads
• Robust: Should be fault tolerant, adapt to failures and recovery fast
• Flexible: Capability to handle changes between cloud vendors and on-prem environments.
• Secured: Communication channels and data at rest should be encrypted.
• Configurable: Define configurable interfaces via web browser.

Generally, Cloud Integration is targeted at the Application level or at the Data level.

• When trying to integrate at the Data level a design pattern called Pipe & Filter could be considered. The piping and the filter code transforms and moves the data from one cloud to another depending on the source (producer/from Cloud) and sink (consumer/to Cloud) data format requirements. Similar to ETL (Extract Transform Load) process.
• Application integration is the key to choosing the best cloud integration option for a given situation so it is required to cover all three could integration possibilities listed above.
  
  • Application to Application integration in the same cloud
    
    • API driven - REST based and Messaging (MQ)
    • REST is a simple lightweight web services communication (often considered as an alternative to CORBA). It's platform and language independent. Does not have security, session, message formats making it less restrictive. It's a client-server model based on resources that are accessed via simple POST/GET/PUT calls. It is a stateless service which means the calls are atomic and do not depend on previous calls. The Server is not concerned about the Client and the Client will have everything required to fulfill the call. REST is used for synchronous communication. 
    • Messaging is required for different business applications to communicate effectively using a common method. when there are multiple applications integrated, there is a need for them to be active/up and running to receive the communication and respond. In reality, applications will go down but it is important to have the communication queued and safely handled without duplicating/missing the communication. This is where Messaging Queues comes into the context that guarantees "assured message delivery and Time independent processing". IBM MQ Light is a Messaging software that is available as a developer sandbox and in IBM Bluemix. A developer can seamlessly deploy the application developed & tested in the sandbox, into IBM Bluemix (or even IBM Websphere MQ). Mostly considered for asynchronous communication.
  • Application in a Cloud to on-prem resource
    
    • A common scenario for this integration is, SOE (System of Engagement) hosted in a cloud integrated with SOR (System of Records) that is hosted on on-prem. Some examples of SOE are UI for a bank that is used by the customers to check their bank accounts. Example of a SOR could be the relational database that stores financial transactions/customer information.
    • IBM Secure Gateway in IBM Bluemix is the preferred way to connect a public cloud application to the on-prem resource. This establishes a secure tunnel between IBM Bluemix organization and remote on-prem resource.
  • Applications hosted on different Cloud vendors
    
    • Integration between two SaaS applications. Considering the advancements happening the PaaS and SaaS areas, most of the cloud vendors are self-sufficient or very quickly enhancing with new/missing services, therefore this integration is not popular in the market. However, IBM Websphere Cast Iron Live in IBM Bluemix could be leveraged for integrating two SaaS applications hosted in different Cloud vendors.
    • One example scenario could be to synchronize events in CRM calendar system with customers Google Calendar to enable customers to see when a meeting is scheduled on the CRM ticket etc.,

To conclude, here are the considerations that could be adopted for Cloud Integration situations

• SaaS to SaaS integration - consider IBM Websphere Cast Iron Live
• SaaS to on-prem - consider IBM Websphere Cast Iron Live
• Public Bluemix to on-prem - consider IBM Secure Gateway
• Public Bluemix to IaaS - consider IBM Secure Gateway

 

IBM Watson Analytics for data insights - explained with a home infrastructure monitoring system

Gone are the days when we would spend countless hours developing intuitive dashboard software that displays/monitor key performance indicator (KPIs) to help the executives steer the company. IBM Watson Analytics moved several steps ahead by digesting any type of data for any domain, deriving relationships between the data, often making it a child's play to explore the data and finally providing a way to predict the future based on existing data patterns.

If you know what your data is about, leave the rest to Watson. All of this is on the cloud (no installation required) and free to try. Trust me it will not take more than 10 to 15 minutes to experience the power. Here is where you need to click https://www.ibm.com/watson-analytics and try it for FREE!!

To understand more let’s scale down from a corporate dataset to a home-based dataset. Like electric power and cooking gas, the internet became the new normal and essential supply for most homes. Though busy with our day jobs and personal work at home, it becomes important for us to monitor all of these essential services that will make our living comfortable and help us deliver better. 

We often have questions like …

- When was my electric power supply hit peek voltage, how many times it happened in the last year and when can it happen again this year so that I would keep my high-value electric equipment safe.

- How is my broadband speed changing over a time period, when was it last disconnected, how many times it got disconnected so that I could measure the quality of service?

- How are the temperature and humidity changing in my house and does that have any relationship to the seasonal flu

- How are my security cameras performing, its essential to get a notification when any of it is disconnected

At a high level, there are two components that could be explored to answer the questions like the above

1. A Raspberry Pi doing all the 24X7 work for you. Attached with SenseHAT for temperature and humidity sensors for continuous monitoring, logging, and reporting in real-time

2. IBM Watson Analytics to load the historical data, explore and predict future outcomes.

Raspberry Pi uses

- Node-Red service with Dashboard and SenseHAT modules added

- speedtest_cli to measure the bandwidth speed from command line

- OpenWeatherMap REST APIs for reporting the location weather

- email / SMS integration for notifications.

Node-Red orchestration displays a live real-time dashboard and logs  the data into CSV files

In the Node-Red drag-drop the required nodes and build the flow that …

- Checks the internet status at regular time intervals and log the ‘Internet UP/DOWN” status

- Collects the bandwidth upload and download speeds on regular intervals and logs them

- Collects the Raspberry Pi temperature on regular intervals and logs them

- Collects the ambient temperature and humidity on regular intervals and logs them

- Collects the location temperature and humidity details on regular intervals by calling the OpenWeatherMap REST API

- Constantly keeps track of local disk space and automatically moves CSV files to an external disk

This list can go on and on like monitoring the security cameras and automatically clean up the cloud storage space that stores the recording/motion detected photos etc.,

The Node-Red Dashboard module is used to build a simple auto-refreshing dashboard that is a  single page view of the key indicators.Here is a github project of the same that could be used for free: https://github.com/srinivasgnv/cmu

Here is an example of the real-time dashboard

When all of this data get’s collected for, let’s say for a few months/year(s) users can perform analytics on this data to derive insights and answer cognitive questions like above. To do this register to IBM Watson Analytics and follow these simple steps.

- Upload the CSV files (make sure there is a heading for each file describing the columns )

- Use the Explore option where Watson digests the dataset and automatically comes up with interesting relationships in the data

- Provides an option to ask questions and get answers

- Predict future outcomes.

All of this is without writing a single line of code. Yes, you read it correctly. You have to try this to experience the power of Watson Analytics.