Complex Event Processing (CEP)

In his keynote speech at Java One, Thomas Kurian, Oracle’s vice President, covered three main facets of Oracle’s Fusion Middleware: Grid Computing, SCA based SOA and EDA.

While I have already written on SCA and SOA, in this post I will quickly introduce EDA in general and Complex Event Processing is specific.

While Oracle already offering EDA based technologies for last few years in the form of RFID- EDA, event based monitoring in BAM, and event-condition-action (ECA) in a form of ESB and Business Event System(BES), CEP is distinctly newer and an important addition.

In their research paper based on their work in Stanford University, David Luckham and Brian Frasca, defined Complex event processing (CEP) as a processing technology for extracting and analyzing information from any kid of distributed message-based system. In their work on this topic, they demonstrated use of CEP for: Casual event histories, Event Patterns, Event Filtering and Event Aggregation. They demonstrated these use cases using Rapide toolsets. A much more work in this field with "different" approaches have been done at various academic including Berkley, UC-LA, as well as industrial institutes.

In my opinion some basic attributes of a stream based CEP systems are:

Streaming related attributes 1. A very high volume 2. a stream based events 3. in a period of time

and processing related attributes (ability to apply processing logics/functions)1. correlations, 2. statistical/aggregation/set processing, 3. pattern matching or 4. predictive Analysis

Today there are many applications of CEP in enterprises from a low level monitoring to intelligent processing. Some practical and familiar use cases can be listed as:

Medical pandemic detection

Fraud detection

Intrusion detection

Algorithmic trading

Traffic hotspots detection

RFID based shipmen/arrival/sales patterns

Stock trading patterns

And many more

A couple of years back, Oracle too joined the efforts and made a significant value addition in this field. A colleague and a good friend of mine, Shailendra Mishra, is leading the work. Oracle now jointly contributing with academicians and other industries to develop SQL like standard language for a stream based event processing. This work is an extension based on academic efforts to develop a language for Continuos Query Language(CQL). When I was working with him to develop a demo for JavaOne, he also mentioned that how easily he could solve the algorithmic trading problem using our tool. Though CEP is already being used for the algorithmic trading, our distinction is in an easy and expressive(declarative) way to solve the problem. Moreover, performance and scalability are two more important aspects. I would say we have done a great work! And we would need to continue to do more.

Moreover, a much can be written and needs to be written to cover the depth of the topic, however, for this entry in todays blog, I will take a pause...

Meanwhile here are some excepts from our PR on this topic:

Complex Event Processing for EDA

Organizations are increasingly adopting EDA as a platform to effectively manage the increasing number of events generated from IT systems, business processes and physical sensors such as RFID. With this in mind, Oracle is developing new Complex Event Processing (CEP) capabilities that support high-volume, programmatic analysis of events to identify patterns and correlations across multiple heterogeneous event sources. These capabilities will complement Oracle's existing EDA offerings, such as Oracle Business Activity Monitoring, that provides real-time operational dashboards for tracking business key performance indicators, multi-channel alerting, and invoking automated or manual response actions.

CEP