Quick Summary of Business Events Support in SOA As11

Oracle's SOA Suite AS11 is a rendezvous of SOA and EDA models. The EDA functionality is served by Business Events Framework (called Event Delivery Network EDN) and Complex Event Process (CEP) are two main components.

EDN: EDN supports a loosely coupled event pub-sub model. From another J2EE server, a publisher can publish the events either programmatically using Java API from a J2EE container or declaratively via ADFbc Entity Objects. These events can be subscribed by zero or more mediator(s) or bpel(s). Both Mediator and BPEL can publish events too. EDN's event pub-sub model supports m:n relations. EDN has been implemented on top of JMS. It supports delivery policies like AtLeastOnce, OneAndOnlyOne, and ExactlyOnce.

EDL: Events are described using EDL (Event Description Language). Each event has a event name as QName and event payload as a xml element. Events carry properties, headers and a payload in runtime.

SCA: We have also made events as a first-class-citizen of SCA. In SCA composite, event publishing and subscribing can be specified.

Subscription: Event subscriptions are defined with event name and (optionally) Xpath based condition filter. While multiple components from one or more composites can subscribe an event, there can be only one subscription per subscribing component.

Testing: You can fire a test event using SOA console in Enterprise Manager (EM)

Monitoring: Events can be monitored by SOA console in EM or http://soaserver:port/soa-infra/events/edn-db-log.html

Sun ( Open ESB) joins CQL

OpenESB joined CQL (continuous query language) that is being promoted and enriched by Oracle's CEP team for a while. Sun's addition would be good for our efforts towards getting CQL standardized.

BEA too entered in CEP market ..

A report from Gartner had listed CEP in a list of rising technologies in their Hype Cycle. Per Gartner, CEP is important to give a competitive advantage in 2007 and it will be "requirement" in 2008. However, per Gartner, it would take a few years to mature the technology. Currently there are different players in the market ranging from niche vendors to one-shop vendors. Oracle, Apama(Progress), IBM, Tibco, Streambase, Coral8, Apsoft, KnowNow, are few to mention. Additionally, there are open source solutions like Esper and academic projects in various universities. Last week BEA too announced a first beta of CEP. It is an indication of growing interest in the field. More vendors entering into the market is generally better for maturity of the field. A standardization of language like CQL would be an important step towards achieving the maturity.
CEP

CEP and ECA / ESB routing services

Event Driven Architecture can be broadly classified into two distinct architectures: CEPs and ECA. Here are some basic distinctions. (some people classify into three: ECA/Simple events, Stream based and CEP thus separating streambased and CEP).

Streambased Complex Event Processing (CEP) systems process high volume of stream-based events received in a defined time window. On contrary, rule based or condition based systems respond to external situation/triggers, called as events, and typically ECAs deal with each event individually. ESBs routing service is an example of ECA.

Streams based systems typically process/joins a series of events to match a pattern or deduce an inference or predict a trend. In contrast, rule based ECAs typically follow a decision tree/table/if-then-else rules to forward the event some destinations or to trigger some actions. ECAs often deal with transformation of data and fan-outs.

Stream based system need to be optimized to give high performing required for a high volume of incoming stream of events.

In summary, stream based event processors try to detect a pattern from incoming data in a given TIME frame. Such processors most commonly do not "act" on "AN" event. They "process" a series of incoming data received in a given time frame. In other words, the stream based event processors "identifies situations" or generates events. (however, Some stream based systems work on "individual" event by setting the time window very small or via an attribute dictating the system to act on individual events as against a series of events.)

In contrast, rule based or ECAs “act” on individual events.

CEP

ECA

Complex Event Processing (CEP)

In his keynote speech at Java One, Thomas Kurian, Oracle’s vice President, covered three main facets of Oracle’s Fusion Middleware: Grid Computing, SCA based SOA and EDA.

While I have already written on SCA and SOA, in this post I will quickly introduce EDA in general and Complex Event Processing is specific.

While Oracle already offering EDA based technologies for last few years in the form of RFID- EDA, event based monitoring in BAM, and event-condition-action (ECA) in a form of ESB and Business Event System(BES), CEP is distinctly newer and an important addition.

In their research paper based on their work in Stanford University, David Luckham and Brian Frasca, defined Complex event processing (CEP) as a processing technology for extracting and analyzing information from any kid of distributed message-based system. In their work on this topic, they demonstrated use of CEP for: Casual event histories, Event Patterns, Event Filtering and Event Aggregation. They demonstrated these use cases using Rapide toolsets. A much more work in this field with "different" approaches have been done at various academic including Berkley, UC-LA, as well as industrial institutes.

In my opinion some basic attributes of a stream based CEP systems are:

Streaming related attributes 1. A very high volume 2. a stream based events 3. in a period of time

and processing related attributes (ability to apply processing logics/functions)1. correlations, 2. statistical/aggregation/set processing, 3. pattern matching or 4. predictive Analysis

Today there are many applications of CEP in enterprises from a low level monitoring to intelligent processing. Some practical and familiar use cases can be listed as:

Medical pandemic detection

Fraud detection

Intrusion detection

Algorithmic trading

Traffic hotspots detection

RFID based shipmen/arrival/sales patterns

Stock trading patterns

And many more

A couple of years back, Oracle too joined the efforts and made a significant value addition in this field. A colleague and a good friend of mine, Shailendra Mishra, is leading the work. Oracle now jointly contributing with academicians and other industries to develop SQL like standard language for a stream based event processing. This work is an extension based on academic efforts to develop a language for Continuos Query Language(CQL). When I was working with him to develop a demo for JavaOne, he also mentioned that how easily he could solve the algorithmic trading problem using our tool. Though CEP is already being used for the algorithmic trading, our distinction is in an easy and expressive(declarative) way to solve the problem. Moreover, performance and scalability are two more important aspects. I would say we have done a great work! And we would need to continue to do more.

Moreover, a much can be written and needs to be written to cover the depth of the topic, however, for this entry in todays blog, I will take a pause...

Meanwhile here are some excepts from our PR on this topic:

Complex Event Processing for EDA

Organizations are increasingly adopting EDA as a platform to effectively manage the increasing number of events generated from IT systems, business processes and physical sensors such as RFID. With this in mind, Oracle is developing new Complex Event Processing (CEP) capabilities that support high-volume, programmatic analysis of events to identify patterns and correlations across multiple heterogeneous event sources. These capabilities will complement Oracle's existing EDA offerings, such as Oracle Business Activity Monitoring, that provides real-time operational dashboards for tracking business key performance indicators, multi-channel alerting, and invoking automated or manual response actions.

CEP