My recently published SOA World article covering OSGi's state of art and future

SOA World recently published another article co-authored by me. In this article on OSGi, we, Dave Chappell and me, cover some basics of OSGi and discuss its relationship with other emerging technologies and frameworks like SCA, Spring, etc. We also cover some work in the area of distributed OSGi and future direction of OSGi.

InfoQ: SCA Interview

A recently published interview of SCA TC members by InfoQ covered many debated issues. Most of the issues are already debated outside and many like myself already commented on the same. However, it would be informative to read perspective from SCA drivers (just FYI I am too member of SCA-Assembly, SCA-BPEL and SCA-Binding TC). Some of the important issues discussed in the interview are: SCA and JBI, SCA and WCF, OASIS vs W3C, SCA and J2EE etc.

Here are some excerpts(verbatim from above InfoQ interview):
SCA & JBI:
Technically there is no overlap. SCA can be used on a JBI runtime, but it is also possible to use SCA without JBI and JBI without SCA. JBI is engine-to-engine. ...SCA looks at defining abstract notions of service invocations at design-time.

SCA & WCF:
Windows Communication Framework (WCF) has some of the features of the SCA service component model, but there is no real equivalent of the SCA assembly model for the composition of applications.

SCA & JEE:
SCA works with JEE rather than competing....(however)..in the context of SOA, or composite applications, there is a need for more facilities and capabilities than JavaEE provides, elements that are beyond the scope of JavaEE....Using SCA, The runtime might still use JMS and one of the JAX SOAP stacks, but their APIs would not be visible to the developer.

SCA-J not being in JCP:
1. SCA as a whole is not a Java specification
2. making liaison between the Java group and the other groups more difficult.
3. It also keeps all the SCA specifications available under one, easy to understand license.
4. the SCA-J team really is looking at how to map language independent notions onto Java

Choice between OASIS and W3C:
OASIS are about business and implementation standards more than W3C.

Conversations in SCA (Service Component Architecture)

Services often interact multiple times to complete a business activity. Such interactions are said to be part of a conversation. In the Database world, a series of database activities are grouped as a part of a transaction. The notion of transaction was extended to Distributed Computing too. In the world of distributed computing, many frameworks and services came up to support distributed transactions. When webservices evolved, standards like WS-Transaction and WS-Transaction Management (WS-TXM) tried to solve the transaction problem. However, a conversation is not a transaction. It does satisfy ACID test of transactions. Rather a conversation is just a way of identifying a series of interactions as a part of one larger activity. In other words, it is like correlating various service invocations related to a single business activity. Hence, conversations and transactions are handling with different constructs in SCA.

Multiple efforts have already been made to standardize conversations in Web services. A long time back, in 2002, HP submitted a specification to define a Web Service Conversation language (WSCL), which was a supposed to be supplementary to WSDL. However, it did not receive any support. Last few years parallel efforts were made at OASIS to standardize two competing specifications, WS-Context and WS-Coordination. Both of them at a very higher level defined notions of context and frameworks to manage them. ebXML supported its own way of conversational trading interactions. BEA implemented annotations based conversations for JWS. SCA’s proposal can be viewed as evolved from annotation based conversation and additional declaration of the conversation intent in policies.

The concept of conversation in SCA is very simple. A conversation may start with calling a method of conversational service (annotated as @Conversational). The conversation continues till a service reference created at start of the conversation is being used. Finally, the conversation ends with invoking a method annotated as @EndsConversation or the conversation timeouts due to being idle for a long time or continued more than a maximum age set for the conversational service. Any further attempt to participate in the “ended” conversation results into an exception, namely, ConversationEndedException.

To make this working in SCA, in addition to annotations in WSDL or Java sources, a conversational intent needs to be declared in policies for the conversational interfaces. Policy interceptors and binding components would then handle the work of populating the conversational context and interact with conversation framework. Thus the developers of services would be spared from the overload of managing conversation.

However, SCA does specify any implementation framework for managing conversation. Rightly so it does not even define a format for conversation identifier. SCA conversation intended to be complementary to various existing standards. Why re-invent the wheel? So one may use a sequence of WS-RM, or correlation id of JMS, context of WS-Context or WS-C, or MessageID of WS-Addressing as a conversation indetifier and use policy/binding implementations based on WS-Addressing, WS-RM, WS-Coordination, WS-Context for conversations.

Like BPEL, SCA assumes that conversations may also be long running. In case of BPEL, a conversation is maintained by using a concept of correlationSets, which can be mapped to a payload. Oracle’s BPEL PM additionally supports WS-addressing based correlations. BPEL does not support explicit end or aging of the conversations. SCA containers would need a framework to handle end of conversations. Meanwhile, a lightweight conversations based only on conversation ID may be sufficient for many applications.

I am attaching a simple diagram representing a conversation between two services. It is based on a business interaction to configure and purchase a PC.

SCA

JSR-312 launched to provide JBI 2.0 : (Too?) ambitious to deliver in the given time fulfill all goals?

A technical committee (experts group) has been formed for JSR-312 to define the next version of JBI, version 2.0. Though software vendors like Sun, Borland, Iona, Tibco, Redhat, Webmethods (now Software AG) are participating in writing this specification, most of the major Application Server as well as ESB vendors are missing from the list. BEA and IBM were not in the camp of the supporters of JBI 1.0 too. Some other vendors from the JBI 1.0 are also dropped out because of their focus on SCA (now OCSA). Without a critical support from key vendors, a survial of such technologies becomes difficult(if not irrelevant). However, let me not prematurely declare a death. I would be happy to see some good results, like bridging gaps between various technologies, coming out of this JSR and such efforts.

I noticed many desired features in the wish list of this JSR. As noted in the JSR draft, the JBI 2.0 intends to be a target runtime for SCA. I had mentioned a possibility of such architecture in my previous posts. However, OSGI may too target to provide a runtime for SCA. There are some overlaps between OSGI’s Admin wire - DS specifications and metadata with SCA. Additionally OSGI provides features like lifecycle management, class loading, etc. Hence, the die-hard supporters of OSGI see OSGI to be evolving not as a runtime for SCA but replacing a need of SCA. However, OSGI's DS specifications still need to be enhanced to support enterprise features in multi-JVM environment. Interestingly JBI's JSR too mentions OSGi as “Enhancements to support full compatibility with OSGi, without necessarily requiring OSGi”. Since the Experts group is yet to meet and flush out the details, I do not give more information.

Coming back to the JBI2.0, it seems that the specification may add much wanted integration features like XSL, integrations with orchestrations (BPEL), Rules, etc to keep whatever limited i momentum it has in the market of ESB vendors where JBI1.0 got some traction with atleast few of the implementations supporting it. However, the scope and lack of technical clarity to provide a platform for some conflicting and some overlapping standards, (JBI1.0, SCA, OSGI, WS-CDL, BPEL,) and goals concerns me about a feasibility of this JSR to provide a “simple architecture in a timely manner” (as stated in JSR). I hope that in F2F meeting the expert group can have a clarity of realistic goals and a higher level architecture. If I could not participate in this JSR, I would at least keep a close watch on its technical direction.

JBI 2.0 would be targeted both for J2SE and J2EE. Leaders of JSR-312 and JSR-313 are yet to align their plans. I already raised this concern to both of both to Ron+Peter and Bill Shannon (leaders of the two JSRs)

JSR-313 for J2EE 6.0 received a setback (temporary)

An effort to standardize the next version of J2EE (6) as JSR-313 received a “temporary” setback at JCP, mainly due to licensing concerns. (Just to re-iterate my disclaimer: opinion/observations expressed in this blog are my own personal and does not necessarily reflect my employer’s views.)

A review ballot for JSR-313 was filed on April 3rd and was to be approved on April 16th. However, it was withdrawn on Friday the 13th!

From a sequence of events, I see interesting coincidences.

On April 10th, ASF sent an open letter to Sun over licensing issues related to J2SE 5.0. ASF has completed almost 95% compatible implementation and wanted to TCK license. However, the efforts failed to get mutually acceptable resolution. Hence, ASF went public with an open letter.

Though it is hard to establish the influence of the open letter on the fate of JSR-311. It might be more than a mere coincidence that ASF sent this open letter in the time window when this JSR was filled. More interesting coincidence is the comments concerning the licensing terms were expressed around the day when the letter went to a public. The letter created awareness and expressed the concerns of many others.

Anyway, since the next version on J2EE is an inevitable thing, I was told by the specifications leads that the JSR will be resubmitted after alienating the licensing concerns. J2EE6.0 would continue a Java's momentum to provide various SOA artifacts. This JSR specifically mentions that it would consider SCA artifacts. However did not mention JSR-235 or SDO. As you may know that JSR-235 was also withdrawn (by BEA and IBM) due to licensing issues. I understand that the SDO specification is also going to be resubmitted. I hope that it would also get added in this JSR.

The JSR proposes to include the following new JSRs:
* JSR-196 Java Authentication SPI for Containers
* JSR-236 Timer for Application Servers
* JSR-237 Work Manager for Application Servers
* JSR-299 Web Beans
* JSR-311 JAX-RS: Java API for RESTful Web Services

Other than these JSRs, there are some impacts on JAX-WS and JBI specifications too.

1. JSR-313 filing
http://jcp.org/en/jsr/detail?id=313

2. Comments and votes on JSR-313
http://jcp.org/en/jsr/results?id=4216

3. Open letter from ASF
http://www.apache.org/jcp/sunopenletter.html

4. ASF FAQ
http://www.apache.org/jcp/sunopenletterfaq.html

Announcements from WS standardizations world at OASIS

This month has been eventful for SOA related standardizations at OASIS, the international standards consortium. It has standardized WS-Trust, WS-Secure Conversation, WS-Context and WS-BPEL. The announcements came one after another… on March 27^th 2007 WS-Trust and Ws-Secure Communication, April 2^nd, WS-Context, and WS-BPEL on April 12^th. All these specifications took around 3 years of hectic joint efforts by their respective technical committees whose members came from various software vendors

WS-trust and WS-Secure Conversations provide additional building blocks on top of WS-Security. WS-Trust provides primitives and extensions for the issuance, exchange and validation of security tokens and dissemination of credentials within different trust domains. It will help to establish a programmatic trust for a secure communication between the communicating. The Web Services Secure Conversation Language defines mechanisms for establishing and sharing security contexts, and deriving keys from security contexts.

WS-Context is an outcome of an initiative called WS-CAF. The efforts began some time in December 2003. WS-Context provides a session construct in a decoupled SOA world. It is yet another but an important building block in WS architecture. Implementations of other runtimes e.g. WS-BPEL can leverage. Currently many implementations depend on features like Endpoint reference etc from WS-Addressing to represent some conversation or sessions in the WS world. As WS-Context provides a more generalized session model than WSA, WS-Context may find place in some of these usages. However, there is a competing standard WS-Coordination which is also OASIS specification. WS-Coordination is targetted to solve the similar problem and it has more momentum. It is interesting to note that it seems OASIS allows competing standards to be OASIS standards as long as they follow proper standardization process.

The OASIS also announced an initiative to standardize Service Component Architecture (SCA) as Open Composite Service Architecture. Considering the fact that SCA is fairly mature specification, I hope that it may take a little bit less to be the OASIS standard.

Service Data Object (SDO): with rich DAO/DTO featureset, a standardization efforts, and support from J2EE AS vendors, ready to take off?

Service Data Object (SDO) specification provides a uniform access to heterogeneous data sources like XML, database, web services etc. Even though there are already plenty access mechanisms and specifications like JDBC, JAXB, JDO, ADO, Entity EJBs, etc, SDO still stands out due to some useful features.

SDO specifications provides:

• Uniform access APIs to heterogeneous data sources
• Multi-language
  
• Maintains a disconnected data graph.
• A dynamic APIs
• Generate static APIs from the data source’s schema or UML.
• Xpath based navigation through data graph
• Change summary
• Validations and constraints
• Metadata access which is useful for tool builders.

SDO comparision with other data programming technologies
(table source Ref . 2 Next Generation Data Programming http://www.osoa.org/download/attachments/287/Next-Gen-Data-Programming-Whitepaper.pdf?version=1)

	Model	API	Data Source	Metadata	Query language
JDBC Rowset	Connected	Dynamic	Relational	Relational	SQL
JDBC cached Rowset	Disconnected	Dynamic	Relational	Relational	SQL
Entity EJB	Connected	Static	Relational	Java introspection	EJBQL
JDO	Connected	Static	Relational + Object	Java Introspection	JDOQL
JCA	Disconnected	Dynamic	Record Based	Undefined	Undefined
DOM & SAX	NA	Static	XML	XML Infoset	Xpath, XQuery
JAXB	NA	Static	XML	Java Introspection	NA
JAX-RPC	NA	Static	XML	Java Introspection	NA
SDO	Disconnected	Both	Any	SDO	Any

These rich features offer several benefits to different players:

For software architects and programmers it is useful to have a uniform representation of various data sources. As many other APIs a separation of data source specific APIs and business logic is highly desirable. With SDO, the interaction with data source is abstracted from application developers. Those who handle the persistent layer or provide a mediation framework would deal data sources.

A uniform access, metadata and dynamic APIs are very useful features for tools developers.

A disconnected data graph, a data change summary and optimistic concurrency would help application builders to build SOA oriented applications where disconnected client can manipulate data and then save to the data source.

Standardization of SDO:

In Nov 2003, JSR 235 was filed to standardize SDO in JCP. Unfortunately, due to some legal issues this JSR never made any progress.

However, in addition to BEA and IBM, many other firms like Oracle, SAP, etc joined the efforts to develop SCA and SDO specifications. As a result of this collaboration, Open Service Oriented Architecture (OSOA), a much mature version 2.0 was introduced in 2005. While this collaboration made a very good progress, the specification would not be standardized immediately. It would need to be submitted to some standardization body like OASIS, which would follow its own process to standardize. However, with the agreement of all the collaborating companies, the specification would serve as an intermediate but ‘de facto’ standard.

With Sun Microsystems joining the SDO and SCA efforts in July 2006, I hoped a revival of JSR-235. However, the SDO 2.0 was never submitted to JSR 235. Moreover, with multi-language support, the SDO 2.0 specification differed from SDO1.0.

Implementations of SDOs: In WAS6.0, IBM converted its WDO to SDO 1.0, while BEA added a SDO 1.0 based implementation in its Liquid Data. Some other vendors like Rouge Wave (HydraSDO) and Xcalia (XIC), SAP (NetWeaver J2EE 5 AS) introduced products supporting SDO1.0 while Oracle and others announced works based on OSOA’s latest specifications of SCA and SDO. Oracle added SDO support in its recently announced opensource Toplink.

Though SDO 1.0 introduced basic architecture and interfaces like DataObject, datagraph, it was incomplete due to a lack of specifications for Data mediation Services, a key architecture module, and other features. Thus it lost some portability across the implementations.

An Open source community is currently working on a project named Tuscany that would provide an implementation of SCA and SDO. Though there are high hopes about this effort, it still has a lot to be added.

Industry adoption of SDO:

For many reason’s SDO took a long time to gear a momentum. Most importantly it is a specifications, which did not get immediately standardized when it was introduced. Moreover, in the initial period, not all J2EE application vendors supported it. So the implementations based on the SDO were not portable across the J2EE application servers. Since the JSR-235 was stalled, the SDO lacked a wider visibility in Java community.

SDO: A plane ready to take off?

SDO version 2.0.1 is much more mature. It has added more languages like C++, PHP etc. There are plans to support C and Cobol too. However, there are some pieces in SDO architecture that are still scoped out of specifications. Most important scoped out feature is Data Access Service. If different vendors implement the DAS in making SDO importable on other application server, there would be roadblocks in its momentum. The success of SDO depends on its support on all J2EE Application servers.

SDO offer richer functionalities like change summary, dynamic APIs and metadata however these rich features should be implemented so that a performance of SDOs in access, updates and serializations is not affected.

As we already know, there are a lot of competing technologies and alternatives to access Data. Microsoft has ADO in its stack. WCF specific implementations may continue with the same. Many applications in Java spectrum may still prefer POJO,JDO, JDBC, JAXB for optimal direct access. However, features like disconnected model, change summary, multi-datasources would provide a sweet spot to SDO in the SOA world. SAgain, in that case, SDO need to have proper integration story working with current popular frameworks like Axis, WSIF, JAX-WS.

SDO may see a better support in SCA based solutions. Specifically some SCA implementation and ESBs may increase the adoption of SDOs. BTW, SCA itself does not strongly advocate SDO. One can have an implementation of SCA without SDO. However, within a SCA composite, one may find optimal use of SDOs as a DTOs transferring data on 'wires'.

Since applications leaders like Oracle and SAP are on board of SCA and SDO, if their applications also get aligned with these technologies, there would be a boost in arm.

Today, SCA and SDO have received a wide support from all major vendors of J2EE Application Server. If they deliver on their promises by adding SCA & SDO in their stacks, usage of SDO itself proves a "practical" advantage over other DTO/DAOs and SCA-SDO specifications get standardized, we would see a wide availability of products, tools and resources that would enable a large pool of developers and architects to develop solutions and products based on SDO. Finally, it seems that most of the stars are getting aligned! Hope for the best.

References:

[1] SDO2.0 specifications

http://osoa.org/display/Main/Service+Data+Objects+Specifications

[2] SDO whitepaper

http://www.osoa.org/download/attachments/287/Next-Gen-Data-Programming-Whitepaper.pdf?version=1

[3] SDO1.0 specifications:

http://xml.coverpages.org/IBM-BEA-SDOv10.doc

SDO
JSR-235

ESB vendors’s response to SCA

Most of the key ESB vendors are already participating in finalizing SCA 1.0 specifications. The list contains on Oracle, IBM, BEA, Tibco, Sun, Iona, Progress (Sonic), Jboss (Redhat), CapeClear, and more. (You may find a latest list at http://www.osoa.org/display/Main/Service+Component+Architecture+Partners)

Most of them see a value in SCA as a promise to have standard for a simplified and powerful Component model to assemble SOA based applications/integrations. They understand a need of a common framework to connecte and assemble SOA components together so as to deploy on various middleware including their ESBs.

(http://www.osoa.org/display/Main/Partner+Motivations)

The commitments from different participants may vary significantly. Some may participate to provide inputs and be part of standardizations with two fold focus: one to influence the direction and another to align their products. Some of them may start implementations based on SCA specifications. The early participants like IBM, BEA and Oracle would offer solutions based on SCA earlier than others. Apache open source’s Servicemix allows deploying SCA composite in its container. Other vendors may work on aligning their products to support SCA. However, some vendors may play wait and watch to see a market adaption and provide a choice when customers demand.

Irrespective to their commitment, they may support SCA differently.

There are three broader possibilities.

1. Build a platform based of SCA foundation: A compatible to SCA in a way SCA as a first class citizen. Such platform can be a monolithic or a modular with different engines for different component types like BPEL, Java, etc. Such implementation would directly use SCA assembly metadata.
   
2. Deploy SCA on their container: support deployment of SCA composite in their container. In this model, as a part of deployment, a container would consume SCA composite but may transform to its own runtime metadata.
3. Integrate: Integrate with SCA based container but the SCA based container is outside the core platform.

Many of ESB vendors took one of these approaches (or a last one “do nothing”) for JBI. For example, ServiceMix is based on JBI while Oracle ESB 10.1.3 provides a JBI container to integrate with ESB.

Here, I am giving a reference to JBI as an example of ESB world’s response to a standard specification. JBI and SCA have some overlapping and some complementary functionality. JBI and SCA is another subject to discuss in detail. Since we are on the subject, I would make a passing note that JBI provides a framework and runtime specifications mainly for integration architects and middleware runtime platform (SPIs) so that runtime containers can be interconnected. SCA specification does not dictate a runtime implementation. One may enhance a JBI based runtime for SCA. There is a little overlap but a very good value addition.

In Summary, SCA is a very good specifications for ESBs to provide SOA platform. We may see some innovation from ESB vendors while implementing it.

SCA

ESB

BPEL in SCA assembly model

(For basics about SCA visit:

http://khanderaotech.blogspot.com/2007/02/basics-of-service-component.html)

SCA and BPEL are complementary technologies. Though both have a mechanism to capture relationship between services, both do it differently. Moreover, BPEL is an execution language while SCA captures only the dependencies.

BPEL process can be an implementation type of service(s) within SCA. One may define service using BPEL and deploy the composite.

BPEL capture relationship between a process and an interactive web service as a partnerlink with different roles tied to port types. SCA maps the partnerlink with a single role (port type) to a reference. Thus, mapping between SCA and BPEL is straightforward.

Let us assemble a composite containing a BPEL process. I am using a stripped down version of a PurchaseOrderProcess given in proposed WS-BPEL 2.0 specifications. This process invokes two other services one to compute price and another to ship the product. These relationships are captured in three partnerlinks.

In case of WS-BPEL, a partnerLink defines two roles, one for the WS-BPEL process and one for the partner. In SCA, Depending upon a message flow direction, one of them becomes a reference and another becomes a service.

In our example, the BPEL process’s interface is exposed as a service entry point of the composite while other partner links are mapped as references.

Invoice (computePrice) service and Shipping service can be local or external to the composite. I am making both of them as external webservices. Thus, our composite looks like:

You may notice, wires between component and references. The BPEL process does not know what kind of implementation those references and what kind of binding it has. This loose coupling and flexibility is a power of SCA architecture.

Finally, here is the SCA composite representation for this assembly.

Disclaimer: The specs are still being evolved and may change. The representation is simplified for explanation hence it is not complete.

References:

1. WS-BPEL 2.0 : http://docs.oasis-open.org/wsbpel/2.0/wsbpel-specification-draft.pdf

2. SCA whitepaper: http://www.osoa.org/display/Main/SCA+BPEL+White+Paper

3. SCA - BPEL specs: http://www.osoa.org/download/attachments/35/SCA_ClientAndImplementationModelforBPEL_v0.95.pdf?version=2

SCA

BPEL