AMQP in 10 mins : Part1 - Introduction

Advanced Message Queing Protocol is a an open standard with royalty free use. It has a strong focus on the financial services industry and provides the performance and reliability required by the said industry.

AMQP Working Group
The AMQP working group is responsible for defining and maintaining the sepcification. The website www.amqp.org provides details on licensing, FAQ, JIRA and a list contributors and a download page.

Version Numbering
The AMQP Version Numbering scheme consists of a major and minor revision number. Ex: 0-10.
Major versions > 0 will provide backward compatibility between minor versions. The current version of the protocol is 0-9. The AMQP WG is expected to release the 0-10 version shortly. Currently the specification is in a rapid growth stage.

Interoperability
AMQP provides complete interoperability via

• A network wire-level protocol
• A defined set of messaging capabilities a.k.a AMQP model

One can partially imply the semantics from the wire-protocol. However we need to define the semantics explicitly in order to guarantee exact behaviour in each broker implementation.

Reliability
The 0-10 (in-progress) version has provided functionality to ensure Guaranteed Delivery to satisfy the level of reliability required by the financial services industry. I will cover this topic in detail in a future blog post as part of the AMQP in 10 mins series.

Full Transaction Support
The 0-10 spec also provides protocol level functionality for distributed transactions. This topic will also be covered in detail in a future blog post.

Security
Security is the main focus for 0-11 version of the protocol. Currently there is only rudimentary support provided via TLS. Planned features include but not limited to partial encryption and payload signing.

Known Implementations
AMQP is platform agnostic and language neutral. Currently we have implementations from a wide variety of languages from procedural to object oriented to functional.

The Apache Qpid project has broker implementations in java and c++ and client implementations in java, c++, python, ruby and .NET.

RabbitMQ provides a broker implementation in Erlang and a client implementation in java

Open AMQ provides broker and client implementations in C.

What about JMS?
JMS can be implemented on top of AMQP. The AMQP WG is also working on standardising the AMQP-JMS mapping and is scoped for 0-11 version of the protocol. This will enable two different JMS over AMQP implementations to behave exactly the same way.
The Apache Qpid project provides an implementation of JMS over AMQP.

Next Part : Part2 - Achieving Interoperability And Avoiding Vendor Lock-in

Prev Part : Part0

AMQP in 10 mins : Part0

From time to time people ask me what is AMQP? and how it is different from JMS? or why do we need another standard ..etc On the other hand the AMQP spec groups is finalizing it's work on the 0-10 version.

So I decided to write a series of blog posts titled "AMQP in 10 mins", where each part can be read and understood in 10 mins. I hope to write a part every working day, or more depending on my mood. Comments and criticisms are equally encouraged.

I am going to use the following terminology in my posts. Most of these terms have established definitions within the industry, but nevertheless I am specifying them for completeness.

Broker/Server :  Accepts connections and implements AMQP queuing and routing functionality

Client  :  Initiates a connection

Peer  :  Either party in a connection

Producer  :  A client that publishes messages

Consumer  :  A client that consumes messages

Message Queue  :  A named entity that holds messages in memory or disk

Message/Content  :  Application data

Durable  :  Entity survives a server restart

Transient  :  Entity doesn't survive a restart

Next Part, Part1 - Introduction.

Axis2 Clustering Support

Axis2 1.3 release has experimental clustering support enabling you to deploy your services in a cluster for Scalability, Failover support and HA. In simple terms, we provide replication for Service Context, Service Group Context and ConfigurationContext. This enables you to deploy your services in a cluster and share state. Please give it a spin and provide feedback via the mailing lists.
Please check the Axis2 Clustering Guide for a good introduction.



In Summary, here are a few points worth remembering

• For Stateful Web Services it is recommended to deploy in pairs using the primary/secondary model. i.e Each Node will replicate to one or more backups (secondary nodes) for failover.
  
• There will be only one active node at any given time. This is to avoid conflicts as we don't have a distributed locking mechanism.
  
• To achieve Scalability, you need to deploy several of these pairs. In other words your cluster will consist of multiple pairs with one active node in each - and your pairs are isolated sub groups with no replication between them.
  
• If your cluster consists of more than one pair, you need to use some sort of partitioning to avoid a user being serviced by more than one pair within a given session in order to avoid potential conflicts. You could use session affinity to achieve this.
  
• There is no such restriction for Stateless Web Services.
  
• If your Services read shared state more frequently and update very rarely, you could experiment with more than one active node. However conflicts are still possible.
  

Sanjiva asked me to help over a year ago and it got postponed month after month due to other commitments until I was finally able to produce a prototype. Many thanks to Afkam Azeez, Chamikara, Snajay and Sanjiva for the support they provided. Azeez and Chamikara was primarily responsible for bringing the clustering support to it's current state after I contributed an initial implementation. If not for those two it would have been next to impossible to get it included in the release. Special thanks to Philip Hanik for answering questions on Apache Tribes even while on vacation.

SCA vs Spring (A reply to Dan's post)

In Dans blog entry, he compares SCA vs Spring with an excellent example which highlights the similarities.

I do not view them as competing technologies rather complementing as I see some synergy between these technologies.

Clearly SCA and Spring share several design principles and it is no surprise that the SCA spec group views Spring as an implementation technology for components and composites. In fact the SCA spec group has gone as far as formalizing this approach with SCA Spring Component Implementation Specification

So Ke Jin's comment on InfoQ was right on the money when he said,

"As the matter of fact, the so-called SCA assembly model is merely a DSL (domain specific language) that can easily be realized on top of Spring or any decent POJO IoC containers, with few hundreds lines of code and half day of work"

On his InfoQ comment, Dan asks?

But why wouldn't I just use Spring to begin with?

The simple answer is that SCA could become a standard and the ability to deploy my composites in any SCA runtime. The other SCA runtime could be using Spring or the next great thing under the covers.

Mike Edwards points out on the same InfoQ thread

What the SCA Assembly model then brings to the party is a organized and standard way of describing how the different components in your (distributed) system are linked together to build a particular application.

This will only add value if and when SCA becomes a standard through OASIS and is adopted across different languages by the masses. Spring is great and I really like it. However distributed systems are not built only using java. It would be nice if we have a standard way of expressing how different components are linked together to build your application.

Also unlike the EJB's if SCA composites can be truly portable across different runtimes (Ex Tuscany java runtime to ABC java runtime) with zero code changes it can be a very good selling point. Vendors shouldn't spoil the party with vendor specific addons in their runtime which can lock-in unsuspecting users.

Dan asked

Someone show me where Ke Jin is wrong and where SCA expands on Spring capabilties.

Technically the capabilities are more or less the same, albeit spring being a more simple model. The standardization and portability of SCA (if achieved) is where it can exapnd or add value on top of Spring. I am not advocating to replace Spring with SCA. You can continue to use Spring to realize SCA. They are certainly complementary. This white paper explains how SCA can be used with OSGi and Spring.

P.S I conveniently ignored the complexity issue. IMHO it is a red herring.
Distributed systems are complex in nature. Thinking we can provide simple solutions is the biggest misconception.
EJB, WS-*, REST, SCA they all prove this one after the other. That's a good topic for my next blog entry :)

Being a single parent (for 2 weeks)

I had to go to Sri Lanka for my brothers wedding for 2 weeks. My wife was unable to make it due to exams as she is taking summer school. So I decided to go alone with my 3 yr old daughter. I survived the long journey to Sri Lanka thanks to my daughter putting up her best behaviour. The next day we had the wedding and again the little angle held her composure while wearing her dress in extremely hot weather during the photo shoot and church ceremony. We spent the next couple of days going shopping, a trip to the zoo and the beach. I had an extremely good time with my daughter and stayed away from my laptop (except during the Axis2 clustering hackathon at the WSO2 office in Colombo) for two weeks. I rarely checked email :) , instead I played with my kid and made the most of the time. She loved to go in three wheelers (tuk tuk) and insisted they were much more exciting than the kiddies roller coaster at Wonderland :). The return journey was a bit tough due to flight cancellations. We got stranded in Paris for a day, but used that time to relax and rest a bit before making it home.
I am well versed with diaper changing, feeding and putting her to sleep, as I take care of her when ever I work from home. But washing her at least 3 times a day and feeding 3 meals and putting her to sleep was extremely tiring. I wonder how my wife does it everyday while being a full time student. My daughter thinks Sri Lanka is much more exciting and charming with ants, flies and butterflies to chase around, cows and stray dogs all over the place, bumpy rides in three wheelers and grand parents who are ready to care for all her whim's and fancies. Sri Lanka is indeed a charming place.

He was a visionary.....

I think Sanjiva already blogged on this, but I feel obliged to do so as well.
If not for his efforts and most importantly his vision, Sri Lanka would not have progressed this far in IT. Especially his efforts in the university circuit opened the doors for many kids who otherwise would have missed the opportunity to study IT. I have met him once or twice at (ICT) Institute for Computing Technology at the Colombo university while I was a student when we were organizing an IT exhibition. If I am not mistaken, the ICT itself was his brain child. He was a pioneer in getting the ".lk" domain name.
Sometimes his work was disrupted by political pressure from various sources which didn't like the contributions the he was making. However he continued to push towards his vision until his last breath.
Well done sir!!!! You made us all proud.

Mahela, I thought u will pull us through !!!

I know enough has been written about the world cup dissapointment, but sadly I haven't got over it yet. It was a very bad day for us. It started with rain and a shortened game which suited the aussi style of power hitting. The ICC which is quickly turning out to be a joke, and is run by a bunch of idiots who are pass their sell by date, should have postphoned the game. It's a shame that the 100 overs was not played in a world cup final !!!!
We were at a huge disadvantage with the light deteriorating by the minute and the batsmen under pressure to stay on top of the duckworth lewis score. But yet we displayed courage and fought till the end. However credit should be given where it is due and I have to say Gillchrist played a one of a kind ininings.
I knew mahela from school days and people thought he would be playing for Sri Lanka when they saw him batting at under 13 level. I was in a match with Nalanda college at under 15 level. And Mahela single handedly took that game away from us. He was always a big game player and thought he would pull us through, just like in the semi finals. But a bad umpiring decesion ended all that.
I am looking forward to 2011!!!!

My take on JSR 311, Java API for RESTful Web Services

I think doing this inside the JAX-WS space seems to give the wrong impression. I heard that they were proposing to tunnel REST services through JAX-WS endpoints. But Dan sent me the following link to a blog post by Marc Hadley which gave me some hope.

Sun always messed it up with Web Services. JAX-RPC and JAX-WS were horribly complicated with no apparent benefit other than to say the implementer is J2EE compliant. What a price to pay to have that check box in your marketing literature. They may very well make the REST API equally useless by tying it to those JAX-WS API's.

Anyways we already have an API for REST in the form of servlets and JSP, but I agree that they weren't good enough. IMHO they should have improved these APIs and add REST support over there, not inside JAX-WS which will fuel the myth, that REST is an alternative to SOAP, instead of REST is an alternative to SOA.

All REST API's I have seen so far, are high level annotation driven API's on top of a HTTP binding (servlets). For example what Dan has in XFire. I strongly believe that REST support should go into an HTTP API (servlets) instead of a WS API.

Benchmark Wars

The recent posts by Paul, Dims and Dan have provided benchmarks pitting Axis2 vs XFire. I think that is a very good step and provides end users some good information. Also I would like if everybody provided the source code for these tests so non believers and end users can run it for themselves.

However, racist ranting with no performance numbers to back it up is not acceptable at all. If a person cannot explain a technical argument without subjecting him self to such a low level, then I don't think the rest of world should care.

Dan I welcome your effort to work out the performance numbers. That is the right approach. Healthy competition will only promote WS-* and help dispell the myth that SOAP is slow. If either Axis2 or XFire is slow in this round then let the respective teams make sure that they make an effort to beat the other in the next round. Now that is whats going to make each other better, not unwarranted racist comments that shows your sorry upbringing and publicly disgracing your parents.

However Dan I believe that most enteprise systems do care about RM and Security (Addressing too). So there will be SOAPHeaders. With more SOAP stacks supporting these specs there will be an increase in the adoption of these standards in the enterprise world and so IMHO the SOAPHeaders should be part of the performance tests. However I also recognize the need for being efficient in the POX case too. As Dan said the POX use case maybe a popular one with SME's. If Axis2 is lagging behind in that area then we as a team can work on the performance.

As for Hani, he is still motivated to spew out his garbage for the very same reason why Howard Stern is still in business. There is and will always be an audience for these type of people. Some would call it "Freedom of expression", others like me would see it as an abuse of the that very sacred principle.

I am gratefull to see the new year !!! (Please wear seat belts)

I am extreamly gratefull and feel blessed to see the new year considering the motor vehicle accident, I was involved with on the 30th Dec 06. It a was multi-vehicle accident (1 van and 3 cars) that was nothing short of a miracle considering every person involved made it through with only whiplash and one person with mild shock.
It was a routine grocery trip to costco and we were heading back home. I stopped at the lights with just one vehicle in front of us. I heard a bang behind us and before I could even think about it there was a massive bang and our car was violently thrust forward and rear ended the car in front of us. As it turned out a van had rear ended the car behind us and that in turn rear ended our car.
When I looked around I saw the car that hit us has rolled into the ditch beside the road. The car was squashed so bad I was expecting the worse. The van was thrown on to the lane right of us and has stopped parallel to us with the engine collapsed and fuel leaking towards our car.
I quickly got out and yelled to my wife to take the baby out of the car seat and we all moved on to the ditch as there was a risk of fire. The ememergency vehicles arrived in 5 mins and the firefightes took control of the situation.
All passengers were saved bcos of seat belts and considering the damaged to the rear of my vehicle I am glad that my daughter and wife were ok. Three vehicles including ours were totaled. Property can always be replaced but not human lives. So please wear seat belts and be alert when you are on the road.

Last year was the toughest of my life so far. It was also the year which I learned so much about life and realized how precious your family is. But I am proud of my fighting sprit and the effort to keep the ship steady through all the troubled times. Humans have a remarkable ability to adapt and fightback when the going gets tough.

Wish you all a Happy and a Blessed New Year.

What is a RESTful Web Service ?

I have to admit that I am very confused about the term "RESTful Web Services". REST is one of the most highly missunderstood concepts out there and certainly one the most highly abused terms. For most REST means POX/HTTP GET or POST, simillar to some folks, who think they are doing "Web Services" simply bcos they are using SOAP, WSDL etc while there is utter disregard to loose coupling, service orientation ..etc.

I also see the endless debates on SOAP vs REST. I guess this stems from the fact that most think, REST is POX/HTTP GET or POST. While SOAP is a message format, REST is an architectural style. So there is absolutely no point in comparing apples to oranges.

So what is a RESTful Web Service ?

I have been talking to Roy Fielding on email trying to understand REST and have also been bugging Sanjiva on IM trying to clarify the facts. REST takes a "resource oriented approach" while Web Services (SOA) takes a "service oriented approach". One can argue that a service is a resource (from a REST POV) and a resource can be exposed as a service (from a WS POV). But there are some fundamental differences in these approaches. IMHO a service should be a more coarse grained concept than a resource. A service may encompass one or more resources. I assume what RESTful services means, is that the respective Web Service obeys the REST principles. How possible is that ? To me this is still a grey area and I will continue to ask this question until I understand this properly.

Web Services is NOT a Distributed Object access Architecture

It is surprising that many people still think of Web Services as a Distributed Object access architecture like CORBA or COM.

This is quoted from the post by Leonard Richardson http://www.crummy.com/writing/REST-Web-Services/

“The answer is that "Web Services” aren’t the web. They’re a heavyweight architecture for distributed object access, like COM and CORBA"

This is very misleading. Web Services is *NOT* a distributed object architecture.
WS is not object centric and has no notion of an object reference or distributed garbage collection.

So it is wrong to say *LIKE* CORBA or COM

A Case Study for Synapse

I wrote a Case Study for Synapse and it's available here

ApacheCon US 06

Finnaly I settled down to write something about Apache Con US 06, eventhough it's been about 2 weeks since I've been there. It was held in Austin, TX and I have to say it was very good. I went out for dinner a couple of times, and enjoyed all the live music that was going on, on 6th Street. Well they say Austin is the live music capital of the world....not sure I agree there.

Anyways I had a good time and met a lot of friends that I have only communicated through email/IM. I also presented on Axis2. The focus was on the usage of Axis2 with spring, pojo support, embedding being the main topics I coverd during the presentation.

I loved the keynote by Dr. Cliff Stoll and I also managed to get Howard Taylor a cartoonist to draw a small caricature of me with a cricket bat.

I met Roy Fielding the guy who invented HTTP and REST. But what really spoiled the whole thing was when I realized that somebody has taken my digital camera. I am pretty sure it was the hotel staff.

My First Podcast

I finnaly started to blog again. The last time I bloged was in January and then I slacked off.
So this time hopefully I will keep up to it. Lots of things happend between January and now and I feel bad that I didn't blog about it.

Anyways I was fortunate enough to do a podcast about Apache Axis2 with Rich Bowen for feather cast. This was done at the Apache Con Asia 06 where I was a speaker for the first time at a conference of this magnitutde. Here is the link to my podcast http://feathercast.org/?p=26