Andrew, great post. Obviously, part of the answer is that in addition to being the leading ecommerce player, Amazon is a serious infrastructure provider with Amazon Web Services, i.e. they know technology, scale, etc.

Now, I'm going to show my technical ignorance though. Is Amazon.com doing a feat so different than half a dozen of the other most sophisticated sites - the "portal" pages of the Googles, Yahoos, MSN, etc?

And where does caching fit into this? Is caching much more associated with the feed-oriented info of the portals, where Amazon is doing live web service calls?

Could it be that the vast majority of Amazon web service calls are to a handful of their partners with whom they share some sort of private fiber network?

My question is partially about Amazon.com and partially about SOA practice here is the biggest-little.

I first learned about Amazon's SOA approach when I read the ACMQueue interview with CTO Werner Vogels. I have been a part of teams that have written distributed systems and so have seen some of the implementation approaches but they were always more tightly coupled than those espoused in SOA. My feeling is that I have not done real SOA yet.

Complicating my distributed systems view is Pat Helland's paper on reconciling data distributed within a system. The key idea is that when receiving outside-data you must integrate it with your inside-data rather than simply inserting it. The state of the requesters inside-data is not necessarily the same as it was when the request was made for the outside-data. Further, the outside-data may no longer reflect the state of the sender because it only reflects the state of the sender at the time of sending and not the time of receiving. There is no global consistent state: There is only an advancement towards similar states. A settling down.

My head is beginning to emerge from one client's projects and so I am at a point where exploring new technologies is fun -- rather than the feelings of frustration stemming from being "stuck" with your project's existing technologies. Is the SOA grass greener?

I think SOA definitely makes sense when scale becomes a concern. I worked at amazon from 97-05 and got to experience the move from monolithic 2 tiered to the distributed system it is now.

Really what they have done is make it logical for many small teams to be productive and work together at the same time. Each team is in charge of a service with a published API that others program to if they need to interact with it. For amazon it makes a lot of sense where you literally have 3-5K developers. It just gets to be too much having them all try to work on one system. SOA to me is more distributing your work force than anything.

For batchblue it has not made sense yet since we have 2 developers. Distributing things up would probably be more of a headache than it is worth right now. The work has not required us to distribute things much but as we hopefully grow that could change. I am looking at rolling a separate 'service' to handle data syncing and it would very much be in the style of amazon. Basically an internal webservice that our main application can interact with but the biz logic is maintained by the sync service. Doing this would greatly decouple a sync service which does not need an http interface to work so there is no need to cram it into our existing application. 

As far as data integrity it can become compromised at times but I think you got to live with that in some cases. A great example of these was amazon's shopping cart. Back in the earlier system you were looking at numbers  as high as 50% drop rate of customers shopping cart information. Just completely gone which is not a great customer experience, especially if you spent all day adding xmas presents to your cart.  A great deal of time was spent to get that number to 100% but it never changed. Perfection in this case was a red herring. 

In came a distributed model using Werner's whisper client protocol where distributed servers would replicate messages out across the system. Each server would then pass the message along to the next until everyone on the system eventually became notified of a change in the customers cart. So now with the distributed model you were pretty much guaranteed a server in the system had at least had some of the customers cart information if not in most cases all of it. The dropped cart number went down to 2%. Not perfect but probably close enough. Obviously you need to be better when it comes to handling money and such but it is not such a bad thing to look at your serve and see where you can handle some failure. I know a lot of folks wish twitter failed with a little degradation of service rather than the whole thing always shutting down...

Jack is also right on to think about the network. Amazons network and system engineers deserve as much credit as the developers for amazon speed and ability to scale. A lot of innovation has/is happened with how amazon packages data and leverages the network to keep things fast.  Amazon makes good use of router technology and every packet matters. This is the part where it gets hard for small companies to compete and why google and amazon are so ahead of the pack. They are leveraging every conceivable part of the system.

Anyway I babble and it is late...