05/27/08 :: [SOA] Architecture of a $7B Loss [permalink]
I wrote this news for InfoQ last night. When you see the picture of the trader's P&L you realize how far a little CRUD can go.
He didn't know a thing about SQL injection, but I bet that over the years the systems "opened up" one modification at a time to eventually let anyone CRUD what ever they needed to make the system(s) of record look like what ever they wanted.
I admit that trading application are a bit extreme and time-to-market is usually prioritized over security or QA. But at the end of the day, this is what happens in CRUD-oriented synchronous client/server model.
In a COSC/S the developer can only guess what the user is doing when a particular line of code is executed: Human tasks and Resource Lifecycles (state machines) are not explicit. Nothing can really be audited or reported on unless you put a lot of energy into it. Authorization mechanisms are ad hoc.
If you take one of the technique the trader used: the "technical" transaction of entering a provision for the purpose of simulation, you realize how a simple lifecycle would have easily caught the problem. "Expirations" are annoying to code, you don't have a user waiting to see things that expire, so you need a notification system (the cheapest being an email), not to mention that you have to harvest periodically the things that expire. This is the prototypical example of how bidirectional interfaces and orchestration can help implement this type of functionality. This is just a few lines of BPEL when in a COSC/S this could translate into hundreds of LOC, at best.
Same thing is true of the 947 fake operations the trader entered in the system. I don't know much about the world of finance, but you would think that these fake operations have counterparts and therefore, if, as a resource, an operation could send event notifications to a coordinator which job is to make sure that all transactions have counterparts, nothing of that sort could have happened. A COSC/S programming model requires that you build such mechanism from scratch all the time: everything that's outbound, asynchronous and long running is such a pain to write that you implement these elements of functionality when you can afford it or have enough time.
Our world is bidirectional, asynchronous and long running, as long as we will continue using a programming model that does not take these aspects into account we will expose ourselves to this kind of CRUD.