Revisiting the Conway Law


 This is a cross-post from my new blog "unRelated". 

The Conway Law seems to be getting a renewed interest lately. In 1968, Mel Conway, then, a manager of peripheral systems research at Univac, devised:

"organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations"

This seminal paper is definitely worth a careful read. Mel's predicates are so visionary that we could easily believe that our ability to create complex systems has not improved in over half a century.

That being said, it might be timely to ask whether the boundaries of the Conway Law are still valid and if its context has changed. 

Insight Before Communication


Leadership tends to focus on the culture and structure of an organization to drive towards desired business outcomes with the expectation is that a better culture and enhanced structure will lead to an effective communication. Actually we are so desperate in our quest to find a better culture that some people go as far as suggesting that "stealing" (ideas) become socially acceptable.

It is interesting to see that in the process nearly everyone rounds off insight, often assuming a perfect ability to gain the correct level of insight into a system design or a question. Mel touches that topic slightly before focusing exclusively on the relation between communication channels and the structural aspects of an organization:

It is a natural temptation of the initial designer to delegate tasks when the apparent complexity of the system approaches his limits of comprehension. This is the turning point in the course of the design. Either he struggles to reduce the system to comprehensibility and wins, or else, he loses control.

I understand that culturally, anyone who questions his or her insight, let alone someone else's insight, will pay a high price for it. We often cover up a deficit of insight as a mere communication disconnect as everyone seeks to appear intellectually sufficient. Mel's predicate is both foundational and consequential because it anchors our perception that insight can't be elaborated and somehow, like knowledge, directly correlates with power. That thought is pervasive in modern organizations where the Sinofskys of the world strive as long as their levels of comprehension and control enables the organization to deliver something. I find it fascinating that 50 years apart, Mel was searching to answer the question "How Do Committees Invent?" and the prevalent culture in corporations like Microsoft or Apple abhors the "Design by Committee" process.

Can we still afford to round off insight? How and what can we really communicate without the proper insight? Don't we spend way more time communicating our insight rather than elaborating it? How many companies fail to benefit from the collective intelligence of their organization?

The Context of Design has Changed Significantly


I don't want to appear condescending to an era of incredible achievements but systems do evolve and new kinds of systems require new levels of insight to elaborate proper designs. Back then, most innovations were product centric. From the mid 80s and well into the late 2000s, innovation became service oriented, and today we are rapidly moving towards an "activity oriented" innovation model.


fig 1. From products, to services to activities

In a product-oriented world, consumers are left to compose individual products to create higher value use cases. In a service-oriented world, businesses identify some of these high value combinations and deliver them "as-a-service" to their customers. The advent of Mobile platforms makes it now possible to design systems that integrate readily with the activities customers are trying to accomplish.

In an activity oriented world, the "edges" of a system take on a disproportionate importance in the design compared to their relative sizes. The focus is no longer on systems and subsystems or even their orchestration into valuable services. The focus is now on understanding every activity customers are trying to accomplish and delivering a set of products and services that will directly integrate with these activities. The design of that integration, that edge, could be, actually, will pretty much always be, far more complex than the design of products and services that support them. You don't design "User Experience" as you design systems from subsystems. Designs need to start from the appreciation of the point of view of the end user, not just from the perceived value of a service or a product.

That change is profound because the design of an edge requires all constituents to cooperate and depart from the unidimensional mindset of product and service centric organizations. Any inefficiency in the scope or variety of edges will have a huge impact on the success or failure of the overall design. In an activity-oriented world, the ability to harness the collective intelligence of your organization directly correlates to your success.

This evolution towards more integrated systems has lead to the emergence of a new homomorphism between the type of insight different groups of people can elaborate and the structure of their organization. Even the very structure of our education system and hence hiring policies are now driven by that homomorphism.

It may not have been true when Mel wrote his paper, but today it is all too common for a group to formulate requirements based their insight to drive the design of other teams. As this new homomorphism developed  the relationship between systems and subsystems, as well as the relationships between products, services and activities made it nearly impossible to devise a structure that would nurture an appropriate level of communication with the goal of achieving a shared understanding.


fig 2. From insight to shared insight

As the levels of shared or aligned insight decrease so does the ability to create compelling and sound designs. Insight gaps develop as organizations tend to focus on building what they can understand as a subset of what their leaders can comprehend. Worst of all, some elements of the design can be built undetected until they fail their organization entirely.

A deficit of insight can be so costly to an organization and its shareholders that it should be accounted for in its balance sheet.

The Tools we Use to Communicate Impact Designs Negatively


The tools we commonly use to communicate would certainly influence Mel's conclusions. Popular knowledge tools (office suite, mind maps, requirements management solution ...) allow us to express ourselves with a greater productivity, but they may also hamper communication much more than we think. Why? The very structure that we use to communicate interferes with our ability to elaborate insight and ultimately designs.

For instance mind maps assume a hierarchical set of relationships, this is unfortunate because most of the relationships in the physical world are not hierarchical. That view seems to be directly inherited from Mel's era where we could still design products in terms of systems and subsystems. Similarly, when we use some "slide-ware" every system looks like a set of layers and when we use "row-ware" such as Excel or any Requirements Management tool to capture and communicate requirements, we cannot effectively represent dependencies, let alone track the elements of the design impacted by these requirements. These artificial knowledge structures are biased and negatively impact our ability to design a system.

Probably, the most important issue introduced by knowledge tools is that none of views that we create are representing any kind of dynamic behavior. How can we create an effective shared understanding between such a wide spectrum of people when everything that we communicate is using the wrong kinds of relationships and lacking even the most elementary dynamic view? Especially when we consider that human languages are poorly equipped to communicate relationships and dynamic behaviors.

A Call for Action


We must consider Insight and Communication separately. Mel's proposition looks attractive:

Research which leads to techniques permitting more efficient communication among designers will play an extremely important role in the technology of system management.

Yet, research that would lead to techniques permitting a more efficient elaboration of insight would have a far greater impact on the system designs.

Organizations should value teams that excel at elaborating insight. This is where education and HR policies should focus.

We have to collectively drive towards closing any insight gap and expand our shared understanding to a level that is compatible with success.

We need new tools and approaches that enhance our ability to elaborate insight.

The path to insight is not as hard as it looks, from federating intuition, to developing perception to growing appreciation and ultimately formulating the vision. Never again, should an individual feel that a group is limiting the design of a system. Never again, should the design of a system be limited by an individual. Never again, should a design reflect only a fraction of the insight of the overall group of designers.


My Favorite Quotes from Mel's Article

there's never enough time to do something right, but there's always enough time to do it over.

there is a homomorphism from the linear graph of a system to the linear graph of its design organization the realization by the initial designers that the system will be large, together with certain pressures in their organization, make irresistible the temptation to assign too many people to a design effort

One fallacy behind [the Accounting theory of management] is the property of linearity which says that two men working for a year or one hundred men working for a week (at the same hourly cost per man) are resources of equal value.

As long as the manager's prestige and power are tied to the size of his budget, he will be motivated to expand his organization.

Probably the greatest single common factor behind many poorly designed systems now in existence has been the availability of a design organization in need of work.

Elementary probability theory tells us that the number of possible communication paths in an organization is approximately half the square of the number of people in the organization. Even in a moderately small organization it becomes necessary to restrict communication in order that people can get some "work" done.

To the extent that organizational protocol restricts communication along lines of command, the communication structure of an organization will resemble its administrative structure. This is one reason why military-style organizations design systems which look like their organization charts.

Research which leads to techniques permitting more efficient communication among designers will play an extremely important role in the technology of system management.

the very act of organizing a design team means that certain design decisions have already been made, explicitly or otherwise. Given any design team organization, there is a class of design alternatives which cannot be effectively pursued by such an organization because the necessary communication paths do not exist. Therefore, there is no such thing as a design group which is both organized and unbiased. Every time a delegation is made and somebody's scope of inquiry is narrowed, the class of design alternatives which can be effectively pursued is also narrowed.

Coordination among task groups, although it appears to lower the productivity of the individual in the small group, provides the only possibility that the separate task groups will be able to consolidate their efforts into a unified system design.

It might conceivably reorganize upon discovery of a new, and obviously superior, design concept; but such an appearance of uncertainty is unflattering, and the very act of voluntarily abandoning a creation is painful and expensive.

Data services are often an essential part of mobile solutions. It can be pretty tedious to manage a data access layer (say written in PHP) and a database schema (say MySQL).

Over the past few months, we have built a number of prototypes and a fairly large scale project that is about to go in production using MongoDB hosted by MongoLab.

What we noticed is that all we had to do was to create a collection for each entity and everything else would happen in mdsl. I must admit this was a nice side effect of mdsl, not necessarily something that I had designed for, I was simply exploring what you could do with MongoDB. Of course, this works well for CRUD operations since there is no data access layer per se, but still, it seems to fit lots of mobile solutions like a glove. The freedom of not having a rigid schema and the ability to add attributes and rows by hand as needed makes a big difference when you are building a solution: all the boiler plate code that keeps the client, data access layer and schema goes away. I cannot emphasize enough how easy and fast developing a connected a mobile solution becomes.

Here is a typical mdsl connection definition to a MongoDB database:

The first operation fetches the entire collection of patients (of course not very realistic). The createPatient operation POSTs a new patient to the patients’s collection. The JSON body is constructed  automatically by the generated code within a “row” element as shown on the right.
Canappi’s knows readily how to create, read, update or delete these types of record.

The third operation, getPatientByEmail, implements a very simple query, using MongoDB query language and assuming the row element name is “row”. Here is what the resulting json document looks like in MongoDB, using the MongoLab console:

Here is how a join between two collections can be defined, assuming an event has a foreign key, itemid that points to an item detail document:

That's it !






Mobile Diet

No, I am not going to leave my iPhone or iPad alone for some months or scale down on using them. I am really talking about food diets. A completely different topic. As most people probably guessed by now, I am a gadget junky. I don't buy big toys (like cars), but occasionally, I like to have the latest gadget, just because it's cool. So I invested last June in a Withings scale. This is a beautiful equipment, easy to set up and completely seamless to operate. You step on the scale and it wirelessly sends you data to their server. You can display graphs and data on your favorite mobile phone or tablet.

Last November I reached 240 pounds (109 Kg). I never weighted as much before and when I saw the video of a talk I gave at l'Ecole des Mines de Nantes, I felt it was time to do something. But What ? yes eat less / better, exercise, ...

I have lost a bunch of weight twice in my life: by exercising for a whole summer and after I got maried when my wife cooked specials low cal meals. But now with two kids, jobs, and other stuff going on, nothing I could do really worked.

I even bought a fitbit to understand how much more I would have to exercise.  I would not recommend anyone buying this device, this is the most expensive useless gadget I ever bought ($99). It is basically a glorified pedometer and nothing more.

So since the end of November, in about 3 months, I lost 22 pounds (10 kg), but what's most important is that the withings scale taught me how to eat and ultimately how to control my weight. So basically I dropped calories from my meals until my weight started going south. I lost all this weight without going to the gym or starving. As you can see in the graph below, when I got the scale in June, I somehow eat a bit less, but my weight crawled up until November without really understanding why. I was fighting it, but without much success.

So here is my diet (I am sure you can find yours):

  • I eat the same thing at Breakfast and Lunch everyday:
    • An 8 grain roll and a coffee at Starbucks and a Honey Oat Veggy sandwich at Subway.
    • When I get home around five I eat a fruit or a yogurt.
    • I eat small portions in the evening.
    • If I can also do a veggy sandwich in the evening I do it.
  • Weigh yourself everyday, my weight curve was a huge motivator
  • Walk a bit more here and there

I never felt hungry the whole time.

That's it, nothing else, nothing more. No need to buy a book or tapes, cook special meals, kill yourself at the gym, weight anything. Each time I did a "real meal" you can see a step in the curve. This is when we had friends at home or we went out.

Yes, and my blood pressure went down too. I am still heavy (99 Kg) so I expected it will drop more on my way to 80 Kg.

So what I learned?

  • Eating out is bad, really bad for your health. We used to eat out at least twice a week, not to mention lunch. I am sorry to say that to the millions of restaurant and fast food owner, but what is convenient and fun is basically a silent killer.
  • It is amazing to see how few resources the human body can use to run all day
    • I can't imagine how much food is wasted each day for absolutely no reason
  • Salt does bring your blood pressure up
  • I know today how to control my weight for life. This is priceless.

Yes, my energy levels are very high, I feel really good and every morning I can't "weight" to jump on the scale.

I hope you'll find that helpful.

Full story »



Subbu wrote a post earlier this month on the Architect role, here he goes:

  • You always talk about the big picture.
  • You think you know how the system ought to be built.
  • You are unhappy that the team is not executing your ideas the way you want them.
  • You don’t have a working build.
  • You spend a lot of time on documents that are not code.
  • You can prototype – but your code is not production worthy.
  • You spend too much time in meetings.
  • The best code you wrote is a few years old.
  • When asked for opinions you tend to speak in general terms.
  • Your team members secretly joke about you.
  • You start to take analysts and tech blogs too seriously.
  • You are a dinosaur.

It's hard to disagree, though I find the list widely incomplete, specially when it comes to "everyone but your mum has an opinion on what the architecture should be", "vendors tell your boss you suck at archicture (unless you buy their product)"...

I am a bit more at odds with his conclusion: Code. Don’t wiki. Don’t powerpoint.

To code or not to code, that seems to be the question: in our industry these days, you code, therefore you are.

Just take any interview with Microsoft, Amazon, Facebook, Google ... and you'll face insipid Comp Sci 101 questions such as: how do you compare two binary trees? So, you bring 25 years of experience and passion, and the best and first thing these companies can do is to have a 25 year old kid ask you to write 5 lines of code. Everything else you have done is irrelevant. That is Subbu's world.

Allow me to take a real example to illustrate how a "coder" architects. A few years ago I was invited at Microsoft for a pubic event, and in a rare insight, a Microsoft architect was proudly explaining why Windows 2003 (R1 if I recall correctly) did not scale as a web server? The developer in charge of the connection manager had used a  ... linked list to manage the connections. I bet this guy had brilliantly passed all the interview CS 101 questions ... Shall we also speak about the kids at Facebook who code your privacy away? Isn't a primary key a convenient way to fetch your favorite piece of information?

The reality, is that Subbu makes the deeply and totally erroneous assumption that we have reached a level of maturity that gives us the perfect communication tools and hence with that in mind of course, writing code shall be the answer.

If you think that a) communicating, b) how and c) when you communicate is irrelevant, I would strongly suggest you watch this presentation from start to finish. Don't get me wrong, coding is great, I love coding, specially metaprogramming, but the question still remains, is an architect, JUST-A developer, hence Archaic, or is his or her role more essential (as in essence), i.e an Archetype ?

Why not think out of the box slide for once? beyond the trees, and the queues, the arrays and the lists, the maps and the sets? Why not ask how Model Driven Engineering could shape the role of an architect and establish a strong and precise articulation between architecture and coding? and with it, enable architecture refactoring ...

Let's review Subbu's list in the light of MDE:

You always talk about the big picture In MDE, you talk about the Solution Model with the highest degree of precision possible. There is no "big" picture, there is an objective, traceable representation of the blueprint of what you are building.
You think you know how the system ought to be built You don't know, with MDE you rapidly try different technical architectures and enable architecture refactoring, just in case.
You are unhappy that the team is not executing your ideas the way you want them A solution model can go a very long way in bringing people on the same page and eventually, the quicker you get on the same page, the faster you execute. Heck, you can even implement part of the solution model yourself.
You don’t have a working build With MDE, the build starts with the solution model
You spend a lot of time on documents that are not code With MDE, you spend all your time having a meaningful impact on the code.
You can prototype – but your code is not production worthy Your solution model is production worthy and drives the code of your team.
You spend too much time in meetings you don't have to spend much time in meetings. With a clear solution model, you facilitate meetings  and rapidly drive to the right conclusion be it technical or functional. How many people "scrumly" hash and rehash the same arguments for weeks, months and eventually request to build something that is not what they had in mind while cutting the scope to the bone, just to claim they shipped something? Who doesn't want more precision in meetings? Are PPTs or Wikis bringing that precision? Is code bringing that precision?
The best code you wrote is a few years old Your best models are still ahead, you love what you do and you can't have enough of 24 hours per day.
When asked for opinions you tend to speak in general terms when asked your opinion, you can articulate your ideas and the ones of others with the highest degree of precision, creating a clear understanding of their architectural impact and the corresponding level of effort.
Your team members secretly joke about you you feel part of the team and you communicate effectively your ideas. The code developers write is 10x more interesting to write.
You start to take analysts and tech blogs too seriously Analysts and tech bloggers have not idea what MDE is, by the time they catch up, you'll be retired.
You are a dinosaur yeah right ...

Shall I also mention that the ultimate beauty of MDE is "No Middleware"?

So, for a RESTafarian, who recently admitted:

Hypertext made a lot of sense when I was looking from the server. A few months spent writing real-world client code changed all that.

I am not really sure that Subbu is in the position to teach us some lessons about what to do, or not to do. Very few "Coders" seek to understand the context in which they code, or poke up to the solution model, and I am not even talking about the problem model. So yes I am disappointed, but not surprised to see someone the caliber of Subbu, thinking that we have to return to the early sources (so to speak) of software engineering. The RESTafarians have already thrown us back 15 years, what's another 15 years on top of that?

It's time to let the Architect surface the Arche (In ancient Greek Philosophy, Aristotle foregrounded the meaning of arche as the element or principle of a thing, which although undemonstrable and intangible in itself, provides the conditions of the possibility of that thing.)


I must admit, MDE is a joy to work with, especially as an architect. MDE allows you to express how a certain class of solutions should be built, far away from the intricacies of general purpose languages, conventional type systems and SDKs. It allows you to solve problems, in ways that simply would never have any room in a GPL world. I may never know if MDE would have changed the face of software engineering, but for an industry that has looked for the holy grail of programming languages for so long, a paradigm shift seems innevitable. If for nothing else, all our software development technologies and processes have been aligned along a "monolithic" assumption, when today solutions are composite and often requires several architecture variants to support to different use cases.

Let me take 3 examples from the Canappi 1.2 release. to show where and how MDE makes a powerful difference.

1) MDE enables solution architecture variants : one of the key features of 1.2 is an enhanced level of support for Universal Binary iPad/iPhone applications. You can now define device specific layouts which will be displayed when the application runs on that type of device. The kitchensink demo has been updated to take advantage of that feature. Here Canappi uses tablet variants, but it would be just as easy to add it for different orientations.

layout sessionDetailLayout_iPad {
    // iPad specific layout


layout sessionDetailLayout {
	text  sessionId	'' (-20,-20, 10,20) //hidden parameter
	text  sessionTitle '' (7, 45, 305, 65) { Left ; }
	text  sessionPresenter '' (7, 110,290, 20) { Left ; }
	tablet sessionDetailLayout_iPad ;

There is simply no other mechanism that allows you to do that so elegantly because you are constrained by the underlying architecture of the  SDK you are building your solution onto. When you look at Apple and Android SDKs, you can clearly see that the support for different form factors simultaneously is an "after-thought", not a key architectural foundation.

Furthermore, any architectural deviation from a given SDK translates into lots of boiler plate code, scattered across the entire solution's code base. MDE completely hides this boiler plate code from the solution model.

2) MDE enables the creation/addition of new logic semantics. Mobile apps often need to call more than one API to get the data you need to populate a user interface. For instance, when you need to display the thumbnails of a Flickr gallery, you would think Yahoo's Web API GET /gallery/{id} would return all you need? wrong, it only returns some gallery information with just the photo ids, you need then to make a second call for each photo_id to get the thumbnail URLs to download the thumbnail image files.

So in this case, not only do you need a solution architecture variant (call 2-N APIs given a call to a single API), but you also need to express the logic of that call. No mystery here, that kind of logic is probably a few thousand of years old, and in modern day computing, this is simply called a join:

connection flickr {

	operation init getPhotos GET 'http://api.flickr.com/services/rest/?method=flickr.galleries.getPhotos&api_key=API_KEY&format=rest&gallery_id=GALLERY_ID' {
		resultSet 'photos' ;
		join getSizes on photo_id = id where _label = 'Square';

	operation getSizes  GET 'http://api.flickr.com/services/rest/?method=flickr.photos.getSizes&api_key=API_KEY&format=rest' photo_id {
		resultSet 'sizes' ;

Yes, it's that simple ! With just a bit of code written once for Flickr and abstracted a tiny bit, I can implement most mashups.

But that's not all, a "side effect" of MDE, in this case, is the ability to refactor your architecture. I happen to believe that mashups should mostly run on the client, unless there are some reason not to bring the data on the client (e.g. the end user is not allowed to see part of the data, bandwidth constraints...), but this point is irrelevant, there will always be the need to do some mashup both on the server or the client. Subbu's team has come up with a nice way to do that on the server side, ql.io.

So how would I enable such a massive architecture refactoring? a simple keyword and a tiny bit of code in the code generator (that generates ql.io scripts and some code on the client that invokes the mashup). I'll try to illustrate that for the next release.

join getSizes on photo_id = id where _label = 'Square' with qlio ;

3) MDE enables you to create solution oriented type systems. Many would argue that the main purpose of a computer is not to "compute", but to manage state persistently. One essential aspect of managing state is data structures and the building blocks of data structures are described by a type system.

One of the challenges when building apps and mobile apps in general is that the UI is quite varied. Take the example of a Map for instance, it often comes with points of interest. How do you bind a data set to a Map such that it displays push pins? just like you want to display rows of values in a table or pictures and picture information in a gallery?

In the case of Canappi we built a simple binding framework that lets you express that the result set from a Web API call is bound to a "layout" (a set of controls).  You can optionally use a fromKey / toKey mapping when the attribute names of the result set are different from the names of the control.

layout myMap bindings pointOfInterest with mapping aSimpleDataMapping ; 

In this statement,pointOfInterest points to one (or more) APIs that gather the data that is automatically bound to the map control which supports binding to a single push pin (lat, long, title and subtitle) or an array of pushpins. The reason why we make it that simple, is because we took the decision early to normalize the response before we bind it to the UI, i.e. we built an implicit type to which Web API response formats map to, on one end, and from which UI views (with any control) bind to.

Ok, I understand this is quite basic, but incredibly productive for at least 95% of the data that moves from a server to a client and back: I don't know any programming language that allows this type of consision (e.g. a twitter app in 30 lines of code). Yet, hundreds of thousands of developers, every day, write some code that picks up some data from some kind of back-end API call and puts it in some kind of UI element.

These examples are the bread and butter of MDE, they are very easy to implement and I would argue that they add less than 20-30% overhead to your first implementation of a variant, type of business logic or data binding. That means on your second implementation you already gained overall 20-30% of your time. So next time you look at the architecture of your solution why not ask these questions:

a) Is there any variant in your solution architecture?

b) Does your architecture deviates from the architecture of the underlying SDK / Platfrom on which it is built?

c) Are you using some business logic that is not easy to express with the programming language that you are using?

d) Would you benefit from extending the type system of your underlying programming model?

If you answer yes, to one or more of these questions, I would certainly take a good look at MDE.


Is HTML 5 losing mindshare?

Robert Scoble thinks so.

The new Path? The one that won a Crunchie last night for great design? It’s not done in HTML 5.

This morning I saw something new coming soon from Storify. Not done in HTML 5. This morning I visited Foodspotting which just shipped hot new apps on iOS, Android, and Blackberry. Not done in HTML 5.

More and more I’m hearing that designers and developers are ignoring HTML 5, especially for the high end of apps.

After building Canappi, I can only wholeheartedly agree with Bob. HTML5 has certainly a place in the mobile landscape, if for nothing else to "mobilize" your web site, but when it comes to user experience, we don't see how it can compete with Native Frameworks at least in the years to come: For every successful mobile web app, someone will build a superior native experience that users will adopt. There is simply no room to compete.

I am currently building Canappi's jQuery Mobile code generator and from what I can tell, it is extremely limited compared to the native iOS and Android SDKs.

The fundamental reason for Web Apps to emerge in the late 90s and 2000s was the "client update" problem and to a lesser degree the number of platforms on which the app should run. In the 90s, assistants used to roam the buildings with CD-ROM (and Floppy disks) to update every single client.

Today, App Stores have completely solved the update client problem (for mobile and desktop apps), and running on multiple platforms really means running on iOS and Android (although the fragmentation of Android is a bit of a concern).

Web Apps have many fundamental flaws, from flowing layouts which simply don't work in small form factors, to cluncky off line experience, not to mention bandwidth and power consumption.

If you combine a horrible developer experience to a horrible user experience, (Mobile) Web Apps have simply no room to grow from here.

<< 1 ... 4 5 6 7 8 9 10 11 12 13 14 ... 22 >>



blog engine