Saturday, April 4, 2009

Managing Complexity

Given that high cohesion and low coupling are necessary but insufficient properties of a well-designed system, what else is required?

We need at least one more - namely 'conceptual coherence'.

I'll attempt to explain myself.

A few months ago I had the pleasure of reading a book by Roger Sessions on Enterprise Architecture that contained a bunch of ideas that resonated deeply with me. The one Big Idea in the book that I liked very much was the concept of complexity, and in particular the importance of minimising it. Its a principle I've tried to follow closely since entering this field (sometimes unsuccessfully, particularly during the early years of my career), and it has served me well.

Much of the advice contained in the book follows from his "Five Laws of Partitions", where a partition is a means to compartmentalize a mathematical set. The primary aim when applying these 'laws' is to minimise complexity. In addition to providing a consistent and rational means to analyse a system, the 'laws' provide a foundation for managing and removing complexity and result in heuristics on how to go about analysing and partitioning an existing system.

Those familiar with his background won't be surprised that Roger also makes use of Type (or Classification) Theory - concepts that will be familiar to any skilled (or unskilled, for that matter) OO practitioner. Combining Type Theory with the "Five Laws of Partitions" is really just a means to discover system properties that are 'invariant under analysis' (IUA) - different people analysing the same system or organisation will, using his methods, theoretically devise very similar architectures and be able to rationally explain the reason for its structure.

System properties will be IUA is if the identified types exhibit a high degree of conceptual coherence. To put it a different way - each identified Type in the system must have properties (concepts) that are tightly bound to each other, and simultaneously be loosely bound to other sets of properties. Conceptual coherence is important when conveying information to other people, as conceptually coherent concepts are easy to convey. For example, "Four-legged cat" is a concept everybody will understand. "3-legged cat with gills" doesn't gell too well as the concept of "cat" is being strained.

Now, coming back to why low coupling and high cohesion are important - devising and/or analysing a system architecture using the above rules, while adhering to the principles of high coherence and low coupling, is related to Principal Component Analysis, The Spectral Theory, Eigendecomposition, and Canonical Forms. In particular by analysing a system using a Design Structure Matrix an optimally designed system will ideally have a lower diagonal structure. The ability to determine those aspects of the system that are IUA in essence reflect a 'canonical' form of the system.

When you think about it, reality adheres strictly to this model; elements and compounds are built on molecules, which are built on atoms, that depends on baryons, which in turn are made from quarks. Biological systems are an even better example; with the aid of natural selection and millions of years of evolution, nature has created systems of phenomenal complexity and yet astounding simplicity.