Service Design Principles (The Contract Story Continues)

Circling back to the theme of service definition and contracts, let's look at the principles that are important to keep in mind when designing "good" services. As with most design pursuits, there is rarely an "ideal" model or an absolute truth. It's frequently necessary to balance influences to achieve the most appropriate and valuable design. Striking the right balance requires good experience and the ability to apply effective judgment. Fortunately, there are a number of well-established principles and patterns that help encourage good decisions about services, interfaces, and relationships. Leveraging this shared perspective leads to better designs.

The principles discussed here build on the most fundamental characteristic of a quality service definition: explicit boundaries. Our service definitions must precisely define the interface syntax and semantics - what it looks like and how it behaves. The definition forms a "contract" - a commitment on the part of the service - that the specified interface and behavior will continue to be honored until which time the "contract" terms are explicitly modified.

The remaining design quality attributes build upon this contract foundation and make solutions easier to build, support, and enhance. We'll briefly define several of these attributes and consider their impact:

1. Coupling
2. Encapsulation
3. Autonomy
4. Orthogonality
5. Idempotency
6. Cohesion

Coupling

The Loose Coupling design goal encourages the design of an interface that makes as few assumptions as possible about the service consumer or other factors external to the component exposing the interface.

The objective is to encourage modular and flexible design by reducing the interdependencies among elements and reducing the risk that changes in one element will necessitate changes in another. Adhering to this principle reduces the overall complexity of the system and significantly enhances flexibility.

Encapsulation

The encapsulation principle motivates us to hide behavioral characteristics and data constructs behind the elements interface. In a well-encapsulated service, the implementation is clearly distinguished from the service contract and hidden from view (or concern). Strong encapsulation is an important tool for creating loosely-coupled designs.

As described, encapsulation is strongly aligned with "information hiding". Some have argued that these are distinct characteristics, emphasizing the role of encapsulation as one of keeping the data packaged with the operations that can be performed on that data. There is a fair argument to be made, but I tend to think of it as stated and emphasize the latter concern as a distinct principle called Autonomy (see below).

While applied in somewhat of a different context, network protocols and the OSI model provide an easy way to think about encapsulation. From Wikipedia:

In computer networking, "encapsulation" is to include data from an upper layer protocol into a lower layer protocol... This is a method of abstraction for networking by allowing different layers to add features/functionality.

The motivation in service design is the analogous to that described here for network protocol design. Allow a service (or any type of design element) to add features and functionality, replace or modify implementation of existing functionality, or in other ways make changes to the implementation without interfering with other dependent elements.

Autonomy

The American Heritage Dictionary defines the word autonomous with expressions such as the following:

• Self-government or the right of self-government; self-determination.
• Not controlled by others or by outside forces; independent:
• Self-directed

As it applies to service design, this principle is closely aligned with "separation of concerns" and emphasizes the responsibility and right of a service to assume full control and authority over the capabilities it delivers. This principle encourages a design whereby two factors are assumed and emphasized by the producer of a service:

• There is full authority to change any internal aspect of the service implementation, so long as the external contract is strictly maintained
• There is no authority to change the responsibilities of another service

In this environment, where each participating service is autonomous and each is able to evolve independently of the other, we realize a much greater ability to build, comprehend, and manage complex interdependent systems.

Orthogonality

A service exhibits good orthogonality when it's interface provides:

• one straightforward way to accomplish a given task
• no redundancy
• no side effects

Wikipedia describes it like this:

Orthogonality is a system design property facilitating feasibility and compactness of complex designs. Orthogonality guarantees that modifying the technical effect produced by a component of a system neither creates nor propagates side effects to other components of the system. The emergent behavior of a system consisting of components should be controlled strictly by formal definitions of its logic and not by side effects resulting from poor integration, i.e. non-orthogonal design of modules and interfaces. Orthogonality reduces testing and development time because it is easier to verify designs that neither cause side effects nor depend on them.

For example, a car has orthogonal components and controls (e.g. accelerating the vehicle does not influence anything else but the components involved exclusively with the acceleration function). On the other hand, a non-orthogonal design might have its steering influence its braking (e.g. Electronic Stability Control), or its speed tweak its suspension.

There's an amusing and effective description of orthogonality in Pragmatic Programmer.¹ I wont' go into the whole story, but Dave Thomas describes how the controls on a helicopter are anything but orthogonal (unlike the car mentioned in the Wikipedia description). If you adjust one of the controls to influence, say, the pitch, you must also adjust others to change things like the tail rotor and the throttle, or you won't achieve the desired results. Essentially, a change in one characteristic of the helicopter's flight has side-effects on others, and those side-effects must be anticipated and accounted for to obtain the desired behavior.

In designing systems, it's essential that we provide orthogonal controls that can be easily understood - by the consumers who will use the service and by the developers who must make future changes to its implementation. Orthogonal designs lead to significantly reduced development and test time, greater flexibility, and enhanced opportunities for reuse.

Idempotency

An idempotent design is one that has the same effect if an operation is used multiple times as it does if used only once. This is especially important in the types of highly-distributed systems we frequently build these days, where interdependent design elements may be isolated on separate networks, potentially positioned in different geographical locations (increasing the chances of communication failures).

Cohesion

There's a common maxim in organization (at least I hear it a lot on those home improvement shows):

"A place for everything and everything's in its place"

I may be taking a bit too much liberty with the principle, but I think this is a reasonable way to describe high cohesion. When a service exhibits high cohesion, its responsibilities are intuitive, unambiguous and focused. They're also well-organized. The service offers the "right" set of related capabilities and stays focused - not trying to become all things.

High cohesion reduces complexity and increases reuse, in part because of the ease of understanding the role and purpose of the element.

Understanding these principles and energetically applying them when designing software systems will lead to products that are delivered more quickly with higher quality, exhibit greater flexibility and adaptability, encourage greater levels of reuse, hold up more reliably under the pressures of time and influences of evolution, and overall add greater value to the business.

¹Hunt, Andrew, and David Thomas. The Pragmatic Programmer: From Journeyman to Master. Addison-Wesley, 1999.

The State of the IT-Business Relationship

Alastair Bathgate at WorkForceInABox wrote a compelling piece titled IT vs The Business - the Final Countdown? in which he asserts the following.

"IT functions have now implicitly turned into two types:

a) the “can’t do” department, focussing only on operations and not strategy (I mean business strategy not IT strategy) and who buy blade servers and occasionally put out a tender for some outsourced development to compete against an SI proposal.

b) the “can do” department that focuses on business strategy, acts as an interpreter between the business and the IT world, and perhaps most importantly focuses on solving business problems. Yes, of course the operations, governance, security and compliance still needs looking after - but that’s all overhead. The exciting bits are the bits that drive the business forwards and deliver value, not cost. And if you’re not at the centre of the business you are on the periphery - and that is no exciting, rewarding, or secure place to be…"

Let's be sure we're the "can do" department that focuses on business strategy. Indeed, Let's be the bilingual partner, actually participating in the definition of the business strategy, translating between business and IT perspectives, and establishing an IT strategy that supports the business. It's a challenging role, but I bet we're up to it. Ready? Set? Go!

Contracts Are The Thing

For as long as I can remember, I've been obsessed with the importance of contracts - deliberate definition/specification of component interface boundaries. In fact, I take a fair share of grief for climbing atop my soapbox on this topic. I suppose that's the price to pay for championing the cause.

The Risk

I was recently skimming through Software Systems Architecture: Working With Stakeholders Using Viewpoints and Perspectives and came across the author’s assertion that that the most important consideration in the Functional Viewpoint (logical model) is poor interface definitions:

"Many architects define their elements, responsibilities, and interelement relationships well, yet totally neglect their interface definitions. Defining interelement interfaces clearly can often be something of a chore. However, it is on of the most important tasks you can perform for the system. Without good interface definitions, major misunderstandings will occur between subsystem development teams, leading to a range of problems from obviously incorrect behavior to subtle, occasional system unreliability."

What’s a Contract?

Bertand Meyer, the designer of the Eiffel programming language, was one of the early proponents of the importance of contracts. In “A Conversation with Bertrand Meyer, Part II - Design by Contract” (http://www.artima.com/intv/contracts.html), he puts it this way, "The main purpose of contracts is to help us build better software by organizing the communication between software elements through specifying, as precisely as possible, the mutual obligations and benefits that are involved in those communications."

Taking it somewhat deeper, an effective contract specification provides an unambiguous description of the interface syntax and semantics - what it looks like and how it behaves. The contract defines the operations offered by the interface, the inputs, the outputs, the effects, and the nature or style of communication (web services, messaging, rpc, etc).

Contracts should exhibit well-known design quality attributes such as separation of concerns, high-cohesion, loose-coupling, granularity and degree of element interactions and others. We’ll come back to those in detail another day.

For today, let’s just emphasize one: the separation of concerns which engenders the ability to organize the development effort around the architecture. Effective contract definition allows individuals or teams to be assigned responsibility for delivery of specific subsystems or components, understanding how that functionality is integrated to the overall system. In other words, contracts help us to divide and conquer.

Who’s on the Hook?

It is squarely the responsibility of the Architect to lead the definition of contracts, and as mentioned above, it’s a common execution weakness. It’s essential that the Architect have ultimate responsibility and accountability for this aspect of the architecture. But the Architect is not a solo act and relies on the engagement of many team members or even multiple teams to achieve results. It’s important for the team members to understand not only “what”, but “why”. In other words, the rationale is also important. As such, successful execution of contract definition will typically require a command of the technical solution, awareness of good design attributes, and ability to orchestrate communication among interested parties - and facilitate negotiation when necessary.

As an aside, you can see a couple of important characteristics of an effective Architect in this description. In particular, a successful Architect is a strong leader and a good communicator. It is not enough to make great decisions and define great contracts. The Architect is not successful unless the contracts are well-understood and agreed to by the relevant stakeholders.

Turning back to “Software Systems Architecture”, I’ll close with the following checklist to help guide our effort and make sure we deliver on this important responsibility:

• Define your interfaces clearly and as early as possible
• Review them frequently to ensure that they are clearly understood
• Do not consider element definition complete until interfaces have been designed
• Make sure that interface definitions include the operations, their semantics, and the examples where possible.

¹Rozanski, Nick; Woods, Eoin. Software Systems Architecture. Upper Saddle River, NJ.: Addison Wesley, 2005

Kruchten: Antipatterns

Picking up on the theme of antipatterns, let's look at a list provided by Philippe Kruchten in his whitepaper "What do software architects do?". This paper, by the way, is another good source of perspective on the emerging discipline of Software Architecture.

Antipattern: Creating a perfect architecture, for the wrong system.

A software architect that is not communicating regularly with the customer, the end users, or whomever represent them (the product manager) is likely to miss the target, in particular as the target is moving, or rather, as the target is only gradually understood.

Antipattern: Creating a perfect architecture, but too hard to implement.

A software architect who does not understand the (maybe limited) skills, capability and experience of the implementation team(s) that will continue and finish the work will create enormous level of stress and frustration, and likely not deliver a quality product in time. The architectural effort has turned into a computer science research project.

Antipattern: Ivory tower

The worse combination is the architecture team that lives isolated in some other part of the organization - another floor, another building, another country - and who comes up after some months with a complete architecture, out of the blue. To their complete surprise, they will experience rejection: an apparent misfit on both front, functional and implementation. This is especially the case if the developers (the non architects) had a few months to make some progress and they have in some ways made some architectural decisions, under some other name. A special case of this antipattern is the architecture group that only scouts technologies and provides recommendations to other groups, but is not making design decisions and is not accountable, as I have witnessed in two large telecommunication companies, the architecture watch..

There is another issue that cannot be completely ignored; it has to do with whom you have chosen to be the architects. It is very likely that you have in this role some of your most talented staff: good at manipulating abstractions, wide experience of a range of systems and technologies, good communication skills, domain knowledge, etc. and you may want to use some of these skills for other tasks than just building architectural views. You want them to speak to the new prospective customers, to show off the organization technical expertise, you want them to help this or that team that experiences a difficult technical issue, you want them to review the architecture of another project, to take part of a due diligence process to acquire a company, to present papers a conference to strut your stuff, or merely extinguishing some nasty fire. But if you are not careful, this leads to another "antipattern:"

Antipattern: The absent architects

No or little architecture design progress is made: the architects are always away, doing fascinating things, or fighting fires. It is very easy to slip in this mode, especially after some initial good progress and early successes, which brought some fame on the architects.

Have you seen examples of these anti-patterns in action? What were the results?

Enterprise Modeling Anti-Patterns

Scott Ambler's Agile Modeling site has a good article on Anti-Patterns for Enterprise Modeling.

Enterprise Modeling Anti-Patterns: "A process anti-pattern is a common strategy which sounds good in theory but in practice proves to be harmful, if not outright disastrous. This article overviews a collection of anti-patterns pertaining to enterprise architecture efforts within an IT organization. These anti-patterns are:

* 30,000 Feet and Climbing
* Bleeding Edge
* Brain Trust Parking Lot
* Buzzword-Driven Architecture
* Detailed Enterprise Modeling
* Enterprise Parking Lot
* Goldplating
* Ivory Tower Architecture
* Modeling for Modeling's Sake
* One Truth Above All Else
* Real-World Disconnect
* Strive for Perfection
* Stuck in the Weeds
* Technology Above All
* Tomorrow Suffers From Today
* Ungrounded Future
* Yesterday's Enterprise Model"

Each of these Anti-Patterns is well worth taking a look, but I'd like to highlight a couple that seem to pop up quite often.

30,000 Feet and Climbing - The enterprise model is so high-level that it is of limited or no practical use to application teams.

Solution: Develop reference architectures which provide working examples of individual aspects of your architecture. Roll up your sleeves and get actively involved with the development teams. Act on feedback from the development teams to evolve your model(s).

Ivory Tower Architecture - Your enterprise architecture model(s) reflect a wishful, perfect world scenario instead of the realities of your actual environment.

Solution: See the solution to 30,000 Feet and Climbing.

Stuck in the Weeds - You are too far into the details, attempting to do all the work for application teams.

Solution: Mentor developers in the architecture, work with them as equal partners on the development teams, and, after they've gotten going, focus on providing them with guidance and advice as needed.

Brain Trust Parking Lot - Your enterprise modeling group is composed of a lot of very smart people who don't fit in well anywhere else within IT but you don't want to lose their knowledge.

Solution: Find a way to make them useful to existing project teams, perhaps as a modeling mentor or as a subject matter expert (SME) in the business domain.

Keep Your Tail Short

Doing some research this evening, I came across a post related to Test Driven Development (TDD) by Jeremy Miller. This was a nice way to describe the importance of keeping the responsibilities of a component very specific and its coupling precisely controlled.

Keep Your Tail Short. The best description for this general concept I've ever heard comes from Stuart Holloway - "When you pull a class off the shelf, what else is coming with it?." Think of it this way, if I want to test my business rules, or my display logic for that matter, what other areas of the code do I have to pull as well? By and large you want to answer that question with "not much." Writing test code just to placate my database or web server when I'm testing business logic is purely overhead -- both in terms of the mechanics of creating the test and the intellectual overhead of understanding the test.

While this refers to an Object-Oriented context, the principle applies perfectly well in just about any development environment, and we see evidence in this scenario of the fact that good design is essential to good testing.

Let's make sure we're able to take just what we need off the shelf... no strings attached.

Who Do You Think Created The Chaos? :-)

Let's have a little fun this morning. No matter how many times I hear this, it still cracks me up.

A physician, a civil engineer, and a computer scientist were arguing about what was the oldest profession in the world. The physician remarked, "Well, in the Bible, it says that God created Eve from a rib taken out of Adam. This clearly required surgery, and so I can rightly claim that mine is the oldest profession in the world." The civil engineer interrupted, and said, "But even earlier in the book of Genesis, it states that God created the order of the heavens and the earth from out of the chaos. This was the first and certainly the most spectacular application of civil engineering. Therefore, fair doctor, you are wrong: mine is the oldest profession in the world." The computer scientist leaned back in her chair, smiled, and then said confidently, "Ah, but who do you think created the chaos?"

As is almost always the case, this tickles our funny bone because there's at least a shred of truth in it. Hmmm... Who's job is it to navigate into clear waters and circumvent the creation of chaos? ( hint)

By the way, I stumbled into this today as I scanned through Booch's recent release of the 3rd Edition of "Object-Oriented Analysis and Design with Applications" and compared it to my dusty copy of the 14 year old 2nd Edition. More on that another day.