Organize Around the Architecture (#1)

In a post titled “Contracts and Integration Tests for Component Interfaces” at Object Mentor, Dean Wampler discussed agile development and the communications between teams. As usual, Dean makes a number of good observations, and I’d like to pick up and expound on a couple of his thoughts.

For many years I led teams that had fully adopted agile practices. We had refined our development approach (and continually did so) to be extremely productive. Our responsibilities were almost exclusively for common services that were used at the enterprise level, which included everything from Credit Card Processing and Campaign Management to Subscriber Identity Verification. In those days, the overwhelming majority of the teams we worked with were still fully rooted in traditional/waterfall methods, and just like the teams Dean described, these teams were very reliant on BDUF.

Working with those teams, we were very quickly able to get ourselves into a position where we stayed barely ahead of them. We performed independent analysis in the business domain and consistently developed the services that would ultimately be needed. Of course, we always made sure not to get too far out in front where we might run too high a risk of disconnecting from or misunderstanding the true business objectives, needs, and priorities. Operating in this mode, there was at least a theoretical danger of building things we didn’t need, but in practice, over more than five years, that didn’t happen a single time. By maintaining close relationships with the “customers” (which included both the true business customer and the other development teams that would consume the services) and constantly communicating about their needs, we consistently blazed the trail just in front of where they wanted to go and made it possible for them to keep marching unencumbered towards their goals.

From a technology standpoint, we did something pretty similar to what Dean suggests. Though they’re absolutely a prerequisite, static definitions of contracts are not sufficient for reliable communications and assurance of unified expectations (see also Contracts are the Thing and Service Design Principles (The Contract Story Continues)). In our case, we delivered in four steps and provided the following in rapid succession to each team: 1) static definition of the contract, 2) client-side mock, 3) server-side mock, and 4) operational service. Sometimes this would all happen within just a sprint or two, and it always allowed the teams to actively participate in the evolving understanding of the dependency and how it could be incorporated into their design and delivery.

By the way, in those days (for calibrated reasons) we were exposing these capabilities as J2EE services, so part of our practice was to deliver a simple client library that encapsulated all the details about communicating with the service. This design was something we first delivered in 2001. It was in this thin library that the client-side mock was implemented. It allowed developers on those teams to start calling the service very early on, when they were ready, even if we were not. That mock remained available in the client library for future testing and development purposes and was typically activated by qualifying the service endpoint. With barely sufficient richness in the communication with the mock (e.g., the type of responses available), the dependent team was able to recognize potential integration pitfalls very early. This led to much greater accuracy, reliability, and validation of the interfaces.

You’ll notice that this approach heavily emphasizes the principle of “Organizing Around the Architecture,” which promotes segmenting a project or program into manageable sub-parts (and corresponding sub-teams) with clear and defined roles and responsibilities that in aggregate accomplish the goals of the overall initiative. Specifically, the dependency between each team should be in alignment with the architecture and in some fashion documented by the “contract” for the services being provided by those teams. For large-scale development programs, I consider this sort of organization to be essential. In fact, I think we all understand the importance of subdividing any large effort into manageable structures (and accountabilities).

Which brings me to my last point. Dean mentions the use of “virtual feature teams.” While that's a common approach and is often used successfully (especially for small to medium efforts where scope and scale are more easily managed), there’s an inherent risk in any approach that doesn’t respect the natural correlation between the structure of the organization (the people) and the structure of the architecture (the product). It’s been proven time and time again that the architecture of a product will tend to follow the organization of the people designing and delivering that product. I’ll talk more about this in a future post, but for now, as we think about the relationships between teams, how they’re structured, and how they communicate, let’s be sure to take the “Organize Around the Architecture” principle into account.

My First Computer or Two

Why I'd want to share this is beyond me. Why date myself?!? Inspired by a post today from Todd Bliske (My First Computer(s)), I thought I'd follow suit and reminisce about my early computers. A few fun memories.

While there were a few steps along the way, I really think of two worth mentioning. As a teenager, there were many hours of coding pleasure at the keyboard of a Tandy TRS-80 Model I. The one I had was quite similar to this one described very nicely by the good folks over at "Vintage Computer".


I hate to think how much time I spent hacking at Basic programs on that thing. See the device under the monitor? The one that's about the size of a dvd player or something like that? That was an expansion interface that enabled the connection of a floppy disk. Before adding that brilliant upgrade, all the persistent storage was handled via cassette!

Fast forward a few years to good times in the U.S. Air Force, and things got a little bigger - but marginally more capable. For several years, I had the distinct pleasure of developing on a TI-980B. Check out this awesome front panel.


This was a beast, but what a lot of fun. Featuring a switch-initiated ROM bootstrap loader, one would walk up to the machine and execute a flurry of load instructions via those toggle switches, and voila! The beast stirs to life, completing its load from good ole 9-track tape. Armed with a 4 mhz clock and a whopping 16k of dynamic MOS/LSI semiconductor memory (spanning two full-size circuit boards), almost anything was possible. One of the things I remember most was a single DMA port with 6 full circuit boards for I/O expansion. As I recall, there were literally 26 cards in the disk controller. Oh, and that disk. You know the one. Remember these?


If memory serves, that whirlpool-sized "mass" storage device provided 100mb of storage and involved a pack with 10 platters and 406 cylinders. It looked something like this one at Wikipedia: Removable Disk Pack. In those days (relatively speaking, it's really not that long ago), head crashes were a notable risk. And what a mess that was. Metal shavings everywhere! It wasn't pretty.

Not too long after that, things got a little simpler, and I remember going through a Commodore 64 and a Commodore 128 before landing on a machine that was a lot of fun for a number of years. The Amiga was a system truly ahead of its time. Both my kids learned how to use a computer (and fling floppies all over the room) on a proud Amiga 1000. It brings a smile to my face even now to think about my 2 year old selecting and inserting floppies to play his "First Shapes from First Byte" game. I can even still hear the robot voice... "First Shapes from First Byte."

Anyone remember the game "Tower Toppler?" Fun times.

How about you? What did those early days look like for you?

Who Has the Power?

Today's post over at The Heart of Innovation is titled "Managers Need to Become Innovation Coaches" and offers guidance that good architects will do well to apply. While the article (and the short extract below) focuses on the role of the "manager", these principles are more aptly described as characteristics of good leadership in nearly any role - especially that of the architect.

Most managers, unfortunately, perceive new ideas as problems. [Instead] they foist their ideas on others and can't figure out why things aren't happening faster.

That's not how change happens. If people are only acting out somebody else's ideas, it's only a matter of time before they feel discounted, disempowered and... well...just plain dissed. People are more than hired hands; they are hired minds and hearts, as well.

If you want to empower people, honor their ideas. Give them room to challenge the status quo. Give them room to move -- and, by extension, move mountains.

Who has the power in an organization? The people who are allowed to think for themselves and then act on their ideas! Who doesn't have power? The people who have to continually check-in with others.

The idea of empowerment is essential. No matter how smart and capable the architect, the true "brilliance" of a team or an organization lies in the collective mind (Leadership - The Secret Sauce). This brilliance can be tapped only through legitimate empowerment, and that means we all should ask ourselves "who has the power" in an organization, on a team, or on a project. Good things happen when the architect provides leadership - and the team provides power.

Progressive Refinement of Estimates (aka The Transmission Repair)

Not too long ago, I had to have the transmission in my vehicle rebuilt. While car troubles are never a pleasant experience, this one created an experience that I think we can apply to many of our software development initiatives. You see, when I walked in to my friendly neighborhood repair shop, I was a bit anxious and really "needed" to know right there on the spot what was wrong with my car and how much it would cost to get it fixed. Oh, and when I would get it back. As you can imagine, it didn't quite play out that way. You see, the transmission guy, being the experienced professional that he is, knew he wasn't in a position to reliably answer those questions - yet. But he also knew how to work with me through a series of activities that would progressively refine our collective awareness of the answers - and ultimately be mutually beneficial. Here's how the story played out.

The Transmission Repair Story

Day 1 - 10AM - The Contact
Transmission's been slipping for a few days, so I head up to see "Ray" the transmission doctor. Walked in the front door and was greeted cheerfully. Explained the symptoms to Ray and asked the obvious questions: 1) What's wrong?, 2) How much will it cost?, and 3) When will it be done? Of course, Ray's been through this with hundreds of customers, and he knows he's not in a position to answer those "ever-so-important" questions. But he knows just what to do, so he explains how the process will work - and how it will meet my needs and get my car back on the road in a jiffy. Ray tells me the next step is to allow him to do a road inspection. He'll drive the car and observe the behavior himself. He'll even do some investigation with the car up on the rack in his shop. He says once he's completed that assessment, he'll give me a call and let me know where we stand.

At my prompting, he lets me know that the trouble could be something simple like contaminated fluid or mechanical adjustments, or it could be a more complicated problem requiring more extensive repair. On the lower end, Ray says it might cost as little as a few hundred dollars. He cautiously points out, however, that major repairs can be well over two thousand.

I anxiously awaited his call.

Day 1 - 3PM - The Engagement
At about 3 that afternoon, Ray calls to give me the update. It's a little bit of bad news, as the early indication is that there is indeed damage in the transmission, and it will need to be repaired. Ray gives me a bit of confidence by explaining what he observed and what that might mean. He is able to correlate his observations to many past experiences and makes me feel good that I'm on a path that many have successfully gone down before. According to Ray, repairs are going to cost between $1100 and $1800, and the correlation to previous work makes me feel confident with this range.

Ray goes on to explain that the next step is for him to get the transmission off the vehicle and onto the bench so he can open it up and look inside. Once on the inside, he'll be able to determine more clearly which parts need to be replaced and what work needs to be done. With that new information, he'll be in a position to give me a more reliable estimate.

Ray says that it's a lot of work to pull the transmission out to take it apart and do the detailed diagnosis. For this work, there will be a charge of $600. If I choose to follow through with the repair, that charge is simply part of the $1100-$1800 estimate. However, after Ray does the diagnosis and comes up with the updated price, if I choose not to proceed with the work (because, for example, cousin Charlie says he'll do it for a couple six-packs), then I'll owe Ray the $600 for his work.

While I'm not too crazy about this idea (it limits my options and makes me commit), I really can't argue against it. It only seems fair. I give the go ahead, and Ray promises to call me with an update within 48 hours.

Day 3 - 11AM - The Commitment
A little less than two days later, Ray has completed the detailed assessment and calls with the news. Along with several other parts, Ray will need to replace the governor pressure solenoid, the servo piston, and the boost valve. I have no idea what any of those things do, but Ray offers to explain in as much detail as I'd like. He wants me to be comfortable and confident in the estimate. With these new parts, his labor, and a couple pieces he has to send out to the machine shop for reconditioning, the revised total estimate is $1700 - $1850 and I can have the car back within 3 days. I give Ray the authorization, and he gets to work.

Day 4 - 2PM - The Change Request
Ray called to tell me he was making good progress on the transmission. It'll be ready for me to pick up tomorrow. That's good news!

He also wanted to let me know about something he'd learned during the repair. Once he had the transmission all apart and cleaned, he noticed that the #4 gear had some pitting on the inner surface of the teeth. He explains that this isn't something that will cause trouble any time soon, but he wanted to present me with the option of taking care of it now while everything is already out on the bench, potentially avoiding a second, more expensive repair at a later date. The part would cost an extra $75 plus $50 for labor. If I opt for the additional work, my revised total grows to $1825 - $1975.

After careful consideration, I decline the additional repair.

Day 5 - 4PM - The Delivery
The car's ready. The total came to $1797, and I can pick it up anytime before 6.

The Lesson

Before getting into the lessons learned in this experience, it might be a good idea to review Ray's process:

1. The Contact - Customer explains the trouble. Ray agrees to spend a couple hours investigating. This investigation will result in an understanding of scope and constraints and provide both Ray and the customer the information they need to make decisions about how to move forward.

2. The Engagement - Ray explains the results of the initial investigation and provides options. Customer agrees to a minimal investment to conduct the more in-depth assessment that will include removing and disassembling the transmission in order to generate a detailed work plan.

3. The Commitment - Ray explains what he learned during the in-depth assessment and provides a reliable estimate for the repair along with details about when he can deliver. Customer agrees to the terms of Ray's plan.

4. The Delivery - Ray completes the work and delivers on-time and on-budget. Customer is satisfied.

It seems Ray has this process nailed down. He avoided making a commitment (or asking me to) before there was a reasonable level of understanding of the work that would be required, and he walked me through each step of the process right up to the final successful delivery. While there were steps in that process that I really wish could have worked differently, I knew I had to be reasonable. For example, in that first interaction ("The Contact"), it wouldn't have done a bit of good for me to pitch a fit and throw a temper tantrum in an attempt to get Ray to cave and tell me exactly what was wrong and what it would cost. Likewise, Ray knew it would be just as unreasonable to avoid giving me a cost estimate as early in the process as possible. If he couldn't (or wouldn't) give me an estimate (even a wide range) at "The Engagement" stage, I surely wouldn't enter into the arrangement, and I'd have to take my business elsewhere. Throughout the process, we agreed to collaboratively work through a series of well-specified milestones. Each step resulted in the addition of new information and yielded an increasing ability for each of us to understand what we were getting ourselves into - and what we could each expect to receive in return.

Applying the Lesson

Let's be encouraged to apply the same reasonable and responsible sort of process to software delivery. Our customers can understand why precise estimates are not possible when there is little or no data, and we can understand that it is absolutely essential that we provide estimates that can be used for coarse-grained business planning as early as possible. All too often, this devolves into an emotional tug-of-war where nobody wins.

We all come out ahead when we design processes like Ray's and work collaboratively to step through a progression of estimates that become more and more reliable as a project unfolds and new data becomes available.

One last thing. You may be interested in comparing Ray's process to the Rational Unified Process, in which you'll immediately be able to draw simple parallels:

"The Engagement" = The Life Cycle Objective Milestone (LCO)
"The Commitment" = The Life Cycle Architecture Milestone (LCA)
"The Delivery" = The Initial Operational Capability Milestone (IOC)

These milestones give us a good framework for the reasonable and responsible collaboration that makes Ray's process a success.

Service Design Principles And Governance

A good architecture governance capability has at it's heart a focus on developing people and encouraging excellence (see Governance Without Goodwill is Dead). I like to think of it in terms of a leadership capability of the sort alluded to in this Dwight Eisenhower quote:

"Leadership is the art of getting someone else to do something you want done because he wants to do it."

With that backdrop, and as part of an emerging governance program, I've recently had the need to introduce or reinforce a set of service design principles to a large audience and begin developing a shared and actionable understanding of these practices. The goal is to increase community awareness and support for good design and to further develop good design skills throughout the community.

A good source for this material is Thomas Erl's SOA Principles of Service Design, and many in the community have been encouraged to buy the book. In fact, we've given away a couple dozen of them. To further jump-start the process, it has been helpful to create a high-level representation of these principles that can be used in introductory sessions and training. Doing so has begun to instill a common language and frame of reference.

You can download a copy of the design principles document. It's a bit large due to the graphics. Following is a quick preview.



Standardized Contract - Implement a standardized contract

Services within the same service inventory are in compliance with the same contract design standards

Loose Coupling - Minimize dependencies

Service contracts impose low consumer coupling requirements and are themselves decoupled from their surrounding environment

Abstraction - Minimize the availability of meta information

Service contracts only contain essential information and information about services is limited to what is published in service contracts

Reusability - Implement generic and reusable logic and contract

Services contain and express agnostic logic and can be positioned as reusable enterprise resources

Autonomy - Implement independent functional boundary and runtime environment

Services exercise a high level of control over their underlying runtime execution environment

Composability - Maximize composability

Services are effective composition participants, regardless of the size and complexity of the composition

Statelessness - Implement adaptive and state management-free logic

Services minimize resource consumption by deferring the management of state information when necessary

Discoverability - Implement communicative meta information

Services are supplemented with communicative meta data by which they can be effectively discovered and interpreted

Hope you find this useful. To learn more about these principles, be sure to get a copy of SOA Principles of Service Design and download a copy of the design principles document.

Right Ways of Working with the Left Brain

"The Heart of Innovation" is the blog for Idea Champions, the highly-respected consulting and training company specializing in creativity, innovation, team building, and leadership. A recent post titled "Right Ways of Working with the Left Brain" offered some valuable suggestions for helping to lead left-brained people (like most of us) through creative activities. The article opens with:

"If your job requires you to lead meetings, brainstorming sessions, or problem solving gatherings of any kind, chances are good that most of the people you come in contact with are left-brain dominant: analytical, logical, linear folks with a passion for results and a gnawing fear that the meeting you are about to lead will end with a rousing chorus of kumbaya."

I can relate.

The 10 practices are listed here, but check out the article for details - and fine-tune your ability to provide leadership throughout the creative process. As discussed in Architect as Advocate of the Business and The Proactive Architect, these sorts of leadership skills help distinguish world-class architects.

Ten Tips for Leading Creative Activities:

1. Diffuse the fear of ambiguity by continually clarifying the process
2. Get people talking about AHAS! they've had in their own lives
3. Identify (and transform) limiting assumptions
4. Encourage idea fluency
5. Invite humor
6. Do the right brain/ left brain two-step
7. Periodically mention that chaos precedes creative breakthroughs
8. Establish criteria for evaluation
9. Be a referee when you have to
10. Consult with the tough people on the breaks

Check out the full article here: Right Ways of Working with the Left Brain

Think Bigger!

Jeff had a very nice post recently about the all too common problem of Engineers becoming inappropriately fixated with things that are important to Engineers (as opposed to things important to the business and to the users). He points out that he gets "frustrated with the depth of our obsession" over things that are very important and necessary (such as unit tests that are absolutely foundational to professional development) but are sometimes allowed to overshadow the things that are MOST important. Jeff remarks, "the ultimate unit test is whether or not users want to use your application" and continues with, "I want to run up to my fellow programmers and physically shake them: think bigger!" And I want to shout, Amen!

But then something odd happens. In response to his post, developers (apparently suffering from precisely the ailment Jeff just described) completely validate his entire premise (that sometimes we fail to see the forest for the trees). One after another weighs in with a comment such as, "unit testing has nothing to do with usability" (boldly paraphrasing).

Finally, Kyle pipes up with the voice of reason:

"While [it's true] Engineers should be concerned with whether they are building the product right (verification), their overriding concern should be whether they are building the right product (validation).

It is important for engineers to get their head above the water and see the bigger picture sometimes."

Kudos to Jeff, Kyle, and all of you who remind us to "think bigger", "get [our] head above water", and remember that our real mission is (as Jeff concludes):

"The ultimate unit test is whether or not users want to use your application. All the other tests you write are totally irrelevant until you can get that one to pass."

Building the product right is very important. Building the right product is essential!