Software Engineering: Dead?

Over on Coding Horror, Jeff Atwood pointed us to a recent publication by Tom DeMarco titled, "Software Engineering: An Idea Whose Time Has Come and Gone?" As Jeff points out, DeMarco is someone who has shaped the thoughts and minds of the software world for a couple of decades, and when he unabashedly claims that "software engineering" is dead... well, that falls under the category of "things that make you go huh!?"

Jeff does a nice job of putting DeMarco's comments into perspective and gives us all some things to think about. In particular, he encourages the focus on the "craftsmanship" aspect of our profession. This "passion for the craft" of software development is something many of us are enthusiastic about and spend a great deal of energy fostering within our organizations and teams. Jeff wraps it up nicely, concluding, "The guys and gals who show up every day eager to hone their craft, who are passionate about building stuff that matters to them, and perhaps in some small way, to the rest of the world -- those are the people and projects that will ultimately succeed. Everything else is just noise."

I'm not one to abandon the engineering disciplines - and I'm pretty sure that's not exactly what Jeff or Tom are suggesting - but it's high time we all recognize that much of what we do is a "craft." Crafts are performed by people who care about what they do. When engineering attempts to take the place of heart and passion... we fail.

Check it out at Software Engineering: Dead?

A Customer-Driven Architecture for Banks - Learning from iPhone

Glenbrook Partners discusses a recent report from Javelin Strategy and Research that elevates awareness to the changing behavior and expectations among electronic banking customers in the "iPhone-era". Javelin explains that these customers will tend to be much more interactive with their financial institutions and will insist on a greater level of control and visibility. In some ways, this report speaks to the shift underway in application design, but it applies that transformation specifically to the needs of online banking. All-in-all, this is another example of how consumers are (appropriately) shifting into a position of even stronger influence over the applications we deliver.

Check it out at A Customer-Driven Architecture for Banks - Learning from iPhone

Mistake Proofing and The Role of QA

Roland Cuellar at LitheSpeed recently highlighted some of the characteristics of the QA role and the importance of rethinking how this role integrates throughout the development process to add even more value. Taking a position similar to what I propose in "One Team - Developers and QA" (http://blog.softwarearchitecture.com/2007/04/one-team-developers-and-qa.html), Roland borrows from some common "Lean" practices and encourages a deeper relationship with development and a focus on keeping defects out of the product right from the start. Specific recommendations include more detailed and interactive communications/partnership and the elevation of test cases to first-class requirements.

Check it out at Mistake Proofing and The Role of QA

Enterprise 2.0 Isn't a Checklist

The FASTForward blog posted a nice perspective on "Enterprise 2.0" that deserves a look. Yes, this term (and the pursuit it represents) has its own set of risks, but it also embodies a shift in corporate climate that deserves clear focus and understanding. Summing up the purpose, Paula Thornton writes:

So what IS Enterprise 2.0 focused on? People: tapping the human potential, helping to change the way business gets done by optimizing it not to the systems but to the people. Not shaping the people (via training and documentation) to the systems and the business, but changing the systems and the business to optimize the potential of the people.

Under the heading of "How to get to 'Enterprise 2.0'", the article goes on to describe a number of the critical success factors:

1. Truly Utilize Resources - Determine the typical scenarios for problem solving and recognize that departments or hierarchies do not hold the answers to business problems/issues: people do.

2. Shorten Distances - Simplify all aspects of "doing" business. Repeatedly ask: What can we stop doing? Leverage what's working (from the perspective of all individuals impacted, not just those with "management" responsibility to execute) - bypass the rest.

3. Embrace Organic - Organic is not chaotic. Nature has order, but that order is under rapid cycles of repeated construction and destruction.

4. Shift Focus - It's not necessary or beneficial to make things "binary". Good designers are comfortable with the "squishiness" of heuristics.

5. Shift Thinking - Design Thinking requires a different approach: it focuses on trying out multiple possibilities (fail fast) to test an algoritm - a problem statement. Many solutions fail because they either 1) started with the wrong question (the solution is the answer to the question) or 2) did not adapt to change the question (the problem statement) as more was learned along the way.

6. Shift Culture - A company that has been optimized for "machine" design (command and control), will have a culture that reinforces such behaviors. Such a culture will undermine E2.0 potential. It will seek to eliminate the efforts as a "foreign body". A different culture is not a prerequisite, it's a corequisite.

Some things to take note of.

Check it out at Enterprise 2.0 Isn't a Checklist

The Process of Managing Change

Change is hard. I've had the opportunity to lead several programs that brought significant change into an organization, and these efforts have taught me a healthy respect for the challenges (and blessings) associated with organizational change. I've also learned from these experiences that there are some basic truths that should be remembered (I highlighted a collection of these principles in 10 Principles of Change Management) to set our change programs on solid footing. Building on that solid foundation, we can significantly improve our performance in establishing and sustaining meaningful organizational change by leveraging a structured methodology such as the work put forth by John Kotter in 1996 in Leading Change. He followed up this work with The Heart of Change: Real-Life Stories of How People Change Their Organizations which brilliantly emphasizes the human/emotional aspect of the methodology. This book has become the corner stone of many successful change practices.

I find the topic of Change Management more relevant than ever in today's corporate climate. It's been my recent experience that bigger leaps - bigger strategic bets - are required in organizational performance. Gradual and organic improvement is frequently not sufficient if we're to remain competitive. Indeed, a company's survival can depend on an ability to take big leaps in organizational capabilities or in bringing significant new products to market. This reminds me of the following quote shared recently by a very dear friend:

"King Saul thought Goliath was too big to fight, David thought he was too big to miss."

We need more Davids.

The change management discipline doesn't apply only to large-scale organizational change. Even on average software development projects where one of the roles of the Architect (see What is A Software Architect? and Most Important Competencies of the Software Architect) is to lead the creation of an architecture and then champion that solution among the stakeholders, we're well served by taking these important concepts into account.

The following offers a high-level overview of Kotter's change methodology. Experience suggests that trying to save time and effort by bypassing any of these eight steps will inevitably lead to even greater effort and frustration as change stalls and results become elusive. I highly recommend you master this material.

The Eight Stages of Successful Change

1. Increase urgency - People start telling each other, “let’s go, we need to change things!”
2. Build the guiding team - A group powerful enough to guide a big change is formed and they start to work together well.
3. Get the vision right - The guiding team develops the right vision and strategy for the change effort.
4. Communicate for buy-in - People begin to buy into the change, and this shows in their behavior
5. Empower action - More people feel able to act, and do act, on the vision
6. Create short-term wins - Momentum builds as people try to fulfill the vision, while fewer and fewer resist change.
7. Don’t let up - People make wave after wave of changes until the vision is fulfilled.
8. Make change stick - New and winning behavior continues despite the pull of tradition, turnover of change leaders, etc.

Step 1 – Increase Urgency
Raise a feeling of urgency so that people say “let’s go,” making a change effort well positioned for launch. Go after the emotions with concrete and almost smell-able evidence, not just the abstractions so favored by the rational mind.

What Works

• Showing others the need for change with an object that they can actually see, touch and feel.
• Showing people valid and dramatic evidence from outside the organization that demonstrates that change is required.
• Looking constantly for cheap and easy ways to reduce complacency.
• Never underestimating how much complacency, fear and anger exists, even in good organizations.

What Does Not Work

• Focusing exclusively on building a “rational” business case, getting top management approval, and racing ahead while mostly ignoring all the feelings that are blocking change.
• Ignoring a lack of urgency and jumping immediately to creating a vision and strategy.
• Believing that without a crisis or burning platform you can go nowhere.
• Thinking that you can do little if you’re not the head person.

Step 2 – Building the Guiding Team
Help form a group that has the capability – in membership and method of operating – to guide a very difficult change process.

What Works

• Showing enthusiasm and commitment (or helping someone do so) to help draw the right people into the group.
• Modeling the trust and teamwork needed in the group (or helping someone to do that).
• Structuring meeting formats for the guiding team so as to minimize frustration and increase trust.
• Putting your energy into step 1 (raising urgency) if you cannot take on the step 2 challenge and if the right people will not.

What Does Not Work

• Guiding change with weak task forces, single individuals, complex governance structures, or fragmented top teams.
• Not confronting the situation when momentum and entrenched power centers undermine the creation of the right group.
• Trying to leave out or work around the head of the unit to be changed because he or she is “hopeless”.

Step 3 – Get the Vision Right
Create the right vision and strategies to guide action in all of the remaining stages of change.

What Works

• Trying to see – literally – possible futures.
• Visions that are so clear that they can be articulated in one minute or written up on one page.
• Visions that are moving – such as a commitment to serving people.
• Strategies that are bold enough to make bold visions a reality.
• Paying careful attention to the strategic question of how quickly to introduce change.

What Does Not Work

• Assuming that linear or logical plans and budgets alone adequately guide behavior when you’re trying to leap into the future.
• Overly analytic, financially based vision exercises.
• Visions of slashing costs, which can be emotionally depressing and anxiety creating.
• Giving people fifty-four logical reasons why they need to create strategies that are bolder than they have ever created before.

Step 4 – Communicate for Buy-In
Communicate change visions and strategies effectively so as to create both understanding and a gut-level buy-in.

What Works

• Keeping communication simple and heartfelt, not complex and technocratic.
• Doing your homework before communicating, especially to understand what people are feeling.
• Speaking to anxieties, confusion, anger, and distrust.
• Ridding communication channels of junk so that important messages can get through.
• Using new technologies to help people see the vision (intranet, satellites, etc.).

What Does Not Work

• Under communicating, which happens all the time.
• Speaking as though you are only transferring information.
• Accidentally fostering cynicism by not walking the talk.

Step 5 – Empower Action
Deal effectively with obstacles that block action, especially disempowering bosses, lack of information, the wrong performance measurement and reward systems, and lack of self-confidence.

What Works

• Finding individuals with change experience who can bolster people’s self-confidence with we-won-you-can-too anecdotes.
• Recognition and reward systems that inspire, promote optimism, and build self-confidence.
• Feedback that can help people make better vision-related decisions.
• “Retooling” disempowering managers by giving them new jobs that clearly show the need for change.

What Does Not Work

• Ignoring bosses who seriously disempower their subordinates.
• Solving the boss problem by taking away their power (making them mad and scared) and giving it to their subordinates.
• Trying to remove all the barriers at once.
• Giving in to your own pessimism and fears.

Step 6 – Create Short-Term Wins
Produce sufficient short-term wins, sufficiently fast, to energize the change helpers, enlighten the pessimists, defuse the cynics, and build momentum for the effort.

What Works

• Early wins that come fast.
• Wins that are as visible as possible to as many people as possible.
• Wins that penetrate emotional defenses by being unambiguous.
• Wins that are meaningful to others – the more deeply meaningful the better.
• Early wins that speak to powerful players whose support you need and do not yet have.
• Wins that can be achieved cheaply and easily, even if they seem small compared with the grand vision.

What Does Not Work

• Launching fifty projects all at once.
• Providing the first win too slowly.
• Stretching the truth.

Step 7 – Don’t Let Up
Continue with wave after wave of change, not stopping until the vision is a reality, despite seemingly intractable problems.

What Works

• Aggressively ridding yourself of work that wears you down – tasks that were relevant in the past but not now, tasks that can be delegated.
• Looking constantly for ways to keep urgency up.
• Using new situations opportunistically (as in “The Street”) to launch the next wave of change.
• As always – show ‘em, show ‘em, show ‘em.

What Does Not Work

• Developing a rigid four-year plan (be more opportunistic).
• Convincing yourself that you’re done when you aren’t.
• Convincing yourself that you can get the job done without confronting some of the more embedded bureaucratic and political behaviors.
• Working so hard you physically and emotionally collapse (or sacrifice your off-the-job life).

Step 8 – Make Change Stick
Be sure the changes are embedded in the very culture of the enterprise so that the new way of operating will stick.

What Works

• Not stopping at step 7 – it isn’t over until the changes have roots.
• Using new employee orientation to compellingly show recruits what the organization really cares about.
• Using the promotions process to place people who act according to the new norms into influential and visible positions.
• Telling vivid stories over and over about the new organization, what it does, and why it succeeds.
• Making absolutely sure you have the continuity of behavior and results that help a new culture grow.

What Does Not Work

• Relying on a boss or a compensation scheme, or anything but culture, to hold a big change in place.
• Trying to change culture as the first step in the transformation process.

10 Principles of Change Management

Strategy and Business magazine recently posted a "Resilience Report" titled "10 Principles of Change Management". As we've discussed (What is A Software Architect?, Most Important Competencies of the Software Architect, Governance Without Goodwill Is Dead, Service Design Principles and Governance), communication and leadership are critical skills for the Architect. These skills are really put to the test as part of initiatives with a large Change Management component - the sort of transformation programs the Architect is often asked to lead.

Strategy and Business identifies the following 10 principles of Change Management. These principles articulate what you might consider to be fundamental truths and to varying degrees should inform nearly every change initiative. I'll follow up with a post about the process of managing change. A solid understanding of these principles and processes pays great dividends.

1. Address the “human side” systematically. Any significant transformation creates “people issues.” New leaders will be asked to step up, jobs will be changed, new skills and capabilities must be developed, and employees will be uncertain and resistant. Dealing with these issues on a reactive, case-by-case basis puts speed, morale, and results at risk. A formal approach for managing change — beginning with the leadership team and then engaging key stakeholders and leaders — should be developed early, and adapted often as change moves through the organization. This demands as much data collection and analysis, planning, and implementation discipline as does a redesign of strategy, systems, or processes. The change-management approach should be fully integrated into program design and decision making, both informing and enabling strategic direction. It should be based on a realistic assessment of the organization’s history, readiness, and capacity to change.

2. Start at the top. Because change is inherently unsettling for people at all levels of an organization, when it is on the horizon, all eyes will turn to the CEO and the leadership team for strength, support, and direction. The leaders themselves must embrace the new approaches first, both to challenge and to motivate the rest of the institution. They must speak with one voice and model the desired behaviors. The executive team also needs to understand that, although its public face may be one of unity, it, too, is composed of individuals who are going through stressful times and need to be supported.

Executive teams that work well together are best positioned for success. They are aligned and committed to the direction of change, understand the culture and behaviors the changes intend to introduce, and can model those changes themselves. At one large transportation company, the senior team rolled out an initiative to improve the efficiency and performance of its corporate and field staff before addressing change issues at the officer level. The initiative realized initial cost savings but stalled as employees began to question the leadership team’s vision and commitment. Only after the leadership team went through the process of aligning and committing to the change initiative was the work force able to deliver downstream results.

3. Involve every layer. As transformation programs progress from defining strategy and setting targets to design and implementation, they affect different levels of the organization. Change efforts must include plans for identifying leaders throughout the company and pushing responsibility for design and implementation down, so that change “cascades” through the organization. At each layer of the organization, the leaders who are identified and trained must be aligned to the company’s vision, equipped to execute their specific mission, and motivated to make change happen.

A major multiline insurer with consistently flat earnings decided to change performance and behavior in preparation for going public. The company followed this “cascading leadership” methodology, training and supporting teams at each stage. First, 10 officers set the strategy, vision, and targets. Next, more than 60 senior executives and managers designed the core of the change initiative. Then 500 leaders from the field drove implementation. The structure remained in place throughout the change program, which doubled the company’s earnings far ahead of schedule. This approach is also a superb way for a company to identify its next generation of leadership.

4. Make the formal case. Individuals are inherently rational and will question to what extent change is needed, whether the company is headed in the right direction, and whether they want to commit personally to making change happen. They will look to the leadership for answers. The articulation of a formal case for change and the creation of a written vision statement are invaluable opportunities to create or compel leadership-team alignment.

Three steps should be followed in developing the case: First, confront reality and articulate a convincing need for change. Second, demonstrate faith that the company has a viable future and the leadership to get there. Finally, provide a road map to guide behavior and decision making. Leaders must then customize this message for various internal audiences, describing the pending change in terms that matter to the individuals.

A consumer packaged-goods company experiencing years of steadily declining earnings determined that it needed to significantly restructure its operations — instituting, among other things, a 30 percent work force reduction — to remain competitive. In a series of offsite meetings, the executive team built a brutally honest business case that downsizing was the only way to keep the business viable, and drew on the company’s proud heritage to craft a compelling vision to lead the company forward. By confronting reality and helping employees understand the necessity for change, leaders were able to motivate the organization to follow the new direction in the midst of the largest downsizing in the company’s history. Instead of being shell-shocked and demoralized, those who stayed felt a renewed resolve to help the enterprise advance.

5. Create ownership. Leaders of large change programs must overperform during the transformation and be the zealots who create a critical mass among the work force in favor of change. This requires more than mere buy-in or passive agreement that the direction of change is acceptable. It demands ownership by leaders willing to accept responsibility for making change happen in all of the areas they influence or control. Ownership is often best created by involving people in identifying problems and crafting solutions. It is reinforced by incentives and rewards. These can be tangible (for example, financial compensation) or psychological (for example, camaraderie and a sense of shared destiny).

At a large health-care organization that was moving to a shared-services model for administrative support, the first department to create detailed designs for the new organization was human resources. Its personnel worked with advisors in cross-functional teams for more than six months. But as the designs were being finalized, top departmental executives began to resist the move to implementation. While agreeing that the work was top-notch, the executives realized they hadn’t invested enough individual time in the design process to feel the ownership required to begin implementation. On the basis of their feedback, the process was modified to include a “deep dive.” The departmental executives worked with the design teams to learn more, and get further exposure to changes that would occur. This was the turning point; the transition then happened quickly. It also created a forum for top executives to work as a team, creating a sense of alignment and unity that the group hadn’t felt before.

6. Communicate the message. Too often, change leaders make the mistake of believing that others understand the issues, feel the need to change, and see the new direction as clearly as they do. The best change programs reinforce core messages through regular, timely advice that is both inspirational and practicable. Communications flow in from the bottom and out from the top, and are targeted to provide employees the right information at the right time and to solicit their input and feedback. Often this will require overcommunication through multiple, redundant channels.

In the late 1990s, the commissioner of the Internal Revenue Service, Charles O. Rossotti, had a vision: The IRS could treat taxpayers as customers and turn a feared bureaucracy into a world-class service organization. Getting more than 100,000 employees to think and act differently required more than just systems redesign and process change. IRS leadership designed and executed an ambitious communications program including daily voice mails from the commissioner and his top staff, training sessions, videotapes, newsletters, and town hall meetings that continued through the transformation. Timely, constant, practical communication was at the heart of the program, which brought the IRS’s customer ratings from the lowest in various surveys to its current ranking above the likes of McDonald’s and most airlines.

7. Assess the cultural landscape. Successful change programs pick up speed and intensity as they cascade down, making it critically important that leaders understand and account for culture and behaviors at each level of the organization. Companies often make the mistake of assessing culture either too late or not at all. Thorough cultural diagnostics can assess organizational readiness to change, bring major problems to the surface, identify conflicts, and define factors that can recognize and influence sources of leadership and resistance. These diagnostics identify the core values, beliefs, behaviors, and perceptions that must be taken into account for successful change to occur. They serve as the common baseline for designing essential change elements, such as the new corporate vision, and building the infrastructure and programs needed to drive change.

8. Address culture explicitly. Once the culture is understood, it should be addressed as thoroughly as any other area in a change program. Leaders should be explicit about the culture and underlying behaviors that will best support the new way of doing business, and find opportunities to model and reward those behaviors. This requires developing a baseline, defining an explicit end-state or desired culture, and devising detailed plans to make the transition.

Company culture is an amalgam of shared history, explicit values and beliefs, and common attitudes and behaviors. Change programs can involve creating a culture (in new companies or those built through multiple acquisitions), combining cultures (in mergers or acquisitions of large companies), or reinforcing cultures (in, say, long-established consumer goods or manufacturing companies). Understanding that all companies have a cultural center — the locus of thought, activity, influence, or personal identification — is often an effective way to jump-start culture change.

A consumer goods company with a suite of premium brands determined that business realities demanded a greater focus on profitability and bottom-line accountability. In addition to redesigning metrics and incentives, it developed a plan to systematically change the company’s culture, beginning with marketing, the company’s historical center. It brought the marketing staff into the process early to create enthusiasts for the new philosophy who adapted marketing campaigns, spending plans, and incentive programs to be more accountable. Seeing these culture leaders grab onto the new program, the rest of the company quickly fell in line.

9. Prepare for the unexpected. No change program goes completely according to plan. People react in unexpected ways; areas of anticipated resistance fall away; and the external environment shifts. Effectively managing change requires continual reassessment of its impact and the organization’s willingness and ability to adopt the next wave of transformation. Fed by real data from the field and supported by information and solid decision-making processes, change leaders can then make the adjustments necessary to maintain momentum and drive results.

A leading U.S. health-care company was facing competitive and financial pressures from its inability to react to changes in the marketplace. A diagnosis revealed shortcomings in its organizational structure and governance, and the company decided to implement a new operating model. In the midst of detailed design, a new CEO and leadership team took over. The new team was initially skeptical, but was ultimately convinced that a solid case for change, grounded in facts and supported by the organization at large, existed. Some adjustments were made to the speed and sequence of implementation, but the fundamentals of the new operating model remained unchanged.

10. Speak to the individual. Change is both an institutional journey and a very personal one. People spend many hours each week at work; many think of their colleagues as a second family. Individuals (or teams of individuals) need to know how their work will change, what is expected of them during and after the change program, how they will be measured, and what success or failure will mean for them and those around them. Team leaders should be as honest and explicit as possible. People will react to what they see and hear around them, and need to be involved in the change process. Highly visible rewards, such as promotion, recognition, and bonuses, should be provided as dramatic reinforcement for embracing change. Sanction or removal of people standing in the way of change will reinforce the institution’s commitment.

Most leaders contemplating change know that people matter. It is all too tempting, however, to dwell on the plans and processes, which don’t talk back and don’t respond emotionally, rather than face up to the more difficult and more critical human issues. But mastering the “soft” side of change management needn’t be a mystery."

Watch for the follow up on the process of change and look for every opportunity to learn more about Change Management principles. Great Architects are distinguished by practical skills in this area.

Hottest Posts in 2008

2008 was a good year for the practice of software architecture. Good progress is being made throughout the industry in recognition of Architecture as a distinct discipline, and the maturity of the discipline was advanced by the excellent writings of a host of passionate practitioners. Looking back at the year on SoftwareArchitecture.com, I found it interesting to note the most popular posts of 2008. People are clearly interested in understanding the profession and reinforcing their practice. Let's keep it up!

The top 10 most popular posts of 2008:

1. What Is Software Architecture?
2. Good Architecture Descriptions are Explosive
3. Service Design Principles (the Contract Story Continues)
4. Most Important Competencies of the Software Architect
5. What is a Software Architect?
6. The Architect is Accountable!
7. Progressive Refinement of Estimates (aka The Transmission Repair)
8. The Proactive Architect
9. Governance Without Goodwill is Dead
10. The Making of a Governance Anti-Pattern?

I hope you have found something thought-provoking. As always, I love hearing from you.