Friday, June 5, 2009

Defining Project Success

For decades, project managers preached the mantra of the "Iron Triangle" to project teams and stakeholders: Cost, Time, and Scope, also known as the "Triple Constraint."


The Iron Triangle

There has always been some variation in the terms used for the "legs" of the Triple Constraint and many practitioners recognize more than three constraints including PMI who added "legs" in the most recent revision (fourth edition) of PMBOK. Nevertheless, the Iron Triangle still stands today as a guiding principle of practice with the majority of project managers.

Define "success"

Many organizations defined project success based on some variation of the Triple Constraint for decades as well, some placing more emphasis or value on one leg or some combination of legs. In actual practice, there is no "standard" measure of project success across all organizations so generalizations about as well as comparisons of project success metrics are difficult. However, most are based in whole or in part on some flavor of the Triple Constraint.

Software projects, in particular, have a very poor reputation for success (Standish and others). This is largely the result of "waterfall thinking" and poor survey design but the mud slung at software projects still sticks to the wall. A great deal of that mud should be washed away and the concept of how "success" is defined in technology projects is in need of a make-over.

Why?

Because surveys and other data gathered to determine success or failure of projects are, most often, based on comparing final (closed) project metrics with original (baseline) estimates of cost, schedule, and scope. In the case of a building, road, or bridge project, this approach makes a certain amount of sense -- the thing (product) is designed up front and delivered according to design. In the case of most software projects, the thing (product) is an amorphous concept at best that can and does change radically in design as development proceeds.

In the case of a bridge construction project, there may be schedule delays or cost over-runs due to weather, late delivery of materials, lack of resources on site, and the like but there is never any doubt about what the finished product is or what it should do. It's "this wide" and "this tall" and is "this type" with "these materials" and carries "X" lanes of traffic. That is known not only before the start of construction but before anyone bids the job. Both the problem and the solution, defined in great detail, are known.

That is almost never the case with software development projects.

Problem?

You bet!

Many software projects declared a "success" based on meeting the original scope, time, and cost criteria are actually failures because the product delivered didn't meet the business needs. This may seem oxymoronic at first blush but is most assuredly the case. It means the criteria for measuring success is probably wrong, incomplete, or inconsistent with reality... the wrong things are measured; the wrong criteria are managed.

Define "failure"

If a project delivers 25% behind schedule and 30% over budget but the business need is met and the numbers still "crunch" in terms of ROI, this project is still declared a failure according to most measurement methods. The project manager didn't meet the scheduled delivery dates and there was a serious (30%) cost over run. The business could make or save millions of dollars as a result of the effort but it goes down in the record book as an failure. The stakeholders are disappointed, the project manager is squirming on the hook, and the delivery team is demoralized. Another apparent oxymoron and an issue with the criteria used.

This is an example of 19th and early 20th century "waterfall thinking" at its worst. The project team is expected to know the unknowable from the start before anything is known, then make an estimate and plan based on both the unknown and the unknowable, and finally live or die by the planned (baseline) scope, schedule, and cost. It's no wonder software development and technology people have stress issues.

Where's the product?

In the Triple Constraint, the only leg specifically applicable to product is Scope with the caveat that Quality also plays in the product space. However, quality is most often measured by how well the project adheres to whatever "process" is mandated and the number of defects detected -- Quality Assurance and Quality Control.

In the case of defects in software projects, they are determined by comparing results of test cases (which do not cover 100% of the possibilities) with the product design specifications. If the design specifications do not meet the business needs, this is not a "defect."

In the case of "process adherence," the product is never measured or considered; customer satisfaction is not a criteria.

The business results from the product produced are reviewed (if at all) in a post-implementation review, usually some months after the project ends. By the time this review process occurs, the project is usually already on the books as a success or failure.

Another problem?

Uh huh.

Negotiation or criteria?

The Triple Constraint is more about negotiation than anything else. Negotiation is intuitive; no degrees or certifications required. Everyone "gets it" and uses it when making a significant purchase - a car, refrigerator, or home, for example. Some people are better at negotiation than others, but everyone understands.

No one wants to pay more than it's worth, wait any longer than they have to for delivery, accept a scaled down version, or pay for added features that are not needed. That's the Triple Constraint in a nutshell -- a simple negotiation model. "You can get siding instead of brick for much less," or "We have this color on the lot right now; if you want that color it will take three weeks for delivery." It's negotiation.

Granted, the Triple Constraint works well in negotiation; at least as a starting point for negotiation. And, granted, negotiation plays a substantial role in a project managers life. However, when the concept of the problem/solution/product is still being elaborated... still being negotiated... still fluid and may remain so until almost the end of the project, how well does the Triple Constraint baseline concept work as the criteria for determining the success of the effort?

Not very well.

It works great if what is needed is already known -- like ordering a car or a hamburger. The "problem" is known (lack of transportation, or hunger) and the "solution" is obvious (a car or hamburger). Only negotiating the price, features, and delivery schedule remains. Using the terms of the final negotiation as the criteria for success, arguably perhaps, makes a lot of sense in these cases. But, can a business "place an order" with its IT department to solve a problem not fully understand by implementing a solution that hasn't been created?

Not likely.

If almost nothing (or somewhere between nothing and something) is known about the problem/solution/product up front, those things can not be negotiated to a baseline up front. They must be continuously negotiated, continuously elaborated and defined/refined as the project proceeds. If that's the case (and it certainly is with many software development and technological innovation efforts), then what is the criteria for success and what do project managers manage to in terms of project objectives or goals?

Software and technology project managers are obviously not defining the correct criteria -- or at least not including a full set of criteria needed to manage software and technology development efforts.

How does that improve?

By asking people what they want and need... And, continuously asking them for those needs and feedback based on actual results as the project proceeds. By engaging and collaborated with everyone throughout the project to ensure their needs are met.

What does success look like?

On the Dr. Dobbs web site (yes, they still exist ;~), there is an article on defining project success; a nice one. It's brief (3 pages) so read it here if you have a few minutes.

In the article, Scott W. Ambler (of Agile and IBM Rational fame), publishes poll results based on using five criteria for project success. Here's what respondents considered important.

  • Schedule: 61.3 percent of respondents said that it is more important to deliver a system when it is ready to be shipped than to deliver it on time.
  • Scope: 87.3 percent said that meeting the actual needs of stakeholders is more important than building the system to specification.
  • Money: 79.6 percent said that providing the best return on investment (ROI) is more important than delivering a system under budget.
  • Quality: 87.3 percent said that delivering high quality is more important than delivering on time and on budget.
  • Staff: 75.8 percent said that having a healthy, both mentally and physically, workplace is more important than delivering on time and on budget.

Note: Read more about the number and type of respondents in the article on page 3.

Wrap it up

Scott's acquired and compiled data is excellent information and puts the success criteria for technology development efforts in much better perspective. It is difficult to convey just how refreshing this is to a project manager who has battled the Triple Constraint success criteria for many years.

  • ROI is more important than budget.
  • Meeting the needs of people (stakeholders) is more important than specifications.
  • Readiness is more important than schedule.
  • Quality is more important than schedule or cost.
  • Mentally and physically healthy people are more important than schedule or cost.

Sound familiar?

Read the Agile Manifesto recently?

There is great emphasis from survey respondents on the condition of the final product as opposed to arbitrary milestones or deadlines. Does the product actually fill the (changing, evolving) needs of the customer when delivered? Is the product really ready for delivery or is this just meeting an arbitrary deadline with an unfinished product? Is the quality of the product at a level where it can be placed in users hands or on the market with confidence or does it require further refinement?

There is great emphasis from survey respondents on the condition of people as opposed to resource allocation, task assignments, or effort/hours. "The needs of people (stakeholders)", and "Mentally and physically healthy people." Products meet and are defined by people's needs and people create extraordinary products out of joy and passion; collaboration and understanding; mutual respect and teamwork.

Obviously it seems the real success criteria and priorities for project stakeholders lie in determining/defining the state of the final product and the state/condition of all the people involved -- stakeholders, users, team members, managers; everyone.

ROI is a number managers can use. Readiness is quantifiable. "Needs" can be determined, continuously elaborated, refined and met. Quality is definable. Healthy, happy, passionate people are something managers can monitor, mentor, and support.

There is still room for negotiation around the Triple Constraint. For example, when a customer has an undefined need but a budgetary constraint, the project can be designed to deliver whatever scope is achievable within the budget. A "time-box" can also be applied limiting the development effort to the scope achievable within a set time frame. In most cases where the team (number of people on the development team) is known, a time-box has the same effect as a budget constraint.

Too often, when project managers say "we get results," they mean meeting arbitrary deadlines instead of producing products that meet people's needs. The sacrifices people make meeting meaningless milestones, jumping through hoops to stay within purposeless budgets, and achieving arbitrary deadlines add nothing of value to the product and are inherently demeaning. Those sacrifices have tremendous negative impact to their family and friends, their health and well being, the health and safety of the work place as well as the quality, readiness, and suitability of the product. Sad, worn-out, desperate, unhappy people do not make extraordinary products.

This is not to say that baselines, budgets, schedules, processes, techniques, and tools are not important. They are. But only if and when they are based on the right criteria and balanced with a focus on people and the final product.

When do managers start giving people (stakeholders, project teams, and users) what they really want and need?

How about right now: Today is good.

Posted by: William W. (Woody) Williams

Add to Technorati Favorites Subscribe

No comments: