A decent proposal: how to evaluate your agency or service provider's proposal or statement of work

Bill Haeck, Vice President of Program Management | One To One Interactive
June 26, 2002

The wining and dining are over, relationships have been established, and you and your agency are both fired up about the new project you've been talking about. Ideas have flourished, enthusiasm is high, and you're ready to watch the project fly. If you're like many companies, your primary attention turns to getting a price, lining up resources, and getting the project moving as quickly as possible. Resist the temptation to just check the price and pass the document on to your legal department — and consider the following facts. The Standish Group estimates that US $145 billion are lost each year due to failed or challenged projects. The Meta Group estimates that number to be US $180 billion. According to the Standish Group's 1995 release of The CHAOS Report ( www.standishgroup.com), "for every 100 projects, there are 94 restarts," and "only 9% of projects for large companies come in on-time and on-budget." A great many of those failures can be directly attributed to the failure of companies to effectively navigate the contracting/proposal process.

The review of the Statement of Work (SOW) or proposal is an often overlooked or ignored step in the project initiation process. For many, the SOW is simply viewed as a contract, a formality to create a binding legal obligation. This may be its primary purpose, but if you look for and demand a well-written SOW, your chances of avoiding surprises and joining the long list of failed projects are dramatically improved. So how can you use a proposal as a predictor for project success, and what should you be looking for in your agency's proposal to maximize your chances of success? Here are five recommendations of what to look for in any proposal you receive.

1. Does the SOW reflect thoughtful planning on the part of the agency?

Vendors who rush to put a contract in front of you without taking the time to think through the details of your project are likely to take the same approach once a project starts. Why look for an agency that has a tendency to look before it leaps? "Projects that skimp on the upstream activities typically have to do the same work downstream at anywhere from 10 to 100 times the cost of doing it earlier." (Boehm & Papaccio 1998). In short, think of the SOW as a mini-project in and of itself, with the SOW as the deliverable. If the SOW presented to you reflects thoughtful planning and attention to detail, you can generally expect the same from the project. Thoughtful planning generally leads to successful project execution.

So what are some key signs that the SOW reflects good planning on the part of the agency? Look closely at the sections describing the project. Does the approach described reflect some detailed thought about your project, its inherent risks and success metrics, and an understanding of your business objectives? Or is it a cookie cutter methodology with your company's name inserted in the blank? Chances are, if it's the latter, your half-baked SOW will be followed by half-baked project. Is there a "Roles and Responsibilities" section that reflects that the agency has put some thought into the right project governance structure and how to work specifically with your organization? Finally, what about an "Assumptions" section? Frequently, the assumptions section is overlooked, but in many ways it can be an excellent indicator of how much thought and effort has gone into a SOW. Unlike some of the more explicit SOW items like cost or schedule, identifying the subtle assumptions which so often derail projects and serve as the source for misguided expectations generally requires a great deal of thought and planning and doesn't lend itself to a spreadsheet. In short, the assumptions section can often be a great bellwether for determining how much thought the proposal's authors put into planning your project.

2. Does the scope section clearly communicate the boundaries of the project?

According to the British Computer Society in 2001, improper scope definition was one of the top five causes of failed project delivery. On all projects, but especially on fixed price projects, failure to adequately define the proper project scope can doom projects before they start. Read through the "Scope" section carefully, looking wherever possible for specific numbers. Will the agency supply you with one report or two? Will they be optimizing weekly, biweekly, or monthly? How many tests will they actually perform? If your agency is looking to do some work on your website, will they be responsible for all of the applications on the site? Who is responsible for content? Who will be hosting and maintaining the site? Failure to identify and come to terms with scope items like these can open the door for ballooning budgets, missed schedules, and ultimately, project failure. The SOW should be the primary source for identifying and coming to terms with those issues, so read the scope section carefully. If these seemingly minor boundaries haven't been properly identified in the SOW, prepare for a project beset by scope changes, countless negotiations, and multiple surprises.

3. Do the contracting terms of the proposal match the type of project?

There are a multitude of contract types that have evolved to meet the wide variety projects that are executed today. They range from pure fixed price projects, where the agency or service provider assumes all the risk, to time and materials projects, where the bulk of the risk is assumed by the contracting agency. There are volumes of material written about contracting and contract types, but here are a few basic pointers.

First, always try to match the risk of the project with the contract type. Fixed price work places all of the risk on the agency or service provider, while time and materials type contracts displace more of the risk back to the contracting agency. So why not insist on always getting fixed price contracts? If an agency is forced to write a fixed price contract for an inherently risky project (one which features new technology, is multi-phased or difficult to scope, or one which is particularly complex), they will, by necessity, build a great deal of buffer in the fee to account for risk and unknowns. As a result, you may end up balking at the price or paying too much when, in fact, the project might have been able to be executed for much less. Time and material contracts are much more appropriate for these type of projects. True, you'll need to manage the project closer to mitigate your end of the risk, but ultimately, both parties end up with a better chance of walking away with a successful project.

Second, try to match the contracting style with your management style. Although both contract types can be modified to meet your needs, in general, time and materials projects are geared for support managers who like lots of reporting, lots of information, and tight control of ongoing projects. Correspondingly, fixed price projects call for and generally operate with considerably less interaction between the agency and the contracting company. Since the agency has assumed all of the risk, there is less need for regular reports, ongoing budget reviews, etc. Thus, one of the inherent advantages of fixed price projects is that they generally require less overhead and, therefore, can be executed for less money.

There are a multitude of other factors to consider regarding contract selection type, including size of the project, incentive elements, relationship with your agency, etc., that are far to numerous to discuss here. So, before you rush to sign a proposal or try to get an agency to follow a "one size fits all" approach to contract types, understand the subtleties of your SOW, and work with your agency or contracting department to make sure you're establishing a fit between your contract type and the project. By properly matching the contract types to the risk of the project and your management style, you'll end up increasing your chances of getting the cheapest price, as well as a successfully executed project.

4. Are you absolutely clear about what you're paying for?

The SOW should provide you with a very clear indication of the deliverables for the project. In some cases, like that of a media plan, it's very clear what you get for your money. In many cases, however, this can be an area open for great misinterpretation. For example, your deliverable may be listed as a report or a prototype. These can be dangerous terms which can come back to haunt both parties later on. Wherever possible, deliverables should be defined in some degree of detail. Even better, if possible, have the agency provide copies of sample reports or deliverables so there's no room for misunderstanding. There may very well be cases where your agency or service provider isn't yet sure of the exact composition of the final deliverable. There's nothing inherently wrong with this approach. By noting this, however, you can identify this lack of definition as a key risk, which can be monitored and mitigated as the project progresses. In doing so, you can prevent the very real risk of arriving at the final day of reckoning, the presentation of the deliverable, only to discover that there's been a big disconnect between your definition of the deliverable and that of your agencies.

5. Does the SOW articulate your role in the project?

Finally, after reading the SOW, it should be abundantly clear how you will participate on the project. More often than not, projects are characterized by a fair to high level of interaction between agencies and their contracting organizations. Failure to understand those interactions ahead of time can lead to big problems. The SOW is a great source for gaining this key understanding. Roles and responsibilities should clearly define your ongoing management and governance roles, from deliverable signoff to sponsor to program manager. The assumptions and scope sections should detail items like what you are expected to provide, who you are supposed to have available to the project team, etc. Finally, the approach section should provide insight into how your team fits into the overall scheme of the project (who will do QA, who will take the handoff from the agency's design team to your development team). Together all of the sections should leave you with no doubt as to how you and your partner will work together to make the project a success. If you read the SOW and walk away with anything less than a detailed, crystal clear picture of how you're going to participate on the project, the alarm bell should be ringing.

In summary, use the SOWs you receive as the valuable tool they are. Ignore the urge to view them as simple contracts and think of them as crystal balls for looking at the future of your project. The SOW is probably one of the best predictors you've got for determining if your project is slated to become a success or just another project gone bad.

back to White Papers