The trouble with estimates

By its very nature, estimation is a game of unknowns. When we make projections for budgets, revenue or returns, we base them on a series of wobbly assumptions. Each assumption in itself might be somewhat reasonable, but the more of them we put together, the shakier the foundation is on which we build our estimate… and the more likely the estimate is to collapse in a heap.

The fact that we base so much of our decision-making on these estimates ought to be worrisome, when we consider how they’re actually built in the first place.


Unknown requirements

We all know that the cost part of estimating relies on a series of guesstimates.

What we also know, but don’t like to admit out loud, is that those “guesstimates” usually include some or all of the following:

  • We’ve never done this before and we’re going to learn how to do it on the job.
  • We have no idea what this thing is yet – no specs, no parameters, nothing.
  • Someone needed the estimate right away and wouldn’t stop bugging us for it, so we just invented a random number to make them go away.
  • We’re basing our estimate on someone else’s equally wild-guess estimates.
  • The tools we’re using to estimate are giving us unreliable data.
  • We don’t care how much it costs us to do; we just care how much the client is willing to pay for it.

If that sounds familiar, there’s a reason. It’s because we’ve erased virtually all accountability for the accuracy of our estimates, and with that, we’ve eliminated the important steps that it takes to cost out a project more accurately.

This happens all the way down the line: Impatient clients can’t wait for things to be scoped out in order to lock down their budgets; account teams pressure technical or creative folks to jump the gun on their estimation process; agency management doesn’t want to invest in the man-hours to scope out a project properly prior to getting the client to sign off on the budget. And so, the chicken-or-the-egg trap of estimation continues.

Some shops are starting to recognize the value in investing in estimation. Particularly in cases where projects vary widely or where there is a lot of new learning to be done on every project, the value of this investment can pay off in spades if the estimation work is more accurate.

But often, agencies just don’t care, because pricing tends to be value-based as opposed to cost-based. “We’ll win the business and then we’ll figure out how to deliver at a profit” is a mantra that we hear time and again. Sometimes it takes being badly burned by inaccurate estimation often enough for a company to change.

Unknown performance

The other side to forecasting the future is calculating projected revenues. And here, the estimation game goes even further afield. At least with costs, we exert some degree of control over the things we do and what we charge for them. But as for how a program will perform? That’s often so completely out of everyone’s hands that it’s almost impossible to predict.

Like with the stock market, in the marketing world, past performance is not a guarantee of future performance. Still, for lack of better options, we tend to base our performance forecasts on how things did in the past.

With brand new clients or programs without history, it’s an even bigger gamble. Too often, we use “industry averages”, which are usually so average that they’re basically useless, in order to make predictions about programs that may fall well outside the norms for a variety of reasons.

Clients do themselves a disservice by asking for hard numbers from an agency for a new program, too. Not only does it create unrealistic expectations, but they’re liable to base important decisions – like hiring, corporate expansion or investment – on those forecasts.

One way around this trap is to set up a test period for the purposes of establishing benchmarks. After that initial test period, it’s likely possible to create some more reliable projections for a program’s performance moving forward. Of course, a lot will still be unknown – and there are never guarantees – but it’s better than going in completely blind.

As we see more agencies move to performance-based compensation models, the accuracy of these projections will improve. It’ll have to. Agencies have been far too content in the past to promise pie-in-the-sky results to clients, knowing that they would never be held accountable for them. But when their paycheques start to depend on delivering those results, we’ll probably see things start to shift.

Unknown X factors

What will change in the digital world? Where will the market be by this time next year? What will the competition do? Will there be an unanticipated PR nightmare for the company? Will someone get bought out, merge or fold?

In the digital world, we get comfortable with a rapid pace of change. We pretty much have to. But our projections can’t realistically be expected to take this level of unpredictability into account.  A client who wants to know in 2011 how his program will be performing in 2015 is making a series of dangerous assumptions, starting with the fact that his program will even still exist in 2015 — when in all likelihood, it will have been replaced by something else entirely by then.

The further into the future you go, the less accurate your projections are. That’s just life. And in the digital world, sometimes 10 months – or even 10 days – can be considered “far” into the future. If we know what’s going to happen in the next 10 minutes, sometimes we’re doing pretty well.

But you can’t run a business like that either, nor can you make smart decisions about marketing investment with that attitude. The only solution is to run the forecasts, but expect them to change. We also should be advising our clients to amortize their marketing investments over shorter and shorter periods of time.

Ultimately, this will probably lead to a more nimble, less clunky decision-making process. More smaller investments and rapid launches of tactical initiatives, and fewer giant campaign overhauls that require months of planning and development. Because while it’s important to estimate, but it’s even more important to be able to optimize as you go along. It’s evolution for the digital age: the most adaptable are the most likely to survive.