Accuracy or Precision?

 

It is the mark of an educated mind to rest satisfied with the degree of precision which the nature of the subject admits and not to seek exactness where only an approximation is possible. – Aristotle (384 BC – 322 BC)

barn

My favourite part about writing these articles is finding just the right quote to introduce them. Before we get started, go back and read the quote from Aristotle above if you happened to skip past it – I think it both accurately and precisely summarizes my argument.

Now in the context of for the , let's talk about what each of these terms mean:

Accuracy: Ability to hit the target (i.e. how close is the actual to the forecast?)

Precision: Size of the target you're aiming at (i.e. specificity of product, place and timing of the forecast)

I'm sorry to be a total downer, but the reason this article is titled Accuracy or Precision is because you can't have both. The upper right quadrant in the illustration above ain't happening (a bit more on that later).

In the world of forecasting, people seem obsessed with accuracy and often ask questions like:

  • What level of are you achieving?
  • How should we be benchmarking our forecast accuracy?
  • Are we accurate enough? How can we be more accurate?

The problem here is that any discussion about forecast accuracy that does not at the same time account for precision is a complete waste of time.

For example, one tried and true method for increasing forecast accuracy is by harnessing the mystical properties of The Law of Large Numbers.

To put it another way – by sacrificing precision.

Or to put it in the most cheeky way possible (many thanks to Richard Sherman for this gem, which I quote often):


Sherman's Law:
Forecast accuracy improves in direct correlation to its distance from usefulness.

So how do we manage the tradeoff between precision and accuracy in forecasting?You must choose the level of precision that is required (and no more precise than that) and accept that in doing so, you may be sacrificing accuracy.

For a retailer, the only demand that is truly independent is customer demand at the point of sale. Customers choose specific items in specific locations on specific days. That's how the business works.

This means that the precision of the forecasting must be by item by location by day – full stop.

Would you be able to make a more accurate prediction by forecasting in aggregate for an item (or a group of items) across all locations by month? Without a doubt.

Will that help you figure out when you need to replenish stock for a 4 pack of 9.5 watt A19 LED light bulbs at store #1378 in Wichita, Kansas?

Nope. Useless.

I can almost see the wincing and hear the heart palpitations that this declaration will cause.

“Oh God! You'll NEVER be able to get accurate forecasts at that level of precision!” To that I say two things:

  1. It depends on what level of accuracy is actually required at that level of precision.
  2. Too damn bad. That's the requirement as per your customers' expectation.
With regard to the first point, keep in mind that it's not uncommon for an item in a retail store to sell fewer than 20 units per YEAR. On top of that, there are minimum display quantities and pack rounding that will ultimately dictate how much will be available to customers to a much greater degree than the forecast.Forecasts by item/location/day are still necessary to plan and schedule the upstream supply chain properly, but it's only necessary for forecasts at that level of precision to be reasonable, not accurate in the traditional sense of the word. This is especially true if you also replan daily with refreshed sales and inventory numbers for every item at every location.

There are those out there who would argue that my entire premise is flawed. That I'm not considering the fact that with advances in , and , it will actually be possible to process trillions of data elements simultaneously to achieve both precision and accuracy. That I shouldn't even be constraining my thinking to daily forecasting – soon, we'll be able to forecast hourly.

Let's go back to the example I mentioned earlier – an item that sells 20 units (give or take) in a location throughout the course of a year. Assuming that store is open for business 12 hours out of every day and closed 5 days per year for holidays, there are 4,320 hours in which those 20 units will sell. Are we to believe that collecting tons of noise (whoops, I meant “data”) from social media, weather forecasting services and the hourly movement of soybean prices (I mean, why not, right?) will actually be able to predict with accuracy the precise hour for each of those 20 units in that location over the next year? Out of 4,320 hours to choose from? Really?

(Let's put aside the fact that no retailer that I've ever seen even measures how accurate their on hand records are right now, let alone thinking they can predict sales by hour).

I sometimes have a tendency to walk the middle line on these types of predictions. “I don't see it happening anytime soon, but who knows? Maybe someday…”

Well, not this time.

This is utter BS. Unless all of the laws of statistics have been debunked recently without my noticing, degrees of freedom are still degrees of freedom.

Yes, I'm a loud and proud naysayer on this one and if anyone ever actually implements something like that and demonstrates the benefits they're pitching, I will gleefully eat a wheelbarrow of live crickets when that time comes (assuming I'm not long dead).

In the meantime, I'm willing to bet my flying car, my personal jetpack and my timeshare on the moon colony (all of which were supposed to be ubiquitous by now) that this will eventually be exposed as total nonsense.

Leave a Reply

Your email address will not be published. Required fields are marked *