Asking questions of data

Percent of what?

Anyone can get an answer now. Almost nobody checks the question. This is the check, and it takes about ten seconds.

A friend of mine runs three cafés, and last spring she came out of a meeting worried. Someone had shown her a chart and said the words: coffee is nearly half of it, and it's falling fast.

Coffee falling by half sounds like the end of a café. She spent a fortnight on it: new beans, a new supplier, a price change.

The number was true. It was also coffee sold through delivery apps, which is 9% of what the shop takes. Coffee was 46% of that 9%, and in the business as a whole that was four pence in every pound. Meanwhile the drive-through, which is a quarter of everything, had been drifting down for months and nobody mentioned it.

Nobody lied, nobody made an arithmetic mistake, and the chart was correct. One question was never asked, and asking it takes ten seconds.

Percent of what?

This piece is about that question, and the handful of habits that come with it. There is no software involved and no maths beyond addition.


The whole job

Nearly every question about numbers is a split

Watch anyone dig into a business and you will see the same move over and over: take a total, break it into parts, look at the parts, pick one, and break that into parts.

Sales by how people order, delivery orders by what they buy, coffee sales by time of day. It is the same move at every level, and the whole of the analysis is a chain of them.

Which means the quality of the thinking rests on one thing: whether the parts still add up to the total.

That sounds too simple to matter. It is the check almost nobody runs, and it is the difference between a fact and a fiction that looks exactly like a fact.

A real split THE TOTAL THE PARTS 42%26%17%15%

No gaps and no overlaps: the parts add to 100, so a share means something.

Not a split THE TOTAL THE PARTS counted twice counted never

Every single number here is right. The split is wrong, and nothing on the screen says so.

The thing to fear is not a wrong number. It is a right number sitting on a broken split, because that looks exactly like a right number.


The method

Five things a good split needs

Two of these make a split true and three make it useful. A split missing any of them will still draw a chart, and the chart will still convince a room.

Nothing counted twice

One sale belongs to one part. This breaks quietly whenever a thing can carry more than one label. A customer marked both "regular" and "office account" gets counted in both, and now your parts add to more than the shop.

Nothing left out

Including the dull parts: the odds and ends, the sales with a missing label, the till that was not reconciled. An "everything else" bucket is not laziness, it is the receipt. Its size tells you something too: 2% is housekeeping, 30% means you are splitting by the wrong thing.

Always a share of the same total

This is the café story. Two levels down, people start quoting percentages of percentages, and the words sound identical. Keep every share measured against the number at the top, and "nearly half" stops being able to mean four pence in the pound.

The order is your choice

Shop, then drink, then hour is one way down. Drink, then hour, then shop is another way down the same sales, and it can leave you with a different opinion. Neither order came from the data: you picked it, usually without noticing.

Every part is a trend

A single month tells you size and hides direction. Two parts the same size, one growing and one dying, look identical in every snapshot ever printed.


The third rule, made touchable

The same number, two meanings

Here is the café. Click into delivery apps, then use the toggle to change what the percentages are measured against. Same sales, same arithmetic, two completely different impressions of the shop.

Sales

Two things worth sitting with. The four parts of delivery add to 9.0%, which is exactly what delivery is worth. Go as deep as you like and the addition still works. That is the check from the last section, holding across a level.

Then press the toggle. Coffee is either 46% or 4.1%, and both are correct. One makes it the story, the other makes it a rounding error. My friend lost a fortnight to the gap between those two true sentences.


Where to look

Big and interesting are not the same thing

The five rules give you a map you can trust. They say nothing about where to walk, and a trustworthy map of somewhere irrelevant still wastes your morning.

Real businesses are lopsided. In the café, the counter and the drive-through are five sixths of everything, and the smallest slice is worth about a penny in the pound. So the decision about where to dig is made before you dig, and it is made almost entirely on size.

Where the money actually is

One measure, so colour shows size rather than identity. The slice we went digging into is picked out in rust.

Counter Drive-through Delivery apps Office catering Everything else 58% 27% 9% 5% 1%

Illustrative figures for a small café chain, chosen to be clean. They add to 100.

Two rules fall out of a picture like that.

  • Never open a slice without knowing its size first. A dramatic collapse inside 1% of sales is a dramatic collapse inside 1% of sales.
  • Something big moving a little usually beats something small moving a lot. The drive-through slipping two points is worth more money than office catering vanishing altogether.

The second one is hard, because small things moving violently are far more fun to look at. A slice that halves makes a better chart than a slice that softens by 3%, and the softening slice is often worth ten times as much.

That is not an argument for ignoring the small stuff. Delivery was worth looking at, and looking at it found something real. It is an argument for the size being on screen at the moment you choose, rather than worked out afterwards to justify a choice you already made.


A different question

"Why did it change?" is not the same as "what is it made of?"

Everything so far splits up a total as it stands today. The question people actually walk in with is a change: it went down, and why.

A change splits too, and the same rule applies: the pieces have to add up to the movement. But size and blame are unrelated. A big slice can explain nothing, because it did not move, and a tiny slice can explain everything, because it fell off a cliff.

Why sales fell last quarter

Each slice's contribution to the change, biggest first. They add up to the fall, or the answer is not finished.

pushed sales up pulled them down
Last quarter Delivery Catering Drive-through Counter Else This quarter +2.1 +0.4 -3.6 -2.0 -0.9 +2.1 +0.4 -3.6 -2.0 -0.9 = -4.0  ·  IT ADDS UP

Illustrative. Note that the drive-through does most of the damage and nobody had mentioned it, while delivery, the slice everyone was talking about, was actually growing.

There is a nastier version of this, worth knowing because it will one day be used on you. A total can move while every single part of it improves.

The café's average spend per order fell all year. Counter spend rose, delivery spend rose, and nobody got worse at anything. Delivery is cheaper per order and it grew, so the blend shifted, and the average fell on its own.

Both went up. The average went down.

Spend per order in each channel, and the two blended together.

counter delivery both together
up down up Q1Q2Q3Q4 SPEND PER ORDER DELIVERY GREW FROM 4% OF ORDERS TO 14%

Illustrative, and extremely common in any business with a growing cheap channel. Report only the dashed line and you will describe a decline that happened to nobody.

The defence is not cleverness. If the split covers everything and the shares are on screen next to the values, a shift in the blend is visible as a shift in the blend. Look only at the average and you cannot tell the difference between the business getting worse and the business changing shape. Those need opposite responses.


What it looks like in practice

Five questions, one after another

Nobody good asks one question. They ask four or five in a row, and each one is chosen by what the last one showed. Here is the café, the way it actually went.

One sitting, about ten minutes

The answer lives in question five, and you cannot get to question five from a standing start. You would never think to ask about spend per order until the first four had shown you that sales alone were hiding something.

Question three found nothing, and that was not waste. Ruling a slice out is how you earn the right to trust the slice you keep.


Honest limits

Four places the addition stops working

Real data is messy, and pretending otherwise is how methods like this get a bad name. Four things to watch for.

Averages do not add up

Sales add up, so sales by channel add to total sales. Spend per order does not, and neither does margin, or the share of orders that get a refund. Averaging the averages gives you a number that is simply wrong.

InsteadSplit the top and the bottom of the fraction separately, then divide at the level you care about. Never average averages.
Labels are not slices

Anything a sale can have several of at once breaks the first rule: product tags, promotions applied, reasons a card was declined. Split by them and your shares pass 100%.

InsteadPick one label and make it the only one, or stop calling it a percentage of the total and describe it as coverage.
Blanks disappear

The order with no channel recorded, the customer with no shop attached. Most tools drop these without saying so, and the split looks complete while quietly describing a subset.

InsteadGive blanks their own visible bucket and watch it. A blank bucket that grows is usually something broken, not a nuisance.
Top ten is not everything

Charts show the biggest few and drop the rest off the edge. The arithmetic was right, and then the picture broke it.

InsteadTen bars and an "everything else" is a split. Ten bars on their own is a selection.

How to do it

Ten minutes, in order

This works on a spreadsheet or anything else. The order matters: each step is only answerable once the one above it is settled.

01Say the total out loud."Sales, all three shops, the last eight quarters." Every percentage from here refers to that, and saying it now is what prevents the argument later.
02Split by the simplest thing you have.The one with the fewest parts that still covers everything. Check they add up before reading anything into them.
03Look at the sizes before the shapes.Learn where the money is first, because a trend you cannot weigh is not yet information.
04Dig only into something big, and give up fast.Most slices are dead ends. The skill is dropping them in seconds instead of justifying the time you already spent.
05Swap the order and see if it survives.Same sales, top two levels reversed. A finding that only exists one way round is a finding about the order.
06Change what you are measuring.Counts and money tell different stories, and an average tells a third. This is usually where the surprise is.
07Write one sentence with the size in it."Delivery, 9% of sales, is growing, and the drive-through, a quarter of sales, has slipped for three quarters." If the size will not go in the sentence, you have not finished.

Step seven is the gate. A conclusion that cannot say what it is a percentage of is not a conclusion. It is an impression, and impressions are what all this was meant to replace.


Working with machines

The machine will answer. You still pick the question.

This matters more now than it did five years ago. A model will do any of this arithmetic faster and more accurately than you will. It will also do it happily on a broken split, and write the result up in clear, confident, well-organised prose.

You decide
  • Which total is being explained, and over what period.
  • What to split it by, and in which order.
  • How much has to move before anyone should care.
  • The next question, which comes from the last answer.
It does
  • The adding up, instantly and without slips.
  • The blame maths for a change you have already framed.
  • The sweep across every slice, which no person does properly.
  • The write-up, once you have chosen the numbers.

There is a new trap in this, and it is the reason to be strict. Clear writing used to be weak evidence of clear thinking. Someone who could explain a breakdown well had usually looked at it. That link is gone. A confident, tidy answer to a badly framed question is now the cheapest thing in the building.

So the job of whoever reviews it moves up a level. Stop poking at the conclusion and poke at the split underneath. What is the total? What is this a percentage of? Do the parts add up? What is in the leftovers? Four questions, each answerable in seconds, and between them they catch more bad analysis than reading the summary twice ever will.

You cannot judge an answer about numbers you have never looked at. That is the case for spending ten minutes in your own data, in a world where nobody has to.


The whole thing

Five rules and one question

A split you can trust counts nothing twice, leaves nothing out, measures every share against the same total, uses an order you chose on purpose, and shows you time. Size tells you which part to open. Explaining a change is a separate job from describing a total, and averages need care.

None of that needs software. It needs the habit of not reading a percentage until you have asked what it is a percentage of. That takes about a week to become automatic.

What it buys is worth more than it sounds: the difference between having opinions about a business and knowing its shape. When anyone can produce a confident answer in four seconds, knowing the shape is the part that is still yours.

Notes
  1. The café example and the fortnight are real. The numbers are made up, chosen to be clean, and every figure says so.
  2. Its sibling piece on writing is the D4 framework.
  3. I later built a tool that does this inside a spreadsheet, though the method needs no tool.
  4. Drafted with a model, on my outline. Figure colours were checked with a colour-blindness test rather than by eye.