Asking questions of data

Knowing the shape

Anyone can get an answer now. Almost nobody has looked at the shape of it. You cannot judge an answer about numbers whose shape you have never looked at, and ten minutes buys you that.

The scene

The answer that was true and useless

Open the banking app and everything is there. Every account connected, every card linked, nothing hidden behind a second login you forgot about. The app says eating out is 46% of your spending this month.

Someone reads that number aloud like it settles something: "Nearly half of what we spend is meals out."

It isn't half of what you spend. It's 46% of the app's "discretionary" view, the slice left over once housing, bills and the rest are stripped out, and discretionary itself is about 9% of everything leaving your accounts. Multiply the two and eating out comes to roughly 4p in every pound that leaves the house.

Nothing is missing here. Every transaction is in the app and every account is linked, so this isn't a case of connecting one more card and getting a truer number. The 46% was true. It was true of a total that was never the whole of anything, and reading it as though it were is the entire mistake.

So you cut the lunches out, a full month of packed sandwiches with no exceptions. At the end of the month the total leaving your accounts hasn't fallen, and if anything it's a little higher than it was.

The reason sits somewhere the app never points a finger at. The weekly shop, close to a fifth of everything you spend, has been quietly climbing for months, and none of that increase came from eating out.

This is the move behind almost every question anyone asks of a number. Take a total, split it into parts, pick the interesting one, and split that into parts too. Every piece of analysis you have ever read is a chain of exactly this, several links long, and the quality of the thinking rests on whether every link still adds up to the total above it.

The thing to fear is not a wrong number. It is a right number sitting on a broken split, because the two look identical on the screen.

This piece takes the total, everything leaving your accounts this month, and the cut, a month without lunches out, as given, and interrogates the number that told you to try the cut in the first place. Say the total out loud and say what it leaves out: it excludes anything moved into your own savings, because that money never left the household, it only changed pockets.

Nobody lied, nobody made an arithmetic mistake, and the app hadn't missed a thing. One question was never asked, and asking it takes about ten seconds.


The method

The shape of a total

A split is only as good as three things, and each one makes it true rather than merely tidy. One thing belongs to one part. Nothing gets left out. And every share is measured against the same total.

That third rule is the one the opening scene broke. A share quoted against an unstated base isn't a true statement about your spending, it's a true statement about a different, smaller total that nobody named out loud. It belongs in the tier that makes a split true, not the tier that makes it useful, because a useful split built on the wrong base is still wrong.

Here is the same money, arranged so you can watch the rule break and un-break with one click.

Money out

Discretionary spending is 9.0% of everything leaving your accounts, and its four parts, eating out, entertainment, shopping and subscriptions, add up to exactly that. Press the toggle and the same four numbers re-base against the slice instead of the whole. Eating out becomes 46%, and it hasn't changed by a penny. Both sentences are true. One makes eating out the story. The other makes it a rounding error, and a month of packed sandwiches sat squarely inside the gap between them.

What's left, once a split is true, is what makes it useful: time on the screen. One month's numbers tell you size and hide direction entirely. Two categories the same size, one growing steadily and one shrinking just as steadily, look identical in a single snapshot.

There is a rule of thumb worth keeping for the bucket you cannot categorise. Compare its size to the movement you are trying to explain. A leftovers bucket bigger than that movement can hide the whole answer inside itself, and no amount of care with the labelled parts will find it there.

Real bank data breaks these rules in ordinary, boring ways. Three are worth knowing by name.

Cash withdrawals disappear

A cash machine withdrawal shows up as a single unlabelled outflow. Whatever it actually bought never gets categorised, so a real slice of spending sits in a bucket that tells you nothing.

InsteadGive the blank bucket its own visible line and watch its size. Large and growing means the split has stopped covering what you actually do.
A transfer to yourself is not a spend

Move money into your own savings account and the app happily counts it as an outflow, in the same column as rent. Nothing left the household, it just moved to a different pot.

InsteadStrip out transfers between your own accounts before you call the rest spending. Otherwise every part is right and the total is a fiction.
One trip, two labels

A big supermarket sells petrol too. One trip, one receipt, two things that belong in different slices, the weekly shop and transport. Left as a single row, it inflates whichever category the app happened to guess.

InsteadSplit the receipt, or accept the row is unreliable and say so, rather than trusting the app's single guess.

Where to look

Where the money actually is

A true split is a map you can trust. It says nothing about where on that map to walk, and a trustworthy map of somewhere irrelevant still wastes your evening.

The order you split by is a choice, usually made without noticing. Category then account is one way down the same money. Account then category is another way down it, and the two can leave you with different opinions about where the problem sits.

Where the money actually is

One measure, so colour shows size rather than identity. The slice we dug into is picked out in rust.

Housing Weekly shop Bills and utilities Transport Discretionary Savings and transfers Everything else 31% 22% 14% 11% 9% 8% 5%

Illustrative figures for a household budget, chosen to be clean. They add to 100.

Household spending is lopsided in the same way almost everything is. A handful of categories are most of it, and the smallest is worth about a penny in the pound.

  • Never open a slice without knowing its size first. A dramatic swing inside 1% of your spending is a dramatic swing inside 1% of your spending.
  • Something big moving a little usually beats something small moving a lot, and this is the hard one, because the small thing moving violently makes the better chart.

For a total as it stands, that is the whole rule: size decides where you dig. A change over time needs one more idea, which is where the next section goes.

None of this is an argument for ignoring the small stuff. Eating out is 4p in the pound and it has doubled in six months, from around £60 to around £120. That is not nothing, and this piece cannot pretend it is. Put next to it, the weekly shop rose by more in the same six months, from £580 to £660, an £80 climb against eating out's £60. The smaller line doubled and made the better story. The bigger line moved further and made no story at all.


A different question

What moved, and what only looks like it moved

Everything so far describes a total as it stands today. The question most people actually walk in with is a change: it went up, or down, and why.

A change splits too, and the pieces have to add up to the movement. But size and blame are unrelated. A big line can explain nothing, because it did not move. A small line can explain everything, because it fell off a cliff.

Say contributions in pounds, not percentages. When a total barely moves, contribution percentages run past 100 and swing negative while every underlying number is perfectly correct, which is confusing rather than useful.

The cut worked. The total still rose.

Each category's contribution to the change, in pounds. They add up to the movement, or the answer is not finished.

pushed spending up pulled it down
Last month Weekly shop Discretionary Transport Bills Else This month +£46 −£15 +£12 +£9 +£8 £2,940 +46 -15 +12 +9 +8 = £3,000  ·  IT ADDS UP

Illustrative. Cutting the lunches out saved £15, a real fall. The weekly shop alone added three times that back, and the total rose regardless.

Averages, rates and margins do not add up the way spending does, and this is where that matters. Averaging two averages is not simply wrong, it answers a different question, and the two only agree when the groups being averaged are the same size. Split the top and the bottom of the fraction separately, then divide once, at the level you actually care about.

Here is the hard one, and the most valuable idea in this piece. A total can get worse while every part of it genuinely improves. Price per item fell at the supermarket over the year, and it fell at the corner shop too. Nobody paid more for anything. Spend per trip crept up anyway, and it would be easy to blame the shops for that. It was not the shops. It was which one you were in: more of the shopping shifted to the pricier corner shop, so the blend rose even though nothing got dearer.

Both prices fell. The blend rose.

Price per item at each shop, and the two blended together.

supermarket corner shop both together
down up down Q1Q2Q3Q4 PRICE PER ITEM CORNER SHOP GREW FROM 20% TO 50% OF ITEMS BOUGHT

Illustrative. Supermarket £1.80 to £1.65, corner shop £2.60 to £2.40, blended £1.96 to £2.03. Report only the dashed line and you will describe prices rising that never rose anywhere.

The defence against this is not cleverness, and it is not automatic either. Shares sitting next to values only reveal a shift along the dimension you actually split by. Here that dimension was which shop, and splitting by it is what made the shift visible. A blend that moves while every part you can see is improving is the signal that the shift lives on a dimension you have not split by yet, and the next move is to go and find it.

Getting worse and changing shape need opposite responses, and a total on its own cannot tell you which one you are looking at.


Working with a model

Ten minutes, and why they still matter

A model does this arithmetic faster and more accurately than you ever will, and it will do it just as happily on a broken split, then write the result up in confident, well-organised prose that gives no sign anything is wrong.

You decide
  • Which total is being explained, and over what period.
  • What to split it by, and in which order.
  • How much has to move before it is worth caring about.
  • The next question, which comes from the last answer.
It does
  • The adding up, instantly and without slips.
  • The blame maths for a change you have already framed.
  • The sweep across every slice, which nobody does properly by hand.
  • The write-up, once the numbers are chosen.

There is a trap in this, worth naming plainly. Clear writing used to be weak evidence of clear thinking, because someone who could explain a breakdown well had usually looked at it first. That link is gone. A confident, tidy answer to a badly framed question is now the cheapest thing in the room.

The same four questions work just as well on someone else's numbers: a deck, a factsheet, a chart in the news. It is the same arithmetic a company runs on its own accounts, with fewer zeroes and a banking app instead of a finance team.

  • What is the total?
  • What is this a percentage of?
  • Do the parts add up?
  • What is in the leftovers?

Ask them of the source, not of the answer. If the only thing that can confirm the parts add up is the thing that produced them in the first place, you have checked nothing at all.

None of this means the reviewer's job moves up a level or down one. Plainly: stop questioning the conclusion, and start questioning the split underneath it.

Ten minutes is enough to do this properly, in roughly this order.

Say the total out loud."Everything leaving our accounts, the last six months." Every percentage from here refers to that, and saying it now is what prevents the argument later.
Split by the simplest thing you have.The one with the fewest parts that still covers everything. Check they add up before reading anything into them.
Look at the sizes before the shapes.Learn where the money is first, because a trend you cannot weigh is not yet information.
Dig only into something big, and give up fast.Most slices are dead ends. The skill is dropping them in seconds instead of justifying the time you already spent.
Swap the order and see if it survives.Same money, top two levels reversed. A finding that only exists one way round is a finding about the order.
Change what you are measuring.Totals, counts and price per item tell different stories, and an average tells a fourth. This is usually where the surprise is.
Write one sentence with the size in it."Discretionary, 9% of spending, includes a real cut in eating out, and the weekly shop, over a fifth of spending, has climbed for six months." If the size will not go in the sentence, you have not finished.

This is not a script, it is a starting shape. Each step's answer picks the next one, and a slice that turns up nothing is progress, not a wasted step. That is the one thing the fold from ten sections to five very nearly dropped, and it earns its paragraph back here.

A conclusion that cannot say what it is a percentage of is not a conclusion. It is an impression.

Notes
  1. The household and its banking app are composed, not lived. There is no real reader behind these numbers. They are illustrative, chosen to be clean, and every figure says so.
  2. Its sibling piece on writing is the D4 framework.
  3. I later built a tool that does this in a spreadsheet, though the method needs none.
  4. Drafted with a model, on my outline. Figure colours were checked with a colour-blindness test rather than by eye.