Start here · Lesson 1.5
Measuring What to Automate With AI: Four Numbers to Track
Once you have picked what to automate with AI, the next question is whether it is working. Most people measure activity: how many prompts they ran, how many drafts came out. None of that tells you whether your week got better. A handful of outcome numbers does, and you can track them in a notes app in two minutes per run.
This guide covers four numbers and a weekly check. The figures in the examples are invented to show the arithmetic. Lesson 1.5, Find the jobs to hand to AI, covers choosing and briefing the job that these numbers then measure.
Track net minutes saved, errors caught, edit effort and whether you still use it. If those hold up for a few weeks, the hand-off is working.
Activity versus outcome
Activity numbers are easy and misleading. "I generated 40 drafts this month" says nothing about whether any of them were used. Outcome numbers ask what changed in your work:
- Did the job take less of your time, counting the checking?
- Did mistakes get through?
- Did you need to rewrite most of it?
- Are you still using it a month later?
These are the only ones worth the effort for a small business or solo creator.
The four numbers
| Number | What to record each run | What it tells you |
|---|---|---|
| Net minutes | Time by hand (baseline) minus time to supply inputs and check | Whether it saves anything |
| Errors caught | Count of mistakes you found before sending | How much you must keep checking |
| Edit effort | Light, medium or heavy rewrite | Whether the brief is good enough |
| Still used | Yes or no at week four | Whether it earned a place |
1. Net minutes
Before you start, time the job by hand twice and take the average. That is your baseline. After a hand-off run, time everything you do: pasting inputs, running, reading, fixing. Subtract. Count the setup separately, since it is a one off, and note how many runs it will take to pay back.
Illustration with invented numbers: baseline 30 minutes. With AI, 5 minutes to supply inputs and 7 to check and edit, so 12. Net saving is 18 minutes a run. If setup took 60 minutes, it is repaid after about four runs. Your numbers will differ. If net minutes are zero or negative after a few runs, the job or the brief needs work.
2. Errors caught
Every run, note how many mistakes you found: wrong names, wrong numbers, invented details, wrong tone. AI tools can make mistakes, and the vendors say so on their own pages, so a count of zero for a long time is more likely a sign you stopped looking than a sign of perfection. Keep the count. If it falls over several runs, your brief is improving. If it rises, something changed, perhaps the inputs.
Also record any error that got past you. One is enough to tighten your checking step.
3. Edit effort
After each run, mark it light (a few words changed), medium (a paragraph reworked) or heavy (rewritten). This is a quick proxy for brief quality. Anthropic's prompting guidance lists examples among the most reliable ways to steer format, tone and structure, and says to give context about the audience and goal. If your edits are mostly heavy, add a better example or more context before blaming the job.
4. Still used at week four
This is the honest number. Did you use the hand-off this week without being nagged by your own list? A job you abandon after two weeks was either a poor choice or too much effort to check. That is information, not failure.
What not to measure
- Number of prompts or drafts: activity, not outcome.
- Hours saved per year extrapolated from one run: too speculative to trust.
- How impressive the output looks: judge it against your standard instead.
- Promised industry averages: we have no verified figure for your work and neither does a generic article.
Your weekly tracking sheet
Keep one row per run. A notes app or spreadsheet is fine.
- Date and jobWrite the date and job name.
- Minutes by hand (baseline)Fill in once, then copy down.
- Minutes with AIInputs, run, check and edit combined.
- Errors caughtA count, and a note on any that got through.
- Edit effortLight, medium or heavy.
At the end of each week, add a line: what changed in the brief, and whether you will keep running the job.
A weekly review in ten minutes
Once a week, read your sheet and ask:
- Are net minutes positive and steady?
- Are errors going down or staying low?
- Is edit effort light for most runs?
- Did anything leak out that I should have caught?
- Do I still want this job handed over?
If three or more answers are yes, keep going. If net minutes are flat and edits are heavy, fix the brief, usually the example or the "what to avoid" part. For help, read How to Brief AI Like a Senior Hire.
Deciding when to stop or move on
- Stop if net minutes stay at or below zero after several runs and a brief rewrite.
- Hold if numbers are fine but you are still catching errors. Keep checking.
- Move on to a new job only when the current one has been stable for two or three weeks. Choose the next one from your scored list; the method is in the Find framework.
If you use a tool that works on files on your own computer, such as Claude Code, which its documentation describes as able to edit files and run commands, track the same four numbers. A more capable tool makes the error count more important.
Common measuring mistakes
- Forgetting to time the manual baseline, so there is nothing to compare.
- Leaving out the checking time, which flatters the result.
- Judging after one run, when the first run carries the setup cost.
- Changing the brief and the job in the same week, so you cannot tell which helped.
Questions people ask
Do I need to track all four numbers?
Start with net minutes and edit effort. They take seconds and tell you most.
How many runs before I judge?
Three to five. The first run carries the setup cost and the brief usually needs a fix.
Is there a benchmark I should hit?
We know of no verified benchmark for your type of work. Compare with your own baseline.
What if net minutes are positive but I dislike the output?
Then quality matters more than time for that job. Improve the example in the brief or leave it manual.
Where to go next
Time your baseline this week, then run one hand-off and fill in one row. For a guided way to choose and brief the job you will measure, Lesson 1.5 walks you through it: Find the jobs to hand to AI.