Run your day · Lesson 6.3
A One Week Trial of AI Meeting Notes: Plan and Scorecard
Does AI meeting notes work outside a demo? The honest way to find out is to run a short trial on your own calls and write down what happens. This post gives you a seven day plan, a scorecard, and an illustrative example so you know what a trial can look like.
One thing first. The example below is made up to show the method. It is not a real result from anyone, and the numbers are placeholders you will replace with your own.
Run the trial on real calls for seven days, time the work, count the mistakes, and decide on the numbers you wrote down, not on how it felt.
Before day one: set the rules
Pick the calls you will include. Five to ten calls is enough. Decide up front what counts as a success, for example "notes sent within the hour on most calls" and "fewer than one wrong next step per week". Write it down now, because people who decide afterwards tend to find that whatever happened was a success.
Choose one capture method and one AI assistant and stay with them all week. Changing tools mid trial makes the results hard to read. Also settle consent: tell guests you are recording and get a yes.
The seven day plan
- Day 1, test the captureRecord a short call or a voice memo of yourself talking. Listen back. Fix volume or placement if you cannot hear it well.
- Day 2, first real callRun the full routine on one call. Time each stage: capture, transcript, AI notes, checking, sending.
- Day 3, standard instructionsSave the instructions that worked and reuse them. Note what the AI got wrong and add a line to prevent it.
- Day 4, normal dayUse it on every call. Do not skip any, even if rushed. Skipping hides the hard cases.
- Day 5, find the weak spotLook at which part took longest or went wrong most. Adjust that part only.
- Day 6, test the follow upSend notes to everyone within the hour and note how many reply or correct something.
- Day 7, reviewFill in the scorecard and decide: keep, change or drop.
The scorecard
Make one row per call and fill in these columns on paper or in a spreadsheet.
| Call | Minutes of notes work | Errors found in next steps | Sent within an hour? | Did anyone reply or correct? |
|---|---|---|---|---|
| Example call A | your number | your number | yes or no | yes or no |
| Example call B | your number | your number | yes or no | yes or no |
Before the trial, estimate how long you normally spend on notes per call, or time two or three calls the old way. That gives you something to compare to.
An illustrative example
Say you are a freelance designer with six client calls in a week. This is how the trial could go, with invented figures used only to show the shape.
On day one, the audio is poor because the laptop is far from you, so you move it closer. On day two, the first set of notes mixes up two people's tasks. You add a line to your instructions: "Where you cannot tell who said something, write unknown." On day three the problem goes away. On day five you notice that checking takes longer than the AI step, so you start reading the next steps first and skimming the rest. On day six, one client replies to correct a date. You are glad the follow up line asking for corrections was there.
At the end, you look at your scorecard. If checking time plus setup time is below what you used to spend, and the errors were caught before sending, you keep the routine. If it is not, you try to fix the weakest stage or decide that handwriting suits your type of calls.
None of those moments are impressive. That is the point: a trial turns vague feelings into a short list of things to change.
What can go wrong in a trial
Poor audio is the most common problem. A slow checking step is second. A third is forgetting to ask for consent on a call, which can be a legal issue depending on where you are, and is always a trust issue. A fourth is judging on one unusual call, for example a very long group meeting, and assuming all calls are like that. Try to include a mix.
Notice also what the AI cannot know. It does not know that "Thursday" meant next Thursday, or that someone was joking. That is why the human review step stays.
Reading the result
After seven days, ask three questions. Did the notes go out faster than before? Were errors caught before they reached anyone? Would I be happy to do this for the next month? If the answers are yes, yes, yes, keep it. If you got two yes answers, fix the weak point and run another short week. If you got none, drop it without guilt.
Do not worry if the answer is mixed. Many people keep AI notes for long calls and use handwriting for short ones. Our post on AI notes versus handwriting helps you decide per call type.
After the trial
Turn what you learned into a standing routine. Keep the instructions that worked, keep the folder where you store notes, and keep a weekly five minute look at open next steps. The lesson Notes and next steps from every call covers the full routine, and the numbers you collect in this trial feed into the measures in the metrics post.
The full routine is taught in the lesson Notes and next steps from every call.
Questions people ask
How many calls do I need for a fair trial?
Five to ten is a sensible minimum. Include different types, such as short and long, one on one and group.
What should I compare against?
Your own old routine. Time two or three calls the old way before you start.
What if I only have one call this week?
Run the trial for longer. A trial needs enough calls to show a pattern.
Should I tell guests about the trial?
In many places you must tell them if you record, and it is good practice everywhere. You do not need to explain the trial, but be honest if they ask.