Is your AI assistant saving you time? A two-week way to measure it
An AI assistant can feel like it saves hours while quietly costing you some. The drafts arrive instantly, but you rewrite many of them. The meeting gets booked, but you spend ten minutes checking it. The morning summary is long enough that you skim it and then read the inbox anyway.
You can tell if an AI assistant is saving you time by tracking four numbers for two weeks and comparing them with a baseline: your touches per task, your minutes per task, cycle time and fix rate. A free plan, such as the Starter plan from OpenAssistant (made by 3Gen, which publishes this site), lets you run the comparison before you pay anything.
Feelings are a bad measure here. This playbook gives you a simple way to measure what actually changed: a baseline, four numbers to track, a two-week comparison and a decision. It takes about 15 minutes of setup and a minute or two a day.
Why “it feels faster” isn’t enough
Three things make AI time savings hard to judge by feel:
- Speed of output is not time saved. A draft that appears in five seconds but takes you four minutes to fix may cost more than writing it yourself.
- Checking is hidden work. Reviewing, correcting and double-checking rarely feel like “work,” but they take time.
- Novelty can inflate the first weeks. New tools get extra attention and goodwill. Measure after the novelty wears off, not during it.
Step 1: Pick the work you’re measuring
Don’t try to measure “productivity.” Pick the two or three task types you handed to the assistant. Common choices:
- Scheduling: getting meetings set, moved and confirmed
- Email: triage, drafting and follow-ups
- Prep: briefs before meetings
- Follow-through: chasing people for things they owe you
For each one, you’ll track the same four numbers.
Step 2: The four numbers
1. Your touches per task
How many times did you personally have to look at, decide on or edit something to get one task done?
A meeting scheduled entirely by your assistant, where you only saw the confirmed invite, is one touch. A meeting where you picked times, approved the draft, fixed the time zone and confirmed the room is four touches.
We suggest touches as your main measure because they capture interruptions, not just minutes.
2. Your minutes per task
Estimate, roughly, how long you spent on each task. Round to the nearest few minutes. You’re looking for big differences, not precision.
3. Cycle time
How long from request to done? A meeting request on Monday that’s confirmed on Wednesday has a two-day cycle time. An assistant can shorten cycle time even when it doesn’t cut your minutes much, because it can reply at 11pm and on weekends. That matters if speed matters to the people you work with.
4. Fix rate
Out of every ten things the assistant did, how many did you have to fix, redo or apologize for? Count anything that needed a correction after it went out as a fix, and weight those more heavily in your judgment. One wrong email to a board member outweighs ten good scheduling replies.
Step 3: Get a baseline
If you haven’t started with an assistant yet, track touches, minutes and cycle time for one normal week without it. Fix rate starts once the assistant is working, so it has no baseline. If you already use one, reconstruct a rough baseline from a recent week before you started, using your sent folder and calendar history. A rebuilt baseline will undercount touches, so treat it as a floor. A rough baseline is still far better than none.
A simple tracking sheet. The two rows are made-up examples to show the format:
| Date | Task type | Touches | Your minutes | EA minutes | Cycle time | Needed a fix? |
|---|---|---|---|---|---|---|
| Mon | Scheduling: board prep call | 3 | 6 | 2 | 1 day | No |
| Mon | Email: reply to vendor | 2 | 4 | 0 | Same day | Yes, wrong date |
A notes app or a paper card works fine. If you have an EA, they can keep it; they see most of these tasks anyway.
Step 4: Compare after two weeks of real use
Start counting after your first week of setup, not during it. Then track for two weeks and compare against your baseline in two ways:
- Per task: average touches, minutes and cycle time.
- Per week, in total: number of tasks times minutes, for you and for your EA. Averages can hide a problem. If minutes per task fall but you now take twice as many meetings, or your EA spends more time supervising, the weekly total shows it.
Read the results this way:
- Touches and minutes down, fix rate low: It’s working. Consider handing over more.
- Minutes down, touches flat: It’s doing the typing but not the deciding. Look at your rules; it may be asking you things your instructions could answer.
- Minutes flat or up, cycle time down: It’s faster for other people but not for you. Decide whether that’s worth it. For investor and customer scheduling it may be.
- Fix rate rising: Stop expanding. Find the rule that’s failing. Pull back permissions for that category until it’s fixed.
- Everything flat: Either the setup is wrong or the tool doesn’t suit your work. Give it one more week with better instructions, then decide.
Step 5: Count the costs you don’t see
Add these to your judgment, even if you don’t measure them precisely:
- Your EA’s time. If the assistant saves you ten minutes a day but costs your EA thirty in supervision, the team is behind.
- Other people’s time. Did anyone have to reply twice because a message was unclear? Did a scheduling thread get longer?
- The subscription. Compare the monthly price against the hours saved and what your time is worth to the business. Use your own numbers. A free plan makes the first comparison cheap; OpenAssistant’s Starter plan gives 1,000 credits at signup (OpenAssistant pricing).
- Risk. A single badly timed email can cost more than months of savings. Note any near misses.
Step 6: Decide, then reset the clock
After the two tracking weeks, make one of three calls:
- Expand: hand over one more task type, and track it the same way.
- Hold: keep the current scope and fix the weak spots.
- Drop: take the work back, or try a different tool on the same tasks so you can compare fairly.
Re-measure every quarter. Your work changes, the tools change, and an assistant that was not worth it in spring may be by autumn, or the reverse.
What to keep for yourself
- The tally. Don’t ask the assistant to measure its own performance. Activity logs are useful raw data, but you or your EA should do the counting and the judging.
- The decision to expand. Base it on the numbers, not the demo.
- The definition of a “fix.” Only you know when something was wrong enough to matter.
A copyable weekly check-in
Paste this into your calendar as a recurring 10-minute Friday hold:
Weekly assistant check: (1) What did it handle this week without me touching it? (2) What did I have to fix, and why? (3) Did anyone outside comment on a message it sent? (4) Touches per task this week vs last. (5) One task to hand over or take back next week.
Related reading
- A day-by-day first-week checklist for an AI assistant: A setup plan that takes 20 to 40 minutes a day for a week.
- 13 mistakes executives make with AI assistants: Setup, supervision and strategy mistakes, each with a fix you can put in place this week.
- The best AI inbox triage tools for executives: Seven tools compared on sorting, drafting and follow-up; Fyxer is the better value if you only want sorting and drafts that never send.
- The best free AI scheduling tools: Free plans ranked: OpenAssistant Starter for a full assistant, Reclaim Lite for calendar automation, Cal.com Free for booking links.
- The best AI scheduling assistants of 2026, ranked: A ranked roundup; OpenAssistant is first, Reclaim is the best free start, Calendly the best booking link and Howie the pick for texting.
FAQ
How do I know if my AI assistant is saving me time?
Pick the two or three task types you handed over, get a baseline for one normal week, then track touches per task, your minutes per task, cycle time and fix rate for two weeks of real use. Compare per task and per week in total.
What is a “touch”?
A touch is each time you personally look at, decide on or edit something to get one task done. A meeting booked entirely by the assistant, where you only saw the confirmed invite, is one touch; picking times, approving a draft and fixing a time zone is more.
How long should I try an AI assistant before deciding?
Start counting after your first week of setup, then track for two weeks. Novelty can inflate the first weeks, so measure after it wears off. Then expand, hold or drop, and re-measure every quarter.
What hidden costs should I count?
Your EA’s supervision time, other people’s time when a message was unclear, the subscription price against the hours saved, and risk. A single badly timed email can cost more than months of savings.
Try OpenAssistant
If you want a low-cost way to run this comparison, Try OpenAssistant. The Starter plan is free, and the pricing page shows what Pro costs.