When I started building TAWK, a macOS menu bar app that converts voice to text, the first real architectural decision I had to make was how to handle speech recognition. The two paths were clear: send audio to a cloud API and get back a transcript, or run a model locally on the user's machine and never let the audio leave the device. I chose local. Specifically, I chose OpenAI's Whisper model running on-device. It was the best decision I made on the entire project, and it was not even close.
This is not a theoretical comparison. I prototyped both approaches, benchmarked them, and shipped one. Here is exactly what I found, why I made the choice I did, and what I would tell anyone facing the same decision today.
The Contenders
When I evaluated speech-to-text options in early 2025, the serious contenders were:
- Google Cloud Speech-to-Text — the industry standard, mature API, excellent language support, real-time streaming capability
- AWS Transcribe — Amazon's offering, strong enterprise features, good accuracy, pay-per-second pricing
- OpenAI Whisper API — OpenAI's own hosted version of Whisper, simple API, competitive pricing
- OpenAI Whisper (local) — the open-source model running entirely on the user's machine, no network required
I tested all four. I recorded 50 audio clips ranging from 5 seconds to 2 minutes, covering different speaking speeds, accents, and background noise levels. Then I ran each one through every service and compared accuracy, latency, and the overall developer experience.
Accuracy: Closer Than You Would Expect
The first surprise was accuracy. I expected cloud APIs to dominate because they have access to massive compute and continuously updated models. And for some edge cases, they did. Google Cloud Speech handled heavy background noise noticeably better, and AWS Transcribe was slightly more reliable with uncommon proper nouns.
But for the core use case of TAWK — a single speaker dictating text in a reasonably quiet environment — the Whisper small model was remarkably competitive. On clean English audio, I measured a word error rate of roughly 6% with Whisper small, compared to about 4-5% for Google Cloud Speech and 5% for the OpenAI Whisper API (which uses a larger model). AWS Transcribe landed around 5% as well.
A 1-2% difference in word error rate sounds like it matters, and in some applications it does. But for dictation on a desktop app, the user is right there. They can see the text as it appears and make quick corrections. The accuracy gap between local Whisper and the best cloud API was not large enough to justify the trade-offs that come with cloud.
Latency: Where Local Wins Decisively
This is where on-device transcription pulled ahead in a way that changed everything. With a cloud API, the audio processing pipeline looks like this: record audio, compress it, send it over the network, wait for the server to process it, receive the response, parse it, and display the text. Even on a fast connection, that round trip adds 500-1500 milliseconds of latency for a short clip. On a slow or congested network, it can be 2-3 seconds.
With Whisper running locally on an Apple Silicon Mac, the pipeline is: record audio, feed it directly to the model, get the text back. On my M1 MacBook Pro, a 10-second audio clip processed in about 1.5 seconds. A 30-second clip took roughly 4 seconds. There is no network round trip, no compression overhead, no server queue. The model has direct access to the audio buffer in memory.
The latency difference between local and cloud is not measured in percentages. It is measured in the difference between an app that feels instant and one that feels like it is thinking.
For a dictation tool, perceived speed is everything. When you stop speaking and expect to see text, every additional second of waiting erodes trust in the tool. Users do not care that your server is fast. They care that the text appears. Local processing gave TAWK a responsiveness that no cloud API could match, regardless of how fast their servers were.
Privacy: The Non-Negotiable
TAWK is a voice-to-text tool. People use it to dictate emails, write documents, compose messages. That audio contains sensitive information — business conversations, personal thoughts, passwords spoken aloud, private medical details mentioned in dictation. Sending all of that to a third-party server was something I was not willing to do.
With cloud APIs, every word the user speaks is sent to Google, Amazon, or OpenAI's servers. Yes, these companies have privacy policies. Yes, the data is encrypted in transit. But the fundamental reality is that the audio leaves the user's device and enters someone else's infrastructure. For a developer tool that people integrate into their daily workflow, that is a significant ask.
With Whisper running locally, the audio never leaves the Mac. It is recorded, processed, and discarded entirely on the user's machine. There is no network request, no server log, no data retention policy to worry about. The privacy guarantee is not based on a company's promises — it is based on physics. The data simply never goes anywhere.
This turned out to be one of TAWK's strongest selling points. Users who would never use a cloud-based dictation tool are comfortable with TAWK because they can verify the privacy claim themselves. There are no outbound network connections during transcription. That is a level of trust that no privacy policy can match.
Cost: Zero Marginal Cost Per Transcription
Cloud speech APIs charge per minute of audio processed. Google Cloud Speech-to-Text costs approximately $0.006 per 15 seconds. AWS Transcribe charges about $0.024 per minute. The OpenAI Whisper API charges $0.006 per minute. These costs seem small individually, but they compound.
A power user who dictates 30 minutes of audio per day would cost roughly $0.18 per day with Google, $0.72 per day with AWS, or $0.18 per day with OpenAI's API. Over a month, that is $5.40 to $21.60 per user. Over a year, $65 to $260 per user. For a $19 one-time purchase app, that math simply does not work.
With local Whisper, the cost per transcription is exactly zero. The model runs on the user's own hardware. There is no API key, no billing account, no usage limits. I do not have to worry about a popular user costing me money or about implementing usage caps that degrade the experience. The business model is clean: the user pays once, and the software works forever.
Reliability: Works Without Internet
Cloud APIs require an internet connection. That is obvious, but the implications are subtle. It means your app is only as reliable as the user's network. It means transcription fails on airplanes, in basements, in rural areas, and during the brief moments when Wi-Fi drops between access points.
Whisper running locally needs no network at all. Once the model is loaded into memory, it works everywhere. I have used TAWK on flights, in coffee shops with terrible Wi-Fi, and in my office when the internet went down for maintenance. It never skipped a beat.
There is also the reliability of the service itself. Cloud APIs have outages. Google Cloud had multiple Speech-to-Text incidents in the past year. AWS Transcribe has had degraded performance windows. When your core feature depends on someone else's uptime, you inherit their reliability problems. With local inference, the only thing that needs to work is the user's Mac. That is a much simpler reliability equation.
The Trade-offs: What You Give Up
I would be dishonest if I pretended local inference was strictly better. There are real trade-offs, and you need to understand them before choosing this path.
App Size
The Whisper small model adds approximately 461 MB to the application bundle. That is significant. TAWK's total app size is around 500 MB, and the vast majority of that is the model weights. A cloud API version of the same app would be under 20 MB. For distribution, this means longer downloads and more disk space. I mitigated this by using the small model rather than medium or large, which was the right accuracy-size trade-off for dictation.
Initial Load Time
When TAWK launches, it needs to load the Whisper model into memory. On an M1 Mac, this takes 3-5 seconds. On older Intel Macs, it can take 8-10 seconds. This is a cold start penalty that cloud APIs do not have. I addressed this by loading the model at app launch (since TAWK runs as a menu bar app, it starts with the system) and showing a brief loading indicator. Most users never notice because the model is ready by the time they first trigger dictation.
No Real-Time Streaming
This was the biggest functional trade-off. Cloud APIs like Google Cloud Speech offer real-time streaming transcription — you see words appearing as you speak. Whisper processes audio in batches. You record a chunk, then Whisper transcribes the whole chunk at once. For TAWK, this means you speak, stop, and then the text appears. It is not a live transcription experience.
For some applications, this is a dealbreaker. Live captioning, real-time subtitles, and voice assistants that need to respond mid-sentence all require streaming. But for dictation — speak a thought, see it as text — batch processing works perfectly. Users quickly develop a natural rhythm of speaking in short bursts and pausing, which aligns well with Whisper's batch model.
Hardware Requirements
Running inference on-device means the user's hardware matters. The Whisper small model runs comfortably on any Apple Silicon Mac, but older Intel Macs with limited RAM can struggle. I had to set minimum system requirements and accept that some older machines would not provide a good experience. Cloud APIs offload this entirely — even a ten-year-old laptop can send audio to a server.
PyInstaller, Signing, and the Bundling Reality
Choosing local Whisper also meant dealing with the practical challenges of bundling a machine learning model inside a macOS app. TAWK is built with Python, packaged with PyInstaller, and distributed as a signed and notarized .app bundle. Getting Whisper to work inside that pipeline was nontrivial.
PyInstaller does not automatically bundle Whisper's asset files — things like mel_filters.npz and the model weights. I had to explicitly add these to the datas list in the spec file. Then came code signing: macOS requires every Mach-O binary inside an app to be individually signed. Whisper's dependencies (PyTorch, NumPy, and their compiled extensions) include dozens of .so and .dylib files, plus standalone executables. Each one needs to be signed individually. The --deep flag for codesign is tempting but unreliable for notarization — I had to sign everything from the inside out.
Apple's notarization process also has quirks with large ML apps. The upload can take a while due to the model size, and I have seen notarization get stuck "In Progress" for extended periods with PyTorch-heavy bundles. These are solvable problems, but they add real development time that you would not face with a lightweight cloud API client.
The Verdict: Local Was the Clear Winner
For TAWK's specific use case — a desktop dictation tool sold as a one-time purchase — local Whisper was the right choice by a wide margin. The privacy guarantee, zero marginal cost, offline reliability, and low latency aligned perfectly with what the product needed to be. The trade-offs (app size, cold start, no streaming) were manageable or irrelevant for the use case.
But I want to be clear: this is not a universal recommendation. If I were building a real-time transcription service, a voice assistant, or a mobile app where storage is constrained, I would likely choose a cloud API. The right answer depends on your use case, your business model, and your users' expectations.
What I do recommend universally is prototyping both approaches before committing. I spent two days benchmarking cloud APIs against local Whisper with real audio samples. That investment saved me from making assumptions and gave me hard data to base the decision on. Whatever you are building, test with your actual audio, your actual hardware, and your actual network conditions. The benchmarks that matter are yours, not someone else's blog post — including this one.