Offline therapy notes — how on-device transcription actually works

Offline transcription stopped being a fantasy in 2024. A clinician's walk-through of what runs on the iPhone, why the audio doesn't have to leave it, and how to verify a vendor's claim with a thirty-second test.

The phrase offline therapy notes sounded, until recently, like a contradiction.

Therapy notes, in the modern sense, are AI-drafted from a transcript of the session. The transcription requires a speech model. The drafting requires a language model. Until 2024 those models were too big to run on a phone. So when a therapist asked a vendor “does this work offline?”, the honest answer was always no — even when the marketing copy implied otherwise.

That answer changed in the last twenty-four months. Offline therapy notes are now a real product category. This post is the version of the explanation a clinician should expect, not the engineering deep-dive.

What “offline” actually means here

A tool is offline if it produces a transcript and a draft note without contacting any server. Not just “encrypted in transit.” Not just “HIPAA-compliant.” Not just “your data is safe.” The traffic counter is zero.

The honest test takes thirty seconds: turn on Airplane Mode, record a short clip, and tap Transcribe. If the transcript appears, the tool is offline. If it spins or errors, the tool was uploading the audio somewhere. The full version of that test, with packet inspection, is on the glossary page.

What runs on the device

There are two places work happens during an offline therapy note: the transcription step and the note-drafting step. Both fit on a recent iPhone.

The transcription step

Two on-device speech models are now production-grade for therapy:

1. Apple's Speech framework — the system-level speech recognizer that ships with iOS. The same engine that powers Siri and dictation. Accurate, fast, and tightly integrated with the Apple stack. Available on every supported device.
2. Parakeet TDT v3 — NVIDIA's open-source ASR model, ported to Apple Silicon by the FluidAudio team. On a 120-session talk-therapy corpus, Parakeet runs at roughly 210x real-time and produces a Word Error Rate of about 6.4% — within striking distance of cloud-grade Whisper. A 50-minute session transcribes in well under a minute.

Both models execute on the iPhone's Neural Engine. The audio is processed in chunks, the model produces text, and the transcript appears with each speaker labeled. Nothing crosses a network boundary.

The note-drafting step

The drafting model has to be smaller than the transcription model — language models are bigger than speech models — but it also has to be good enough to write a clinical note that a licensed therapist will sign. Two options now meet that bar on-device:

1. Apple Foundation Models (iOS 26+, iPhone 15 Pro and newer M-series iPads) — Apple's system-level language model, with structured-output support tuned for templates. Drafts a SOAP note in seconds without leaving the device.
2. Phi-4-mini (Microsoft, ~3.8B parameters) — a compact reasoning model that runs at acceptable speed on every supported iPhone, used as the fallback when Foundation Models is not available.

The drafting model reads only the transcript, never the audio. The transcript is your ground truth; the note is a structured re-rendering of it into SOAP, DAP, BIRP, EMDR, Intake, or Progress.

Why this matters for therapy notes specifically

Therapy sessions contain some of the most sensitive content in healthcare. Trauma history, current suicidal ideation, custody-relevant disclosures, EMDR processing blocks. The asymmetry of harm from a breach is severe — and the natural question for any session-recording workflow is “what happens to this audio?”

With cloud transcription, the answer is some variant of “it sits on a vendor's server, encrypted at rest, retained per their policy, accessible to their employees with appropriate role-based access, and recoverable under subpoena.” With offline transcription, the answer is “it never left your device.”

The two answers carry different risk profiles for the patient and different liability profiles for the practice. Privacy by architecture vs HIPAA is the long version of why those answers are not equivalent.

The trade you are making

Offline therapy notes are not strictly better than cloud therapy notes. There is a real trade. Three pieces of friction come with the architecture:

1. No API push to your EHR. The signed note is copy-pasted into SimplePractice or TherapyNotes. An automatic push would require an authenticated handle on the EHR, which would re-introduce a server in the loop. Therapists in beta tell us the paste takes about six seconds and they stopped noticing it after a week.
2. Storage on the device. The speech and language models are big — somewhere between three and eight gigabytes depending on which models you use. Cloud tools download nothing because the work happens on someone else's machine. Offline tools require the storage on yours.
3. Device-bound history. Your session history lives on the iPhone. If you replace your phone, restoring from encrypted iCloud Backup brings the locked notes with you, but Apple does not have the keys — only your passcode and FaceID can unlock them. That is the right answer for confidentiality. It also means a lost-and-not-restored phone is a lost record.

The trade is real. We think — and the therapists we've sold to think — it is the right one for this category.

How to evaluate an offline therapy notes app

Five things to check before you trust one:

1. The Airplane Mode test passes. Record, transcribe, and draft a note in Airplane Mode. If any step fails, the tool is not offline.
2. The privacy proof is verifiable, not asserted. A live network counter inside the app is better than a sentence on a marketing page.
3. The transcription model is named. “On-device AI” is too vague. Apple Speech and Parakeet TDT v3 are specific, falsifiable claims.
4. The drafting model is named. Same reason. Apple Foundation Models, Phi-4-mini, Gemma — these are specific, you can check the licenses, you can know what you're running.
5. The compliance posture matches the architecture. An offline tool that asks you to sign a BAA is signaling that some part of the workflow still touches their server. The honest offline product needs no BAA because it never receives PHI.

Bryl is built to those five tests. The architectural attestation is on the security page; the side-by-side with cloud-based competitors is at Bryl vs Mentalyc, Upheal, Heidi, Blueprint, Eleos, Twofold.

If you are looking for an offline therapy notes app and the tool you are evaluating fails any of those five tests, you are looking at a cloud product with a privacy-flavored marketing skin. The architecture is the only part of this category that matters.